I still wish it was a little better, but there's hope for another model checkpoint (maybe with some of GLM 5.2's goodness distilled into it, that would be nice).

Edit:

> GPT-5 does the job.

I bring up DeepSeek V4 Flash a lot on HN, but I want to mention that according to Artificial Analysis, it trades blows with GPT-5 (high) (from August, 2025) [0]

[0]: https://artificialanalysis.ai/models/comparisons/deepseek-v4...

The trick is not bringing React into the terminal.

(FWIW, I have a link to a TUI harness in my profile that uses 50MB of ram and about 1% CPU while streaming, even in giant contexts)

I think you’re looking for an answer you’re not going to get unfortunately. I think there actually is a higher than average risk of data leakage with the insane optimizations that go into model serving - GLM5.1 had an issue of going into jibberish when their infra was under high load, and it turned out to be a cross-request KV cache contamination issue.[1]

Personally, my effort has been to use local models only as of late, and it’s gone pretty well!

[1]: https://z.ai/blog/scaling-pain

I'm not willing to outsource the understanding how things work part of myself. That part of myself is what got me into computing in the first place.

If this work becomes simply a matter of describing intent to a machine (probably through an Issue, like a user), and going to check on the result when you get the 'done' notification: I'm done.

It's possible to use the tools to do awesome things without letting go of full system understanding of the parts that you look after.

That sounds like a lot of work. I don't want to sign up to this much work for every product I own that I want an iota of control over.

So I would argue if this is "better" by any stretch of the word

To be clear: I agree fundamentally that there is no safe way to have agents connected to the world in a way that allows them to take irreversible actions. Deployments where agents can take destructive actions are deployments where the agent will, eventually, take destructive action.

You draft n tokens, and you verify them in a single forward pass.

Here's the vLLM flag:

    --speculative-config '{{"method":"mtp","num_speculative_tokens":2}}'

It works great. I'll keep my increased performance, and

> so i don't know why you are punching these documents into the chatbot, and asking it questions about them, and then it gives you the wrong answers

you keep whatever this is. I posted direct quotes from their papers which say "it speeds up inference" (paraphrasing). I don't feel there is anything I can do to turn this into a good-faith discussion. Beep boop.

(emphasis mine)

> Instead of predicting just the next single token, DeepSeek-V3 predicts the next 2 tokens through the MTP technique. Combined with the framework of speculative decoding (Leviathan et al., 2023; Xia et al., 2023), it can significantly accelerate the decoding speed of the model.[2]

> As DeepSeek-V3, DeepSeek-V4 series also set MTP modules and objectives. Given that the MTP strategy has been validated in DeepSeek-V3, we adopt the same strategy for DeepSeek-V4 series without modification.[3]

[1]: https://arxiv.org/pdf/2412.19437#subsection.2.2

[2]: https://arxiv.org/pdf/2412.19437#subsubsection.5.4.3

[3]: https://arxiv.org/pdf/2606.19348v1#subsection.2.1

Side comment: I feel you may be too cynical towards your fellow commenters.

Moving to 2 sparks meant switching to vLLM with 2-way tensor parallelism and working multi-token prediction. The parallelism and MTP on top of better tuned kernels[1] gave an extremely nice boost! I was quite pleased. I've seen bursts up to 60tok/s at ~150k context - sometimes the MTP seems to really kick in (i.e. high acceptance rate on its tokens)

Currently running a custom vLLM build put together by some folks on the Nvidia forums[2], which speaks to how early support for the model is.

[1]: https://github.com/lukealonso/b12x

[2]: https://forums.developer.nvidia.com/t/372268

Now at 40-50tok/s generation and ~2000 tok/s prefill with a model that I've seen reason through race conditions and be able to trivially pull off any straight-forward coding task, and remain coherent at 500k context. With a preview checkpoint of the weights!

I'm excited for the future of local LLMs. There is some buy-in but apparently not an extreme amount to get access to models that can stand in the for the giants on all but the most challenging and/or hands-off coding tasks.

[1]: https://github.com/antirez/ds4

> Moving forward, the industry cannot continue to train bigger and bigger models since their intelligence not only plateaus but often will get worse

These are wild claims - why are we concluding that bigger models and more data = more hallucination? That’s actually the opposite of what’s been happening over the last couple years. Some models may still hallucinate more but they all hallucinate much less than the original 175B ChatGPT which was smaller and trained on (much) less data than anything current.

Edit: My mention of data comes from this quote:

> A shift is happening among major AI labs, who are becoming increasingly skeptical of endless parameter count and training data scaling

My take on the current situation: it seems clear that the industry has seen that there is still a lot left to squeeze out of sub-1T models. But for that you do need more, high-quality data in the distribution which you want to unlock capabilities for.

I'm not sure if we'll become less intelligent. I think our sacks of neurons are gonna keep on making associations, just across a different set of topics.