- Reasoning effort = 1 is now actually usable and produces 2.5x less tokens than old weights, without intelligence lost.

- Overall 20% less tokens at each reasoning effort.

- Same intelligence and same tool-calling pattern.

More details in blog post https://sference.com/blog/deepseek-v41f-1017-benchmark

The fix turns that single slot into a set, capped per request by long_prefill_token_threshold. Most of the work was in the places that assumed only one partial prefill could exist: KV reservation, output streaming, parking chunks under memory pressure, and spec-decode tail tokens.

In that scenario the big request slowed from 105s to 112s. The six requests behind it went from about 105s to between 0.8s and 2.4s.

It isn't free. If every request is huge, FCFS is already the right policy and chunking only adds overhead. A lone long prefill still pays the per-round cap even when nothing else wants the budget. Peak KV usage also rises somewhat. The threshold is a ceiling today, and I'd rather it were a floor, so concurrency adapts at runtime without tuning.

Upstream PR: https://github.com/sgl-project/sglang/pull/34623

I'm happy to go into scheduler details. I'd especially like to hear from anyone who has handled this differently, for example with prefill/decode disaggregation or priority queues.

Comments URL: https://news.ycombinator.com/item?id=49856359

Points: 1

# Comments: 1

I'm getting negative reaction for providing an obvious solution. Anthropic stores all your prompts as well and uses it for training.

Engineering shouldn't be an academic pursuit of "good engineering" for its own sake. It is fundamentally a business function. The objective is to achieve real world goals and build things people actually want, even if that means the solution is scrappy and unpolished today, so long as it solves an important problem cheaply and effectively.

In large organizations, accountability for actual value delivery is so diluted across the org chart that it practically disappears.

Leaders gravitate toward building bloated "internal platforms" that offer little external utility. Engineers over-engineer simple problems because their performance reviews reward technical complexity, not business impact. It becomes a systemic shield, allowing the old guard to justify their headcount without delivering new value.

Software engineering is uniquely plagued by this. Civil engineers understand their objective: build a bridge that safely carries traffic within a specific budget. They don't invent a novel, hyper-complex suspension architecture when a standard short-span bridge will do the job.