Based on this finding, I suppose the better way is to rely on local hardware whenever possible?

DeepSeek got MLA, and then DSA. Qwen got gated delta-net. These inventions allow efficient inference both at home and at scale. If Anthropic got nothing here, then their inference cost can be much higher.

DeepSeek also got https://github.com/deepseek-ai/3FS that makes cached reads a lot cheaper with way longer TTL. If Anthropic didn't need to invent and uses some expensive solution like Redis, as indicated by the crappy TTL, then that also contributes to higher inference cost.

35B A3B is faster but didn't do too well in my limited testing.

For Qwen3.5 27B, I got good result with --temp 1.0 --top-p 1.0 --top-k 40 --min-p 0.2, without penalty. It allows the model to explore (temp, top-p, top-k) without going off the rail (min-p) during reasoning. No loop so far.

* Haiku 4.5 by default doesn't think, i.e. it has a default thinking budget of 0.

* By setting a non-zero thinking budget, Haiku 4.5 can think. My guess is that Claude Code may set this differently for different tasks, e.g. thinking for Explore, no thinking for Compact.

* This hybrid thinking is different from the adaptive thinking introduced in Opus 4.6, which when enabled, can automatically adjust the thinking level based on task difficulty.

Will check your updated ranking on Monday.

(Big fan of SQLAlchemy)

Let's see how long it will take for DeepSeek to crack this.

You don't have to work for a frontier lab to know that. You just have to be GPU poor.

Question: Have you tried using LLM as a compiler?

Well, I sort of did, as a fun exercise. I came up with a very elaborate ~5000 tokens prompt, such that when fed with a ~500 tokens function, I will get back a ~600 tokens rewritten function.

The prompt contains 10+ examples, such that the model will learn the steps from the context. Then, it will start by going through a series of yes/no questions, to decide what's the correct rewrite pattern to apply. The tricky part here is the lack of precision, such that the "else" clause has to be reserved for the condition that is the hardest to communicate clearly in English. Then it will extract the part that needs to be rewritten and introspect the formatting, again with a series of simple questions. Lastly, it will proceed, confidently, with the rewrite.

With this, I did some testing with 50+ randomly chosen functions, and I could get back the exact same rewritten functions, from about 20 models that are good in coding, down to the newlines and indentations. With a strong model, there might only be 1~2 output tokens in the whole test where the probability was less than 80%, so the lack of batch invariance wasn't even a problem. (temperature=0 usually messes up logprobs, go with top_k=1 or top_p=0.01)

So input + English = output, works for multiple models from multiple companies.

But what's the point of writing so much English, in hope that it leaves no room for ambiguity? For now, I will stick with mitchellh's style of (occasional) LLM assisted programming, jumping in to write the code when precision is needed.