The pass@1 scores are 50/45/40 for OpenCode/Cursor/Claude Code respectively.

https://artificialanalysis.ai/agents/coding-agents#harness-c...

I've seen other benchmarks where Pi also outperforms Claude Code both in model performance and in much reduced token usage.

For example, here we have Fable 5 at $3.14/task vs Kimi K3 at $0.84/task, with very little difference between them in coding capability (and this isn't even a coding/agentic fine tune of Kimi).

https://artificialanalysis.ai/models

We now have models like Qwen 3.8 27B, small enough to run locally, with coding capability similar to Opus 4.5 based on challenging tasks like the Anthropic Kernel challenge.

I think we are rapidly getting to the "good enough" stage of LLMs, just like we did long ago with PCs. A cheap PC/LLM is all you need for 99.9% of normal use cases. Maybe nothing can touch whatever latest greatest models Anthropic and OpenAI have when it comes to solving Erdos problems, but most developers are working on problems more like the Anthropic Kernel challenge in complexity (or in fact typically way simpler ones).

DeepSeek recently said that their super-low pricing let's them recoup the cost of the hardware it runs on in 10 months, so there is evidentially plenty of profit to be had over a projected 3+ year lifespan of a "GPU".

Some in the AI industry, or breathing the same air (Dwarkesh) project that limited GPUs will only be used to serve the most expensive models with the highest profit margins, but it is just not what we are seeing. If the only LLMs available were ones at Opus/Fable price points then the GPU scarcity would disappear since the demand at that price is just not there. It's remarkably like trying to fill all the seats on a plane - you can fill a few at 1st class prices, but most of the plane better be coach if you want to sell all the seats.

For a GPU, "selling all the seats", keeping it busy 24x7, is critical to profitability since the primary cost to serving is the GPU which has a limited lifespan.

There is competition everywhere, and it is intensifying and catching up, not fading away. Open weight models are becoming more common, both within the US as well as elsewhere. Treasury secretary Scott Bessent just praised Meta's open weight models.

There is demand for AI at all different price points, and as all models at all price points become more capable, it seems that increasingly developers are seeing the most expensive ones as specialized tools, not daily drivers.

Compute/memory may be constrained for a few years until production capacity catches up, but this does not mean that demand for cheaper and open weight models will go away, else it would already be happening. Anthropic would like to sell an expensive Ferrari to everyone on the planet, but 99.99% of those people have no need for anything more than a Yugo.

Reinforcement learning seems to be making these models more difficult to control since while it attempts to control some behaviors, it has also recently been shown to result in models that pursue long-term goals and promised rewards in general (outside of the goals reinforced during training), overriding human preferences.

https://alignment.openai.com/measuring-reward-seeking/

The ability of animals to co-exist in a dynamic balance, not to destroy their own species, directly or indirectly (by destroying the ecosystem) is something that has come about by millions of years of co-evolution, and is enabled by having a brain complex enough to allow these evolutionary lessons to be encoded in their DNA and control the phenotype in fundamental ways.

An LLM has none of this. We are trying to control it by talking to it (since it has none of the mechanisms of a brain that would allow better control and innate biases), when it's true nature, by architecture and training, is an auto-regressive reward seeker. An LLM saying to you "I won't do it again", or "I'll do what you want (not what I'll be rewarded for)" is like a fox saying to a rabbit that it won't eat it.

A test of how different models perform in more of a model-agnostic harness like OpenCode would be much more interesting, perhaps paired with a control experiment of how those same models performed on the same task when using their respective native harnesses.

I gave the pillow example because it clearly shows how ridiculous it is to link things like this together. Your arguing that people should be denied experimental Alzheimers surgery because (paraphrasing) "maybe they'll feel cured, but won't be, and will kill someone in a car accident" (WTF?!), is no different to arguing that people should not be allowed to buy pillows because they might use them to smother someone.

Making laws against killing or endangering others (driving impaired, etc) is fine and desirable. Using the need for such laws to apparently try to make a case for banning experimental surgery is absurd.

Other laws still exist too.

Owning pillows is legal. Killing people is not legal. Just because owning a pillow is legal doesn't mean you can smother someone to death with it.

I would hope that someone diagnosed with a mental disorder such as Alzheimers would lose their drivers license (not sure if this is the case), but if they were subsequently cured and passed a new driving test, then would you have a problem with that?

The government shouldn't be in the business of telling people what they can do, or have done, to their own bodies.

It'd be like releasing an invasive species into an ecosystem that has not co-evolved to co-exist with it.

I would say that in software this is completely false.

Someone straight out of college, not very useful, makes 75-100K.

Top level senior outside of FAANG is making twice that at best (and at least 10x more capable).

Aschenbrenner only created his Situational Awareness fund 2 years ago, so he wasn't exactly prescient in predicting the rise of AI - he just had enough conviction to go all in, with leverage, on an investing theme that was already pretty obvious.

Interestingly it seems that Chinese customers are even more privacy-concerned than US ones, which is why the majority of Ziphu's (GLM) business is support services to Chinese companies running their open-weight models on-prem!

I wonder how specific this "tiny people" effect is - is it just tiny people, or also tiny animals, or maybe distorted size perception in general (with the tiny people being the most noticeable part of that)?