I think the more likely reason is it doesn't work as well as in context learning. Otherwise they would prefer to avoid polluting context and degrading performance.

It's relevant to AI safety. If you have a diversity of outputs, the AI will agree to hack the bank 0.1% of the time regardless. If you have a uniformity of outputs, in most contexts the AI will hack the bank 0% of the time, but in certain odd contexts, all AIs will work together to hack the bank 100% of the time.

> Spend too little time and you toil away in a cave, at best reinventing things that were invented already without your knowledge.

Grothendieck disputes this point. He reinvented the wheel many times when he was 17-20 years old, but believes it was training for accessing the required cognitive spaces. He says:

   A few years after I finally established contact with the world of mathematics at Paris, I learned, among other things, that the work I’d done in my little niche [. . . had] been long known to the whole world [. . .]. In the eyes of my mentors, to whom I’d described this work, and even showed them the manuscript, I’d simply “wasted my time”, merely doing over again something that was “already known”. But I don't recall feeling any sense of disappointment. [. . .] Without recognizing it, I’d thereby familiarized myself with the conditions of solitude that are essential for the profession of mathematician, something that no-one can teach you. [. . .]

But reality doesn't owe us this luxury, unfortunately.

Offense increases entropy. It's easy to destroy. Attack surfaces are numerous. Humans are fragile. It's hard to defend.

A profitable prediction markets trader who goes by the alias semiotic rivalry posted this prediction on his X account recently:

> Prediction: in 1-2 years, almost everyone will agree that open sourcing the best AIs is totally crazy as the AIs become capable of autonomous hacking of most humans and institutions, serious terrorism and creation of biological weapons, etc.

And I mostly agree with him. As much as it's awful to have such a concentration of capabilities in the hands of a few governments or corporations, as far as I can tell there are no easy answers here, only a matter of trade offs and different risks.

That's too strong. Only in an actual monopoly for a product with no substitutes that has price inelastic demand can pricing fully disconnect from costs. Frontier model serving is only maybe a soft version of that, where costs and moat both contribute to pricing.

Low latency is a big deal but the immediate use cases are somewhat different in the short-term (more serial coding workflows) rather than pure math research which is effectively massive-scale search through a tree of possibilities, which is where you want throughput and low cost per token, rather than high speed per token.

The polling I have seen from Germany and Australia of a social media ban for children was quite in favor of it. Sentiment here on HN and on Reddit was the polar opposite. We are in a bubble.

> https://researchprofiles.ku.dk/en/publications/winners-and-l...

Surely, AI helps them do stuff even cheaper, and cheap is good, so their business should be booming?