There are some transformation approaches to re-use the kv cache across inferences, but none are in wide use due to accuracy concerns following the transformation.

A frontier model team having to fight their board on whether to monetize the datacenters directly or continue to invest in AI work is going to have a hard time.

If John and Mary were first, how long until everyone is a descendent of john and Mary?

An IPO with massive insider selling counterbalanced by a flood of index fund rebalancing is entirely different.

As a holder of index funds such as the S&P, I'd much prefer that these vehicles are excluded for at least some period of time to ensure that the greater fool isn't simply my index portfolio.

Paying 2k for something that you use 100 hours of is quite expensive. Having the capability built into your existing silicon which you would buy for 1k is cheap. Paying 200 dollars a month for 2 years give a present value of $4200 dollars. Meaning that that paying 2k upfront cuts your overall spend in half.

I spent 6k on codex last month which, if repeated, implies a present value of ~144k.

Billions spent on RL may be good enough to beat human performance.

The problem is that expectations rise in datacenters, hardware/power/security/availability guarantees cost real money. Then the operator providing these guarantees expects some margin.

You can see this most clearly with "developer desktops", a gcp instance costs about 10x a hetzner instance which costs between 5 and 10x the same hardware sitting in the back of an office somewhere. While all of these premiums matter for 24/7 systems under active development, they don't really matter for ephemeral small scale workloads.

This is a hobby version of a project, but you can imagine commercial versions of the same prompt for new databases, genomics studies, material analysis, operating systems etc.

There are many problems I want to work on which require billions of tokens. These are completely inaccessible without corporate project sponsorship at the moment. An asic generation machine which can pump out a few 10s of thousands of tokens per second at opus4.6 quality is more than sufficient.

We are going to see a new generation of models which effectively “solves” these problems for most businesses. Likely within the next two years- then we’ll talk about some other problems which limit adoption.

There is only 1 top law firm, financiers of law firms have no interest in starting a race to the bottom. Foundation model labs will take a significant portion of the value, the remainder will be captured by entrenched monopolies.

While we could claim that such a company can grow by consuming a larger share of the GDP ... this would not bode well for future political stability, and nationalization would be a major topic.

So your left with a fast take off scenario, a job apocalypse, or a massively reduced growth rate.

It's better to just ask codex to do the search for me, but this is much slower - but increasingly my go to. I wish there was a fast search api codex could hook into to answer internet questions faster.

Coding is an interesting case as [1] the pace of progress has been absurd and [2] it's hard to put an upper bound on required capability. However hard to put a bound on and will are different, it's quite possible that the average engineer will cease to see the benefit of rapid progress - or that their employer will be satisfied with lower tier models.

How smart of a model do you need to build a high quality CRUD app for internal users? Or build a scalable web service?