1) low latency desired, long user prompt 2) function runs many parallel requests, but is not fired with common prefix very often. OpenAI was very inconsistent about properly caching the prefix for use across all requests, but with Anthropic it’s very easy to pre-fire

Indeed in seems in most language model RL there is not even process supervision, so a long way from NTP

Strategy matters a lot and good strategy is worth at least a few positions in a race.

Insta was a huge hit for sure but since then Meta Capital allocation has been a disaster including a lot of badly timed buybacks

In fact it’s a great illustration of why the hype around agents is misplaced!

Finally, saying that agents can have predictable behavior is wrong (except on simple tasks where you shouldn’t be using an agent anyway). Agents loop and compound their input, making them highly non-deterministic even for the same prompt.

In the long run the definition used by OpenAI, Anthropic et al will win out so can we just all switch to that?

They claim that a small Chinese hedge fund could acquire $1bln in GPUs, with no state support, including many sanctioned chips, then trained a model optimized for a far smaller server compute size, and that they have a source at this very small fund who is willing to admit to export violations. A 40bln param active model is exactly the size you would expect from a server of the size they claim.

What’s more likely - that semianalysis made it up like they have a bunch of other things, or that all the above is true?

Deep Seek R1 is literally an open weight model. It has <40bln active parameters. We know that for a fact. That size of model is definitely roughly optimally trained over the time period and server times claimed. In fact, the 70bln parameter Llama 3 model used almost exactly the same compute as the DeepSeek V3/R1 claims (which makes sense, as you would expect a bit less efficiency for the H800 and for the complex DeepSeek MoE architecture).