Over time the enormous investment in techniques and hardware manufacturing will almost certainly make these runnable in a more practical way. It will be a shame if by the time we get there it’s illegal to distribute them and you have to pay a reg capture premium and feed the machine.

I see this behavior constantly in 5 - the quality of opus and fable have degraded constantly since 4.6 was such a riotous success

In other news, no whales were bothered by this windmill, either.

The only way there is no benefit would be is if the open models are literal copies of the closed model, which unless there was direct theft, is highly improbable.

The future is good and better for everyone the more work is in the open and the more work is in the commercial space. It’s good all around.

I wouldn’t lump this into a west vs east thing as well. This is particularly PRC. I feel comfortable doing business in Japan, Korea, Singapore, Thailand, Malaysia, etc. But it requires some particularly strong willfulness to pretend the PRC isn’t actively and structurally built around economic espionage, and funneling IP through PRC for short term economic gain has been one of the primary factors in their growth over the last 30 years. This just scales it faster.

I wouldn’t expect the USG won’t compel AI companies in the US to disclose and retain data as well - however it’s not a simple thing, the companies are hostile to it themselves, courts are often unsympathetic to the government, and the “machine” for converting it into actionable economic advantage is non existent - and there’s a very significant human component in that all links in the chain are culturally uncomfortable with such things. While it happens and it’s possible it’s very difficult, fraught, and does not scale. The PRC is the opposite - the courts, government, and business culture are all aligned in the goals and processes.

The marginal utility problem is a real one for AI companies. I think the current generations are already saturating marginal utility for 95% of the population. Almost everyone I know outside of my career has no use for a more powerful model. This is a serious problem for the economics of AI and semiconductor investment. This is a bigger problem than Chinese models. It leads to a demand curve problem - that supply outstrips demand.

That said I fully intend to use deepseek hosting for operational agents that are making decisions about non sensitive material. The economics are astounding.

Kimi? The economics aren’t that amazing to merit switching from 5.6. I expect fable will rapidly reappear in subscriptions. Competition is good.

“”” We have been here before. Mozilla exists because one company tried to own the front door to the web, and an open community rose up to make sure it never could. Twenty-five years later, someone is running the same play. We bet on open the first time. Open won. Together, we can do it again. “””

For instance, is solving a math proof by enumerating all permutations exhaustively on a computer cheating? Does it matter that it is not a proof by construction? That its not descriptive? Of course not. The proof of the four color theorem is all that’s necessary and sufficient to prove it. Calculator at an algebra exam? Who cares. This isn’t an exam, this is the real world. The fact an AI can use a physics harness to perfectly achieve ARC-AGI-3 without attaining those goals demonstrates the power of the technique and that the goals are not necessary for that class of problems. Then find another benchmark that actually demands the goals be necessary and sufficient to achieve the benchmark goals. But don’t denigrate the fact we have technology today that yesterday was a fantasy.

I get what you mean in terms of testing the model itself to see its improvement in some domain. However if you can transform the domain to be better adapted to the model and achieve the desired results, this is indeed an accomplishment because a whole domain of problems is shown to be practically feasible with this technique without expensive model improvements. Of course the benchmark still exists without the harness, but the harness also exists which allows these problems to be solved.

As noted elsewhere the models themselves were used to build the harness, which means the models can in fact score this scores without intervention but building a harness for themselves adapted to the domain and using it. Is this cheating by the goal posts you’re setting?

There’s a real tension between “I want to solve problems and this technique shows how to solve the problem domain,” and the “I want to measure how something performs unassisted with other techniques.” Fortunately it’s not a mutually exclusive situation. You can do both simultaneously, gain the benefit of the technique to transform the problem into something tractable and keep measuring using the benchmark.

The next move would be to offer a stripe managed card network that captures the full MDR interchange fee stack, from issuance to processing to network to bank. They could offer deep discounts to merchants hosting their accounts on their bank issuing cards through them and processing through them by cutting out every middle man. It would instantly become the third largest network in terms of points of sale behind visa and Mastercard. They could further carve out heavy rewards for customers through direct relationships with airlines, hotels, and merchants through their existing issuer services. They would become the first fully integrated vertical in payments, and the efficiency that would give them in fees would allow them to crush the hodge podge payment stacks out there that exist. I’d wager in five years they would dominate the market.

The challenge I think for codex is the restriction on context size and the constantly rolling compactions. They are less aggressive or disruptive but it is still annoying you can’t force a 1mm context window on a 1mm context window model.

But it’s recall beyond compaction boundaries is much better than Claude code. The impact of compaction is much less noticeable.

Performance on task work is between fable and opus, but the marginal utility between that gap is not enough to pay extra.

Anthropic painted itself into a corner with fable at many turns and this latest twist is one of the more interesting. Either fable is too expensive to run at scale, or they’re trying to incentivize mega spend on tokens, or whatever - but them locking the frontier model away for the few enterprises willing to spend top dollar while codex is including frontier in the subscription (and I’ve found it also is both less token hungry and the limits are much higher for codex) has finally made me put Claude aside and use it as my backup for very specific tasks, where codex has filled that spot for a long time now.

50% more weekly limit, but no fable. Ok. I might have a refactoring job somewhere for you Claude for those extra tokens.

In fact most mathematicians (myself included!) think the more clever the trick the better the proof! The trick itself being clever is interesting because it often yields a new way of tackling or thinking about your own proofs. A bland explanatory proof that elicits some conceptually “why” is only preferable if it has a reason for doing so - does understanding why yield a new avenue of research? Often then the “why” is quite a clever trick too.

I think it’s a bit the opposite of programming. There you want your solutions to demonstrably not be clever and the code be its own documentation. It’s a different discipline.