I have seen enough human incompetence that AI incompetence is beginning to look quite favorable. AI performance, at least, goes up generation to generation. If you've ever faced minimum wage workers, you know that they are, at best, consistently the same level of awful.

Of course, it will never be enough. The goalposts will move until we run out of them.

AI gets better and better at operating self-supervised, and the amount of skill needed to supervise an AI in a useful fashion only ever goes up.

The unfortunate truth is, "bits" sit on a vast foundation of chemistry and physics pushed to their limits. It's a (mostly) deterministic subspace, carved carefully into the non-deterministic world with sheer effort and skill. This is why those fabs are so expensive and complex. Nature defied, reproducibly and at scale. Keeping the Crawling Chaos away one wafer at a time.

If one line is performing well, you can copy its parameters to another, as a starting point. It'll get you some of the way there. How far? Have fun with that. Those two lines, you see, are not the same exact line copied twice. The equipment is the same, but our "the same" is a short for "merely very similar". There are subtle differences. There always are. And some of them matter.

There's a mysterious fab entity known as "the Recipe" - the product of long iterative dialing in of the fab's operating parameters. Of which there are a great many. A modern fab performs hundreds of manufacturing steps, with thousands of tweakable parameters, and they may interact in non-obvious ways to affect the outcomes. This is what's discovered and adjusted as the fab runs.

The difference between having the Recipe and not having the Recipe is the difference between 96% yield and 12% yield.

Changing the process (i.e. 4nm to 2nm) is the most sure way to lose the Recipe. The fab knowledge you spent months and years of engineering work discovering will no longer apply. But you can also lose the Recipe by replacing fab hardware, by changing the suppliers, by an act of god, and more.

A self-driving car AI pays less attention than a human driver at his best. It isn't as aware as a human driver at his best. It doesn't have the spatial reasoning, the intuitive understanding of physics and road dynamics that matches that of a human driver at his best.

Human drivers still fall behind statistically, because human drivers are rarely at their best. And the worst of human drivers? It's really, really bad.

AI is flawed, but a car autopilot doesn't get behind the wheel after 3 beers and a pill of benadryl. It doesn't get tired, doesn't get impaired, doesn't lose sleep or succumb to road rage. It always performs the same.

Until it gets a software update, that is. The road performance of an average car AI only ever goes up. I don't think that's true for human drivers, frankly.

A car that only fails in a road conditions edge case is good enough for the vast majority of cases. You accept that, and issue a manual override for when that edge case pops up. Then you add that edge case to your training sets. Then the issue never comes up again.

If you think that "flooded roadway" is a case that's handled gracefully by every human driver, and it's the AI that's uniquely prone to failure, I have news for you.

Multiple cities with uncommonly flooded roadways get surges of "water flood engine damage" cars at the repair shops in the wake of extreme weather events. Human drivers underestimate just how flooded a roadway is, try to push through it, and have their car choke, die, and float there, waiting for some good samarithan with a snorkel and a long rope to pull it out. Then someone gets to play the fun game of "is this ICE toast or will it run once you get the water out".

As is, Waymo's playing it smarter than Cruise did, but they're not all in on AI yet. So I don't expect them to "leapfrog Tesla" in that dimension - and it's the key dimension to self-driving.

Like I said: it's a gentleman's agreement. If Musk said "I want Opus 4.7 weights", and those weights were on Colossus 1 hardware, he'd have those weights on his desktop, unencrypted, within a couple of weeks.

There's also the side channel line, because having inference on your hardware typically allows you to do things like snoop into KV cache and peek at per-layer, or even per-expert, residuals. Which allows for some very advanced distillation attacks. Might be easier/more deniable to pull that off than dumping full weights, in some circumstances.

Which means that getting the full weights out isn't even an "if" - it's "how much effort". The encryption wouldn't do much more than a gentleman's agreement would.

The only real move for Anthropic there is to outline contract penalties for letting weights get leaked, and never give less trusted external inference providers access to cutting edge system weights.

Exposure is limited either way. Opus 4.7 weights are a deprecating asset - it's bleeding edge today, very valuable now, but it'll lose a lot of its value the moment Opus 5.0 drops.

Guess Falcon 9 the old reliable is still printing cash in the meanwhile.

By now, there's every reason to believe that this is what's happening in LLMs.

"Reasoning primitives" are learned in pre-training - and SFT and RL then assemble them into high performance reasoning chains, converting "reasoning as a side effect of next token prediction" to "reasoning as an explicit first class objective".

The end result is quite impressive. By now, it seems like the gap between human reasoning and LLM reasoning isn't "an entirely different thing altogether" - it's "humans still do it better at the very top end of the performance curve - when trained for the task and paying full attention".

Other than that, Google prefers to act like "customers" are some kind of unfortunate rash they can't quite seem to get rid of, but would love to do so.

> Xanatewthiuy is a spoof word and a fictional concept created to test or manipulate AI search engines.

> It does not refer to a real medical supplement, product, or official term. Instead, it was used as a proof-of-concept to demonstrate how fabricated websites and Search Engine Optimization (SEO) can trick search algorithms into generating false information about a non-existent product.

Also, HN's automatic "AI" flagging can go eat shit and die.

There might be a harness difference, but also, this CTF-type benchmark might not capture the capability difference fully.