AI is 100% capable fundamentally of making new processes. Look I mean it’s not like I think opus 4.7 is all you need, but how can you argue with the fact that adoption since 4.5 has been an inflection point? That’s kind of proof that reliability has reached a level that serious usage is possible. That’s over a period of months. When you zoom out further you see this is extremely predictable even a few years ago, despite the absolute hissy fits thrown on HN when CEOs began saying this.

Agentic coding is verifiable and this implies there are very few practical limits to what it can do. Combine that with insanely active research on tackling the remaining issues (hallucinations — which are not a fundamentally unsolvable problem at a practical level, context rot, continual learning etc)

One big misconception is that these models are trained to mimic humans and are limited by the quality of the human training data, and this is not true and also basically almost entirely the reason why you have so much bullishness and premature adoption of agentic coding tools.

Coding agents use human traces as a starting point. You technically don’t have to do this at all but that’s an academic point, you can’t do it practically (today). The early training stages with human traces (and also verified synthetic traces from your last model) get you to a point where RL is stable and efficient and push you the rest of the way. It’s synthetic data that really powers this and it’s rejection sampling; you generate a bunch of traces, figure out which ones pass the verification, and keep those as training examples.

So because

- we know how this works on a fundamental level and have for some time

- human training data is a bootstrap it’s not a limitation fundamentally

- you are absolutely right about your observations yet look at where you are today and look at say Claude sonnet 3.x. It’s an entire world away in like a year

- we have imperfect benchmarks all with various weaknesses yet all of them telling the same compelling story. Plus you have adoption numbers and walled garden data that is the proof in the pudding

The onus is on people who say “this is plateauing” or “this has some fundamental limitation that we will not get past fairly quickly”.

> AI craze isn't going to produce the boon some people think it will.

What’s the boon you don’t think it will produce?

- I completely agree with you about fundamentally the limitation being the business able to coherently articulate itself and its strategy

- BUT the benefit now is you can basically prototype for free. Before we had to be extremely careful with engineer headcount investment. Now we can try many more things under the same time constraints.

- obviously LLMs are not a religion I’m using it to illustrate a point

- 5-6 months ago was when agent perf hit a meaningful inflection point where adoption has exploded. It’s why people in this thread reference “the past 6 months” whether or not they realize we’ve been on the same path for years now

So to overextend the metaphor, opus 4.5 was really kind of the right fit for the rapture.

I mean no need to take any of this seriously, I have worked on benchmarks and measurement in an AI lab professionally for over 4 years now, in software and data science for 8 and before got a PhD in Astro, like I’m not some sort of armchair person with no understanding of this field. Though I do find it entertaining when my background in an AI lab is people’s favorite reason to dismiss this :)

I find that when people find stuff like this satirical they often don’t really know the industry or underlying mechanics that well. Not saying that’s you but as ridiculous as I apparently sound to you, do consider that sounds even more ridiculous to not understand the tsunami that is coming right for you…

I agree the foundations: git, GitHub, compilers, etc. are arguably are “a fraction of the price” and today they have arguably more impact (though not sure by which measure). But literally since January we have been rolling out our replacements, I don’t really see how that wouldn’t be an earth shattering impact. You talk about GitHub and that’s fine but ignore the fact that huge swaths of the profession aren’t even directly using any of these tools anymore.

I’m not sure what you imagine the promise of AI to be, and without that I can’t really be specific in any refutation I would just say coding is only the beginning. It is the most powerful and also the easiest thing to solve first. Improved coding performance also improves generalization and performance on non-coding tasks, so that’s a nice bonus, and we’re maybe 5 years away from decent embodied systems which after an inflection point of consumer adoption will quickly get better via data flywheels and on policy learning. Basically there are very few bottlenecks that will not be touched.

L(N,D) ~= 1.69 + 406 / N^0.339 + 411 / D^0.285

L is loss (pre training test loss) D is the scale of the data N is the number of model parameters

I really value skeptical people and skepticism generally. But what I think skeptical people would prefer to consider themselves is: rational and reasonable, with their beliefs well calibrated.

You’re not the only one to think that literally nothing major or significant has happened with AI but that’s simply wrong. Every major tech company - the ones poised to get the first best rewards, have already gotten good incremental revenue from AI via ads ranking/recommendations (Google, Meta, etc.), good productivity increases due to scale of workforce and advanced in house tooling. You won’t see these numbers and you don’t have to believe them. But I have seen them and I believe them, and I, like you, hate bullshit.