I'm starting to get a feeling there is a phenomenon like this with AI - some people just genuinely don't hear the AI "voice" at all. They really can't distinguish why sending AI written text is going to impact the person at the other end. It's going to be an interesting ride as these people start using AI and are completely baffled why people are offended by their perfectly reasonable responses.

Other domains are extracting value but I feel like there's an order of magnitude difference. It raises the question, what other domains fit into these categories where the AI itself has pretty much free reign to verify its own results?

Purely AI written systems will scale to a point of complexity that no human can ever understand and the defect close rate will taper down and the token burn per defect rate scale up and eventually AI changes will cause on average more defects than they close and the whole system will be unstable. It will become a special kind of process to clean room out such a mess and rebuild it fresh (probably still with AI) after distilling out core design principles to avoid catastrophic breakdown.

Somewhere in the future, the new software engineering will be primarily about principles to avoid this in the first, place but it will take us 20 years to learn them, just like original software eng took a lot longer than expected to reach a stable set of design principles (and people still argue about them!).

That depends on where the methodology goes. But more and more it's hands off. If the trajectory continues it won't matter because nobody is sitting their waiting / watching the LLM code anyway. It is all happening in the background. We might see hybrid approaches where the weaker / cheaper agent tries to solve it and just "asks for help" from the more expensive agent when it needs it etc.

Once we hit that point, I am curious how much of Anthropic's current business model falls apart? So far it's always been clear that you just pay for the most intelligent model you can get because it is worth it. It now seems clear to me that there is limited runway on that concept. It is just a question of how long that runway is. I honestly wonder how much of their frantic push to broaden out into enterprise / productivity is because they see this writing on the wall already.

Don't get me wrong, I'm still disappointed. But mainly because it looks so superficial. I was trying to work out what's new and it just looks like an Android device (or Chrome? I can't tell) with some party trick Gemini features sprinkled on it. There isn't anything technically interesting here.

I'm still waiting for someone to ship a truly AI native device - something with the right sandboxing and UI layers to let an AI model truly understand and work with the device natively, but safely. The OS SDK itself should natively incorporate all these elements as first class primitives. And the model would be trained heavily to explicitly understand and work well with them.

In my org, I have to file a form for reimbursement if I bought a pencil for $0.25 but in AWS? spend varies by +/- $5k per month and nobody even questions it. This will definitely make it trivially easy for me to build on Anthropic's services without even telling anybody vs the hoops I would have to jump to get it paid for another way.

that's a nice breakdown

I think there's something key you get at in terms of the combo of dynamic environment + type safety maximising both. With a dynamic environment, the LLM can do a lot of interrogation to understand the problem space on the fly. I've witnessed agents sort out pretty complex issues through `python -c "..."`, `groovy -e "..."`, executing snippets of code with Node etc which is much less accessible if they have to compile it first. They can also inject logging code that interrogates the runtime as well (what type do we really have at line 1003?) etc which works better with runtimes that have deep introspection capabilities.