Put it this way: would you rather this article didn't exist, or had less detail? Because that's the alternative in many cases.

I will say, if the problem is purely code you occasionally see "sparks of AGI" where the model will step back and say "hold on, there is a massive simplification we can make here...". This started happening with Fable-class models. But rarely if ever will the model push for a huge simplification at the product level.

Fair point on the contradiction. I don’t have the answers either.

The meta at this point seems to be mostly around "knowing what to ask for". Eg: I suck at prompting for specific design tweaks, because I don't have the design vocabulary to be specific and articulate what I want. It's a lot of trial and error. I hear people who are developing full scale applications without a tech background run into the issue writ large. For now, I have an advantage over these people when building applications. But perhaps that's just one round of RL away from being obsolete too. Supposedly the model will eventually have such good taste in design/architecture/whatever there will be little point in asking for tweaks.

The only thing that seems relatively future-proof is just being able to communicate clearly what you're trying to do. Not how to do it, or even what success looks like. Perhaps an element of it is even as basic as being honest with yourself.

- Occasionally has strange tics around asking for permission for obvious next-steps, implied actions, etc.

- It's very expensive, both in terms of tokens and % usage on subscription plans.

- Relatedly, effort level is unintuitive. Sometimes it seems like higher effort levels are actually cheaper due to not under-thinking and needing to correct work. But other times they are overkill and send the model into rabbitholes.

That said, it's fantastic as a code-reviewer or "hunter seeker". It's better at finding bugs than Fable and "Get this well articulated task done single-mindedly" is an Astra-shaped task.

Opus 5 has issues too, comment-slop, claude-ish, etc.

5.1 on the other hand can seemingly do no wrong. Easy to work with, writes human-level code. Expensive, yes, but even at Low effort it's well worth it.

For me I added some instructions to speak clearly and it helped marginally and that's fine. There will be a new model out in a few weeks where I'm sure they've laser focused on this issue since nobody can shut the fuck up about it. The same thing happened with GPT if anyone can recall the ancient period of 4-6 months ago.