It's like how psychopaths are eerie because there's nothing behind their eyes. AI-generated code is eerie because there's nothing between the lines. Code is in some sense theory building, and when you read a humans code you can (mostly) feel their theory working in the background. LLMs have no such theory, the code is just facts strewn about. Very weird experience to try and understand it.

What we're building today isn't even remotely close to that.

The question is whether AI will ever become good enough to magically infer information where none is provided.

For instance, I've had this startup idea for an itemized physical storage company. We'll never reach a point where I can simply say "Hey AI, create all the software necessary for an itemized physical storage company". It's not because AI won't continue to improve, it's because there's literally not enough detail in that statement to understand what I mean. It's too vague. I'm sure the AI of tomorrow could do a pretty good job in guessing what I mean by it, but the chance of it capturing my vision is literally 0%.

When I talk about an incoherent mess, I'm talking about something different. I mean that as the codebase grows and matures, subtle details and assumptions naturally shift. But the AI isn't always cleaning up the code that expressed those prior assumptions. These issues compound to the point that the AI itself gets very confused. This is especially dangerous for teams of developers touching the same codebase.

I can't share too much detail here, but some personal experience I ran into recently: we had feature ABC in our platform. Eventually another developer came in, disagreed with the implementation, and combined some aspects of it into a new feature XYZ. Both were AI generated. What _should_ have happened is that feature ABC was deleted from the code or refactored into XYZ. But it wasn't, so now the codebase has two nearly identical modules ABC and XYZ. If you ask Claude to edit the feature, you've got a 50/50 shot on which one it chooses to target, even though feature ABC is now dead, unreachable code.

You might say that resolving the above issue is easy, but these inconsistencies become quite numerous and unsustainable in a codebase if you lean on AI too much, or aren't careful. This is why I say that having a super clear vision up front is important, because it reduces this kind of directional churn.

There's some truth in there that judgement is as important as ever, though I'm not sure I'd call it taste. I'm finding that you have to have an extremely clear product vision, along with an extremely clear language used to describe that product, for AI to be used effectively. Know your terms, know how you want your features to be split up into modules, know what you want the interfaces of those modules to be.

Without the above, you run into the same issue devs would run into before AI - the codebase becomes an incoherent mess, and even AI can't untangle it because the confusion gets embedded into its own context.

The Python ecosystem provides too many nooks and crannies for malware to hide in.