Personally I don't even let my agent run a single shell command without asking for approval. That's partly because I haven't set up a sandbox yet, but even with a sandbox there is a huge "hazard surface" to be mindful of.

I wonder if AI agent harnesses should have some kind of built-in safety measure where instead of simply compacting context and proceeding, they actually shut down the agent and restart it.

That said I also think even the most advanced agents generate code that I would never want to base a business on, so the whole thing seems ridiculous to me. This article has the same energy as losing money on NFTs.

But Railway bears some responsibility too because, at least of the author is to be believed, it looks like they provide no safety tools for users, regardless of whether they use AI or not. You should be able to generate scoped API tokens. That's just good practice. A human isn't likely to have made this particular mistake, but it doesn't seem out of the question either.

So while the AI did something significantly worse than anything a hapless junior engineer might be expected to do, it sounds like the same thing could've resulted from an unsophisticated security breach or accidental source code leak.

Is AI a part of the chain of events? Absolutely. Is it the sole root cause? Seems like no.

In this case they had to customize the model fitting to try to get the error closer to zero specifically on those attributes.

I have good success when I ask the agent to help me debug the harness. "Help me debug why Claude Code is ignoring my hook".

I think the AI companies all stink to high heaven and the whole thing being built on copyright infringement still makes me squirm. But the latest models are stupidly smart in some cases. It's starting to feel like I really do have a sci-fi AI assistant that I can just reach for whenever I need it, either to support hard thinking or to speed up or entirely avoid drudgery and toil.

You don't have to buy into the stupid vibecoding hype to get productivity value out of the technology.

You of course don't have to use it at all. And you don't owe your money to any particular company. Heck for non-code tasks the local-capable models are great. But you can't just look at vibecoding and dismiss the entire category of technology.

It's not just Python too, I've hit the same issue in Common Lisp.

Yes one can run contracts and unit tests and static analysis, but what's a type checker anyway other than a very strict static analysis tool?

Anecdotally like half, depending on the area. Plots of land go for $500k in Boston suburbs and new construction homes go for $1m.