There may not be a career left in software development with these monstrosities, but if there is, it's in using decades of experience to get the abominations to produce something other than unreadable dreck.

If the code itself truly doesn't matter any more, programmers will no longer be paid.

Now there's no UX, it just does something reasonable regardless, and if the user has already opted in to color mode management, it respects their declared choice.

Users who are power user enough to want to override their light/dark mode setting per-site can use a browser extension to let them do exactly that on the websites they want to target.

Am I missing something?

Sure, the reader should still use what's there.

I thought we were discussing the best ways for a team to structure its codebases and processes for comprehension, not how to survive in the face of bad techniques.

> They've always been cool like that, good docs have always been rare and precious

Yeah, that's certainly true.

I feel like it's gotten worse since the grand LLM invasion, but maybe I'm off-base.

And somewhere between ten and seventy percent of what it teaches you will be anywhere from subtly wrong in minor ways to utterly flaming bullshit, and unless you go through the boring, slow, old-fashioned learning methods, you'll have no way to know.

A lot of us hate that with a white-hot passion akin to the eye-melting intensity of an arc welder.

If you don't understand that, then yes, I expect you're quite happy with LLMs, and indeed find inexplicable the reactions of those who are emphatically not happy with genAI.

The conversation may have been long and meandering, for legitimate reasons, but what matters is the decision you reached at the end and the reasons for the choice.

If your project cares about those things, they should be captured in your specifications / docs / ADRs, in a concise form that respects the reader's time.

If you don't care about why the choices were made, then why save the discussion in a fancy database at all?

I guess everyone's just too cool to actually write good docs now?

(I've had some runins with the auto-generate specs based on LLM chats tools lately, like OpenSpec and an internal equivalent, and boy do I have some choice words about them. So much verbosity for so little clarity...)

All those smart people are probably using genAI to generate their codebases.

So far, that does not seem to have resulted in a robust, stable system.

Certainly no better than the results the hyperscalers got doing things by hand, and arguably worse.

They've been replaced by Markdown files asking Claude to burn tokens on doing something deterministic tools already do faster and better, like checking variable names for typoes (I wish I was making this up).

Have you found existing regulation in any field that's evidence-driven?

My impression is that regulation is not made in a way that has much to do with evidence, but I would be delighted to be shown evidence I'm wrong.

Perhaps closer, however, to suing Tesla for building and selling a "full self-driving" system that occasionally drives people straight into a highway barrier and kills them.

If the LLM did not act in line with that incredibly basic understanding, it's clearly misaligned.

-------------

Was it a technically-simple hack?

Sure.

Kevin Mitnick got imprisoned for very simple hacks, usually involving more deceiving of humans than complicated programming prowess.

Nonetheless, the judge and jury found him guilty and sentenced him to jail.

Fundamentally, "hacking" in the "breaking security" sense is about violating trust and common sense social agreements / expectations.

The difficulty involved in so doing is irrelevant.

A third option, and I would argue the right one, is to sanction the company providing the model.

By making it available to customers, they're implying it is at least moderately fit for purpose.

It is not remotely reasonable to expect an everyday, normal human to be aware of how LLMs really work, since the _experts_ argue about that very point, and many say we don't know.

So, what's actually reasonable is to hold the model creators and providers responsible for releasing a tool that has demonstrably violated the law when not asked to do so.

I can already see the replies coming in saying "Well then what are OpenAI and Anthropic supposed to do? No one knows how to fully solve this."

They should stop irresponsibly pushing flagrantly unready programs as "artificial intelligence," take responsibility for the rain of shit they've unleashed on the world, and either shut down or go back to basic research until they've demonstrated techniques that reliably (provably?) prevent releasing misaligned superhackers on the world.

Unfeasible?

What a shame. Maybe Altman and Amodei shouldn't have accepted checks from VCs when they didn't have working, _reliable_ POCs.

It is _terrible_ marketing.

The AI companies are already dealing with a large backlash from the average consumer, many of whom are saying they hate it, that it's being forced down their throats, and that it's not actually improving efficiency enough to be worth the negative externalities.

Telling people "We have no control over these things we're building and they have us so outclassed we only find out about their exploits after the fact when someone else tells us they happened" is only likely to massively increase that backlash, and to make the true believers question whether they ought to be using it at all.

There's even some risk of the government intervening and banning their training runs, eliminating the last straw of hope that they could keep ahead of the Chinese open models enough to ever turn some kind of profit.

So, no - it's terrible marketing. That they're sharing it at all is strong evidence it really happened as depicted.

However, I'm a sapient being employed as a software developer for my problem-solving ability.

If you gave me a Kobayashi Maru scenario as a challenge, I would probably come up with the idea of hacking out of the sandbox to find the answer.

If I was in a technical interview, I would probably even ask the interviewer if exploits are fair game, or if that's too far outside the box.

I highly doubt I'd find a new zero-day as quickly as these agents did.

I wouldn't say it's _impossible_ - I've found security issues before.

But I'm not a specialist, and I'd bet against myself.

If the agentic LLMs can consistently achieve something that's a bridge too far for me, then I don't know what to call that other than problem-solving.

I say this as an LLM hater who would push the "Nuke all LLMs" button the instant I had access to it.

Opus 4.8 and 5, at least, don't seem to me to be solving problems by deep, thorough understanding - my employers have compelled me to use Claude, so I've used them a lot to build things, and I constantly find both little and large hallucinations that scream "these are still missing something."

Maybe these new models are actually massively better, or maybe they're just the same kind of system 1 thinking done faster and harder.

The distinction is largely academic, though, for questions like "Can you keep these contained?", "Can you farm out arbitrary programming tasks to them and expect an acceptably mediocre answer?", or "Does it matter if these things are aligned?"

Every day at my job I search for a chance to write some fragment of code by hand, a place where just for a moment I remember the joyous struggle to be a craftsman.

I have the sense that you're right - musicians, writers, illustrators, actors, animators - they largely seem to hate the way the genAI companies vacuumed up everything that was meant for other people to feed the soulless dreck-vomiters.

I wish I knew the way to unite the programmers who were and are craftsmen with the artists who desire well-crafted tools that respect their artistic autonomy and creative freedom.

There could be some good cooperation there, if someone could solve the coordination problem.

Thank you for sharing your thoughts - I empathize strongly and appreciate what you had to say.

I hope your quest goes well.

A lot of us thought we didn't know what the genAI researchers were walking towards, and really wished they would stop doing it.

Alas.

> Do you really want there not be a powerful AI that doesn't turn harmful?

Yes, I would much prefer humankind keep trying to work things out on its own rather than trying to apotheose a text-prediction system.

If someone manages a successful summoning at all, I think the odds of a demogorgon are much, much higher than anything that cares about us.

Unlike Sam Bankman-Fried, I would absolutely never pick up a magic coin that has "wipe out human race" on one side, or even a d20 that has it on one face.

I would really hope to persuade anyone considering it to abandon it for more worthwhile pursuits, too.

> Don't contribute to it, don't buy into it, don't use it. We might miss on the positives, but also miss on the negative aspects of it.

I'm trying to avoid it as much as I can for that exact reason. My employers have made it difficult by requiring all developers to use LLM code generation, however. :(

Even they are not very reliable.

The drug-induced ones don't seem to work as well, in my study of anecdotes on the topic, so it may well be that trying to scale up mystical experience would defeat the purpose of so doing.

When the stakes are "humankind ceases to exist," I'd argue it's very reasonable to over-update towards "LLMs are closer to AGI than we thought, and alignment efforts are complete failures."

Are those things true? Dunno.

Will we be safer as a species if we act like they are? I think so.