> Money.

For those who didn’t read, the actual response in the text was was:

“The answer is that the very abilities that allow Claude to be used in these attacks also make it crucial in cyber defense.”

Hideous AI-slop-weasel-worded passive-voice way of saying that reason to develop Claude is to protect us from Claude.

Which I believe still does have a large grain of truth.

These things can make us simultaneously dumber and smarter, depending on usage.

https://muldoon.cloud/2025/05/17/frankin-llm.html

My argument is that it’s our job as consumers to align the AIs to our values (which are not all the same) via selection pressure: https://muldoon.cloud/2025/05/22/alignment.html

I would really try to disentangle this.

1. I don't know what my desires are. 2. "Desire" itself is a vague word that can't be measured or quantified; where does my desire for "feeling at peace" get encoded in any hypothetical artificial mind? 3. People have different and opposing desires.

Therefore, Coherent Extrapolated Volition is not coherent to me.

This is kind of where I go when I say that any centralized, top-down "grand plan" for AI safety is a folly. On the other hand, we all contribute to Selection.

I’m being facetious but there ARE ways to decide/act as a society and as subgroups within society that we want to disallow and punish and select out qualities of AIs that we think are unethical.

Not sure what you’re getting at here; pharmaceuticals are also human made. The point in the blog post was that we should also want drugs (for example) to be aligned to our values.

> What I think is absolutely important to understand is that throughout human history "alignment" has never happened.

Agree with that. This is a journey, not a destination. It’s a practice, not a mathematical problem to be solved. With no end in sight. In the same way that “perfect ethics” will never be achieved.

(Disclaimer: fell asleep after 10 minutes of reading the SSC post last night. I know it’s part of the HN Canon and perhaps I’m missing something)

Of course, this gets to the heart of the free will debate (to be settled in a future post ;)). Both are true at the same time - organized people and dictators and other factors simultaneously wrestle for influence in complex ways in which causation is impossible to measure.

My own two cents, though, is that the Categorical Imperative is a tremendously important and underappreciated tool for raising the self-consciousness of groups.

A practical implementation of it is linked at the bottom of the blog post.

Still, “AI existential risk” is practically a different beast from “AI alignment,” and I’m trying to argue that the latter is not just for experts, but that it’s mostly a sociopolitical question of selection.