(edit: This is not at all unique to spacex, of course, but given the nature of Musk's companies and their "fans" it's logical that they would employ this strategy. They are also doing a staggered unlock to avoid upsetting the market when insiders start dumping their shares.)

I don't really understand the comments (apparently) denying the basic logic of this scenario (maybe the article is so confusing that they, or I, am wrong about what it's trying to say). IMO the only real question is how close current technology is to achieving this scenario.

What is the difference? It's easier to make money with the AI you get at each incremental step toward potentially destroying human civilization (though, of course, it's debatable whether these companies really are making money as such).

So what? You are implicitly arguing that human civilization will be unable to resist engaging in a large-scale, coordinated effort to destroy itself, just to make a few bucks along the way. Is this true? I don't know. The point of the nuclear analogy is that we have previously shown that we can, under certain conditions, put the eschaton back on the shelf for some period of time, despite very real pressure to take more incremental steps toward doom. "But AI can write code" is not a refutation of the possibility that we could take a more measured approach to AI development.

> The only variable of disagreement is around AI doom.

The source of our disagreement seems to be your belief that somebody can either a) believe "AI doom" is inevitable, or b) not believe it's possible. This is an obvious false dichotomy that's stunting your ability to engage effectively with what I've written, and also stunting your ability to understand the broader landscape of the issue. You are projecting this dichotomy onto everybody involved and understanding their behavior in that way, which is leading you to make other reductive and honestly bizarre claims—like, for instance, the idea that a sudden change of course from Dario Amodei at this moment in time would be broadly perceived as somebody who was already losing the race cashing out his chips. If you really do believe that, I have to assume it's because you're modeling your hypothetical observers as falling into one of your two extreme mindsets and assuming Dario, being a smart guy who knows a lot, thinks the same way. I believe it is—yes, I'm ready for the dopamine hit—naive to assume all people or even most people fall into one of these two camps. Naive, at best. Yours is a self-limiting framework for thinking about this stuff.

I encourage you to broaden your thinking and engage in less projection and ad hominem stuff in discussions like this. I probably won't reply to whatever you post next unless you can do a better job writing a substantive reply to what I've written here.

I'll mention again the nuclear analogy. It is, believe it or not, possible for great powers, and even adversary great powers, to agree to limit the development and proliferation of dangerous technologies.

> The main secret is out of the bag.

This is not something you can do in a shed with a handful of GPUs just because you know "the main secret". To build something like Mythos you need tens of billions of dollars, massive amounts of power, enormous buildings filled with specialized bleeding edge computer chips that are made by (optimistically) a handful of companies with deep government ties. You need free access to all the intellectual property that humans have created and posted openly on the internet. You need all of this at each step, and to take each next step you (or somebody) needs to have taken the previous one.

For now, there are a million ways for a government to pump the brakes on this cycle.

I think you're reading some subtext into my comment that I didn't intend. Knowing myself, I assume the scare quotes there are just a bit of casual irony re: the insanely high stakes here. The word "concerns" as used by previous commenters doesn't seem equal to the context.

> The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are.

You can, in fact, opt out. You can opt out and do your damndest to stop what's happening, throw every cent you have at it, bend any ear that will listen, make use of the fact that your voice (as Anthropic leadership) has some meaningful weight.

If you really believe that we are heading down a path that's likely to end poorly for most or all of humanity, and you are the kind of person who's inclined to favor saving billions of lives over saving your own skin when the stakes are still relatively distant, abstract, and generally unclear, opting out is obviously on the table as a grand gesture that burns your position in the race to show just how fucking serious you are. The sense of inevitability your comment shares with many others does not seem well founded---we have, for instance, not had a global nuclear war yet. Leaders in the 20th and 21st centuries have shown remarkable restraint.

If today's political and tech leaders are unable to think beyond this inevitability, for whatever reason, the worst outcomes essentially become a self-fulfilling prophecy to the extent that reality bears them out.

---

But yes, these people are acting the way they are for obvious reasons, obviously. My previous comment is reacting to the general disagreement over whether Anthropic actually believes what they say about safety, etc., or whether it's a marketing gimmick. The purpose of my comment is to explain that "it's hard not to be cynical" about actions taken by very rich and powerful people that are claimed to be in everybody's best interests but are indistinguishable from the actions they would take to maximize their future power and wealth. I think everyone ought to agree with that statement. It's not a value judgment; it's simply an observation of how it feels to be on a plane whose pilot appears to be robbing the passengers (including you) at gunpoint and is conspicuously wearing the only parachute on board.

It's like a prisoner's dilemma where one party is loudly lecturing the other about the obvious benefits of cooperation while also obviously working on defecting. They want to have their cake and eat it too. Maybe they really are the pure-of-heart Chosen Ones destined to lead us around the great filter, but I don't see why I should believe that's the case when their behavior is just as easily explained as maneuvering toward being the winner who takes it all.

I'm sure that in your head this is a witty rejoinder, but it really is quite a wild thing to say: that you place no value on the individual variations in how different human writers express themselves. It follows that you really don't care about voice on YouTube either, except in the most basic mechanical sense: you would be happy watching videos written by AI and narrated by the same monotone text-to-speech narrator, video after video, efficiently delivering that densely packed information you crave.

This is actually a thing, isn't it? Like those "shorts" with the AI narration and matching subtitles flashing by in the middle of the screen. I guess you must love those---somebody does, probably a lot of people, or they wouldn't exist.

I'm tempted to frame this as a new kind of illiteracy. People whose brains are so addled by the modern media landscape that to get them to pay attention to anything at all you have to resort to tricks like this; god forbid they ever encounter a writer or narrator who speaks differently, sounds differently, thinks differently, frames differently. Nobody should be surprised, I suppose, that the ability to parse different levels of meaning in Content that falls outside the AI cognitive monoculture is a dying skill.

This is not really meant to single you out, since there's a lot of this going around, but I really don't think this should be a matter of "of course". Why should it be the default to let a tool that doesn't have your context, or your voice, override your own usage of language?

I think a big part of this is that many (most?) modern games are designed to be played on big screens at a distance (e.g. TV <-> couch). The apparent size of the display in that scenario isn't much different from a Steam Deck held naturally.

I just wish the Deck had VRR. That and the general lack of power are my only real issues with it, and the power isn't that big a deal given the massive back catalog it supports.

The way I use it, it's a 1:1 replacement for the way I used Dropbox: a directory of files that are continuously synced across different machines, with a durable central copy that gets regular snapshotted backups. I understand that Dropbox has more features now, but that's pretty much all it was when I started using it; and the fact that Syncthing supports other use cases than mine doesn't mean it isn't a perfect fit for mine. Out of all my experience self-hosting ~replacements for consumer cloud services, it's probably been the most successful by far.

I separate my files in Syncthing into different shared folders that sync to different sets of machines, and put stuff that doesn't need syncing at all on my NAS. On paper this might sound more complicated but in practice for me it's a much lower mental burden.

I paid for Dropbox for a long time specifically because it was an independent option, but over time the feature bloat annoyed me more and more, and their dabbling in genai stuff was the last straw. Now I use syncthing over wireguard tunnels.