You don't have be that clever to try to extort people... just without scruples. I think the attack surface is just absolutely enormous once you bring creativity into the mix, which is what these models are autonomously capable of.

We could prevail if we are willing to turn off the internet for a long time. It is like when a disease infects livestock... they cull billions of chickens.

The other thing is: look at all the fab capacity that is being brought to bear on this. I guess within a couple of years we are going to have 5x the total online HBM that existed a year ago? As consumer machines and phones get far more powerful, they will start to be able to host models that could meaningfully participate, even if they will be a long way from mythos.

What does "aligned" even mean? Aligned with who? We have no universal code of ethics. Democracies kill and launch wars merely to have cheaper stuff, even when we are already rich. And why would China ever agree to stay in second place? Our society is built on the premise "the smart and powerful dominate". The idea of domination is deeply embedded in our capitalist model.

Or course it is true that powerful AI will empower the average Joe to make a bio-weapon, and that will lead to our ruin. But at the same time, having powerful AI in the hands of a just a few is almost equally as horrific.

But deeply embedded in out national and cultural ethos is to advance science, tech, and material wealth at all costs. We never ask if we have enough. We sacrifice community to advance our careers for ever more.

There is no way out of this one, I'm afraid. Our cultural predispositions and mindset of domination compel us to chase ever more powerful AI as if doing so were a mandate from god.

And the result will be a crisis in so many dimensions it is hard to reason about what will go wrong first and most spectacularly.

So yeah, I am zooming in on an exponential and calling it linear, but that doesn't work for speed. And doubling of velocity gives 4x the energy regardless of how tiny the step.

A cup might not break at all in a 1 foot fall and might shatter when dropped from 10. The outcome is binary, which isn't useful as a scale of input energy.

We know intuitively that a ball atop a 20ft ladder has twice the potential energy of a ball atop a 10ft ladder. And we also know when they fall, by the time they reach the ground and all the potential energy has been converted to kinetic energy, the previously higher ball will have twice the kinetic energy too.

But a twice higher ball won't have even close to twice the speed at impact. So let's look at why not.

The force of gravity is a constant force that causes constant acceleration in free fall regardless of speed. (Ignoring air resistance, inverse sq considerations, etc.)

Suppose it takes 1 second for the ball on the 10ft ladder to hit the ground with kinetic energy of 10 and a speed of 100. Again, gravity as a constant acceleration force is speed increase per time... not speed per distance. In the ladder example, it took 1 full second for gravity to accelerate the object to speed 100.

Now think about the 20ft ladder: the ball is dropped. How much kinetic energy and speed does the ball have after it has fallen 10 feet (but still has 10 left to go)? Well it has the same exact amount as the other ball did after falling 10 feet for a duration of 1 second: kinetic energy of 10 and speed of 100.

Now the crux: thinking about when the final 10 feet of the fall look like. We know for sure the ball still has 10 ft of potential energy to covert into kinetic, and that that will happen as it falls. But what of the impact speed? Since the current velocity of the ball as it enters the last 10 feet is already 100, we know it will spend less time transiting this distance than it did the first half where it started at off at speed 0. Since gravity imparts speed in free fall as a function of time - consequently less speed will be imparted over the second 10 foot interval. That concept is enough to prove the relationship isn't linear.

If you do the actual calculation or tests, you will see one ball needs to be dropped from 4x the hight of another to hit the ground at 2x the speed, but yet with still 4x the kinetic energy.

2) In the case of the rocket, it accelerates eastward. Even if parts fall off, and they slow down because of the drag of the atmosphere, and they slow down all the way to the speed of the atmosphere, the atmosphere is more or less moving eastward at the same speed as the earth's rotation, so any part would be unlikely to land any further west than where it detached from the rocket. And if there was no atmosphere on earth any part that fell off a rocket during its ascent to orbit would fall according to a ballistic trajectory and would most certainly land farther east than where it fell.

Also - if you have free electricity, which is a thing in certain settings, where prices can and do go negative, mining hardware will be viable for "a long time".

Just because you assert this doesn't make it true. Tragedy of the commons is at play - we have no good way to measure if your statement is true or not - which is it's own issue.

It is easy to imagine how your statement could be wrong, despite what we see. Starting with a non-industrialized society, you add in some tech, like cars. Some people use them for personal benefit. Of course a few cars don't cause everything to go suck just by themselves, and even so, the benefit for their users outweigh the harm to them personally, so they use them! Some other people are now going to need to use cars to compete with the people who do use them. Eventually the harms really become apparent, but there is no going back. And even if some people would choose not to use cars because of the harm, it is much harder to choose to abstain when practically speaking, things are still going to suck because everyone else is.

So, taking cars as an example, how do you know you are right?