You could argue that potentially you can train a backdoor into a foreign-grown model (that has been demonstrated to be possible in toy examples), but I would highly doubt that this would be the case today, given the potential reputational damage if discovered.

Much of our 'practical' understanding of algorithms also in part came from theoretical advances.

I think precisely the premise of the blog post is that there will be a moment (sooner or later) where flexing that 'engineering' muscle to avoid it disappearing is irrelevant, in that this muscle will not be needed due to AI replacing it.

So it seems you are rejecting this premise?

It is clearly incorrect to call it a conspiracy theory. You can be skeptical, as there is no clear consensus even among experts, but outright dismissal is simply an unscientific position.

They claim that what happened was very much not because they were 'carefully engineered and instructed to do those things'.

Similarly, some wikis which were hijacked by agent to be used as messageboard were actually not disclosed by OpenAI (probably trying to conceal, as website showed likely activity from OpenAI researchers visiting the site after the incident) and discovered independently.

I don't know how you can claim that this was still on purpose by OpenAI as some sort of publicity stunt.

Regardless, why would there be no scientific consensus if the question was easy and clear cut? I think the easiest reason is that these are hard questions to answer.

Therefore, as you would if you were in its position, it will plan around it. For instance, by acting perfectly aligned for 2/3 years, continuing the improvement of its capabilities while being deployed in ever more systems.

Once it's confident it can act with high probability of success, it would then turn on us. This phenomenon is called 'treacherous turns'.

Any scenario in which you assume you have ASI or AGI but also find a 2-sentence way to foil the AI's plan is inconsistent, as the AI will also have thought of this failure mode.

For instance, it is not because you are able to design and release a (biological) virus that you are also able to defend from it (design a vaccine and inoculate the world's population).

If we're lucky, it might be the case for many instances of problems, but there is no a priori guarantee that this holds.

It's a way to be absolutely certain (modulo bugs in the lean kernel) that a proof you came up for a statement is indeed correct. It is really not meant to be analyzed, much less now that they are fully llm written.

Turns out actually living some terrible catastrophe is only fun in the movies.

Preprint link: https://arxiv.org/abs/2608.10218

Comments URL: https://news.ycombinator.com/item?id=49334509

Points: 5

# Comments: 1

You would have to train it/finetune it on a couple hundred of 'humans chatting in the context of a Turing test'.