It's okay to have a strong opinion on this, but if you suggest the answer is 'clearly anything', I will start to suspect you haven't thought about the problem enough.

Although I don't fully agree. With AI the focus now shifts a lot from 'being able to implement the thing' to 'being able to know/figure out exactly what you want the thing to look like', also against the siren song of the LLMs constantly having ideas of what you could _also_ build.

So, I agree that it's not a direct translation from 'pro pizza cook' is 'pro pizza orderer', but I'm quite sure the pro pizza cooks would generally have the better pizzas in the end, if there were infinite variations of pizza on the menu and the ordering agent was constantly trying to upsell you on slathering on another layer of sauce and cheese.

Which is not to say there aren't exceptions in both directions, because as a pro pizza connoisseur you could even be better positioned to 'know exactly what you want', without ever having made a pizza; and as a pro cook you might still be overwhelmed by the ordering process.

Though all of this still assumes of course that we are in the 'complexity region' where we can expect the LLM/the pizza agent to deliver _exactly_ what we wanted, without issue. As soon as we leave that region (though it expands every week, it feels like), the skills of the cook are still very much necessary to make sure things are as they should be, and fix them if not.

- destabilize the world just by what they do: making previously very hard things (generating fake everything, manipulating people, hacking, bio-weapons research) easy enough fast enough that our civilization doesn't cope with it and/or

- accelerate AI research to the point where they make the dangerous ASI thing appear years earlier than would otherwise have happened

(and interactions like these are part of the reason why I <3 HN)

[1]: https://phrack.org

I mean, I can think of one reason. Them being quite far away from the ground most of the time, mainly.