I'm not saying I 100% wouldn't use AI to help me in product searches, but isn't one of the main selling points of AI that it is general-purpose? Why can't I just boot up ChatGPT and ask it what cars have XYZ things I need? Certainly being informed that Capital One's system is "multi-agentic" doesn't tell me much about what is being offered.

I think "some people" might be underselling it, as evidenced by the borderline innumerable fiction books in existence?

> and as such, it's a quite selfish activity

"quite" seems a bit harsh, surely "writing because you enjoy it" is pretty far down the list of all "selfish" activities? I'd imagine many authors also write because they think others will enjoy their works.

I mostly agree if "a human" is just any person we pluck of the street. What I still see with some regularity is the models (right now, primarily Opus 4.6 through Claude Code) making mistakes that humans:

- working in the same field/area as me (nothing particularly exotic, subfield of CS, not theory)

- with even a fraction of the declarative knowledge about the field as the LLM

- with even a fraction of frontier LLM abilities suggested by their perf in mathematical/informatics Olympiads

would never make. Basically, errors I'd never expect to see from a human coworker (or myself). I don't yet consider myself an expert in my subfield, and I'll almost certainly never be a top expert in it. Often the errors seem to present to me as just "really atrocious intuition." If the LLM ran with some of them they would cause huge problems.

In many regards the models are clearly superhuman already.

I don't think that it is likely AT ALL, but it is probably only necessary for China and the US to agree to stop, not all organizations and nation states. It is at least possible given leadership in both countries that see AI as an existential threat.

The hardware needed to run and train SOTA AI can only be made by a very small handful of companies in a small handful of countries that either the US or China have significant influence over. Making AI R&D illegal would stop 99% of it overnight, most of the researchers are in it for money rather than some ideological commitment and there are plenty of other well-paid jobs they could take. Doing local inference in secret with existing models and GPUs would be possible, but training new SOTA models probably wouldn't be.

Maybe this is overly optimistic, but if AI starts to have negative impacts on average people comparable to the plague, it seems like there's a lot more that people can do. In medieval Europe, nobody knew what was causing the plague and nobody knew how to stop it.

On the other hand, if AI quickly replaces half of all jobs, it will be very obvious what and who caused the job loss and associated decrease in living standards. Everybody will have someone they care about affected. AI job loss would quickly eclipse all other political concerns. And at the end of the day, AI can be unplugged (barring robot armies or Elon's space-based data centers I suppose).

I think it makes sense? Unlike small details which are certain to be explicitly part of the training data, "big picture stuff" feels like it would mostly be captured only indirectly.

Seems like the G in GPU is very obsolete now:

https://www.tomshardware.com/news/nvidia-h100-benchmarkedin-...

> As it turns out Nvidia's H100, a card that costs over $30,000 performs worse than integrated GPUs in such benchmarks as 3DMark and Red Dead Redemption 2

I think this is where we're headed, very quickly, and I'm worried about it from a social stability perspective (as well as personal financial security of course). There's probably not a single white-collar job that I'd feel comfortable spending 4+ years training for right now (even assuming I don't have to pay or take out debt for the training). Many people are having skills they spent years building made worthless overnight, without an obvious or realistic pivot available.

Lots and lots of people who did or will do "all the right things," with no benefit earned from it. Even if hypothetically there is something new you can reskill into every five years, how is that sustainable? If you're young and without children, maybe it is possible. Certainly doesn't sound fun, and I say this as someone who joined tech in part because of how fast-paced it was.

Right now I can think of very few white-collar jobs that I would feel comfortable training 4+ years for (let alone spending money or taking on debt to do so). It is far from a guarantee that almost any 4-year degree you enroll in today will have any value in four years. That has basically never before been true, even in tech. Blue collar jobs are clearly safer, but I wouldn't say safe. Robotics is moving fast too.

I really can't imagine the social effects of this reality being positive, absent massive and unprecedented redistribution of the wealth that the productivity of AI enables.

I'm surprised I haven't seen anyone do a case study, having truly non-technical people build apps with these tools. Take a few moderately tech-savvy (can use MS office up to doing basic stuff in excel, understands a filesystem) people who work white collar jobs. Give them a one or two-day crash course on how Claude Code works. See what is the most complicated app which they can develop that is reasonably stable and secure.

Is this saying that Claude matched the best human performance, where the human had two hours? I think that is the correct reading, but I'm not certain they don't mean that Claude had two hours, and matched the best human performance where the human had an arbitrary amount of time. The former is impressive but the later would be even more so.

> People want an Eisenhower doing these kinds of things

Why would people who don't want Trump doing it want an Eisenhower doing it? He helped overthrow democratically elected Árbenz in Guatemala with even weaker justifications than Trump overthrowing Maduro (Maduro at least seems to lack popular support and probably cheated in elections).

Eisenhower:

Overthrow of Árbenz to protect fruit company profits > series of military dictators > 30+ years of civil war where the US-backed government committed a genocide against Maya people

The only thing that IMO would be really hard to regulate would be the distribution of open-weight models existing at the time regulations come into effect, although I imagine even that would be substantially curtailed by severe enough penalties for doing so.

> Yeah, I'll concede, some physical tools like TNT or whatever should probably not be available to Joe Public. But digital tools?

Digital tools can affect the physical world though, or at least seriously affect the people who live in the physical world (stealing money, blackmailing with hacked photos, etc.).

To see if there's some common ground to start a debate from, do you agree that at least in principle there are some kinds of intelligence that are too dangerous to allow public access to? My extreme example would be an AI that could guide an average IQ novice in producing biological weapons.