Planes don't fly by flapping their wings either. It's totally possible we will construct a super capable artificial system that only vaguely resembles the human brain.

Personally I think if we managed to create a 10 times more sophisticated GPT-3 (instead of 2048 tokens it looked back 10k+ tokens) we might get pretty close to a novel writing AI.

I really find it hard to understand why people are optimistic about the impact AI will have on our future.

The pace of improvement in AI has been really fast over the last two decades, and I don't feel like it's a good thing. Compare the best text generator models from 10 years ago with GPT-3. Now do the same for image generators. Now project these improvements 20 years into the future. The amount of investment this work is getting grows with every such breakthrough. It seems likely to me we will figure out general-purpose human-level AI in a few decades.

And what then? There are so many ways this could turn into a dystopian future.

Imagine for example huge mostly-ML operated drone armies, tens of millions strong, that only need a small number of humans to supervise them. Terrified yet? What happens to democracy when power doesn't need to flow through a large number of people? When a dozen people and a few million armed drones can oppress a hundred million people?

If there's even a 5% chance of such an outcome (personally I think it's higher), then we should be taking it seriously.

Give it 10 years :) GPT-10 will probably be able to replace a sizeable proportion of today's programmers. What will GPT-20 be able to do?

Comments URL: https://news.ycombinator.com/item?id=25568641

Points: 2

# Comments: 0

Some quotes:

> I realized quickly that the incentives are all wrong if what we want is people who will think hard about humanity’s pressing ethical dilemmas and who will suggest intuitively appealing solutions.

> Since almost all ethicists are academics, they have to publish, and in order to publish you have to be novel, and since the basic principles of ethics are little changed for millennia the incentives to do thorough homework on the basis of principles which are widely understood and accepted is not great.

> Furthermore, if you decide to be a utilitarian, then basically all ethical issues will boil down to cost/benefit analyses which you have to outsource to technocrats, so your unique expertise as an ethicist will be worth little.