Compared to what??

Actually, someone even commented in that thread about how it was similar to biological mechanisms: https://news.ycombinator.com/item?id=47390619

> A quick note: All of the data visualizations in this roundup are free to use as common content. You are welcome to share and republish them.

Plus if you want to make a robot which can do everything a human can do, without exception, you'll start running into weird edge cases if you make it with a fixed wheeled base. Like suppose you want it to climb a ladder or turn sideways to slip into a narrow space and retrieve something. Or maybe you want it to be able to climb in and out of vehicles designed for human occupants. Would you want to have to buy a special vehicle to bring your robot somewhere with you?

And judging by the name of the company and that they likely aspire towards more than what they've made so far, I'd guess that a long-term goal could be to also make something for people who realize the weakness of their flesh and crave the strength and certainty of steel.

For better or for worse, the equivalent race right now is in AI.

Also, on the graph you linked to, you can click on the line in the graph of population versus year to the right, and if you start back in the 1950s and work your way along to the present, you can see the big lump of the baby boomers getting older. That's why people say the United States has an aging population.

This seems almost comically lemming-like. Fear of missing out . . . on dying?

> Getting frozen and brought back in the distant future, even if we assume you'll be in great health, means all your friends and family are gone.

(Assuming none of your friends and family were also frozen.) But more importantly, I wouldn't suggest that people should aspire to death because of losing all their friends and family. If starting over (in what's likely an unfamiliar society and culture) is all one's got, that's still something.

Mainly I find it grotesque that it might be a way to slip past death, but only for rich people. But I can't bring myself to suggest that it would necessarily be a better universe if they took the normal death route instead.

Penrose tries to respond to this by saying that various things may affect the functioning of a brain and keep it from reliably perceiving such truths, but when brains are working properly, they can perceive the truth of things. Most people would recognize that there's a difference between an idealized version of what humans do and what humans actually do, but for Penrose, this is not an issue, because for him, this truth that humans perceive is an idealized Platonic level of reality which human mathematicians access via non-computational means:

> 6.4 Sometimes there may be errors, but the errors are correctable. What is important is the fact is that there is an impersonal (ideal) standard against which the errors can be measured. Human mathematicians have capabilities for perceiving this standard and they can normally tell, given enough time and perseverance, whether their arguments are indeed correct. How is it, if they themselves are mere computational entities, that they seem to have access to these non-computational ideal concepts? Indeed, the ultimate criterion as to mathematical correctness is measured in relation to this ideal. And it is an ideal that seems to require use of their conscious minds in order for them to relate to it.

> 6.5 However, some AI proponents seem to argue against the very existence of such an ideal . . .

Source:

https://journalpsyche.org/files/0xaa2c.pdf

Penrose is not the first person to try to use Gödel’s incompleteness theorems for this purpose, and as with the people who attempted this before him, the general consensus is that this approach doesn't work:

https://plato.stanford.edu/entries/goedel-incompleteness/#Gd...

Also I think in the US at least, religion plays into it. God over humanity is conceptually the ultimate hierarchy model that Christians buy into. So it seems like Christians often don't have as much issue with embracing the idea of one person being over another person as a reflection of that model. This seems a little bit funny given how anarchic Jesus was, but he never said, "Yo, check it out, I am literally an anarchist and you should be too," and so they seem to have missed that.

> Only on the outside, where they can be appreciated as a whole, will the impression of intelligence emerge. A human brain, too, does not exhibit the intelligence under a neurobiologist's microscope that it does participating in a lively conversation.

We only have fuzzy definitions of "intelligence", not any essential, unambiguous things we can point to at a minute level, like a specific arrangement of certain atoms.

Put another way, we've used the term "intelligent" to refer to people (or not) because we found it useful to describe a complex bundle of traits in a simple way. But now that we're training LLMs to do things that used to be assumed to be exclusively the capacity of humans, the term is getting stretched and twisted and losing some of its usefulness.

Maybe it would be more useful to subdivide the term a bit by referring to "human intelligence" versus "LLM intelligence". And when some new developments in AI seem like they're different from "LLM intelligence", we can call them by whatever distinguishes them, like "Q* intelligence", for example.

This is true ...

> If it becomes impossible to rent properties profitably, then eventually there would just be fewer rentals available.

But this is a leap that doesn't apply to the Georgist scheme. A key feature of the scheme is that taxes on buildings (possibly among other taxes) are removed in favor of the land value tax. It's true that if the tax on some piece of land increases sufficiently, it will eventually become impossible to profitably charge rents on existing buildings on that land--but improving existing buildings or building some superior building (for example, a new building with more units than a pre-existing building) allows landlords to continue profiting.

If the current landlords don't want to improve buildings there, they can sell the land to someone who will. And it's clear that someone will, because if there's enough demand for the land that the land value tax makes it unprofitable to sit on it without improving it, someone is going to want to do something more with it.

The reason we don't already have a Georgist-style economy is quite obvious, of course: the homo sapiens landlordicus is a very soft, squishy, vulnerable subspecies. If they had incentives to improve their properties and thereby possibly even work as hard as the rest of the human species, they might well go extinct. And no one wants that.

> We don't.

I don't mean humans as they presently are, but humans as somewhat-novel GAI entities after upgrading (and humans as an ongoing source: likely there will be some humans that aren't interested in changing themselves in such ways, but who might change their mind over time and/or have children who see things differently).

> (I am speaking in economic terms. We can deeply intensely highly "value" many things, but if we don't actually invest economic value in protecting or nuturing those things, our appreciation has little or no impact.)

I don't think this is true. For example, suppose you deeply value a bunch of things that you don't think have much economic significance, but economic significance is the most important thing to your perspective / philosophy, so you don't take care of the things that you deeply value. This does have a significant impact: you become sad, you become depressed, you become less economically productive. So then you say, oh, I'll just take care of those other things I care about, and then I can be economically productive! So you make a few changes but then find yourself feeling down again not long afterwards because economic value is the cornerstone of how you think about things. And then you realize that from an economic perspective, it's more beneficial economically to stick with a perspective that embraces your values as you discern them (and how those continue developing over time) rather than sticking with a perspective that puts economic value first. And then you are taking your first steps to freedom from the curse of your Bachelor's in Economics. ;)

Ethical GAIs also seem likely to me ... hopefully.

The additional value that humans present would thus be a diversity factor. Much as it might be economically beneficial to cut down the entire Amazon, there's also significant economic benefits in keeping it intact and finding out what kinds of crazy biological stuff comes out of there. And over the long run, we get way more economic benefit from finding crazy things in there than in chopping it all down. There are all kinds of ways to explore uncertain dimensions, but looking at whatever the universe has already managed remains a consistent source of value.

> But transforming ourselves into GAI to keep up (while maintaining continuity of memories, etc.) is going to be a much more expensive proposition than simply making more GAI hardware from scratch.

It's basically a fixed cost because of how slowly the human population grows (if it even continues to grow).

Synthetic GAI entities will be able to use vastly more resources than humans presently do, because they'll be able to create more of themselves as quickly as resources become available, and they won't be stuck operating at a fixed clock speed. It's also probably going to be much easier for them to make use of off-world resources. So the total population of GAI entities could plausibly explode compared with the total human population.

It comes down to timing: if there's technology available for humans to become less ... biological, and it seems very desirable to lots of people, but it's really expensive and there aren't that many GAI entities yet, that might be less pleasant. But if that kind of tech depends sufficiently on AI developments that there's some significant lag time between a GAI population explosion and that kind of human-upgrading tech showing up, the cost of making it available to whatever humans want it might be a drop in the bucket from a total economic perspective.

To be clear, I don't see biological humans "keeping up" over the long term. If we take AGI and BCIs and various Ship of Theseus questions seriously, there could be some significant blurring of lines between "human" and "AI." And if we combine that with GAIs originating from humans, the concept of "keeping up" seems to become less meaningful. Who's keeping up with who?

Lastly, I'm suspicious that GAI entities won't be purely economically motivated, because I don't see any reason that they'll be "purely" anything at all. There is no magical "essence d'intelligence"; instead there's staggering layers of complexity. And every plausible AI safety approach I've seen so far involves training AIs to be inclined towards beneficial acts and away from harmful acts because there's no way to program conceptually pure "motivations" into them.

Which is why no child has ever survived until adulthood.

And TBH I find this an appropriate metaphor because the promise of AGI is that every adult human will be infantile by comparison--but there are also potential technologies that could allow humans to ... "upgrade" themselves and become a mature ... whatever. Sadly brain-computer-interfaces seem like a technology that works best when enabled by AI (to help with interpreting brain signals), so it seems quite unlikely that any biological humans are going to keep up with AI over the medium term.

Consider the relative charisma of the people around him, though.

1. I wouldn't necessarily assume FTL. But even if we assume "arbitrarily technologically advanced," there's no reason to think that any technology will ever have an absolute 0% chance of failure.

2. It only has to be sufficiently peaceful to not wipe itself out.

3. I don't think AI doing things changes either of the previous two points.

What I find harder to believe is that humans would be able to get their hands on a crashed alien craft and keep it.