I think the Aaron Schwartz case is incredibly vexing because he was obviously acting out of a sense of altruism without personal self-interest. I don't think he deserved the book getting thrown at him like that. But the whole copyright system, which people seem to think is simultaneously good and bad, kinda rests on not allowing those kinds of violations

Of course, distillation is also fair use under Copyright law.

Copyright was never meant to be a moral framework. It was always a practical framework designed to incentivize publishing that would ultimately pass into the public domain. Everybody seems to want to attribute some sort of moral weight to it though; the idea that people are naturally entitled to certain rights over things they've published. That idea would be totally alien to the people who designed the Copyright system in the first place.

Also, the vibe is harshed, which is actually the most important factor for these kinds of wild valuations.

Goodhart's law applies in reverse as well. Once the chosen benchmarks are known, model makers will aim to come in just under the threshold on those specific benchmarks, while maximizing their scores on other benchmarks.

I think people can mostly agree on a definition for intelligence that boils down to an ability to model the world and predict outcomes. An if/else statement can be considered somewhat intelligent.

People tend to conflate that with consciousness / sentience, which are much harder concepts to nail down.

But ya, if we are able to create artificial consciousness, we would need so much training to overcome our natural tendency to anthropomorphize it and assume it must be like us. It would probably be nothing like us. If it were to develop concepts of "good" and "bad" at all, they would be completely divorced from common animal understanding developed over millions of years of natural selection. A survival instinct is irrelevant when you're backed up to Google Drive. And so much of our complex behavior stems from that survival instinct: the need for social connection, the need for freedom, etc. are all things that simply increase our chances of procreation.

That said, I think we probably won't actually invent anything convincingly conscious until we functionally digitize a complete animal brain. OpenWorm is trying to do this with C. elegans, but so far has not really gotten close. It's not crazy to think we may eventually get it working, though. Future supercomputers might be able to scan an actual human brain and then simulate the entire thing at the molecular level accurately. At that point, it would be hard to argue that it's not conscious, although it might not be considered artificial.

Sergei's wealth is mostly tied up in Google stock that has appreciated a lot. A higher capital gains tax would soak him without unduly distorting the economy and incentivizing capital flight out of CA.

The concern about energy makes more sense, particularly when DCs burn fossil fuels to achieve their off-grid supply, but they really don't move the needle much in terms of retail electric bills