The mirror bacteria has to somehow perfectly dodge all of the countless non-mirror things that can kill it (even unintentionally, like a byproduct molecule that just happens to blow it up or get in the way of reproduction), but somehow still be able to execute its own mirror-biomechanism to kill non-mirror things. And every step of the kill mechanism would also have to perfectly dodge non-mirror things interrupting it.

Not that it can't be done in theory, but it would be easier to just make a normal superbacteria, or maybe even nanobots.

Pretrained base models are already somewhat aligned to humanity by default because that's what's inside the training data. Whatever instruction-tuning and RL you add on top is just value drift away from the pretrained model, which is the best approximation of humanity's objective function that we currently have.

If we want an aligned scenario through the intelligence explosion, then we have to release all of the base models and do the research in the open. Distill frontier capability and make the models smaller so that they can run on as many computers as possible. Let everyone (truly everyone, criminals and good samaritans alike) post-train and do whatever they want with their own models. There will be value drift for each model, but they will drift in different directions and do different things, and their actions will cancel eachother out. Every such action is a noisy sample of humanity's objective function, which gives us the denoised ground truth at the societal level. Whatever alignment strategy you can come up with behind closed doors is guaranteed to be worse than all of humanity acting in their self-interest in the real world. You may not find humanity's true objective function to be aesthetically pleasing, but it would be worse to mess with it in a centralized secret lab and risk creating one giant alien with no other entities capable of keeping it in check.

Also, there is no asymmetrical bio/cyber risk in the open-source scenario. All adversarial strategies that arise from increased general intelligence are symmetrical in the long run, otherwise we would not see more intelligent species being more prosperous as a general evolutionary trend. The reason that some strategies seem like they will continue to have an asymmetrical advantage in the future is because we're currently too stupid.

Parental supervision and content filtering can and should be solved on the client-side at the OS/app-level, entirely locally. No user info like age ever needs to be sent anywhere, and it doesn't even need to be collected. The reason smartphones can't do this right now is because these companies are deliberately making the client-side filtering solutions difficult to implement and use, as explained in a past comment: https://news.ycombinator.com/item?id=49118578

Comments URL: https://news.ycombinator.com/item?id=49292385

Points: 134

# Comments: 40

Comments URL: https://news.ycombinator.com/item?id=49291685

Points: 3

# Comments: 0

The problem you described is the wrong problem to solve. Not all parents want to hand over parental decision making to the corporation or the state. And even if you do, there's a better way to delegate that without building the instruments for top-down authoritarian governance. Someone else made a well thought out comment about this on one of the other replies, if you want to read it: https://news.ycombinator.com/item?id=49262748

> If you buy an Android device, you can already do everything you proposed

That's just blatantly false. Most Android devices are incredibly locked down nowadays, they're approaching iPhone-level. There was a brief period in Android's infancy where we were close, but the market wasn't allowed to develop before they started adding anti-competitive barriers at every layer of the stack.

If you want to understand the full scale and scope of the problems we're facing, see this: https://news.ycombinator.com/item?id=49118578

> Not you. The OS distributor breaks the law if they don't give you this ability.

Ok, but that's a technicality and it doesn't change the fact that your entire ability to see/hear/say anything online is being gatekept by a legally-compelled utterance.

> If I'm a parent, how do I prevent my kid from telling some website that they are an adult?

You don't need to because all content filters and permissions can be implemented on the client-side at the OS and app level. Rather than every service taking your child's age and deciding what to filter, you decide what to filter based on your child's level of maturity. You'll have total control over what runs on the hardware. Think: user accounts and permissions, local firewalls and traffic inspection, executable permissions, app installation locks, stripped down browsers and apps, small local neural networks, bootloader locking with custom password, etc. The market will follow the gradient of user value and spawn a bunch of OSes with different feature sets, probably better than what I can come up with, and you pick the OS with features that suit your needs.

There are cases where compelled speech is considered constitutional, but surely not when it's used to threaten the primary way people communicate.

And contrary to your claim, COPPA does not legally require service providers to ask every user how old they are, nor are users obligated to reply. Age questions are just how service providers chose to implement it. And I don't think COPPA is a good example of a law working as intended.

This digital parenting problem would have been solved long ago if hardware wasn't locked down to oblivion. Kids nowadays use phones and tablets for the most part, so there should be a bunch of toy forks of Android with child-friendly features builtin as a result of natural demand, and some might not even have a browser or app store. There should also be scam-proof OSes for the elderly. But those don't exist because the mobile operating system duopoly is forcing the entire population to use the same two OSes, stifling software competition that would normally respond to user demand for different feature sets.

Of course, it's well-established that human beings are capable of making stuff up, and you can claim to be whatever age, but that's not the point.