eg consider child-proof packaging and labeling laws for medication, which dramatically reduced child mortality due to accidental drug misuse.

To be clear, this isn’t what Dwarkesh was pointing at, and I think you are using the term “continual learning” differently to him. And he is primarily interested in it because humans do it.

The article introduces a story about how humans learn, and calls it continual learning:

> How do you teach a kid to play a saxophone? You have her try to blow into one, listen to how it sounds, and adjust. Now imagine teaching saxophone this way instead: A student takes one attempt. The moment they make a mistake, you send them away and write detailed instructions about what went wrong. The next student reads your notes and tries to play Charlie Parker cold. When they fail, you refine the instructions for the next student … This just wouldn’t work … Yes, there’s RL fine tuning. But it’s just not a deliberate, adaptive process the way human learning is.

The point I’m making is just that this is bad form: “AIs can’t do X, but humans can. Humans do task X because they have Y, but AIs don’t have Y, so AIs will find X hard.” Consider I replace X with “common sense reasoning” and Y with “embodied experience”. That would have seemed reasonable in 2020, but ultimately would have been a bad bet.

I don’t disagree with anything else in your response. I also buy into bitter lesson (and generally: easier to measure => easier to optimize). I think it’s just different uses of the same terms. And I don’t necessarily think what you’re referring to as continual learning won’t work.

In practice, continual learning has not been an important component of improvement in deep learning history thus far. Instead, large diverse datasets and scale have proven to work the best. I believe a good argument for continual learning being necessary needs to directly address why the massive cross-task learning paradigm will stop working, and ideally make concrete bets on what skills will be hard for AIs to achieve. I think generally, anthropomorphisms lack predictive power.

I think maybe a big real crux is the amount of acceleration you can achieve once you get very competent programming AIs spinning the RL flywheel. The author mentioned uncertainty about this, which is fair, and I share the uncertainty. But it leaves the rest of the piece feeling too overconfident.

And there are now many tasks which I can confidently delegate away to AI, and that set of tasks is growing.

So I agree with the author for most of the programming tasks I can think of. But disagree for some.

[0] https://arxiv.org/abs/2203.03466

So improving uptime involves holding out a set of GPUs to swap out failed ones while they reboot. But also the whole run can just randomly deadlock, so you might solve that by listening to the logs and restarting after a certain amount of inactivity. And you have to be clever with how to save/load checkpoints, since that can start to become a huge bottleneck.

After many layers of self healing, we managed to take a vacation for a few days without any calls :)

Also the author rules out psychology, but I wouldn't, especially since there were multiple confirmed errors in the route preparation, which I expect could reduce one's trust in the fairness of the competition. In the moment, I might start to wonder, "If one hold was out of place, why not more? Is anyone even checking this?" even if untrue / unlikely.

If I had to critique Hinton's claims, I would say his usage of the word "understand" can be vague and communicate assumptions because it's from an ontology used for reasoning about human reasoning, not this new alien form of reasoning which language models embody.