I wonder if the game state could be sent in an analog/continuous encoding rather than as encoding of a digital state. So that the human player could learn to roughly interpret the sounds, or at least figure out that the sounds are correlated with what’s happening.

That might have a different, possibly satisfying feel compared to the current bloops.

State would not be reconstructed bit-perfect but for a model of a physical system like pong, approximate might be ok.

1. The discussion setup where you imagine a counterfactual is given by magic does not entail that magic can do any other thing you choose.

“Imagine I give you $100, what do you spend it on?”

“A McClaren, since you could have given me $300K”

2. Taking your question directly: because these are not closely related problems.

I didn’t make any claims about the relative difficultly.

iRobot, the company that shipped the most robots in history because they focused super hard on finding a cost-effective subset of cleaning tasks, failed to scale up to other even closely related tasks, went bust and was sold off.

But yeah, it's probably straightforward, right?

That trash picker may never compete with a human. There's no reason to assume it will.

edit:

> none of those things strike me as any more challenging than the control software

Maybe the difficulties of motors and batteries are not intuitive. Look at their improvement rates over time. It's a far cry from transistor counts or FLOPS. And look at the performance and efficiency of humans and other animals. We are absurdly strong, light, efficient and cheap to maintain. The gap is huge and not closing quickly.

Imagine I give you for free the perfect trash picking robot control AI software. Works great. Benchmark smashed.

Now imagine what the rest of the robot is like, and how much it would cost. I mean really imagine, not just picture C3PO with a bucket and litter spike. How many motors and what are the torque vs. mass requirements? How is energy stored? It goes on, not all of it is solved, and none of it is cheap. Except compute.

AI likely does accelerate robotics usefully - certainly in sensing and control. But there are major bottlenecks that are not software, that AI can only help with as a generic engineering tool, same as it helps planes and fridges.

No, we are currently unable to do most tasks that require physical manipulation, independent of what anyone wants.

The video talks about accels in a few places, and how they are required for pose estimates.

The customer may not care what the internal contributions are to such a measure. Instead it represents the customers’ overall values in some way.

Good luck guessing ‘grep’ from first principles.