For RL, we are arriving at a similar point https://www.tobyord.com/writing/how-well-does-rl-scale

Next stop is inference scaling with longer context window and longer reasoning. But instead of it being a one-off training cost, it becomes a running cost.

In essence we are chasing ever smaller gains in exchange for exponentially increasing costs. This energy will run out. There needs to be something completely different than LLMs for meaningful further progress.

We are missing the value function that allowed AlphaGo to go from mid range player trained on human moves to superhuman by playing itself. As we have only made progress on unsupervised learning, and RL is constrained as above, I don't see this getting better.

This means that the agent will can have context of previous decisions, something that they currently struggle with as they are always starting from a blank slate.

Vegetarians and vegans have lower T2D incidence on average FWIW.

Same way an opioid pill is still prescribed in cases of cancer or severe pain. Just that there are probably better, milder alternatives that don't have as many side effects that could fit a lot of these people with milder problems.

It feels a bit like the US approach to pain management – instead of massages or physical therapy, some opioid pill is cheaper to prescribe.

If you desire performance close to the chip, you chose the wrong language and should write code in a language closer to the chip. Unless the abstractions and concepts required for your primary work are so different from what you are using for day-to-day work (data science, ML python and C++ bindings for interacting with the GPU)