What exactly are these dozens of breakthroughs? Most frontier models architectures today still look very much like GPT2 at their core. There were various of improvements like instructgpt, finetuning techniques, efficiency improvements with kv caches, faster attention, lora, better tokenizers, etc. Most of these are for making things run faster. The biggest differentiator has probably been data curation and post-training data and the ability to fit more into the model. But I think we had few breakthroughs that would fall into the category of different technologies.

Now I would love to tell you about all the amazing magical healthy benefits that have come with that, but unfortunately there are none. I feel no difference at all.

There is indeed something useful about trying to write elegant code. Not because others read it. But because that's how you learn about the engineering tradeoffs and abstraction that exist everywhere.

I'm curious what their long term vision they pitched investors is.

I was prototyping something with pi under the hood for a personal project, going to switch off it now.

Annoying Shorts. I'm trying to keep my watch history clean to "steer" recommendations, but YT keeps adding things to it that I didn't actually watch just because I happened to hover my mouse over a video, etc.

LLMs are nothing like that. They are probabilistic systems at their very core. Sometimes you get garbage. Sometimes you win. Change a single character and you may get a completely different response. You can't easily build abstractions when the underlying system has so much randomness because you need to verify the output. And you can't verify the output if you have no idea what you are doing or what the output should look like.

Even if you can get an LLM to output good artifacts that don't eventually evolve into slop, which is questionable, it's really not that useful, especially not for a personal wiki.