Of course I don't know if there's really a way for this to be molded in current LLM's (sounds more like diffusion)

The main place they are a bit behind is in the number formats they support natively. Iirc M5 doesn't have native FP8 support, so you will take a speed penalty on quants where other architectures get better acceleration.

And given how fast the hardware and software is evolving, I can easily imagine a future where we all have very capable models running on our own devices for an embedded intelligence layer that's doing most of the day-to-day tasks, and only have to outsource to a super-smart cloud model for specific things.

While we're at the point where traditional software is pretty much fast enough for all but extreme use-cases, with LLM's it feels like we're back to the days where you press compile and go have a coffee or chat to your colleague.

But I think this will eventually be a problem solved at the OS level in a more streamlined way. I.e. there will be fine-grained permissions you need to approve to give an agent access to the system.

I'm not a fan of big-tech acquisition results either, but one benefit can be that a product continues to exist when it would otherwise become insolvent.

Models are already largely hardware agnostic. It would be pretty hard to put that cat back in the bag.

I could imagine them building value-added services on top of HF to advantage Nvidia products (i.e. "run this model on NVIDIA cloud" with one-click), but in this moment it's hard to imagine how they could actively disadvantage models built to run on other platforms.

My use-case is coding, currently working on a project with a Rust backend and TS/React/Vite frontend, with probably tens of thousands of lines of code total (including tests).