In a constrained market, supply and demand favors folks who can most efficiently extract rent. Local models only make sense in a world with abundant compute and energy.

Demand will only go down if people reduce their use of these AI tools. Given how much folks here complain about quotas, I'm very skeptical that will happen willingly.

Surely it is a more efficient use of DRAM to run inference on shared hardware with large batch sizes and more utilization.

I want a robust open-source ecosystem where anyone can take my hardware projects and modify them without needing to deal with licensing friction.

The cost of not being efficient is even higher DRAM costs than we have now, given supply and demand.

This stuff is expensive because supply is much lower than demand. If everyone was to run their own hardware with a batch size of 1, we'd have 100x more demand for inference hardware and electricity than we do now, and people would be even more frustrated. Efficiency is everything, and we need all the economies of scale we can get to meet demand.

Also, supporting those tiny pulses requires large capacitors to hold a charge in between pulses. That plus the required magnetics make PoE sensors way more bulky and expensive to manufacture than old fashioned RS-485 sensors.

https://discreetfx.com/documents/NewTekVideoToasterDeveloper...

I remember using this thing when I was a kid, trying to figure out how all the switching effects worked, so stumbling on this manual many years later was really satisfying...

I recall wasting a lot of time staring at decompiled .NET bytecode trying to understand how to work around many problems with it, and it was clear from the decompiler output that WPF's architecture was awful...

People having easy access to LLMs makes this job much harder. LLMs can create what looks at the surface like expert-written code, but suffers from below-the-surface issues that will reveal themselves as intermittent issues or subtle bugs after being deployed.

Inexperienced devs create huge commits full of such code, and then expect me to waste an entire day searching for such issues, which is miserable.

If the models don't improve significantly in the future, I expect that most high-stakes software teams will fire all the inexperienced devs and have super-experienced engineers work with the bots directly.

https://youtu.be/eMefy5VK9TI - Toto, Montreaux, 1991