For instance recently, media downloads on my NAS became quite a bit slower. I asked it to investigate and it quickly got back to me saying the issue was the ethernet link had been downgraded to 100Mbps from 1Gbps. It even went through journalctl to tell me when this had happened. To fix it I just had to plug an ethernet cable out and back in. Now this is the kind of thing, earlier I would have to spend an evening on. But given an open system like Linux, a well made safe model just makes it so much more accessible than having to RTFM for one off things I am sure to forget in a month.

I dont really let it run loose through my systems, and keep an eye on the thinking traces it puts out and the permissions it asks for. But now I dont have to slog through manuals and deal with gruff people on the internet. I can choose to do so after the fact, depending on how much free time and curiosity I have.

A serious router therefore needs to build up a dataset of how different models responded, end to end, to different tasks on different contexts. I won’t comment on whether the current frontier models can reliably judge these outputs, but I am sceptical about that. And moreover look at the state of evils! They are gamed to hell and keep losing credibility.

A more difficult problem for router builders is that they are working on an opaque system behind an external API. How can you reliably guarantee model behaviour when model behaviour has been shown to deteriorate under arbitrary conditions that have nothing to do with the task given? So much investment only to be an AI company that can get rug pulled by the real AI companies at any given time.

I would think the only people who can come up with good routers for a collection of models are the inference providers themselves. Because theoretically they have full control of how their models are served. And inference is not zero cost or cheap for them either. And going by OpenAI’s experience routing is not an easy problem for them to solve either. And they don’t have the incentive to route you to cheaper models and reduce costs for customers at the same time. Routing objective for them is to increase their own profits.

This makes you, an ai inference user an unwitting AI company with a non scalable product.

The biggest mental trap people have fallen for is the notion of “best” and, always using the frontier model. Instead you should just bite the bulet and choose the cheapest or the best. This routing dance is just tokenmaxxing in disguise.

Edit: Another thing concerning me is that the models themselves are not concrete behind their endpoints. Models can be arbitrarily dumbed down by reducing their inference resources. Once Anthro/OAI release a new great model, they are fully incentivised to dumb down their current models to drive traffic to the new shiny more expensive one. In fact they can do this for any reason. Once they switch things behind the API interface, how useful is your meticulously tuned router? Not much at all.

My man, that is the prize. Article is clearly written for US government elite to exhort them to bully other countries into staying in the system.

Recognise the safety and guardrails BS for what it is. An elaborate smoke and mirrors to get out of responsibility and liability.

And Claude or gpt can make websites just fine. So can the open weight models.