And the disinclination of these companies to push the weights of their cutting edge models into people’s cars where they can be dumped.

This is more generalised.

But also since there’s a huge volume of data it’s too expensive to just keep scaling compute up (per car overhead) so there are necessary tricks involved.

I do think having a large model that can do this means that a small specialised model could be distilled form it though. Which is probably the most feasible path to production IMO.

Its very much {reliable, performant, flexible} pick none.

If you layer and segment correctly you can build atop a secure core and have some decent security.

Large outputs dominate compute time so are more expensive.

IMO input and output token counts are actually still a bad metric since they linearise non linear cost increases and I suspect we’ll see another change in the future where they bucket by context length. XL output contexts may be 20x more expensive instead of 10x.

Without it youre stuck with the basic http firewall, etc which is extremely dangerous and this is maybe the 1 opportunity we have to do this.

Make is useful and buyers will come. The never had issues selling multiple macs frankly.