The "trick" is that cunning powerful people fail forward, so they keep on doing dumb shit with even more impact.

Introspection is basically THE core mechanism for learning. That's HOW one learns on any topic. It's not a "wishy washy hippie feeling" (being provocative here) but rather introspection is (and to be fair I verified with https://plato.stanford.edu/entries/introspection/ just to make sure I wasn't talking out of my own ass) precisely looking at your inner workings. How you function IN ORDER to do better. You notice flaws, inefficient behaviors, things you enjoy, etc THEN you act on it.

Having no introspection is like doing math without verifying. It's like coding without compiling, linting or even executing without looking at the output.

So dumb it hurts.

TL;DR: bet on stuff being removed

the founder does not want to risk money for his own idea

while

funders have simultaneously also too much money while believing they don't have enough.

That very simple dynamic is what is driving investment in the Silicon Valley, itself praised worldwide as the forefront.

That's what bringing our own civilization on the economical (AI bubble), ecological (AI bubble, car brain) and democratic (surveillance capitalism, privacy zuckering) cliff.

I suggest checking Arvind Narayanan's tinkering with his own children. That being said not every parent is a Princeton professor working on AI so tread with caution.

Also on a pragmatic front if you want to tinker on the domain something potentially easy to setup yet relatively open https://www.home-assistant.io/voice-pe/

Nice... so the rush is to delegate power to the large American platform?

... that being said, and I did read https://www.home-assistant.io/blog/2025/09/11/ai-in-home-ass... or even have use a local modal for STT but IMHO it's not good, definitely not worth buying a $4k device for. It doesn't generate good rules, it gets some commands but it's quite basic. So I understand the concept but so far I haven't seen anything good enough to warrant such a purchase for such a very very specific niche. I can imagine literally a couple of persons interested in this but I'd bet they'd also have a GPU laying around that could be available for that too.

TL;DR: technically not impossible, I doubt it's a popular use case.

Well it's not exactly a guide and honestly it's quite outdated (because I stop keeping track as I just don't get the quality of results I hope for versus huge trade offs that aren't worth it for me) but I listed plenty of models and software solutions for self-hosting, at home or in the cloud at https://fabien.benetou.fr/Content/SelfHostingArtificialIntel...

Feels free to check it out and if there is something I can clarify, happy to try.

I don't want to make an ad here but I'm going to point to HuggingFace https://endpoints.huggingface.co (and to avoid singling them out just https://replicate.com/pricing too but I don't know them well) as an example with pricing.

The "beauty" IMHO of such solutions is that again you pay for what you want. If you want to use the endpoint only for 5min to test that the model and its API fits your need? OK. You want the whole month? Sure. You want 1 user, namely you? Fine, not a lot of power, you want your whole organization to use that endpoint? Scale up.

I'm going to give very rough approximation because honestly I'm not really into this so someone please adjust with source :

Apple Mac Studio M3 Ultra 96GB = $4K

~NVIDIA A100 with 80G ~ 10x perf compared to M3 Pro (obviously depends on models)

So on Replicate today a one can get an A100 for ~$5/hr which is ... about a month. But that's for 10x speed and electricity included. So very VERY approximately if you use a Mac Studio for 10 months on AI non stop (days and night) then it's arguably worth it.

If you use it less, say 2hrs/day only for inference, then I imagine it takes few years to have the equivalent and by that time I bet Replicate or HuggingFace is going to rent much faster setup for much cheaper simply because that's what they have ALL done for the last few years.

That being said if I were to be in such a situation, and if somehow the guarantees wouldn't be enough then I'd definitely expect to have the budget to build my own data center with GB300 or TPUs. I can't imagine that running it on a Mac Studio.

There is basically no lock-in, you don't even "move" your image, your data is basically some "context" or a history of prompts which probably fits in a floppy disk (not even being sarcastic) so if you know the basic about containerization (Docker, podman, etc) which most likely the cloud provider even takes care of, then it takes literally minutes to switch from one to another. It's really not more complex that setting up a PHP server, the only difference is the hardware you run on and that's basically a dropdown button on a Web interface (if you don't want to have scripts for that too) then selecting the right image (basically NVIDIA support).

Consequently even if that were to happen (which I have NEVER seen! at worst it's like 15% increase after years) then it would actually not matter to you. It's also very unlikely to happen based of the investment poured into the "industry". Basically everybody is trying to get "you" as a customer to rely on their stack.

... but OK, let's imagine that's not appealing to you, have you not done the comparison of what a Mac Studio (or whatever hardware) could actually buy otherwise?

I really don't get why anybody would want that. What's the use case there?

If someone doesn't care about privacy, they can use for-profit services because they are basically losing money, trying to corner the market.

If they care about privacy, they can rent cloud instances in order to setup, run, close and it will be both cheaper, faster (if they can afford it) but also with no upfront cost per project. This can be done with a lot of scaffolding, e.g. Mistral, HuggingFace, or not, e.g. AWS/Azure/GoogleCloud, etc. The point being that you do NOT purchase the GPU or even dedicated hardware, e.g. Google TPU, but rather rent for what you actually need and when the next gen is up, you're not stuck with "old" gen.

So... what use case if left, somebody who is both technical, very privacy conscious AND want to do so offline despite have 5G or satellite connectivity pretty much anywhere?

I honestly don't get who that's for (and I did try a dozens of local models, so I'm actually curious).

PS: FWIW https://pricepertoken.com might help but not sure it shows the infrastructure each rely on to compare. If you have a better link please share back.

FWIW might want to check https://github.com/wg-easy/wg-easy to remove yet another managed elsewhere piece of your setup.

Oh no, who would think about the big corporations? How is Micro$lop going to survive? /s

- open book, you have access to nearly the whole Internet and resources out of it, e.g. torrents of nearly all books, research paper, etc including the history of all previous tests include those similar to this one

- arguably basically no time limit as it's done at a scale of threads to parallelize access through caching ridiculously

- no shame in submitting a very large amount of wrong answers until you get the "right" one

... so I'm not saying it makes it "easy" but I can definitely say it's not the typical way I used to try to pass tests.

They just store tokens, without other FA at "worst" you get locked of your account but nobody else has access either. You're also supposed to, as good practice, not be limited to token generation and typically have a dozen or so of recovery tokens. Also if they were somewhat not working at doing the 1 task they should do, namely generate tokens, then you won't be able to use them so it won't even be added.

So... I might be missing something, can you please explain what worries you and why I should thus worry too?