I think this misses the actual limits here.

The problem isn't demand it's, "how much people are willing to spend on it".

Cheap AI has to be served on cheap compute, and if inference gets cheap enough to unlock massive usage numbers, by definition it also doesn't require anywhere near as much infrastructure per unit of demand.

Take DeepSeek serving ~100T tokens/day, depending on workload and utilization, you're potentially talking about only a few thousand last-gen GPUs. With current-gen GPUs maybe closer to ~1,000, and with Rubin even fewer I will be damned if I could get my hands on one.

That's the part I think people are missing when they extrapolate token demand into enormous infrastructure or AI revenue.

Yes usage will explode. But if the cost per unit collapses, the revenue doesn't necessarily go up with it.

You can't simultaneously argue that intelligence becomes so cheap that everyone uses enormous amounts of it, while also assuming customers will somehow spend trillions of dollars a year consuming it.

There is no obvious $1T customer-facing AI revenue number at the end of this rainbow in the short/medium term.

The average person isn't going to spend anything remotely comparable to what they spend on a car every year for an AI service. Even businesses have budgets now, huge demand doesn't matter if the willingness to pay isn't there.

The only path I can see to numbers like that is AI consuming existing business domains, even then it's very thin.

Say SaaS + legal + consulting + BPO + various other service industries collectively represent something like $10-20T globally.

Even if AI eventually replaces an enormous portion of that, it's probably not doing so at the same price. Why would customers switch otherwise?

Either the AI product has to be dramatically better, which is difficult for mature workflows, or dramatically cheaper which is much more plausible.

If it replaces $10-20T of existing services at roughly 1/10th or 1/100th (more likely) the cost, then you're looking at maybe a ~$1T AI revenue opportunity after replacing an absurdly large fraction of the existing service economy.

Who are now unemployed and can't pay for shit.

And that's before competition.

I think it's crazy to assume AI companies won't compete aggressively on price. As capabilities diffuse, smaller models catch up, inference hits pareto frontier the open-source alternatives have already improved and caught up, margins on routine intelligence should compress "hard" (emphasis on "hard").

We've already seen how difficult adoption can be even when the technology looks impressive on paper. Cheap here means 100x cheaper for 10x more demand that's a net 10x loss before any software or hardware optimizations.

So yes, I completely agree that cheap intelligence can bring an enormous amount of new usage.

"I just don't think usage means revenue." (you can plaster it on a wall if you want to, "usage doesn't mean revenue", if you want to find that out I have foss software bridge to sell)

The PC analogy actually reinforces this if you really think about it. Compute became "vastly more useful" while the cost per unit of compute collapsed. Society captured enormous value, but all computer companies are literal failing giants without the AI hype. Value got caught by people who provided productionization.

Now if people expect AI to self productize itself I am happy to tell your try it. We all saw how OpenAI fell behind Anthropic because they thought that would work...

Google couldn't productize the search, instead they sold the eye balls and web-real-estate. Maybe that's the AI business model, but that's not $1T worth given you need to unglue people from other stuff.

Unless we get something approaching genuine ASI producing so much additional economic value that entirely new trillions, I don't see a path to $1-2T in direct AI revenue from customers.

The market simply can't absorb that level of spending.

Demand can be effectively infinite at the right price. But I think people are delusional on HN and SF if they think that number is in Trillions like the investments seem to suggest.

I am not saying Nvidia will fall tomorrow but someone will have to pull the breaks before this car goes to hell.

If someone told me that costs for X will keep increasing because they have been increasing rapidly in the last 1.5 years, but they have a history of continuously decreasing for decades before that.

I am not sure if I will take anything they say serious, I am not sure if it's HN or AI but people are delusional if they think compute costs will keep increasing from now on...

Either AI will be really good, hence compute and everything will materially depreciate or it won't be much better than it is today and token volumes will plateau compared to compute.

For instance the amount of token compute that's to come online in 6-12 months is several times what we have today...

Second 3) Compare performance in terms of difficult tasks/$ over the last 6 months, 3 months, etc. Open weights are a ratchet. In terms of intelligence per $, a Spark is never going to be a worse deal tomorrow than it is today, at least until the entire platform is replaced or obsoleted.

This is a bad take because again this assumes DGX Spark will not depreciate in price, we will have something better for far cheaper surely in the next couple years. M5 Max & Ultra are already arguably it, but will have to see.

> 71 days ago the best model you could run on two Sparks was an aggressive Q3 quant of Qwen 3.5 397B (AA 34). 70 days ago it was a mixed-quant of GLM 5.2 (AA 53). 30 days ago it was full fat DeepSeek 4 Flash (AA 53). Today it's GLM 5.3 Flash (AA57) and/or Qwen 3.8 Next (Unknown). Sometime this week it will likely become mixed-quant GLM 5.3 (AA 60).

This has nothing to do with DGX Spark's value, if models get cheaper the API costs also go down, this is not a defensible argument to cost to value.

Are people on HN really not thinking straight?

Tldr; no matter how you do the math compute is only getting more valuable because of a temporary crunch, don't expect this to continue permanently, sure you maybe able to time it and make money but so could you in stocks this is not for investments. Further second hand hardware sells for cheaper than sticker price, outside of a bubble..

And models getting cheaper == APIs getting cheaper == your hardware becoming worse value as your electricity & maintanence costs still remain.

I am not saying local models don't have their place but if someone is trying to use this logic to justify their purchase then I wish them all the best, as someone who is actively working on AI compute/inference/hardware stuff I personally don't have this level of courage.

But this is not a sound investment strategy that if something is going up and seems like it might keep going up, especially when investing in heavily depreciating assets like compute.

That's less than what 40$ at current API rates... So if you are willing to pay 200$ per month you will get much better limits paying API rates.

You can't run large Kimi K3 models on 10K worth of hardware either way, you need to spend like 50K USD minimum.

Just pay for the API rates or get a low cost provider that uses higher batching, you can get shittier tps but much better prices, probably go as low as 20$ for as much usage as you can ever get from a 10K USD machine from GLM 5.3 Flash...

The issue is nothing expensive runs on these devices and cheap stuff isn't worth running locally, eletricity costs ~12cents/kwh in us iirc, so at 330W M5 Ultra will burn around 8 * 0.12 = ~1$ per day extra in electricity so the electricity is going to cost you the same as the API rates(30$ per month).

I truly don't think you are accounting for the costs here properly. But again if money truly doesn't matter it's much better for privacy and better than paying one of the shady AI labs who are doing god knows what with your data.

You will get around 2-4 concurrent streams of aggregate tokens at best for such a model and around 0.5B output tokens per month assuming you use loops and run it when you are sleeping. That's 500 (per mill) * 0.5$ = 250$ only at most.

Then there is maintanence and efficiency costs due to electricity usage and such, any down time, etc.

You will be lucky if you can squeeze more than 200$ of value out of it in a month.

I don't think people should buy local hardware for money reasons, by the time you will pay off a 10K USD machine, 2-3K USD machine will catch up and beat it by a significant margin.

Unless your expectation is that we will be in hardware winter for the next 10+ years. At 200$ per month it will take around 200 * 50 = 10k, that is, 50 months, so around 4-5 years.

Again assuming you are making the most of your hardware somehow, very hard to do in practice.

I don't recommend people to use compute as investment or payoff thing, but if you have the money to burn and can afford it why not, maybe with some software optimizations it will be cheaper but then again Z.ai is currently offering 50% discount and providers will offer cheaper rates for sure.

But either way you will never be able to burn more than 200$ worth of token on a cheap hardware device, because inference becomes more profitable the more you scale it up, you have separate prefill and decode engines/systems, and a lot of nuance, but assume for every 10x increase in infra you increase margins by 5-10%.

So from 10K to 100K to 1M to 10M to 100M.. I don't think this curve continues beyond 100M but I have no idea about that scale unless some AI lab is interested in hiring me lol.

So a 100M infra will have ~30% better margins than you at 10K, then there is software optimizations but that's cheap enough, though some of it is only viable at scale.

Either way assume 10K is the price of privacy if you really want to buy it. Don't worry about making the most out of the usage, you will always be in a net loss but I would assume for you 10K doesn't matter.

Otherwise I think they just meant chinese models not mainstream models.

Like Hy3, Mimo, etc. Also gemini models but not sure if they are mainstream anymore.

That's how all engineering leaders I have worked with have worked.

https://www.blocksandfiles.com/ai-ml/2026/01/20/samsung-sk-h...

Big manufactures have absolutely cut production of lower margin products and it's not like they can re-purpose all of the equipment used for it for HBM yet. edit They can definitely focus more resource because of it on HBM though.

Things have improved a bit since govts are starting to get angry.

Tldr; no one is cutting the production of high margin stuff, but low margin stuff can go and die, because it's not a free market it's a stock market these companies want to show off better margins in every unit there is no reason to keep inefficient things running when not doing it pays more.

They just don't want 30-40% margins anymore. That is not free market that is quite literally serving their own interests because they know there is a monopoly.

All big memory manufacturers are heavily supported by the govt, that's what has made them the monopolies they are Micron supported by USA (they crushed Japan's memory industry for it), and Korean giants are more tangled up with the govt than well most people in US would realize.

Somehow people think free market means anything goes, which honestly is where we stand today so I don't blame anyone.

But using govts to crush competitions and build yourself up on subsidies and similar isn't really free markets. China hasn't been able to grow a significant memory chip industry yet because of lack of EUV machines and similar technologies blocked by the US, which is also not "free market".

As for WD, and Sandisk separating I feel like this was more mutual and they new HDDs or SSDs being together was a distraction for both. Which feels more strategic, think of it like this, the shareholders who had the shares how hold shares in both and together they might have likely been worth less so shareholders made money.

Aka good for free market. I don't see how that relates to the actual memory crunch issue more broadly.

They have slow rolled rollouts and everything, if there wasn't CXMT they would probably still wouldn't have budged on building new fabs for a while longer.

You can absolutely price fix/manufacture a shortage. Ever heard of merchants setting fire to crops after hoarding to sell at a higher price?

The issue is the distributer and producer are the very same people this essentially means producers have no incentive to rationalize supply, they can make much more money of the long drawn out shortage.

Maybe it doesn't matter in a post AI world but perhaps it will allow better abstractions.

No need to yuck someone else's yum.

Azure I am not sure how long they will be around doing MAI thinking models, I feel like they will pivot to smaller simpler enterprise only models. And given they already own npm and github openrouter might have made sense though I think they might buy huggingface, not that I want them to but it just feels like it.

Google IDK what google is doing exactly all my friends I knew in the AI teams are out a while ago, so maybe they are doing something really great we just don't know yet.

But I feel like it makes sense for hyperscalers since they can push their weight around a lot better than openrouter and can offer extra compute when providers are under crunch at higher prices to handle spikes. I think they are the only ones who can truly do fluid compute for GPU/AI in the short term(next couple of years).

Maybe after than we might have other big players in the space. Given the sheer scale of buildout I can almost guarantee we will see this pivot, otherwise there is too much hardware and token prices are too high.

Imagine if Stripe's end game is to control which producers of tokens become visible for a price aka middleman tax for tokens. If AI is as big a change as people claim honestly AWS should have bought openrouter instead.

But either way though 7B$ is just too rich no matter what, because the middleman tax is only worth it if you have a moat which OpenRouter doesn't.

All it takes is one of the bigger players to get serious and they will have a similar platform up and working in months if not weeks. (like Vercel & Cloudflare are already doing but even they are small compared to the true behemoths)

If hyperscalers start their own open routing service and cut off openrouter completely and become price competitive, I feel like they could wipe them out.

7B$ is just too much. Without enough of a big defensible moat. Especially since you can use a router in-front of openrouter and move users piece meal.

Can anyone explain what makes OpenRouter specifically worth 7B$ over everyone else? I say that as someone who has used a lot of similar services with similar results. Openrouter is better but only marginally.

Heck there isn't more than 1 commercial airplane crash in every few years globally.

Trains might be higher but if UK is having 12+ crashes a month I think you should really re think things. But again data?

Same with airplane, but often thats the only solution if cloudflare is your only solution in your engineering problem I would much rather you re-review and reframe that problem.