Hacker News: Bolwin

New comment by Bolwin in "Don't trust large context windows"

Bolwin — Sun, 14 Jun 2026 15:02:08 +0000

It's pretty hard to measure because most context rot comes from related context and the model has to be able to figure which parts are truly relevant, which ones are relevant but stale, which ones to ignore etc.

Each relevant thing is basically a rule. Trying to so something with 500 rules is what's hard.

If you take a standard benchmark and just prepend a random book to it, it will not capture that

New comment by Bolwin in "Don't trust large context windows"

Bolwin — Sun, 14 Jun 2026 14:46:30 +0000

I don't use Claude Code. I use my own handwritten agent (formerly using Pi) and know every token that goes into it. There are zero memories to confuse it. The system prompt is 200 tokens and completely self consistent.

Plus I've found that the only time models go above 100k tokens anyway is when they've started looping at which point it's much better to go back anyway.

Anecdotally most models know their recall is terrible (or have been trained to act as such), that's why they constantly reread files before editing or while reasoning.

New comment by Bolwin in "Don't trust large context windows"

Bolwin — Sun, 14 Jun 2026 08:01:24 +0000

I see this said often and find it insane given how many times I find opus models making basic recall mistakes at <100k tokens.

Personally I consider < 60k to be the smart zone for opus. This is worse for opus 4.7 and 4.8 cause of the more granular tokenizer

New comment by Bolwin in "Kimi K2.7-Code: open-source coding model with better token efficiency"

Bolwin — Fri, 12 Jun 2026 16:47:29 +0000

What do you mean by custom format? Non-json?

New comment by Bolwin in "FrontierCode"

Bolwin — Tue, 09 Jun 2026 01:34:49 +0000

What is the "house" harness for minimax? They haven't released any

New comment by Bolwin in "The OnlyFans Economy of American AI"

Bolwin — Sun, 07 Jun 2026 16:48:53 +0000

If this is what we get without editors I want every thing I read to be without editors

New comment by Bolwin in "Hacker News, Sans AI"

Bolwin — Sat, 06 Jun 2026 05:53:49 +0000

Probably not as fast as a simple regex but static embedding models can get stupid fast e.g https://www.flowercomputer.com/news/fast-static-embedding/

New comment by Bolwin in "Hacker News, Sans AI"

Bolwin — Sat, 06 Jun 2026 05:49:54 +0000

It is a little ironic coming from the most prolific AI poster here

New comment by Bolwin in "The back cover of C++: The Language raises questions not answered by front cover"

Bolwin — Sat, 06 Jun 2026 05:43:01 +0000

Also is this an official Microsoft dev blog?

Probably not a good look back at publishing hq

New comment by Bolwin in "Hacker News, Sans AI"

Bolwin — Fri, 05 Jun 2026 21:54:11 +0000

You can get the best of both worlds with a small embedding model

New comment by Bolwin in "Am I Unc?"

Bolwin — Fri, 05 Jun 2026 17:34:27 +0000

The social section is also just selecting for introverts it asocial people

But they expect a few wrong

New comment by Bolwin in "MAI-Thinking-1"

Bolwin — Wed, 03 Jun 2026 15:34:55 +0000

In my experience above 60k quality noticeably drops.

30k for open source models

New comment by Bolwin in "Step 3.7 Flash"

Bolwin — Fri, 29 May 2026 17:03:23 +0000

What's wrong with the name? The step function is a pretty well known one

New comment by Bolwin in "Xiaomi MiMo-v2.5 Series API Permanent Price Reduction Up to 99%"

Bolwin — Thu, 28 May 2026 21:31:07 +0000

Here's another: https://xcancel.com/FireworksAI_HQ/status/206010388602804673...

Fireworks is processing 30T tokens a day on open models, or about 210T a week. So about 40% of gemini? I'd say that's pretty good.

Two more points: 1. https://openrouter.ai/provider/fireworks ~5B tokens average daily on openrouter from fireworks, which is a ration of ~1:6000 2. https://openrouter.ai/rankings total tokens on openrouter, ~4T daily, and more than half seems to be open. Say 2T.

If other providers' ratio is anywhere close to fireworks, that's on the order of 10 quadrillion open tokens daily.

That said I'd guess the ratio is probably not nearly as high for most providers.

New comment by Bolwin in "Unicode 18.0.0 Beta"

Bolwin — Thu, 28 May 2026 05:19:57 +0000

I've never seen emoji used for subtext. Usually they just repeat or emphasize what's in the text

New comment by Bolwin in "Xiaomi MiMo-v2.5 Series API Permanent Price Reduction Up to 99%"

Bolwin — Wed, 27 May 2026 03:19:01 +0000

OpenRouter is not indicative of volume. Most high volume clients will go to the providers directly. There's not point to paying the 5% OR cut if you know what you want.

New comment by Bolwin in "Xiaomi MiMo-v2.5 Series API Permanent Price Reduction Up to 99%"

Bolwin — Wed, 27 May 2026 03:14:39 +0000

I mean there is a minor moat. Most people don't enjoy switching providers or models. If you can get people to trust you'll stay near frontier, they'll stick around even when you aren't the best. Claude is a prime example of this

New comment by Bolwin in "Xiaomi MiMo-v2.5 Series API Permanent Price Reduction Up to 99%"

Bolwin — Wed, 27 May 2026 03:10:08 +0000

No one is producing one output token though.

And using up gpus for that cache is a pretty big opportunity cost. I highly doubt it's done in vram. That would be insane for the one hour caches.

So its memory + the time it takes to unload/load into vram + the extra cost per output token

Is it a scam? Idk

New comment by Bolwin in "YAML? That's Norway Problem"

Bolwin — Sat, 23 May 2026 01:22:27 +0000

Cause it's very verbose. A lot more syntax to break.

I personally think the best is one of the humanized json ones like https://maml.dev/

New comment by Bolwin in "Google Declaring War on the Web"

Bolwin — Wed, 20 May 2026 22:34:41 +0000

I don't see how being decentralized helps search. Makes it quite harder if the fediverse is any indication