Hacker News: ak_t

New comment by ak_t in "Small Models Have Arrived"

ak_t — Thu, 27 Aug 2026 17:41:06 +0000

Many benchmarks now measure the total cost or energy usage per completed task.

New comment by ak_t in "A simple fix for LLM tail latency"

ak_t — Mon, 17 Aug 2026 23:23:22 +0000

You don't have to send every single request twice, just the ones that are haven't returned in time. Wait until some threshold, such as your p95 latency, and send your backup request after that. Return whichever request comes back first, and it should cut your tail latency without doubling your cost, since it only duplicates the small % of requests at the tail.

Google calls this a 'hedged request': https://cacm.acm.org/research/the-tail-at-scale/

New comment by ak_t in "Qwen3.8-2.4T"

ak_t — Thu, 13 Aug 2026 18:03:01 +0000

It's been pretty consistent, the smaller models (hundreds of billions of params) usually catch up in 6 months or so to their frontier counterparts, at least on benchmarks.

New comment by ak_t in "Qwen3.8-2.4T"

ak_t — Thu, 13 Aug 2026 06:16:36 +0000

I think it is more likely that a smaller model (<400B) with similar intelligence gets developed long before the hardware to serve a 2.4T model gets cheaper than 10k.

New comment by ak_t in "DeepSeek V4 Flash 0731"

ak_t — Fri, 07 Aug 2026 21:25:53 +0000

It doesn't need extra quantization. The official weights are natively mixed precision FP4/FP8, so it fits in ~160GB. The API slowness is probably from being batched with other concurrent user requests. The provider's aggregate throughput gets higher but per-stream speed slows down.

New comment by ak_t in "DeepSeek V4 Flash 0731"

ak_t — Fri, 07 Aug 2026 20:46:53 +0000

Note this is the 07/31 release of DSv4 flash and not the "preview" that they put out a couple months or so ago.

I've been running this model locally for a week, and the preview version before that. This updated one feels like a whole tier up. It's very capable for debugging and analyzing documents/data I upload.

The killer feature, IMO, is the speed. On 2x RTX Pro 6000 Blackwell, its ~8k tok/s prefill and ~250 tok/s on a single stream. I saw 1000 tok/s with ~64 concurrent streams on vLLM.

That's fast enough that you can interactively chat with it without switching tabs while you wait, and its a ~300B (13B active, hence the speed) model so the responses are also very good. It's actually more convenient now for me to direct 95%+ of my day to day usage to my local model, and only use Claude Fable for really big coding tasks.

Until this model was released, I was contemplating spending even more money on hardware to run GLM5.2 (~750B) at reasonable speeds, but I no longer feel that need. This is smart enough, and I think it only gets much better for local models from here.

New comment by ak_t in "The AI Billboards Are Killing SF"

ak_t — Fri, 07 Aug 2026 16:38:58 +0000

I haven't live there since 2020, but before that it was all various SaaS and data pipeline related billboards.

New comment by ak_t in "DeepSeek V4 Flash on a Single AMD MI300X"

ak_t — Tue, 04 Aug 2026 20:13:15 +0000

It's expensive to run it locally at full quality, but at least on my setup, its about 5 times faster than any API, and is completely private.

When is NVLink worth it?

ak_t — Wed, 22 Jul 2026 14:18:11 +0000

Article URL: https://platform-fools.com/posts/2026-04-27-nvlink/

Comments URL: https://news.ycombinator.com/item?id=49007249

Points: 61

# Comments: 9