Comments URL: https://news.ycombinator.com/item?id=49738954

Points: 2

# Comments: 0

One drive gives about 2 tok/s; striped across four drives it reaches 3.5 tok/s with byte-identical output, and our best internal build with a not-yet-published patch does 4.2.

Method and numbers: https://github.com/argonautlabsai/argodrive (built on antirez/ds4).

                        prompt processing      steady decode
    upstream, internal       16.23                 10.59
    ours, internal only      28.04  1.73x          14.38  1.36x
    ours, +1 external        36.88  2.27x          16.05  1.52x
    ours, +2 external        43.62  2.69x          17.38  1.64x

  The first fork row is the one that matters: same single internal SSD, no replicas, no enclosures.

Comments URL: https://news.ycombinator.com/item?id=49722242

Points: 13

# Comments: 2

Comments URL: https://news.ycombinator.com/item?id=49711198

Points: 1

# Comments: 1

I wanted to see whether Kimi runs at all on one machine with the full record published, and for long multi-table finance reasoning I wanted the strongest model I could keep on the machine.

I did some tests against Deep Seek v4 flash results on my reports and Kimi definitely has some advantages.

Each enclosure tops out at about 7.1 GB/s on whole-file reads regardless of the drive inside (a 2 TB SN8100 measures the same as the 1 TB);

the drive behind the hub reads 5.7 GB/s and falls with queue depth.

Details in the README's hardware section

Two things carry over: the experts are read from disk per token either way, and the barrier model — a layer waits for the slowest of its 16 reads — is platform-independent.

Two things don't: the 50 GB resident trunk lives in unified memory here, so on a discrete GPU it would need to fit in VRAM or be streamed too; and a desktop's PCIe lanes let you put NVMe drives on the bus directly rather than behind a ~7 GB/s Thunderbolt enclosure, which is our per-drive wall.

Whether that ends up faster is exactly the kind of thing that wants measuring rather than guessing.

(0.535 vs 1.038 tok/s at 128 tokens), since one fast drive still has to serve all 16 reads per layer while four drives split the load

https://raw.githubusercontent.com/argonautlabsai/deltafin/ma...

The KV cache on this engine grows about 2.8 MiB per token of context, and the machine's 128 GB is already holding the 50.7 GiB resident trunk plus reserves the speculative-verify path needs

(we found the hard way that squeezing those makes the verifier reject wide batches and decode falls to single-token steps).

With the current reserves the engine admits ~4.4k tokens; that's a configuration ceiling you can raise by giving the cache more of the 128 GB and accepting less headroom elsewhere.

K3 itself supports far longer contexts — but see the prefill caveat above: on this setup long prompts cost minutes per 512 tokens until the scheduling fix lands.

Put a five-line TL;DR at the top of the README — what it is, the number, the honest limit, the two findings, credits — with the detail below for anyone who wants it.

Decode was flat from 128 to 512 generated tokens (0.926 → 0.923 tok/s drafter-off), but that's a 6-token prompt plus the output — total context under a thousand.

The current configuration admits about 4.4k tokens of context at all, and at anything like 200k the killer wouldn't be decode, it would be prefill: today it reads each layer's experts once per 64-row pass,

so 200k tokens of prompt would be measured in days, not minutes, until the scheduling fix.

because prefill currently reads each layer's experts once per 64-row pass (~9 TB of reads for a 1.4 TB model).

Fix is scheduling and it's the next thing being built; once prefill reads each expert once per layer, the one-token-out classifier pattern becomes the sweet spot rather than the worst case.

What actually moves the barrier is how fast the slowest of 16 concurrent whole-file reads completes: more drives on direct ports, stable tail behaviour under load, and scheduling. Our ladder shows even that with diminishing returns (one drive ≈52% of four, three ≈90%).

Going from one drive to four (13.6 → ~33 GB/s of combined ceilings) took decode from ~52% to 100% of our number — not 4× — with

Every drive already at 90–100% of its own ceiling. RAID-0 was one of the first things tried and it lost: striping makes every read touch every drive, so the slowest drive sets every barrier.

What moves this is per-read latency and read scheduling, and for long prompts not re-reading each layer's experts eight times.

Numbers in results/SCALING.md and results/PREFILL.md.

https://github.com/argonautlabsai/deltafin/tree/main/k3-publ...

  Kimi K3, 2.78T parameters, ~1.45 TB of MXFP4 experts streamed from four SSDs on an M5 Max / 128 GB. 1.00 tok/s steady over 512 tokens, 1.13 over 128, ~6.3 min to first token on a 512-token prompt. Output token-identical drafter on/off on a given drive layout; the int8 trunk is non-weight-exact per upstream.
  
  The useful bits: one drive gives ≈52% of four, two ≈73%, three ≈90%; and prefill is slow because of ~9 TB of reads for a 1.4 TB model — a scheduling bug with a planned fix.
  
  README with per-run logs: github.com/argonautlabsai/deltafin — a fork of gavamedia/deltafin, who built the engine.