Hacker News: zackangelo

New comment by zackangelo in "Qwen 3.8 27B available on Cerebras at 1500 tokens/s"

zackangelo — Thu, 03 Sep 2026 20:33:27 +0000

Something a lot of model providers don't talk about: any time an engine uses speculative decoding the throughput will depend on how much your output token distribution matches what the draft model was trained on.

The DFlash2 draft model we're using was trained on a lot of code, so if you use it in a coding agent you'll probably notice it run a lot faster (we've seen it break 300 tok/s).

New comment by zackangelo in "Qwen 3.8 27B available on Cerebras at 1500 tokens/s"

zackangelo — Thu, 03 Sep 2026 20:15:27 +0000

just added 8 more H200s to the cluster, if you (or anyone else) runs into issues please feel free to drop me a message: zack at mixlayer.com

New comment by zackangelo in "Qwen 3.8 27B available on Cerebras at 1500 tokens/s"

zackangelo — Thu, 03 Sep 2026 19:34:10 +0000

apologies we just got a sudden burst of new users and traffic, it's scaling up now.

New comment by zackangelo in "Qwen 3.8 27B available on Cerebras at 1500 tokens/s"

zackangelo — Thu, 03 Sep 2026 19:05:10 +0000

We're serving it around 150-200tok/s (uses our new speculative decoding implementation on a DFlash2 draft model).

https://mixlayer.com, LAUNCH-Q38-27B gets you $5 in credits if you want to kick the tires.

New comment by zackangelo in "Rivian OS 2"

zackangelo — Thu, 20 Aug 2026 16:35:27 +0000

What was wrong with your R1S?

I had an R1T Launch Edition for a little over a year and it was hands down one of the best cars I’ve ever owned. The only issue I had was with the fob proximity sensor. The only reason I sold it was it was a bit too big for me.

New comment by zackangelo in "DFlash 2: Keep Drafting Parallel"

zackangelo — Thu, 20 Aug 2026 00:31:26 +0000

DFlash is lossless so this would be a bug in the implementation if it is indeed a regression against the target model.

New comment by zackangelo in "Performance per dollar is getting faster and cheaper"

zackangelo — Sat, 04 Jul 2026 02:25:05 +0000

Blackwell supports nvfp4 natively.

New comment by zackangelo in "Two Qwen3 models on one DGX Spark: the residency math"

zackangelo — Sun, 21 Jun 2026 17:26:18 +0000

what was the concurrency limitation? that node should be able to support a lot more

New comment by zackangelo in "Kimi K2.7-Code: open-source coding model with better token efficiency"

zackangelo — Fri, 12 Jun 2026 17:51:47 +0000

I don't believe safetensors has a native int4 dtype, so they packed 4 int4s into a bf16 in this checkpoint.

New comment by zackangelo in "The real cost of owning a home"

zackangelo — Tue, 26 May 2026 16:58:24 +0000

If you're in SF and weighing this decision, it's easy to get tilted in the buy direction because the rental stock is so horrific. Landlords have very little incentive to update properties or provide basic amenities that people take for granted in other major cities (good luck getting a washer/dryer).

New comment by zackangelo in "Qwen3.7-Max: The Agent Frontier"

zackangelo — Wed, 20 May 2026 15:09:32 +0000

With the 3.5 release, the Plus model was just a rebrand of the open weight 397B. But I suspect that will change going forward. They haven’t released the weights for 3.6 but they did make it available through a few US providers.

New comment by zackangelo in "I’ve joined Anthropic"

zackangelo — Tue, 19 May 2026 17:29:14 +0000

absolutely not, take Kimi K2.6 for a spin

How do agents see your website?

zackangelo — Wed, 13 May 2026 16:11:19 +0000

Article URL: https://what-do-agents-see.runtype.app/

Comments URL: https://news.ycombinator.com/item?id=48123838

Points: 4

# Comments: 0

New comment by zackangelo in "Mistral Medium 3.5"

zackangelo — Wed, 29 Apr 2026 17:56:11 +0000

Isn't Kimi K2.6 natively INT4?

New comment by zackangelo in "HashiCorp co-founder says GitHub 'no longer a place for serious work'"

zackangelo — Wed, 29 Apr 2026 12:54:39 +0000

I don’t think this is true across Blizzard. Overwatch is the best it’s ever been.

New comment by zackangelo in "Parallel agents in Zed"

zackangelo — Wed, 22 Apr 2026 23:35:02 +0000

I give them a try about twice a year. I write a lot of Rust which should be squarely in their wheelhouse.

This last time I was pleasantly surprised to find they mostly fixed their SSH remote editing support. But then it started truncating rustc inline error messages and I couldn’t figure out how to view the whole thing easily. When you’re just trying to get something done little bits like this can add up quickly. Punted back to Cursor for now.

New comment by zackangelo in "Qwen3.6-35B-A3B: Agentic coding power, now open to all"

zackangelo — Thu, 16 Apr 2026 15:00:42 +0000

They are but the IDE needs to be integrated with them.

Qwen specifically calls out FIM (“fill in the middle”) support on the model card and you can see it getting confused and posting the control tokens in the example here.

New comment by zackangelo in "Qwen3.6-35B-A3B: Agentic coding power, now open to all"

zackangelo — Thu, 16 Apr 2026 14:57:47 +0000

17b per token. So when you’re generating a single stream of text (“decoding”) 17b parameters are active.

If you’re decoding multiple streams, it will be 17b per stream (some tokens will use the same expert, so there is some overlap).

When the model is ingesting the prompt (“prefilling”) it’s looking at many tokens at once, so the number of active parameters will be larger.

New comment by zackangelo in "GPU memory snapshots: sub-second startup (2025)"

zackangelo — Sat, 10 Jan 2026 23:56:27 +0000

This uses Nvidia’s CUDA snapshot API under the hood, but you have to pair it with a host side snapshot as well. Modal uses gVisor for this, which is notoriously high overhead.

Does anyone know of a more efficient alternative if you’re running a trusted container?

New comment by zackangelo in "macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt"

zackangelo — Fri, 12 Dec 2025 23:26:39 +0000

You’re right I misunderstood.

I’m not sure if it would be of much utility because this would presumably be for tensor parallel workloads. In that case you want the ranks in your cluster to be uniform or else everything will be forced to run at the speed of the slowest rank.

You could run pipeline parallel but not sure it’d be that much better than what we already have.