I wrote this to try and clear up what I think is a common misconception (speculative decoding is a free lunch), when the picture is more nuanced (and imo, interesting).

Comments URL: https://news.ycombinator.com/item?id=49483231

Points: 2

# Comments: 0

Happy to answer any questions!

Comments URL: https://news.ycombinator.com/item?id=48658847

Points: 2

# Comments: 0

I also think the attention approach (always attend to the image/prefix, with a sliding window for local context) is neat!

I do wish they updated their comparison table to include more recent work (that scores marginally better on OmniDocBench), like dots.mocr.

Gemma 12B, multitoken prediction, and official quants released. Feels like Google is putting real effort into this string of releases, and I'm very excited to see that!

Comments URL: https://news.ycombinator.com/item?id=48179263

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=48149368

Points: 2

# Comments: 0

Comparing the architecture to Qwen3.5, it seems:

- fewer, wider layers

- mixing full attention and conv's, instead of the full+linear attention of Qwen3.5

- the vocab is about 1/4 the size

Comments URL: https://news.ycombinator.com/item?id=47596047

Points: 3

# Comments: 1

I worked through what fp6 (e3m2) would look like, doing manual additions and multiplications, showing cases where the operations are non-associative, etc. and then I wanted something more rigorous to read.

For anyone interested in floating point numbers, I highly recommend working through fp6 as an activity! Felt like I truly came away with a much deeper understanding of floats. Anything less than fp6 felt too simple/constrained, and anything more than fp6 felt like too much to write out by hand. For fp6 you can enumerate all 64 possible values on a small sheet of paper.

For anyone not (yet) interested in floating point numbers, I’d still recommend giving it a shot.

Comments URL: https://news.ycombinator.com/item?id=47343902

Points: 125

# Comments: 56

At the end of the day, I'm here because of all the thoughtful commenters and people sharing interesting stories.

I’ve been a monthly donor since ~the beginning when I was just an undergraduate, and I still read the stories and emails I receive. I’m glad that you opted for the steady growth path, and that you’ve made it a sustainable thing.

*this is incorrect per the author’s response, my apologies.

For instance, it goes into (nano)vLLM internals and doesn’t mention PagedAttention once (one of the core ideas that vLLM is based on)[1].

Also mentions that Part 2 will cover dense vs MoE’s, which is weird because nanovllm hardcodes a dense Qwen3 into the source.

Here are better (imo) explainers about how vLLM works:

- https://hamzaelshafie.bearblog.dev/paged-attention-from-firs...

- https://www.aleksagordic.com/blog/vllm

- https://huggingface.co/blog/continuous_batching

Aleksa’s blog is a bit in the weeds for my taste but it’s really worth working through.

A lot of the magic of vLLM happens in the PagedAttention kernels, which are really succinctly implanted in nanovllm. And the codebase is great and readable by itself!

—

1. https://arxiv.org/abs/2309.06180

For the non-ONNX models there are some inference tricks that generally improve performance, and potentially lowering confidence could help.

It's really gratifying to see people building on the work, and I love that it's possible to do browser-side/on-device.

That’s built on a dataset and paper I wrote called CommonForms, where I scraped CommonCrawl for hundreds of thousands of fillable form pages and used that as a training set:

https://arxiv.org/abs/2509.16506

Next step is training and releasing some DETRs, which I think will drive quality even higher. But the ultimate end goal is working on automatic form accessibility.

Releasing the dataset, paper, models, and (imo most importantly) simple/convenient tooling to automatically prepare any PDF.

Links: Repo: https://github.com/jbarrow/commonforms - Paper: https://arxiv.org/abs/2509.16506

Comments URL: https://news.ycombinator.com/item?id=45450135

Points: 1

# Comments: 1