Comparing the architecture to Qwen3.5, it seems:

- fewer, wider layers

- mixing full attention and conv's, instead of the full+linear attention of Qwen3.5

- the vocab is about 1/4 the size

Comments URL: https://news.ycombinator.com/item?id=47596047

Points: 3

# Comments: 1

I worked through what fp6 (e3m2) would look like, doing manual additions and multiplications, showing cases where the operations are non-associative, etc. and then I wanted something more rigorous to read.

For anyone interested in floating point numbers, I highly recommend working through fp6 as an activity! Felt like I truly came away with a much deeper understanding of floats. Anything less than fp6 felt too simple/constrained, and anything more than fp6 felt like too much to write out by hand. For fp6 you can enumerate all 64 possible values on a small sheet of paper.

For anyone not (yet) interested in floating point numbers, I’d still recommend giving it a shot.

Comments URL: https://news.ycombinator.com/item?id=47343902

Points: 125

# Comments: 56

At the end of the day, I'm here because of all the thoughtful commenters and people sharing interesting stories.

I’ve been a monthly donor since ~the beginning when I was just an undergraduate, and I still read the stories and emails I receive. I’m glad that you opted for the steady growth path, and that you’ve made it a sustainable thing.

*this is incorrect per the author’s response, my apologies.

For instance, it goes into (nano)vLLM internals and doesn’t mention PagedAttention once (one of the core ideas that vLLM is based on)[1].

Also mentions that Part 2 will cover dense vs MoE’s, which is weird because nanovllm hardcodes a dense Qwen3 into the source.

Here are better (imo) explainers about how vLLM works:

- https://hamzaelshafie.bearblog.dev/paged-attention-from-firs...

- https://www.aleksagordic.com/blog/vllm

- https://huggingface.co/blog/continuous_batching

Aleksa’s blog is a bit in the weeds for my taste but it’s really worth working through.

A lot of the magic of vLLM happens in the PagedAttention kernels, which are really succinctly implanted in nanovllm. And the codebase is great and readable by itself!

—

1. https://arxiv.org/abs/2309.06180

For the non-ONNX models there are some inference tricks that generally improve performance, and potentially lowering confidence could help.

It's really gratifying to see people building on the work, and I love that it's possible to do browser-side/on-device.

That’s built on a dataset and paper I wrote called CommonForms, where I scraped CommonCrawl for hundreds of thousands of fillable form pages and used that as a training set:

https://arxiv.org/abs/2509.16506

Next step is training and releasing some DETRs, which I think will drive quality even higher. But the ultimate end goal is working on automatic form accessibility.

Releasing the dataset, paper, models, and (imo most importantly) simple/convenient tooling to automatically prepare any PDF.

Links: Repo: https://github.com/jbarrow/commonforms - Paper: https://arxiv.org/abs/2509.16506

Comments URL: https://news.ycombinator.com/item?id=45450135

Points: 1

# Comments: 1

Interesting to see a big provider entering this space. Originally swapped to Modal because big providers weren’t offering this (e.g. AWS lambdas can’t run on GPU instances). Assuming all providers are going to start moving towards offering this?

1. https://dynamicland.org/

If the PDF hasn't been made accessible, you have to do a lot of inferencing based on the layout about how things are grouped and how they should flow if you want to be able to make meaningful edits. Not impossible (Acrobat does it), but very challenging.

It's part of the legacy of PDF as a format for presentation and print jobs, rather than typesetting.

Fwiw, I do see the issue with being unable to scroll down across both Safari and Chrome.

In addition, did you try the "enhanced" pipeline? It gives each field a meaningful name based on the label, which might help with accessibility.

PDF accessibility is a huge issue that _should_ be easily solved, but isn't, unfortunately.

In my experience it's the best tool for the job across nearly every form. It ended up being super useful and pretty close to free to run (thanks Modal!), so I've hosted it as a free utility at https://detect.penpusher.app.

There are some advanced settings as well, so if you're not getting good results with a specific pipeline you can try tweaking those, or let me know and I'd be happy to take a look. Plus the next round of model training is focused on improving handling of scans.

Hope this is useful and happy to answer any questions!

Comments URL: https://news.ycombinator.com/item?id=43269590

Points: 3

# Comments: 2

Qwen2.5 VL was trained on a special HTML format for doing OCR with bounding boxes. [1] The resulting boxes aren't quite as accurate as something like Textract/Surya, but I've found they're much more accurate than Gemini or any other LLM.

[1] https://qwenlm.github.io/blog/qwen2.5-vl/