Hacker News: rahen

New comment by rahen in "Want to Write a Compiler? Just Read These Two Papers (2008)"

rahen — Wed, 15 Apr 2026 18:32:36 +0000

I'm also writing a compiler and CS6120 from Cornell has helped me a lot: https://www.cs.cornell.edu/courses/cs6120/2025fa/self-guided...

New comment by rahen in "Show HN: I built a tiny LLM to demystify how language models work"

rahen — Mon, 06 Apr 2026 13:36:53 +0000

I don't mean to be 'that guy', but after a quick review, this really feels like low-effort AI slop to me.

There is nothing wrong using AI tools to write code, but nothing here seems to have taken more than a generic 'write me a small LLM in PyTorch' prompt, or any specific human understanding.

The bar for what constitutes an engineering feat on HN seems to have shifted significantly.

New comment by rahen in "Circuit-level PDP-11/34 emulator"

rahen — Sun, 29 Mar 2026 00:55:07 +0000

Out of curiosity, what were those wonderful things you were hearing about the 11/34 back then?

New comment by rahen in "Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer"

rahen — Sat, 28 Mar 2026 21:35:57 +0000

Yes. The Cray supercomputers from the 80s were crazy good matmul machines in particular. The quad-CPU Cray X-MP (1984) could sustain 800 MFLOPS to 1 GFLOPS, and with a 1 GB SSD, had enough computer power and bandwidth to train a 7-10M-parameter language model in about six months, and infer at 18-25 tok/sec.

A mid-90s Cray T3E could have handled GPT-2 124M, 24 years before OpenAI.

I also had a punch-card computer from 1965 learn XOR with backpropagation.

The hardware was never the bottleneck, the ideas were.

New comment by rahen in "Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer"

rahen — Sat, 28 Mar 2026 18:26:19 +0000

I also have a working design for a small Transformer on the original Game Boy. It has around 4000 parameters fitting in the 8 KB cartridge SRAM, where the "saved game" is the trained model. A TI-82 with its 32 KB of RAM would be even more comfortable.

New comment by rahen in "Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer"

rahen — Sat, 28 Mar 2026 18:22:19 +0000

I encouter two main failure modes. First, the bipolar PROMs degrade at the atomic level, the metal ions in the fuses tend to migrate or 'regrow' over decades, causing bit rot. Second, the backplanes suffer from mechanical fatigue. After forty years of thermal expansion and structural flexing, especially when inserting boards, the traces and solder joints develop stress cracks. Both are a pain to repair.

https://retrocmp.com/articles/trying-to-fix-a-dec-pdp-1134-b...

New comment by rahen in "Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer"

rahen — Sat, 28 Mar 2026 18:16:19 +0000

The WASM GUI is probably the easiest way to see the Transformer in action on this machine: https://dbrll.github.io/ll-34/

There's also the original Tetris from 1984 to play.

New comment by rahen in "Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer"

rahen — Sat, 28 Mar 2026 13:57:40 +0000

That thing is a Tamagochi though, it constantly needs attention, pardon the pun. I did most of the development and tuning on ll-34 for that reason.

New comment by rahen in "Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer"

rahen — Sat, 28 Mar 2026 13:49:25 +0000

Thanks for reposting! I'm the author of ATTN-11. Happy to answer any questions about the fixed-point arithmetic, the PDP-11 hardware, or the training process.

Paper Tape Is All You Need – Training a Transformer on a 1976 Minicomputer

rahen — Wed, 25 Mar 2026 15:20:08 +0000

Article URL: https://github.com/dbrll/ATTN-11

Comments URL: https://news.ycombinator.com/item?id=47518568

Points: 145

# Comments: 26

New comment by rahen in "The Connection Machine CM-1 "Feynman" T-shirt"

rahen — Tue, 03 Feb 2026 08:54:33 +0000

Around the same time (1984), there was also another very cool piece of technology that often gets overlooked: the CMU WARP. It wasn’t as flashy as the Crays and the Connection Machine, but it was the first systolic array accelerator (what we’d now call TPUs). It packed as much MFLOPS as a Cray 1.

It's also the computer that powered the Chevrolet Navlab self-driving car in 1986.

New comment by rahen in "Sheaf – A Functional Language for Differentiable Programs"

rahen — Wed, 28 Jan 2026 14:59:58 +0000

I've been building a functional language for differentiable programming that compiles to JAX. The core idea is homoiconicity applied to ML, models are data structures that can inspect and transform themselves.

Sheaf – A Functional Language for Differentiable Programs

rahen — Wed, 28 Jan 2026 14:59:58 +0000

Article URL: https://sheaf-lang.org/

Comments URL: https://news.ycombinator.com/item?id=46796189

Points: 3

# Comments: 1

New comment by rahen in "The Emacs Widget Library: A Critique and Case Study"

rahen — Wed, 14 Jan 2026 09:59:59 +0000

For those interested, this guy is revamping the Emacs widget library with something more modern and platform agnostic, based on SDL: https://appetrosyan.github.io/posts/

His posts are very insightful.

New comment by rahen in "I switched from VSCode to Zed"

rahen — Tue, 06 Jan 2026 12:26:08 +0000

My only complaint regarding the Zed editor is the inability to display two panes of the sidebar one below the other. Not only is it impossible to display them together, but switching between them requires clicking a tiny button in the status bar. To make matters worse, performing a search hides the symbols and the tree view.

So right now I'm sticking to Emacs.

New comment by rahen in "Go away Python"

rahen — Tue, 30 Dec 2025 19:51:53 +0000

Lisp isn't missing anything, it's a natural fit for AI/ML. It’s the ecosystem's tooling that needs catching up.

The code hasn't reached RC yet, but I'll definitely post a Show HN once it's ready for a preview.

New comment by rahen in "Go away Python"

rahen — Tue, 30 Dec 2025 17:51:10 +0000

I doubt it considering there are massive Clojure codebases with large teams collaborating on them every day. The lack of Lisp tooling and the prevalence of Python are more a result of inertia, low barrier to entry and ecosystem lock-in.

New comment by rahen in "Go away Python"

rahen — Tue, 30 Dec 2025 17:35:44 +0000

"> I think all of ML being in Python is a colossal mistake that we'll pay for for years.

Market pressure. Early ML frameworks were in Lisp, then eventually Lua with Torch, but demand dictated the choice of Python because "it's simple" even if the result is cobbled together.

Lisp is arguably still the most suitable language for neural networks for a lot of reasons beyond the scope of this post, but the tooling is missing. I’m developing such a framework right now, though I have no illusions that many will adopt it. Python may not be elegant or efficient, but it's simple, and that's what people want.

New comment by rahen in "Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB"

rahen — Mon, 29 Dec 2025 10:09:59 +0000

I love it, instant Github star. I wrote an MLP in Fortran IV for a punched card machine from the sixties (https://github.com/dbrll/Xortran), so this really speaks to me.

The interaction is surprisingly good despite the lack of attention mechanism and the limitation of the "context" to trigrams from the last sentence.

This could have worked on 60s-era hardware and would have completely changed the world (and science fiction) back then. Great job.

New comment by rahen in "Learning Fortran (2024)"

rahen — Wed, 17 Dec 2025 23:44:45 +0000

The reverse is true though, and I find that fascinating with Fortran.

I recently learned Fortran IV to build a backpropagated neural network for the IBM 1130 (1965) and was amazed to see it compile with no warning on both the punched card compiler from IBM and on a modern Fortran compiler (gfortran).

Some Fortran II conventions, like the arithmetic IF, are now deprecated, but --std=legacy is all it takes to make it work.