Hacker News: ianand

Cognitive Mercantilism: AI panic resembles fears of early capitalism

ianand — Mon, 26 Oct 2026 04:11:40 +0000

Article URL: https://contraptions.venkateshrao.com/p/cognitive-mercantilism

Comments URL: https://news.ycombinator.com/item?id=50239027

Points: 67

# Comments: 68

New comment by ianand in "Transformers Explained Visually"

ianand — Tue, 22 Sep 2026 06:22:57 +0000

Thanks!

New comment by ianand in "Transformers Explained Visually"

ianand — Tue, 22 Sep 2026 05:12:27 +0000

Would be curious if https://spreadsheets-are-all-you-need.ai/gpt2/ works on your Chromebook. You need to download the weights and then drag and drop them back into the page but maximum compatibility was one of the design goals.

New comment by ianand in "Transformers Explained Visually"

ianand — Tue, 22 Sep 2026 05:03:39 +0000

Interestingly, I find absolute positional embeddings easier to explain to a layperson than rope, e.g. [1]. What am I missing?

[1] https://www.youtube.com/watch?v=ZuiJjkbX0Og&t=5712s

New comment by ianand in "CS336: Language Modeling from Scratch"

ianand — Mon, 01 Jun 2026 18:01:01 +0000

In that case I humbly suggest my talk from AI Engineer World's Fair https://www.youtube.com/watch?v=ZuiJjkbX0Og

Gives you the basics on LLM internals in about 90 minutes and includes an already built model in JavaScript that you can step through in browser devtools to get as detailed as you want.

New comment by ianand in "Website streamed live directly from a model"

ianand — Wed, 22 Apr 2026 22:09:45 +0000

It's like "GPT is all you need for the backend" [1] on steroids

[1] https://news.ycombinator.com/item?id=34503418

New comment by ianand in "The spectrum of isolation: From bare metal to WebAssembly"

ianand — Fri, 16 Jan 2026 10:01:40 +0000

Since you mention serverless it might be worth mentioning firecracker and v8 isolates.

New comment by ianand in "The Q, K, V Matrices"

ianand — Thu, 08 Jan 2026 06:32:34 +0000

I'm not a fan of the database lookup analogy either.

The analogy I prefer when teaching attention is celestial mechanics. Tokens are like planets in (latent) space. The attention mechanism is like a kind of "gravity" where each token is influencing each other, pushing and pulling each other around in (latent) space to refine their meaning. But instead of "distance" and "mass", this gravity is proportional to semantic inter-relatedness and instead of physical space this is occurring in a latent space.

https://www.youtube.com/watch?v=ZuiJjkbX0Og&t=3569s

New comment by ianand in "Intel: Winning and Losing"

ianand — Sat, 10 May 2025 15:31:04 +0000

The site’s domain name is the best use of a .fail tld ever.

New comment by ianand in "Show HN: GPT-2 implemented using graphics shaders"

ianand — Sat, 03 May 2025 08:20:22 +0000

Lol. You joke but... https://www.reddit.com/r/GraphicsProgramming/comments/pg9dw8...

New comment by ianand in "Show HN: GPT-2 implemented using graphics shaders"

ianand — Sat, 03 May 2025 08:09:52 +0000

Dude, been forever. Thanks. Will DM you.

New comment by ianand in "Show HN: GPT-2 implemented using graphics shaders"

ianand — Fri, 02 May 2025 17:34:03 +0000

As the guy who did GPT2 in Excel, very cool and kudos!!

Curious why you chose WebGL over WebGPU? Just to show it can be done?

(Also see my other comment about fetching weights from huggingface)

New comment by ianand in "Show HN: GPT-2 implemented using graphics shaders"

ianand — Fri, 02 May 2025 17:30:45 +0000

Checkout https://github.com/jseeio/gpt2-tfjs fetches the weights for GPT2 from huggingface on the fly.

New comment by ianand in "Local LLM inference – impressive but too hard to work with"

ianand — Tue, 22 Apr 2025 06:58:32 +0000

Thanks. Took a quick look and definitely needed to use Google Translate but seems to have worked to get the gist of it.

New comment by ianand in "Local LLM inference – impressive but too hard to work with"

ianand — Mon, 21 Apr 2025 21:40:41 +0000

Sounds like an interesting masters thesis. Is your masters thesis available online somewhere?

New comment by ianand in "Xee: A Modern XPath and XSLT Engine in Rust"

ianand — Fri, 28 Mar 2025 19:33:07 +0000

Fun fact: A decade ago the designer of HAML and Sass created a modern alternative to XSLT. https://en.wikipedia.org/wiki/Tritium_(programming_language)

New comment by ianand in "Tracing the thoughts of a large language model"

ianand — Thu, 27 Mar 2025 19:12:34 +0000

The model architecture is the same during RL but the training algorithm is substantially different.

New comment by ianand in "Tracing the thoughts of a large language model"

ianand — Thu, 27 Mar 2025 19:05:44 +0000

> LLMs that haven't gone through RL are useless to users. They are very unreliable, and will frequently go off the rails spewing garbage, going into repetition loops, etc...RL learning involves training the models on entire responses, not token-by-token loss (1).

Yes. For those who want a visual explanation, I have a video where I walk through this process including what some of the training examples look like: https://www.youtube.com/watch?v=DE6WpzsSvgU&t=320s

New comment by ianand in "SmolGPT: A minimal PyTorch implementation for training a small LLM from scratch"

ianand — Thu, 30 Jan 2025 02:58:10 +0000

hey, creator of spreadsheets-are-all-you-need.ai here. Thanks for mentioning!

I now have a web version of GPT2 implemented in pure JavaScript for web developers at https://spreadsheets-are-all-you-need.ai/gpt2/.

The best part is that you can debug and step through it in the browser dev tools: https://youtube.com/watch?v=cXKJJEzIGy4 (100 second demo). Every single step is is in plain vanilla client side JavaScript (even the matrix multiplications). You don't need python, etc. Heck, you don't even have to leave your browser.

I recently did an updated version of my talk with it for JavaScript developers here: https://youtube.com/watch?v=siGKUyTk9M0 (52 min). That should give you a basic grounding on what's happening inside a Transformer.

New comment by ianand in "RWKV Language Model"

ianand — Thu, 02 Jan 2025 14:34:00 +0000

Reminder that Microsoft ships RWKV with Windows (~1.5 billion devices), making it probably the most widely deployed non transformer model out there. Amazing work! https://blog.rwkv.com/p/rwkvcpp-shipping-to-half-a-billion

ps Eugene you should brag about that on the homepage of RWKV.