Hacker News: sheepseb

New comment by sheepseb in "A tiny world model running games on a Raspberry Pi Pico 2 W"

sheepseb — Fri, 30 Oct 2026 09:45:32 +0000

No, not shadows. I use the heading as a token, in Doom the minimap has a player token and the facing (the current 3D view), I tried using only the frame but it was harder to learn only from the wall patterns. So the agent isn't magically grokking position, its using the map. As for the Pac Man, it's based on mouth and has tokens for "keep going" or N/E/S/W direction. For breakout it uses 4 ball tokens, one per diagonal and it moves one cell per step in that diagonal, and for pong the ball has 6 tokens (combiantion between [up, level, down] with [left, right], e.g up right, level down)

New comment by sheepseb in "A tiny world model running games on a Raspberry Pi Pico 2 W"

sheepseb — Fri, 30 Oct 2026 07:37:59 +0000

Yes, there were some hallucinations, e.g: in breakout hitting a corner would destroy more blocks than expected, in doom-lite it could shoot from miles away and get a kill. The fixed was add some noise to the input frame and keep the target clean.

New comment by sheepseb in "A tiny world model running games on a Raspberry Pi Pico 2 W"

sheepseb — Fri, 30 Oct 2026 07:27:17 +0000

The core of it uses rust with no_std and on the pico we use embassy-rs with no os. At the same time, the core can be build for webassembly but it doesn't use webgpu.

New comment by sheepseb in "A tiny world model running games on a Raspberry Pi Pico 2 W"

sheepseb — Thu, 29 Oct 2026 22:58:11 +0000

For the world model training I've used an RTX 4060, but for the agent playing the game that was directly on the pico using stream-ac

New comment by sheepseb in "A tiny world model running games on a Raspberry Pi Pico 2 W"

sheepseb — Thu, 29 Oct 2026 21:51:03 +0000

It's a self contained net for each game but the data used for training doesn't overlap with the validation. I think it would be easy to merge them into a single model