Hacker News: nee1r

New comment by nee1r in "A General Goal-Conditioned Minecraft Model"

nee1r — Wed, 15 Jul 2026 18:04:33 +0000

how do you pick good goal conditioning images/do you have to hand pick a dataset of good goal images? seems hard if you don't have full context. really cool though!

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 06:45:04 +0000

thanks! a lot of credit to the people who helped write/edit

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 06:44:35 +0000

giving back to the research community! releasing and talking about research helps everyone

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 06:43:23 +0000

thanks! i definitely love diffusion + pushed for it, as a non-causal generative method i think its pretty unique

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 06:42:35 +0000

thanks! got a lot of inspiration from VPT https://arxiv.org/abs/2206.11795 is a great paper, would recommend a read

we all have various backgrounds, me particularly i did a lot of material science x ai research and just fundamental architecture research before

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 01:51:44 +0000

we have an alignment blog post dropping soon! scaling up in the next couple of months, then hopefully opening up an API or licensing it.

Benchmarks are really fun—lots of secret ones. Our main thesis is that you should be using the same benchmarks to measure human ability to use a computer, as you would an AI model. Definitely a suite of continuous long term planning tasks (games) and things such as marking emails as spam etc.

definitely! we are looking into more interp + visualizations in general as we scale up.

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 01:19:46 +0000

planning on instruct tuning soon!

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 01:19:31 +0000

safety was important for the demo, the model didn't have access to the brake or accelerator.

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 01:18:24 +0000

thanks! the math and architecture of the FDM (no video encoder) is pretty simple, its a regular transformer with next-token predictions but with frames interleaved.

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 01:17:15 +0000

yeah! i love the BCO paper, i think its extremely intuitive and these methods are really interesting in a time where data without labels is abundant. i especially like the idea of iteratively making the inverse dynamics better—might lean closer to that in the future

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 01:14:54 +0000

cool thanks for the title idea!! hopefully when we scale up in the next month/two we can update the community

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Thu, 26 Feb 2026 01:13:44 +0000

collected! no synthetic

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Tue, 24 Feb 2026 17:48:29 +0000

thanks! the inverse dynamics model is trained first on 40k hours of data and then frozen to label all 11 million hours. yup! the idea is that it should take a small amount of data to generalize environment dynamics, then you can use a lot of data to understand actions.

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Tue, 24 Feb 2026 17:46:52 +0000

real

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Mon, 23 Feb 2026 18:45:35 +0000

this is honestly an issue for the inverse dynamics (for app specific shortcuts etc.) but for general UI learning we still see promising eval trends

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Mon, 23 Feb 2026 18:31:14 +0000

no finetuning data for the blender task! we actually think its the opposite, there are a lot of video tutorials for complex tasks like onshape/blender/fusion360 but not as much of people idly browsing.

but also at the 11M hour scales it still sees a substantial amount of data

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Mon, 23 Feb 2026 17:27:11 +0000

i actually drove the car (with arrow keys) around south park for around ~45 minutes as finetuning data, no extra labelling other than that. think the car line graph is super cool because you actually see the videegame prior working

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Mon, 23 Feb 2026 17:15:55 +0000

the main chain of experiments was trying causal => non-causal => non-causal with ctc and CE. i think a good intuition here is that you need a generative approach fundamentally because there definitely are multiple correct IDM labels.

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Mon, 23 Feb 2026 17:14:15 +0000

good question! we use exponential binning (map the mouse movements onto a plane with exponentially increasing tick marks https://si.inc/fdm1/exponential_binning.webp) but tried a bunch of other methods (linear creates too many tokens for the model to learn well). Polar coordinates seem like a better solution but empirically didn't work well because the tokens got too coarse too fast.

New comment by nee1r in "The First Fully General Computer Action Model"

nee1r — Mon, 23 Feb 2026 17:10:27 +0000

Hey guys! I’m Neel, been holed up in our south park office for the past year working on model training. excited to share our research!

This is a preview of a very different type of computer use model—we train on the internet. Specifically we have 11 million hours of computer video stored on our storage cluster (previously shared https://news.ycombinator.com/item?id=45438496 !) and the model can work in 30 FPS. Since we match the fundamental form factor of computer-use, we can get our model to do CAD, browse websites, and even drive a car using arrow keys. I’m super excited to see what our model can do as we scale more, it's a fun frontier to work on (not language models :) ).

The team and I will be online responding to the comments, so drop any questions.