I personally want to test spatial, temporal, and causal coherence in an interactive 3D world. Does the model understand where things are, world stays consistent over time and do the consequences make sense? There is a million people generating random games here and there on yt, but I want something that I can reproduce every time a model comes out and gets scored.

For this first run, 9 models received the same three roughly 30-line game prompt. I just added Kimi k3 to the results.

They all ran in high-thinking mode through the same open source harness, with the same sub-agent setup and access to Three.js, Rapier, and Playwright. There is currently one run per model per brief, producing 27 browser-playable games.

Because the qualities I’m interested in are difficult to score automatically, the main evaluation happens through blind pairwise comparisons. You play two games built from the same brief without seeing the model names, then compare their game feel, world design, presentation, completeness, and overall quality.

I’m also publishing the prompts, generated artifacts, generation time, estimated cost, and code size.

Fable produced some of the strongest games from what I could see, but its three runs cost about $756. GPT-5.6 Sol cost about $108, while GLM-5.2 and Grok 4.5 each cost around $19. Opus also felt closer to Fable than I expected, considering the large cost difference. Kimi somehow ended up roughly at the cost of GPT, but performed somewhat similar to Opus (thats just my subjective opinion there)

This first run is small, and as stated above, human preference is subjective. But I plan on running more and hope to evolve the methodology. As long as I can afford all these tokens. Fable is ridiculously expensive.

If you look at it, I'd really appreciate criticism of the task methodology, design, blind evaluation, etc. What would make this rigorous enough to be truly useful.

Comments URL: https://news.ycombinator.com/item?id=48994848

Points: 2

# Comments: 0

Instead of cranking boilerplate, they spend Day 1 reviewing AI diffs. We pair them with a senior for a “why did the agent choose this?” teardown.

What matters is mean-time-to-rollback. If the agent + test harnesses catch breakage faster than a human pair can, 2 × better is already good economics. Reliability engineering beats perfection thresholds.

Syntax, style and most unit-level bugs are now linted or auto-fixed. Humans zoom out to architecture, data contracts, threat models, perf budgets, and “does this change make sense for the product?”. So even juniors are now a lot more involved on the sujective elements of development

So I think that as the easy bugs vanish, new failure modes show up: latency cliffs, subtle privacy leaks, energy use, fairness. The goal posts moves

and as the article said "an LLM who just spent thousands of words explaining why they're not allowed to use thousands of words", its just funny to read.