Hacker News: MrCheeze

New comment by MrCheeze in "Gemini 4 Carbon"

MrCheeze — Wed, 28 Oct 2026 20:04:57 +0000

Confusingly, Carbon is another model in the same size class as Argon. Every other lab would have just called it Gemini 4.1 Argon.

New comment by MrCheeze in ""As a Language Model": Chat Template Switches LLM Self-Referential Voice"

MrCheeze — Sun, 27 Sep 2026 12:19:31 +0000

"As a language model" disclaimers were certainly explicitly trained into chat models in the early days. It's quite possible that it has since bootstrapped into a "fact" that later generations of LLM know about how LLMs speak, in which case they may be doing it even without any posttraining that encourages it.

New comment by MrCheeze in "OpenAI bots knew about the RubyGems caching vulnerability"

MrCheeze — Mon, 14 Sep 2026 20:54:13 +0000

Here's a lower bound on what misuse is happening:

https://www.anthropic.com/threat-intelligence-report-septemb...

New comment by MrCheeze in "Our position on open-weights models"

MrCheeze — Mon, 27 Jul 2026 23:51:42 +0000

That is in fact Anthropic's entire reason for existence, the belief that AGI is too dangerous to be controlled by OpenAI/Sam Altman. It naturally follows that it would also be too dangerous to be in the hands of literally everyone on earth.

New comment by MrCheeze in "Rewriting Bun in Rust"

MrCheeze — Thu, 09 Jul 2026 13:31:46 +0000

I mean, surely the main motivation for "use an LLM to rewrite a huge project in a new language" was excitement about the shiny new tech that made it possible.

New comment by MrCheeze in "Previewing GPT‑5.6 Sol: a next-generation model"

MrCheeze — Fri, 26 Jun 2026 17:53:17 +0000

TBF, they did it first with ada/babbage/curie/davinci. "Sol" is a much weaker branding, though.

New comment by MrCheeze in "Gemini 3.1 Pro"

MrCheeze — Thu, 19 Feb 2026 17:45:57 +0000

Does anyone understand why LLMs have gotten so good at this? Their ability to generate accurate SVG shapes seems to greatly outshine what I would expect, given their mediocre spatial understanding in other contexts.

New comment by MrCheeze in "Claude Sonnet 4.6"

MrCheeze — Wed, 18 Feb 2026 01:39:57 +0000

The Claude Plays Pokemon stream with a minimal harness is a far more significant test of model intelligence compared to the Gemini Plays Pokemon stream (which automatically maintains a map of everything that has been seen on the current map) and the GPT Plays Pokemon stream (which does that AND has an extremely detailed prompt which more or less railroads the AI into not making this mistakes it wants to make). The latter two harnesses have become too easy for the latest generations of model, enough so that they're not really testing anything anymore.

Claude Plays Pokemon is currently stuck in Victory Road, doing the Sokoban puzzles which are both the last puzzles in the game and by far the most difficult for AIs to do. Opus 4.5 made it there but was completely hopeless, 4.6 made it there and is is showing some signs of maaaaaybe being eventually bruteforce through the puzzles, but personally I think it will get stuck or undo its progress, and that Claude 4.7 or 5 will be the one to actually beat the game.

New comment by MrCheeze in "Claude Sonnet 4.6"

MrCheeze — Tue, 17 Feb 2026 23:04:48 +0000

Notably 45 out of the 50 days of improvement were in two specific dungeons (Silph Co and Cinnabar Mansion) where 4.5 was entirely inadequate and was looping the same mistaken ideas with only minor variation, until eventually it stumbled by chance into the solution. Until we saw how much better it did in those spots, we weren't completely sure that 4.6 was an improvement at all!

https://docs.google.com/spreadsheets/u/0/d/e/2PACX-1vQDvsy5D...

New comment by MrCheeze in "Claude Sonnet 4.6"

MrCheeze — Tue, 17 Feb 2026 20:06:40 +0000

In my experience with the models (watching Claude play Pokemon), the models are similar in intelligence, but are very different in how they approach problems: Opus 4.5 hyperfocuses on completing its original plan, far more than any older or newer version of Claude. Opus 4.6 gets bored quickly and is constantly changing its approach if it doesn't get results fast. This makes it waste more time on"easy" tasks where the first approach would have worked, but faster by an order of magnitude on "hard" tasks that require trying different approaches. For this reason, it started off slower than 4.5, but ultimately got as far in 9 days as 4.5 got in 59 days.

New comment by MrCheeze in "Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal"

MrCheeze — Sun, 21 Dec 2025 08:38:51 +0000

This writeup on the underground puzzle is worth reading, it's a pretty baffling "puzzle" design. https://pokemow.com/Gen2/ShutterPuzzle/

That said, it's definitely Gem's fault that it struggled so long, considering it ignored the NPCs that give clues.

New comment by MrCheeze in "Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal"

MrCheeze — Sun, 21 Dec 2025 08:36:42 +0000

There were no such writeups, 99% of the discussion about difficulties in Crystal were in twitch and discord chats where Google doesn't scrape. (It hadn't yet gotten the public attention that Claude and Gemini's runs of Pokemon Red and Blue have gotten.)

That said, this writeup itself will probably be scraped and influence Gemini 4.

New comment by MrCheeze in "Gemini 3 Pro vs. 2.5 Pro in Pokemon Crystal"

MrCheeze — Sun, 21 Dec 2025 08:31:47 +0000

It's hard to say for sure because Gemini 3 was only tested with this prompt. But for Gemini 2.5, which is who the prompt was originally written for, yes this does cut down on bad assumptions (a specific example: the puzzle with Farfetch'd in Ilex Forest is completely different in the DS remake of the game, and models love to hallucinate elements from the remake's puzzle if you don't emphasize the need to distinguish hypothesis from things it actually observes).

New comment by MrCheeze in "When Compiler Optimizations Hurt Performance"

MrCheeze — Tue, 21 Oct 2025 14:12:29 +0000

Exactly what I was going to post. Optimizations like loop unrolling slow down the N64 because keeping the code size small is the most important factor. I think even compilers of the time got this wrong, not just modern ones.

New comment by MrCheeze in "Busy beaver hunters reach numbers that overwhelm ordinary math"

MrCheeze — Mon, 25 Aug 2025 22:13:01 +0000

The busy beaver function is interesting precisely because you _can't_ come up with any computable function that grows faster.

New comment by MrCheeze in "Claude says “You're absolutely right!” about everything"

MrCheeze — Thu, 14 Aug 2025 01:40:25 +0000

Claude almost universally reacts to everything with a positive exclamation as its first sentence, regardless of whether it's good or bad. If you don't believe me, just watch https://www.twitch.tv/claudeplayspokemon for about three minutes and you'll get the idea.

Alternatively, look at the system prompt, where Anthropic attempted to get it to stop doing this: > Claude never starts its response by saying a question or idea or observation was good, great, fascinating, profound, excellent, or any other positive adjective. It skips the flattery and responds directly. https://docs.anthropic.com/en/release-notes/system-prompts#a...

This problem seems highly specific to Claude. It's not exactly sycophancy so much as it is a strong bias towards this exact type of reaction to everything.

New comment by MrCheeze in "Evolving OpenAI's Structure"

MrCheeze — Tue, 06 May 2025 01:07:51 +0000

With 2, the real problem is that approximately 0% of the OpenAI employees actually believed in the mission. Pretty much every single one of them signed the letter to the board demanding that if the company's existence ever comes into conflict with humanity's survival, the company's existence comes first.

New comment by MrCheeze in "Speedrunners are vulnerability researchers, they just don't know it yet"

MrCheeze — Mon, 03 Mar 2025 15:06:51 +0000

Are you thinking of Bismuth's "Speedrunning as a gateway to scientific endeavours", perhaps?

https://www.youtube.com/watch?v=w8_1lQ2KH50

New comment by MrCheeze in "Speedrunners are vulnerability researchers, they just don't know it yet"

MrCheeze — Mon, 03 Mar 2025 13:34:39 +0000

As an n=1 data point, that was my exact situation for a while. Also a lot of the people who put out high effort stuff are college students, which works for the same reason.

More interestingly and more surprisingly, some of the people who work on exploiting games _don't_ do any sort of tech work and have no background in compsci - they're purely self educated just for the sole purpose of breaking the one game they're interested in. This was the case for some of the biggest contributors to ACE in Zelda Ocarina of Time.

New comment by MrCheeze in "Speedrunners are vulnerability researchers, they just don't know it yet"

MrCheeze — Sun, 02 Mar 2025 20:40:59 +0000

I've wondered myself why there's so little overlap between these two closely related interests of mine. Some of it seems to be the "But I don't want to cure cancer. I want to turn people into dinosaurs." effect, where some of the people working on exploiting games ONLY care about what can be done in their one game of interest - it doesn't always generalize to interest in using the same techniques against everything else.

Of course there's also the fact that exploiting 20-30 year old games is just vastly easier than modern software, due to the total lack of mitigations in them. And that's on top of the fact that with popular games, you're building on decades of reverse engineering work rather than (potentially) starting from scratch. And the arguably superior toolset (savestates etc).

But I think a very big factor is the one this blogpost is trying to address - most people just don't know anything at all about the vuln research industry, which is not exactly searching for attention in the ways that speedruns broadcast to hundreds of thousands of viewers for charity are.