Which is: When you think about Google, true, old, "don't be evil", tech excellence Google, you don't think about Demis. You think about Jeff's and Sanjay's geeky, technically uncompromising, faces.

[1] https://retrogames.biz/products/thespectrum/

Comments URL: https://news.ycombinator.com/item?id=49135699

Points: 2

# Comments: 0

This makes sense. At most it converts the question of ability to a question of cost (i.e. fire up a prompt for each file).

This article reduces the hype about Mythos in my mind: A new model that can find 9 new bugs while no previous model can identify them, is a whole different story from what this article demonstrates: that only 2/9 of the detected bugs are new for Mythos.

Great work.

This should have probably been surfaced at the text. To answer my own question: If one uses gemma4-26b-a4b, mimo-v2.5-pro and gemini-3.5-flash then 7/9 bugs are covered, while no model can uncover the remaining two.

> And, you have misunderstood what the benchmark does. It tells the model to audit the file, and it is allowed to look at the rest of the repo. It is not pointed at the bug.

I obviously meant that the model was pointed to the bug report. What would have been the point of providing the actual bug. This is still easier than telling Mythos to generally look at a codebase and find any bug.

It might have been easy to guess, but you didn't really think of it in the past 51 years since the Commodore 128 was introduced, did you?

Also, it seems to me that pointing a model to a bug and asking it to solve it is somewhat easier than what Mythos did, which if I understand correctly, was to generally look at a codebase and find any bug. Even so, non-Mythos models only managed to fix 4/9 of these bugs.

I think the article makes the point that Mythos is at a different level.