Hacker News: johnfn

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sun, 12 Apr 2026 04:03:42 +0000

In this entire thread of conversation, I never said that LLMs would take people's jobs, and that is not something I believe.

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sun, 12 Apr 2026 01:48:15 +0000

Your proof-in-pudding test seems to assume that AI is binary -- either it accelerates everyone's development 100x ("let's rewrite every app into bug-free native applications") or nothing ("there hasn't been anything to show for that in years"). I posit reality is somewhere in between the two.

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sat, 11 Apr 2026 22:55:10 +0000

No one is saying your nested for loop idea because it won't actually work in practice. In short, the signal to noise ratio will be too high - you will need to comb through a ton of false positives in order to find anything valuable, at which point it stops looking like "automated security research" and it starts looking like "normal security research".

If you don't believe me, you should try it yourself, it's only a couple of dollars. Hey, maybe you're right, and you can prove us all wrong. But I'd bet you on great odds that you're not.

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sat, 11 Apr 2026 22:45:13 +0000

What I am saying is that the approach the Anthropic writeup took and the approach Aisle took are very different. The Aisle approach is vastly easier on the LLM. I don't think I need a citation for that. You can just read both writeups.

The "9500" quote is my conjecture of what might happen if they fix their approach, but the burden of proof is definitely not on me to actually fix their writeup and spend a bunch of money to run a new eval! They are the ones making a claim on shaky ground, not me.

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sat, 11 Apr 2026 20:38:35 +0000

You can look at some of the bugs, if you'd like. They are (at least the ones I looked at) fairly self-contained, scoped to a single function, a hundred lines or less. There's no need for a massive amount of context.

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sat, 11 Apr 2026 20:18:56 +0000

Admittedly just vibes from me, having pointed small models at code and asked them questions, no extensive evaluation process or anything. For instance, I recall models thinking that every single use of `eval` in javascript is a security vulnerability, even something obviously benign like `eval("1 + 1")`. But then I'm only posting comments on HN, I'm not the one writing an authoritative thinkpiece saying Mythos actually isn't a big deal :-)

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sat, 11 Apr 2026 19:39:50 +0000

The citation is the Anthropic writeup.

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sat, 11 Apr 2026 18:34:31 +0000

> Wasn't the scaffolding for the Mythos run basically a line of bash that loops through every file of the codebase and prompts the model to find vulnerabilities in it? That sounds pretty close to "any gold there?" to me, only automated.

But the entire value is that it can be automated. If you try to automate a small model to look for vulnerabilities over 10,000 files, it's going to say there are 9,500 vulns. Or none. Both are worthless without human intervention.

I definitely breathed a sigh of relief when I read it was $20,000 to find these vulnerabilities with Mythos. But I also don't think it's hype. $20,000 is, optimistically, a tenth the price of a security researcher, and that shift does change the calculus of how we should think about security vulnerabilities.

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 17:31:22 +0000

If you want to delete your account you can just set your noprocrast to some absurdly large number like 99999999.

New comment by johnfn in "Small models also found the vulnerabilities that Mythos found"

johnfn — Sat, 11 Apr 2026 17:27:16 +0000

The Anthropic writeup addresses this explicitly:

> This was the most critical vulnerability we discovered in OpenBSD with Mythos Preview after a thousand runs through our scaffold. Across a thousand runs through our scaffold, the total cost was under $20,000 and found several dozen more findings. While the specific run that found the bug above cost under $50, that number only makes sense with full hindsight. Like any search process, we can't know in advance which run will succeed.

Mythos scoured the entire continent for gold and found some. For these small models, the authors pointed at a particular acre of land and said "any gold there? eh? eh?" while waggling their eyebrows suggestively.

For a true apples-to-apples comparison, let's see it sweep the entire FreeBSD codebase. I hypothesize it will find the exploit, but it will also turn up so much irrelevant nonsense that it won't matter.

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 08:05:43 +0000

Don't leave dang -- we need you now more than ever. :(

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 05:36:46 +0000

I think he is using "emulate" in a more metaphorical sense, like that it can do similar things that the human brain can do? I'm not trying to be antagonistic, it just seems logical? He says the Turing test won't be passed until 2029 - if we're going by your definition of "emulate" wouldn't it have been passed the instant the brain was "emulated?"

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 05:20:40 +0000

He doesn't say 'simulate' a human brain unless I'm missing it in the summary (cmd-f "simul" has no results) - that would require significantly more capacity than that contained in a brain (think about how much compute it takes to run a VM). He seems to be implying that by 2020s a computer will be about as smart as a human. LLMs seem capable of doing a decent amount of tasks that a human can do? Sure, he's off by a few years, but for something published 20 years ago when that seemed insane, it doesn't seem that bad.

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 04:51:23 +0000

I mean, an LLM isn’t too far away from this? He had the Turing test being defeated in 2029 - if anything, he was too pessimistic.

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 04:06:20 +0000

To be fair, Ray Kurzweil has been the loudest voice in this space, and he's been pretty consistent on 2045 since the publication of his book almost 20 years ago[1].

[1]: https://en.wikipedia.org/wiki/The_Singularity_Is_Near

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 03:57:09 +0000

There are many people who have been saying this far there was any sort of business model in place.

New comment by johnfn in "Sam Altman's response to Molotov cocktail incident"

johnfn — Sat, 11 Apr 2026 00:37:07 +0000

Some people think there will be an exponential takeoff, which means that a 6 month lead effectively rounds up to infinity.

New comment by johnfn in "Molotov cocktail is hurled at home of Sam Altman"

johnfn — Fri, 10 Apr 2026 23:42:00 +0000

I think this is a little too optimistic:

- Go onto a Reddit thread about ICE, everyone in the comment threads says they don't like ICE. That's the obvious statement, not edgy.

- Go onto a Reddit thread about Trump, everyone says they don't like Trump. That's the obvious statement, not edgy.

Why would we think the Sam Altman thread is any different? I unfortunately think the Reddit thread might be the real deal, or at least a little more real than you are saying.

New comment by johnfn in "A compelling title that is cryptic enough to get you to take action on it"

johnfn — Fri, 10 Apr 2026 22:33:23 +0000

The best part about this is that the title "A compelling title that is cryptic enough to get you to take action on it" is perfectly self-descriptive.

New comment by johnfn in "Who is Satoshi Nakamoto? My quest to unmask Bitcoin's creator"

johnfn — Thu, 09 Apr 2026 07:19:16 +0000

The NYT has no authority to dox people. If they or anyone believed that SSC was acting unethically or illegally, that should be processed through proper legal or ethical channels, which exist for a reason. The solution is not that NYT should abuse their power to skip those channels.