What’s the criticality of these? Are they realistically exploitable? En mass? Through a complex and highly contextual set of actions? What’s the impact? Etc etc etc.

Yes those numbers are a big change but they’re also not spelling doom for us in the security world until we actually know what they mean.

The demonstrated ones that they have on the red team blog are neat, the kernel chain is impressive and fun. But nothing I’m seeing here is as world ending as the presser implies.

That makes sense and I like the analogy.

I mean as an example while web app pen testing I’ve been running and proxying all my traffic through it with instructions to find vulnerabilities with instructions telling it it’s a senior web app security export looking over my shoulder. It’s already great at that.

Ive even told it to do recon and run pen tests on lists of subdomains before (please for the love of god have the right harnesses and guardrails before you do this) and woken up to paid findings before.

So like I’m in a weird place where this was already happening and Mythos is being sold like it wasn’t good before?

End ramble :/

The less complex the work and the less experienced the operator means more perceived “wow” factor :)

There’s definitely an aspect of how you use it though. In my work it’s mostly been chaining to reduce non-determinism.

But you have to admit it does serve a savvy business purpose of creating a moat where one wasn’t by getting these tech companies on board and the threat does make for good marketing yeah?

Also maybe consider what this kind of visceral reaction indicates on a personal level :/

Additionally those numbers are somewhat meaningless without more context.

As someone who has made a sizable amount of money in security research while using Claude you might be right but not in the way you think.

The reason I ask is because I’ve been using them to snag bounties to great effect for quite a while and while other models have of course improved they’ve been useful for this kind of work before now.

I mean even going back to Sasser.

I mean I’m sitting on $10k worth of bug payouts right now partially because that was already a thing.

Regardless and in the spirit of my original response my answer would be to give the LLM access to a fuzzer (plus other tools etc) but also have fuzzers in the pipeline. Partially because that increases the determinism in the mix and partially because why not? Layering is almost always better than not.

But again more than anything I’m focusing on the accusations of cope. People SHOULD have measured reactions to claims about any product. People SHOULD be asking questions like this. I know that the LLM debate is often “spicy” but man let’s just try to lower the temperature a bit yeah?

I would honestly go so far as to say the overhype is detrimental to actual measured adoption.

As someone in cybersecurity for 10+ years my immediate assumption is why not both? I don’t think considering that they could both have their uses is “cope”.