I'm not saying you're not going to drive confusion by overloading context, but the number of tokens required to trigger that failure mode in opus is going to be a lot higher than the number for gpt-oss-20b.

I'm pretty sure a model that can run on a cellphone is going to cap out it's context window long before opus or mythos would hit the point of diminishing returns on context overload. I think using a lower quality model with far fewer / noisier weights and less precise attention is going to drive false positives way before adding context to a SOTA model will.

You can even see here, AISLE had to print a retraction because someone checked their work and found that just pointing gpt-oss-20b at the patched version generated FP consistently: https://x.com/ChaseBrowe32432/status/2041953028027379806

If you point your model directly at the thing you want it to assess, and it doesn't have to gather any additional context you're not really testing those things at all.

Say you point kimi and opus at some code and give them an agentic looping harness with code review tools. They're going to start digging into the code gathering context by mapping out references and following leads.

If the bug is really shallow, the model is going to get everything it needs to find it right away, neither of them will have any advantage.

If the bug is deeper, requires a lot more code context, Opus is going to be able to hold onto a lot more information, and it's going to be a lot better at reasoning across all that information. That's a test that would actually compare the models directly.

Mythos is just a bigger model with a larger context window and, presumably, better prioritization and stronger attention mechanisms.

The smaller models can recognize the bug when they're looking right at it, that seems to be verified. And with AISLE's approach you can iteratively feed the models one segment at a time cheaply. But if a bug spans multiple segments, the small model doesn't have the breadth of context to understand those segments in composite.

The advantage of the larger model is that it can retain more context and potentially find bugs that require more code context than one segment at a time.

That said, the bugs showcased in the mythos paper all seemed to be shallow bugs that start and end in a single input segment, which is why AISLE was able to find them. But having more context in the window theoretically puts less shallow bugs within range for the model.

I think the point they are making, that the model doesn't matter as much as the harness, stands for shallow bugs but not for vulnerability discovery in general.

Comments URL: https://news.ycombinator.com/item?id=47523037

Points: 2

# Comments: 0

Proving out a security vulnerability from beginning to end is often very difficult for someone who isn't a domain expert or hasn't seen the code. Many times I've been reasonably confident that an issue was exploitable but unable to prove it, and a 10s interaction with the maintainer was enough to uncover something serious.

Exhausting these report channels is making this unfeasible. But the number of issues that will go undetected, that would have been detected with minimal collaboration between the reporter and the maintainer, is going to be high.

Comments URL: https://news.ycombinator.com/item?id=46571004

Points: 2

# Comments: 1

> When they finally did reply, they seem to have developed some sort of theory that I was interested in “access to PII”, which is entirely false. I have no interest in any PII, commercially or otherwise. As my private email published by Ruby Central demonstrates, my entire proposal was based solely on company-level information, with no information about individuals included in any way. Here’s their response, over three days later.

Imagine an LLM with the ability to read emails, update database records, and destroy database records.

The user instructs the LLM to update a database record, but a malicious injection from one of those emails overrides that with a directive to destroy the database record. Unless the validator understands the users intent somehow, the destructive action would appear perfectly reasonable.

In the context of web-request -> validator -> db query, the purpose of the validator is only to ensure that the request is safe, it doesn't care what the user chose to do as long as it's a reasonable action in the context of the app.

In the context of user -> LLM -> validator -> tool, the validator has to ensure that the request is safe, but the users intention can be changed at the LLM stage. If the user wanted to update a record, but the LLM decides to destroy it, the validator now has to have some way to understand the users initial intention to know whether or not the request is sane.

Especially in this context, where decades have been spent building and improving same origin policy controls. The entire web has been built around the expectation that those controls prevent cross origin data access.

I also don't even think it's that difficult to solve. For one, data in the context window doesn't have to be a string, it can be an array of objects that contain the origin they were pulled from as metadata. Then you can provide selective content to different MCP-B interfaces depending on their origins. That would live in the protocol layer that would help significantly.

If the purpose of the MCP-B tool on mail.com is to summarize your email, then the site needs to allow the agent to pull your email into the context window. Once it's in the context window it's available to any other MCP-B enabled site that can convince the agent to send it along.

This seems really bad to me. There are so many ways for a website to end up in one of my browser tabs without me wanting it there, or even knowing it's there.

If that happens, and that tab just so happens to be a malicious MCP-B enabled page, it could steal all kinds of data from all kinds of different web apps I'm interacting with. I think it should be seen as the responsibility of the framework to enforce some level of data isolation, or at the least opt-in consent mechanisms.

For example, when you use the sandbox attribute on an iframe in a web application, it's not the user that's untrusted, it's some other user that's attempting to trigger actions in your client.