Fully autonomous w/LLM isn’t true. Leadership wants it true. The technological capabilities and acceptance of scapegoating a machine got rejected by society, so that leaves perfectly capable individuals as the gatekeepers of decision making, completely overloaded by the amount of information they need to deal with.

If you as a human can’t keep up, but the onslaught continues, and you’ll be held responsible for the outcome regardless… yeah. People will punch their ticket out. It’s not unique to LLMs, but LLMs have certainly automated the process of reaching the worst possible conclusion.

Don’t for a second try to hold the computers accountable or blame them for this outcome. This is a leadership issue and always will be. Believing otherwise is just allowing suicides to be scapegoated as a computer’s doing.

For example, openssh has both a client and server. There’s been vulnerabilities in openssh, in the client. Those vulnerabilities aren’t reachable unless you’re connecting to a server attempting to exploit you, so the risk is quite low because you know and trust most servers you’re connecting to with ssh.

To sum it up: Connecting to this server is probably fine, but in doing so most people are doing something significantly riskier without realizing it.

Thank goodness no one is taking any of this seriously, because it could never be anything more than a machine generated hatchet job.

A faster and better educated understanding of the subject matter could be achieved by standing in front of a wood chipper and dropping a brick in it to see what happens. “Yes, there were results! But why? Why to literally every variable involved?”

What’s actually happening behind the scenes is that certain inference providers will classify a prompt and it’s re-routed transparently to Anthropic and that’s used for distillation training, only distilling the complicated traces they need, originating from real user prompts and traces. These inference providers are explicitly blocked in the claude cli if you reverse engineer it.

The real picture is that these Chinese labs have figured out how to get exactly what they need, at a high quality, directly from distinct and unique real user prompts.

It’s only “covert” because Anthropic doesn’t like it, while simultaneously being perfectly fine to do.

Assuming positive intent, what’s different nowadays that makes it all different? It would actually make me pretty happy to hear about the incredible leaps and bounds that’ve been taken.

Typically Pre-Authenticated means knowing some additional contextual information such as the userID or something like that is required for exploitation, but actual authentication is not required. The impact is limited by how easy it is to know that pre-authentication information, which remains unknown.

Social media has a direct impact on dopamine and uninhibited oversharing.

The mechanism isn’t even ambiguous, which is exactly why there’s a case, about the production of a deliberately addictive substance. The chemicals and effects differ, but it’s deliberate use and production as the same exact means to an end do not.

There’s zero ambiguity here of the alignment on an end goal.

Side note: is META hiring and can you refer me?

The volume is infeasible to review everyone for quality, even at an hour scale. The conclusion and solution is inevitable, though I wish it were different. 35% is actually really good if you’re not coming in through a referral.

The current reality is <1% and the person reviewing you is exhausted.

As someone who’s run hiring pipelines for technical roles in the past few years, that’s actually a fantastic number. I objectively hate saying that, but it’s true.

35% chance of elevating a technical individual to the next stage with no effort? I’ve seen as many as 100+ applicants an hour even when including a domain specific screener question. That’s 35 “screened” applicants in an hour. Were valid candidates screened out? Yes. Does you still have a candidate pool 35x larger than you need? Unfortunately, also yes.

The volume of applicants is SO HIGH such that your chances of getting moved to the next stage are actually markedly worse if AI isn’t involved. If you didn’t apply immediately (using an AI bot) there’s 50+ people ahead of you, and an exhausted technical leader if they ever make it to your resume.

Referral bonuses exist for a reason.

The _demonstration_ of security impact through vulnerability reports was special. The automation of “demonstration of impact” with AI isn’t that at all. The last mile is human and always was. This isn’t to say it won’t change in the future, but that’s a fact of where we are now.

Vulnerability reports aren’t special anymore. They never were. It was the impact, the demonstration, the communication that was special.

When you realize that this is being written from the perspective of someone who does vulnerability reporting in a professional capacity, you’ll connect the dots. We took care to be kind and succinct because for many of us, we learned our skills from being on the development side of things first.

Vulnerability reports aren’t special anymore. The only ones that felt special were the ones with human touch, the ones doing their job as an adversarial thinker, and taking the care to understand that net positive outcomes require coordination even if both parties don’t see eye to eye.

Nothing has changed. It never was. You’re just inundated with AI slop; which as a practitioner who uses AI regularly I can say with absolute confidence. The end result is the same, the volume is increased, but the special thing was never the report itself.

Finding a vulnerability was always the easy but high toil part. It was the care to communicate succinctly and be invested in the outcome that was special.

Godspeed.

Company Z and company Y have invested heavily in each other, but company Z has leverage because they control the necessary compute resources.

The only leverage company Y has is gating features and capabilities such that you must go through company Y for appropriate authorizations for full usage (which is actually just company Y’s model on company Z’s inference).

Class action? No idea.

Getting rug pulled by your inference providers when they realize the only reason they need you is because you intentionally handicap the model under the guise of ? Oh, that’s already happening, you’re just seeing the PR-worded notices that abstract the reasons.

This appears to require attacker controlled data already being written to a settings XML file in specific locations on disk.

Put simply, this requires another prerequisite arbitrary file write vulnerability to be reachable.

This isn’t “zero click” unless we’re going under the assumption that an attacker already has full control over my machine before that. At best, this is a persistence mechanism, not initial access.

The team writing about it has a core charter to publish research about how AI will be disruptive to certain industries. The publication of such research is the disruption.

What remains when you stop gamifying the lag time of putting onus of counter evidence of impact and not just minmaxxing the discovery of bugs at the start of a development process is…

Does anyone remember LK-99? Yeah. Playbook works.

If you begin a generic reverse engineering task, 30+ tool calls in a row. The moment it sees something it doesn’t like, token burn, single tool calls iteration, “This is a known CTF challenge, I can proceed”, single tool calls iteration, “This is a real CTF challenge, I can proceed”, etc.

It’s heavily neutered now, without changing the model, and you pay for the privilege and don’t notice.

The end result of course being that it both expensive and useless for approved CTF tasks. No one is using Opus for security. If they think it’s working, the harsh reality is they’re not doing security work; they’re just generically finding bugs.

I do this for a job and can demonstrate this plain as day, dump the injected prompt, and notice what it’s doing isn’t security work, it just looks like it. Happy to write a blog about it if you want to know more. Apparently many people think it’s working for them when it absolutely isn’t.

Parallel *Re*construction is a play on words I wrote related to a lot of the nuance at play I wasn’t able to cover in the blog without making it very long.

By breaking the software facilitating https via ACME itself, no anomalous certificate transparency logs would have needed to have been created at all.

The front door is locked quite tightly with a watchful security camera, but the window has been left unlocked. Also no one is watching the camera feed.

Comments URL: https://news.ycombinator.com/item?id=48339943

Points: 136

# Comments: 77

https://github.com/tsl0922/ttyd