Making the change without making it clear though, that's just awful. A clear recipe for catastrophic loss & drip drip drip of news in the vein of "How Backblaze Lost my Stuff"

By training: dusty philosophy & cog sci, less dusty linguistics, translation theory, NLP. More recent work w/ London School of Economics on AI Law, Policy, & Governance. LLMs come naturally, so I'm looking for a change. (Location: NYC area, USA)

For what I can offer:

Over the past few months I've built a toolkit for model interpretability that operates below black-box prompt-output observation and raw numeric activations, collapsing a lot of complexity into something more discrete and tractable, with accuracy that appears, at least from outside the core AI industry, to be harder to come by with many current methods. This includes monitoring and inference-time intervention without retraining or weight modification.

I've used it to improve benchmark performance across modalities- DeBERTa mini boosted overall > 10% in the adversarial HANS dataset through fewer false positives, no retraining or degradation in other performance. Needs testing in any deployment of course. Similarly, MedGemma, MMFLD, Whisper, a few others, with some of their standard benchmarks. Same methodology of exploration and inspection.

Utility tooling along the way includes an intuitive REPL interface for token-by-token exploration of model internals during inference, or optionally post-inference with data capture by SQLlite & LanceDB, analysis with UDF's and python. Other tools for pre-token-gen semantic monitoring and intervention intra inference. I've observed some things that are more speculative, though still promising, for understanding model behavior. All generally grounded in classical Linguistics areas of study that seem less mined for insights than industry has had opportunity to pursue in-depth with but actionable nonetheless.

I'd love to talk further. These are mainly my personal time interests though, so a resume through a job posting doesn't generally cover things adequately, if a different option is available, though I can go that route if you'd prefer.

That... pft. Nevermind, I'm just jealous

That said, reachability and novel strategies are somewhat overlapping areas of consideration, and I don't see many ways in which RL in general, as mainly practiced, improves upon models' reachability. And even when it isn't clipping weights it's just too much of a black box approach.

But none of this takes away from the question of raw model capability on novel strategies, only such with respect to RL.

[0] https://arxiv.org/pdf/2506.14245

Tyler expressed some skepticism of Democracies but nothing like this. The too on-the-nose nature of this often passed along bit of propaganda should also be the giveaway that it might be one of those rare things on the internet that someone may have been less than honest about the origins, and go look and see.

Some middle ground might be there somewhere. But if forced to choose… the choices for interpreting behavioral engineering funded by $billions in research for over a decade + data harvesting on a scale unprecedented, for the purpose of manipulating users:

Doesn’t sound a lot like fishing to me.

The other truths that were not about coding agents:

--Skill Atrophy. (Use it or lose it-- another thing we already know)

--The economics of serving code agents at scale (Ungrounded in actual numbers, only OpenAI's miscellaneous statements and annecdotes. Actual cost of running code agents: last gen's mid-tier gaming gpu's will get you reasonably close to Claude Sonnet if you put just a little time in to an agent harness, and its getting cheaper and cheaper for better and better. So, at scale, with real sysadmins doing the hard engineering to eek out every last bit of performance-- well, infra needed for serving these isn't the cost center)

--Copyright. (This passed on the same bad read of a court ruling half the press has been doing for a few years now. TLDR: The Thaler vs. Perlmutter case, which said nothing about output not being protected by copyright. It denied Thaler's attempt to register *the AI* as the owner of the copyright)

AI is part of the problem with what MS has shoved in to things but it may be part of what can help with the underlying issue of this behavior by corporations.

The average user increasingly will not need to be walked through in certain ways, they’ll only have to be aware something, some way, is possible. Because we are most of usthe average, meaning outsider to knowledge and understanding of things their functioning on a computer. I can strip out tired windows behavior to some extent and certainly stand up a Linux desktop. But I didn’t know how to easily manage retrieval of data from an old disc image that refused to mount. But I knew it was there and not impossible so I asked Claude. A one shot prompt that a few minutes later had Claude reading raw bytes in someway and finding the location of a few files I needed.

So there is potential for AI to fill some gaps in this way and make some things easier and more in reach of average users. It’s potential only though, so continuing to work and ensure open models remain a thing, it’s important. Just like the Internet enabled a lot of things previously out of reach of people. And yeah, that was not an un mixed blessing with the rest, so all the more reason to move forward thoughtfully.

If that was the intent don’t you think it would be stated somewhere, or in the faq?

>“Talking” past

It’s only text, there’s no talking past. You can’t talk past someone when the conversation isn’t spoken. At best, you might ignore what they write and go on and on and on at some length on your own point instead, ever meandering further from the words you didn’t read, widening the scope of the original point to include the closest topic that isn’t completely orthogonal to the one at hand, like the current tendency to look for the newest pattern of LLM output in everyone’s’ comments in an attempt to root out all potential AI generated responses. And eventually exhaust all of their rhetoric and perhaps, just maybe, in the very end, get to the

I have-- I see your info via your HN profile. If I have a spare moment this weekend I'll reach out there, I'll dig up a few examples and take screenshots. I built an exploration tool for investigating a few things I was interested in, and surfacing potential reasoning paths exhibited in the tokens not chosen was one of them.

Part of my background is in Linguistics-- classical not just NLP/comp-- so the pragmatics involved with disfluencies made that "wait..." pattern stand out during just normal interactions with LLM's that showed thought traces. I'd see it not too infrequently eg by expanding the "thinking..." in various LLM chat interfaces.

In humans it's not a disfluency in the typical sense of difficulty with speech production, it's a pragmatic marker, let's the listener know a person is reevaluating something they were about to say. It of course carries over into writing, either in written dialog or less formal self-editing contexts, so it's well represented in any training corpora. As such, being a marker of "rethinking", it stood to reason models' "thinking" modes displayed it-- not unlikely it's specifically trained for.

So it's one of the things I went token-diving to see "close up", so to speak, in non-thinking models too. It's not hard to induce a reversal or at least diversion off whatever it would have said-- if close to a correct answer there's a reasonable chance it will get the correct one instead of pursuing a more likely of the top k. This wasn't with Qwen, it was gemma 3 1b where I did that particular exploration. It wasn't a systematic process I was doing for a study, but I found it pretty much any time I went looking-- I'd spot a decision point and perform the token injection.

If I have the time I'll mockup a simple RAG scenario, just inject the documents that would be retrieved from RAG result similar to your article, and screenshot that in particular. A bit of a toy setup but close enough to "live" that it could point the direction towards more refined testing, however the model responds, and putting aside the publishing side of these sorts of explorations there's a lot of practical value in assisting with debugging the error rates.

My apologies, it wasn’t my intent to convey that as a primary. It isn’t one. It’s simply the first thing you should do, apart from vetting your documents as much as practicality allows, to at least start from a foundation where transparency of such results is possible. In any system whose main functionality is to surface information, transparency and provenance and a chain of custody are paramount.

I can’t stop all bad data, I can maximize the ability to recognize it on site. A model that has a dozen RAG results dropped on its context needs to have a solid capability in doing the same. Depending on a lot of different details of the implementation, the smaller the model, the more important it is that it be one with a “thinking” capability to have some minimal adequacy in this area. The “wait-…” loop and similar that it will do can catch some of this. But the smaller the model and more complex the document—- forget about context size alone, perplexity matters quite a bit— the more a small model’s limited attention budget will get eaten up too much to catch contradictions or factual inaccuracies whose accurate forms were somewhere in its training set or the RAG results.

I’m not sure the extent to which it’s generally understood that complexity of content is a key factor in context decay and collapse. By all means optimize “context engineering” for quota and API calls and cost. But reducing token count without reducing much in the way of information, that increased density in context will still contribute significantly to context decay, not reducing it in a linear 1:1 relationship.

If you aren’t accounting for this sort of dynamic when constructing your workflows and pipelines then— well, if you’re having unexpected failures that don’t seem like they should be happening, but you’re doing some variety of aggressive “context engineering”, that is one very reasonable element to consider in trying to chase down the issue.

My own read of the subtext was something a bit different. Dimon saw something he really didn’t like and my guess would be that more than just a handful of people at JP Morgan were having their next few days or longer personal plans cancelled—- or that it had already settled from something like that— to find whatever they had in the way of cockroaches. And so Dimon’s public statement was a soft nudge to try and get others to do the same, cautiously and slowly without panicking.

It’s tea leaves but the time since then seems to bear that out, with right now’s world economic volatility being a good opportunity for many places to go a little more aggressively in reigning in whatever they have in cockroach’s with some cover from that volatility and distraction to not have to explain too much more or get too much scrutiny that would accelerate things beyond manageable.

Overall, my take was that Dimon is still probably pissed off about SV bank and trying to make sure whatever shape or size this private credit rot may have doesn’t go down quite that haphazardly.

Since then I’ve seen small things indicating lots of people quietly checking their books for such.

In the last week or two this has accelerated. A lot. Every few days there are ratchets tightening things up. Dimon just put some hard limits on some private credit lines and what they could take out. A few other banks trying to take other precautions.

You have to choose model suitably robust is capabilities and design prompts or various post training regimes that are tested against such, where the model will identify the different ones and either choose the correct one on surface both with an appropriately helpful and clear explanation.

At minimum you have to start from a typical model risk perspective and test and backtest the way you would traditional ML.

The fundamental problem is that once you take something outside the realm of law and rule of law in its many facets as the legitimizing principal, you have to go a whole lot further to be coherent and consistent.

You can’t just leave things floating in a few ambiguous things you don’t like and feel “off” to you in some way- not if you’re trying to bring some clarity to your own thoughts, much less others. You don’t have to land on a conclusion either. By all means chew over things, but once you try to settle, things fall apart if you haven’t done the harder work of replacing the framework of law with that of another conceptual structure.

You need to at least be asking “to what ends? What purpose is served by the rule?” Otherwise you’re stuck in things where half the time you end up arguing backwards in ways that put purpose serving rules, the maintenance of the rule with justifications ever further afield pulled in when the rule is questioned and edge cases reached. If you’re asking, essentially, “is the spirit of the rule still there?” You’ve got to stop and fill in what that spirit is or you or people that want to control you or have an agenda will sweep in with their own language and fill the void to their own ends.