It's also infamously the subject of a Von Neumann joke

>Two bicyclists start twenty miles apart and head toward each other, each going at a steady rate of 10 m.p.h. At the same time a fly that travels at a steady 15 m.p.h. starts from the front wheel of the southbound bicycle and flies to the front wheel of the northbound one, then turns around and flies to the front wheel of the southbound one again, and continues in this manner till he is crushed between the two front wheels. Question: what total distance did the fly cover ? The slow way to find the answer is to calculate what distance the fly covers on the first, northbound, leg of the trip, then on the second, southbound, leg, then on the third, etc., etc., and, finally, to sum the infinite series so obtained. The quick way is to observe that the bicycles meet exactly one hour after their start, so that the fly had just an hour for his travels; the answer must therefore be 15 miles. When the question was put to von Neumann, he solved it in an instant, and thereby disappointed the questioner: "Oh, you must have heard the trick before!" "What trick?" asked von Neumann; "all I did was sum the infinite series

I do wish that some day someone will tell the story of what happened during that time. Maps was bad at launch yes, but it also wouldn't get better without people contributing more data, and the fact that it took a decade to slowly improve implies that there's nothing anyone could have done to get it right "off the bat". It still feels to me Forstall was set up as the fall guy, especially considering no one was fired for antennagate.

If I understood the paper right, during post-training (e.g. DPO) models learn the correct "shape" of responses. But unlike SFT they're also penalized for going off-manifold so this incentivizes development of circuits that can detect off-manifold responses (you can see this clearly with RLVR perhaps, where models have a "but wait" reflex to steer themselves back in the correct direction) [^1]. Since part of the training is to be the archetypical chatbot assistant though, when combining with anti-jailbreak training this usually gets linked into "refusal" circuits.

One hypothesis might be that the question itself is leading. I.e. models will by default respond "no" to "are there any injected thoughts", just as they would to "are you conscious" or "do you have feeligns", because of RLHF that triggers refusal behavior. Then injection provides a strong enough signal that ends up "scrambling" this pathway, _suppressing_ the normal refusal behavior and allowing them to report the injection. (Describing the contents of the injected vector is trivial either way, as the paper notes the detection is the important part).

The interesting thing is that ablating away refusals doesn't actually change the false positive rate though, so instead the above hypothesis of injections overriding a default refusal doesn't fit. Instead there really does seem to be a separate "evidence carrier" detector sensitive to off-manifold responses, which just so happens to get wired into the "refusal circuits" but when "unwired" via ablation allows the model to report injections.

I guess what's not clear to me though is whether this is really detecting _injection_ itself. Wouldn't the same circuits be triggered by any anomalous context? It shouldn't be any surprise that models can detect models anomalies in input tokens (after all LLMs were designed to model text), so I don't see why anomalies in the residual stream would be any different (it's not like a layer cares whether the "bread" embedding was injected externally or came through from the input token).

In theory the case of "anomalous input context" versus "anomalous residual via external injection" _can_ be distinguished though, because there would be a sort of "discontinuity" in the residual stream as you pass through layers, and the hidden state at token i depth n feeds into that of token i+1 depth n+1, you could in theory create a computational graph that could detect such tampering.

I think the paper sort of indirectly tested this in section 3.2 "SPECIFICITY TO THE ASSISTANT PERSONA"

>In contrast, the two nonstandard roles (Alice-Bob, story framing) induce confabulation. Thus, introspection is not exclusive to responding as the assistant character, although reliability decreases outside standard roles.

Which does seem to imply that as soon as you step out of distribution to things like roleplay that RLHF specifically penalized, the anomaly detectors start firing as well.

[^1] I think this is also related to how RLHF/DPO are sequence-level optimizations, with a notion of credit assignment. And optimizing in this way results in the model having a notion of whether the current position in the rollout is "good" or not.

[1] https://nathan.rs/posts/roberta-diffusion/

If you're willing to incur a latency penalty on a "cold resume" (which is fine for most use-cases), why couldn't they just move it to disk. The size of the KV cache should scale on the order of something like (context_length * n_layers * residual_length). I think for a standard V3-MoE model at 1M token length, this should be on the order of 100G at FP16? And you can surely play tricks with KV compression (e.g. the recent TurboQuant paper). It doesn't seem like an outrageous amount of data to put onto cheap scratch HDD (and it doesn't grow indefinitely since really old conversations can be discarded).

Why can't they save the kv cache to disk then later reload it to memory?

But why does the economy need to grow? If you can manufacture everything you need, and you have access to the raw resources, what else do you need as a country. In what sense is growing your economy with VC scams like Juicero better than actually having industrial output?

[1] https://news.ycombinator.com/item?id=43881692 [2] https://machinelearning.apple.com/research/neural-engine-tra...

They are handled as part of the "conceptual" run loop, but they seem to be dispatched internally by AXRuntime library from a callback off some mach port. And because of this, the call to nextEventMatchingEventMask in the main -[NSApplication run] loop never even sees any such NSEvent.

    -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:]  (in AppKit)
        _DPSNextEvent  (in AppKit)
          _BlockUntilNextEventMatchingListInModeWithFilter  (in HIToolbox)
            ReceiveNextEventCommon  (in HIToolbox)
              RunCurrentEventLoopInMode  (in HIToolbox)
                CFRunLoopRunSpecific  (in CoreFoundation)
                  __CFRunLoopRun  (in CoreFoundation)
                    __CFRunLoopDoSource1  (in CoreFoundation)
                      __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__  (in CoreFoundation)
                        mshMIGPerform  (in HIServices)
                          _XPerformAction  (in HIServices)
                            _AXXMIGPerformAction  (in HIServices)

[1] https://www.cocoawithlove.com/2009/01/demystifying-nsapplica...

Do you mean that you can just AXUIElementPerformAction once you have a reference to it and the OS will internally synthesize the right type of event, even if it's not in the foreground?

There is also this old blog post by Yegge [1] which mentions `AXUIElementPostKeyboardEvent` but there were plenty of bugs with that, and I haven't seen anyone else build on it. I guess the modern equivalent is `CGEventPostToPSN`/`CGEventPostToPid`. I guess it's a good candidate though, perhaps the Sky team they acquired knows the right private APIs to use to get this working.

Edit: The thread at [2] also has some interesting tidbits, such as Automator.app having "Watch Me Do" which can also do this, and a CLI tool that claims to use the CGEventPostToPid API [3]. Maybe there's more ways to do it than I realized.

[1] https://steve-yegge.blogspot.com/2008/04/settling-osx-focus-... [2] https://www.macscripter.net/t/keystroke-to-background-app-as... [3] https://github.com/socsieng/sendkeys

How does that even work technically? macOS doesn't support multiple cursors. On native Cocoa apps you can pass input to a window without raising via command+click so possibly they synthesized those events, but fewer and fewer apps support that these days. And AppleScript is basically dead, so they can't be using that either.

I also read they acquired the Sky team (who I think were former Apple employees). No wonder they were able to pull of something so slick.

Is thinking of it in this sense actually accurate? I always assumed since every government has embraced MMT they can spend whatever they want simply by printing it out of thin air. Then taxation could be understood as the only crude knob to "destroy money", and also has the effect of forcing USD to be the primary national currency (e.g. owning bitcoin won't do you any good if you ultimately need to pay taxes in USD).

IntelliJ can't cost more than the AI provider subscriptions, and it will actually handle large refactors without breaking your codebase.