If you're willing to incur a latency penalty on a "cold resume" (which is fine for most use-cases), why couldn't they just move it to disk. The size of the KV cache should scale on the order of something like (context_length * n_layers * residual_length). I think for a standard V3-MoE model at 1M token length, this should be on the order of 100G at FP16? And you can surely play tricks with KV compression (e.g. the recent TurboQuant paper). It doesn't seem like an outrageous amount of data to put onto cheap scratch HDD (and it doesn't grow indefinitely since really old conversations can be discarded).

Why can't they save the kv cache to disk then later reload it to memory?

But why does the economy need to grow? If you can manufacture everything you need, and you have access to the raw resources, what else do you need as a country. In what sense is growing your economy with VC scams like Juicero better than actually having industrial output?

[1] https://news.ycombinator.com/item?id=43881692 [2] https://machinelearning.apple.com/research/neural-engine-tra...

They are handled as part of the "conceptual" run loop, but they seem to be dispatched internally by AXRuntime library from a callback off some mach port. And because of this, the call to nextEventMatchingEventMask in the main -[NSApplication run] loop never even sees any such NSEvent.

    -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:]  (in AppKit)
        _DPSNextEvent  (in AppKit)
          _BlockUntilNextEventMatchingListInModeWithFilter  (in HIToolbox)
            ReceiveNextEventCommon  (in HIToolbox)
              RunCurrentEventLoopInMode  (in HIToolbox)
                CFRunLoopRunSpecific  (in CoreFoundation)
                  __CFRunLoopRun  (in CoreFoundation)
                    __CFRunLoopDoSource1  (in CoreFoundation)
                      __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__  (in CoreFoundation)
                        mshMIGPerform  (in HIServices)
                          _XPerformAction  (in HIServices)
                            _AXXMIGPerformAction  (in HIServices)

[1] https://www.cocoawithlove.com/2009/01/demystifying-nsapplica...

Do you mean that you can just AXUIElementPerformAction once you have a reference to it and the OS will internally synthesize the right type of event, even if it's not in the foreground?

There is also this old blog post by Yegge [1] which mentions `AXUIElementPostKeyboardEvent` but there were plenty of bugs with that, and I haven't seen anyone else build on it. I guess the modern equivalent is `CGEventPostToPSN`/`CGEventPostToPid`. I guess it's a good candidate though, perhaps the Sky team they acquired knows the right private APIs to use to get this working.

Edit: The thread at [2] also has some interesting tidbits, such as Automator.app having "Watch Me Do" which can also do this, and a CLI tool that claims to use the CGEventPostToPid API [3]. Maybe there's more ways to do it than I realized.

[1] https://steve-yegge.blogspot.com/2008/04/settling-osx-focus-... [2] https://www.macscripter.net/t/keystroke-to-background-app-as... [3] https://github.com/socsieng/sendkeys

How does that even work technically? macOS doesn't support multiple cursors. On native Cocoa apps you can pass input to a window without raising via command+click so possibly they synthesized those events, but fewer and fewer apps support that these days. And AppleScript is basically dead, so they can't be using that either.

I also read they acquired the Sky team (who I think were former Apple employees). No wonder they were able to pull of something so slick.

Is thinking of it in this sense actually accurate? I always assumed since every government has embraced MMT they can spend whatever they want simply by printing it out of thin air. Then taxation could be understood as the only crude knob to "destroy money", and also has the effect of forcing USD to be the primary national currency (e.g. owning bitcoin won't do you any good if you ultimately need to pay taxes in USD).

IntelliJ can't cost more than the AI provider subscriptions, and it will actually handle large refactors without breaking your codebase.

It's mainly due to pressure from above. People who want to do a good job and are allowed the time to will be fastidious with or without AI. But now AI provides a shortcut and band-aid where things can be papered over or products can be launched quickly. Ship fast and then iterate" doesn't work when you're building on shaky foundations, but good luck convincing people of that.

This collapse in variety matches with what I've seen some studies show that "sloppification" is not present in the base model, and is only introduced during the RL phase.

This is my understanding as well. What most people seem to ultimately be debating is "dedicated tool calls" (which is what MCP boils down to) versus a stateful environment that admits a single uber-tool (bash) that can compose things via scripting.

I guess this is what riles people up, like emacs vs vim. Some people see perfectly good CLI tools lying around and don't see why they need to basically reimplement a client against API. Others closer to the API provider side imagine it cleaner to expose a tailored slim-down surface. Devs that just use claude code on a laptop think anything other than CLI orchestration is overcomplicating it, while others on the enterprise side need a more fine-grain permission model and don't want to spin up an entire sandbox env just to run bash.

It's also not either or. You can can "compose" regular tool calls as well, even without something as heavy weight as an entire linux env. For instance you could have all tools exposed as FFI in QuickJS or something. The agent can invoke and compose tools by writing and executing JS programs. How well this works depends on the post-training of the model though, if agents are RL'd to emit individual tool calls via

    {"myTool": {"arg1": 1}}
    {"myTool": {"arg1": 2}}

   
      const resp1 = myTool(1);
      const resp2 = myTool(2);
      console.log(resp1, resp2);
   

Comments URL: https://news.ycombinator.com/item?id=47733971

Points: 236

# Comments: 177