Now, say you don't want to sign a pre-committed enterprise contract with Anthropic. But oh, you already have such a contract with AWS, and they'll let you use any model you want, and they've implemented KYC and will graciously connect you with a solutions partner who can help you with the IAM systems integrations for key tracking and attribution.

Oh, and all these enterprise contracts will bill by token. We're not talking a small stake in a company selling subscriptions, we're talking immediate revenue at four-figure-per-user levels, and pushing more and more companies to see that as "just part of their AWS bill."

This is worth a significant amount of money to AWS. So the question is: does Hanlon's Razor apply when a $2.5 trillion company is putting its best minds into how to engineer strategic outcomes?

The regulatory capture angle here is, if anything, an implementation detail.

This subthread there is a fascinating explainer about one user's journey into funding and incentivizing research into their own rare form of blood cancer, and how they are able to push forward the state of the art: https://news.ycombinator.com/item?id=48506997 - something of a modern-day (and more accurate) Lorenzo's Oil!

Certainly, one could tamper with the hardware, but could one do it in a way that wouldn't get that machine immediately flagged, removed from the routing pool, and told to wipe its memory immediately, by a watchtower (perhaps even the routing layer itself) that runs in a separate secure Apple datacenter?

I imagine this is part of the impetus behind the Bun acquisition - they have a deep need to push optimization efforts towards the specific patterns that are most relevant to their use cases. (Which are probably good ones for the broader Bun userbase, to be sure, but relative prioritization is something they now have greater control over.)

While this might seem to be true for casual users, I recall that one of the reasons for Anthropic's recent changes for only retaining KV cache for an hour or so, was that many users just have one massive ongoing session that they continue on with multiple unrelated queries (as one would in a single-thread "group chat"). And this is hard to distinguish from someone who wants that context for their seemingly-unrelated query to apply tone etc.

So in practice, there are many casual users who are typing their Google-esque searches against a 100k+ token context window - and it's at that point where things balloon into 300GB+ KV caches to maintain.

I wouldn't be surprised if we see new UX's around subsidized plans starting to encourage resetting the context window more often.

(Some for different aspects of full stack features, some for managing specific client situations advised by the codebase and its tools!)

The CLI is not designed to be lightweight, and it’s easy to get into situations where every CLI session consumes multiple GB of memory alone - stack them up and it’s a lot!

And not all terminal GUIs handle multiple tabs well enough to see all sessions at a glance.

So on top of the plugin features, desktop is a really useful thing to have!

https://www.ici.org/research/stats/combined_active_index_042...

It seems entirely reasonable to say: "if we make a certain decision, we correlate both our reputation and a nontrivial portion of the U.S. economy with the whims of one of the most volatile personalities in industry, and we should likely pay attention to this trial balloon that shows such anticipatory fear of the decision that we might lose our reputation as an index altogether."

(Along those lines, I recall lots of this getting messy in a pre-LLM project the moment someone said "merge these two CRM accounts and their histories, but oh whoops turns out they were different all along, and only some of the updates should have applied" - there's a whole set of interesting challenges around attributing EAV when the very notion of object identity evolves over time. Whether a fact is relevant is really a judgment that can only be made with full context - but we now have tools that eat context for breakfast!)

I've always thought that knowledge graphs/expert systems, and even the broader concept of entity-attribute-value storage, got an unfairly bad reputation because of the 1970s/1980s "AI Winter."

And I think that perhaps this reputation is why so much of the oxygen in the RAG space has been consumed by the notion that "RAG = retrieval of fragments by vector similarity."

The difference now from decades ago, of course, is that now LLMs can do both the job of maintaining that graph at scale, and being able to agentically run successive queries to explore for best practices in any situation! And these have reached the scalability where any small business can build and use their own expert system.

I really want to see this approach win, because I think there's such an opportunity to explore even more data structures and approaches from the past and how their impact can be reimagined. If LLMs do indeed approach AGI, it will be in large part due to the ability to use tools (there's some evolutionary irony there, too) - and we should be trying every kind of underlying storage for those tools that we can, standing on the shoulders of giants.

(And curious what database you use for the knowledge graph - those are also a place where we stand on the shoulders of giants!)

It could require a 48 hour cooldown period on any package update that wants to add an install script that didn't have one before, and has a certain number of downloads. And it could publish the list of these so security researchers have an opportunity to scan them.

It could add an optional key to package.json that allows someone to whitelist which packages can run install scripts.

It could add a Hardened Security program where (1) package maintainers could opt into a program where multi-factor confirmation by maintainers is required on every publish, even those triggered by CI; (2) this hardened package status would be public, and (3) a developer could set a flag in their package.json that causes any npm action to act as if all non-hardened packages had frozen versions.

And so much more.

But these very endowments have been special cased as additionally taxable, despite that status, under the 2025 OBBBA, resulting in research budget cuts [0].

Would independent endowments as you describe them be more immune?

[0] https://www.pbs.org/newshour/education/college-endowment-tax...

And part of my reasoning for this is: the only system capable of actually fixing bugs in vibe-created code is an LLM. If we humans couldn't write it without assistance, we certainly won't be able to debug it without assistance. So there's a real stickiness here.

We're signing pacts with demons - we have to, if we want to outcompete the other warlocks - and those pacts are written in the very size of our codebases.

And context can be extremely tailored to your niche: specific inventory, from a specific supplier, for a specific user of a specific B2B client of a specific business model subtype, who should or shouldn’t see certain features on that specific inventory at certain times.

When you can write your own logic, and just run this in a tight loop as easily and performantly as you can use a constant, it makes your business incredibly agile. Think some text might change for some customers? Just write the code to make it configurable, and you get tests and flags for free.

Sadly, that zero-hop setup requires a sophisticated client execution engine, which it doesn’t appear Cloudflare has implemented here. Makes sense for their memory constrained workers, less sense for traditional infrastructure.

Statsig has an approach here that I quite like:

> To be able to do this, Server SDKs hold the entire ruleset of your project in memory - a representation of each gate or experiment in JSON. On client SDKs, we evaluate all of the gates/experiments when you call initialize - on our servers.

https://docs.statsig.com/sdks/how-evaluation-works

You can also roll your own - just sync your rulesets to a few data structures every few seconds in a background thread and atomically swap the reference to them. Then you just need a CRUD interface over the applicability ruleset dimensions.

Just be careful to have governance on who can play with which would-be constants. Great power and great responsibility and all that!