But in my experience, to actually get where they're going quickly (as opposed to spending hours and hundreds of dollars stumbling around in the dark), coding agents generally need more interactive hand-holding than that. If you just fire off one non-interactive session and wait for it to come to a stop, the problem usually isn't fully+correctly solved at the point at the LLM decides to "finish." And if you then start another non-interactive session to continue the work, the new session will have lost the old session's state/memory/context, and so will stumble through many of the same mistakes / misapprehensions.

What you really want, for a CLI program with a "use coding agent to do X" workflow-step, is for the CLI program to play the role of a human in a temporary durable coding-agent conversation session: prompting the agent; then waiting for it to finish responding (and side-effecting); then either asking the agent itself to evaluate an "am I done yet" predicate with a constrained output syntax; or having the CLI program do its own out-of-band validation of the changes made to the shared state by the agent; where, in either case, if the agent isn't "done yet", then the workflow step must continue poking it — or prompt the human to make a decision on how to proceed (possibly involving providing direct input to the LLM, but this is not ideal; ideally the CLI "abstracts away" the need for the end-user to understand the intricacies of the conversation the program is having with the LLM. Even more ideally, the conversation just whizzes by and the human doesn't have to think about an LLM being involved at all.)

Basically, think of this not as the CLI program saying to an agent "answer me this question" or "edit this file for me", but rather, the CLI program popping open a mini "guided + 99%-of-the-time automated" TUI coding-agent micro-IDE "inside" the workflow, in about the same way that git pops open your EDITOR inside `git commit`.

Not sure how to accomplish this. Anyone have any suggestions? Are there libraries for this yet? (And how would they even work? It feels like, to do this right, there would have to be some background service that CLI software could expect to interact with via a well-known local IPC socket — similar to how e.g. the docker daemon works. But I'm unaware of any coding agent software/frameworks that expose such an IPC capability...)

A general-purpose OS is one to which you can build a stack on top of it for any use-case you can think of, and it will cope with whatever stack you lay on it about equally well, because it hasn't been forced into a particular shape where it's much better at some things but much worse at other things. A "jack of all trades, master of none" OS.

Microsoft would call all consumer and server editions of Windows "general-purpose OSes." But Windows Datacenter Edition and Windows IoT Core would be non-general-purpose OSes — the former only exists to run hypervisors/SANs, and it doesn't support "stripping off" that layer, so if you used it for anything else, that layer would always be there, bloating things up; and the latter only exists to run on embedded devices, and it doesn't support "adding back" the extra frameworks and services regular Windows has, that would be required to use it for "more" than embedded use-cases.

An OS being "tuned" for a particular substrate (what the OS is good at running on), meanwhile, has nothing to do with the OS's use-case (what can be run well on the OS.)

An analogy: each mobile OEM's spin of Android only works on that OEM's own phones, because that OEM's phones have the required hardware wired to the right SoC pins, and the Android spin ships with a BSP that defines a device tree that matches that expected wiring. Thus, those OEM Android spins are "tuned for" those phones.

But in the end, they're all just Android phones, and they can all do the same things. All of these Android spins are "general-purpose OSes." They're all made to enable you to put any Android software you like on top of them, and run it just fine. (Contrast Android spins made by industrial vendors specifically for automotive or kiosk use-cases, where a given car company or kiosk manufacturer then produces a hardware-customized-and-tuned spin of that already-appliance-purposed spin. You wouldn't use a car-infotainment Android upstream for other use-cases; you'd have to undo all the car-infotainment stuff.)

Azure Linux is exactly like a phone-OEM "tuning" of Android (and unlike a vertical-specific Android spin.) Azure Linux is also like, for another example, the vendor-specific Linux "distros" [really, tunings] that ship as (usually binary-only) images for various Single-Board Computers.

In all three cases, a "tuned" fork of an OS is still intended to run anything a user might want to run on the platform the "tuned" fork was forked off of. It exposes a general-purpose surface to the developer — just one that happens to do some of the general-purpose things you ask it to do, more performantly than a non-"tuned" OS would on the same hardware/substrate.

And, in all three cases, the "tuned" fork accomplishes that by relying on device-specific knowledge and capabilities (i.e. drivers, device-tree entries, kernel patches, etc) that have been burned into the "tuned" fork rather than upstreamed. There's still a HAL between you and that stuff; your workload doesn't need to know the "tuned" fork has been tuned. It just benefits automatically, from the OS having a deeper understanding of the hardware/substrate.

That is not a given. There are Linux distributions that run anywhere but are not general-purpose. For example, the various "immutable" Linux distros that exist solely to be used as Kubernetes nodes to host containers.

1. those assets are stored in proprietary formats that only the game code itself understands, and

2. no tool exists in the project to extract the assets from these proprietary formats into open formats, unless that tool itself exists only in source-code form in the codebase, and requires the ROM as an input to compile it (even if in the case of such a tool, the ROM is doing nothing but serving as a "key" to unlock compilation.)

Basically, if you have to prove you have your own copy of the IP in order to make their embedded copy of the IP "legible", then it's very hard to construct an evidence-based DMCA takedown order that actually makes any coherent point about the project "distributing" said IP.

That being said, shipping assets like this at all, even if you "can get away with it", is ultimately just a kind of laziness / shortcut-taking. They do it because there's either no clear/simple/obvious way to automatically extract the given asset data from the ROM (e.g. because the relevant data is split up into various data planes + metadata bits that are stored "exploded" all over the ROM), so they just did it once by hand, committing the results; or because there's no clear/simple/obvious way to store the extracted asset data such that a regular compiler/assembler natively understands how to embed it into the binary in the particular form it was found in the original ROM. (Remember, re-assembling/compiling to the original ROM is always the test these projects use to ensure their disassembly/decompilation is preserving semantics. So they need to replicate every weird layout quirk the original dev tooling imposed upon the original ROM. And sometimes the original dev tooling included special-purpose domain-specific asset-codegen tools that aren't part of regular compiler toolchains.)

What these projects should actually be doing, is taking on the schlep: writing the extract tooling anyway, even if it's just "copy these bytes from here and these bytes from there, and spit them out as hex in an .asm file with this header"; and/or writing matching asset-codegen tooling to the tooling that likely existed in the platform SDK, to run before compile/assemble time, converting the extracted ROM asset files into a form (probably a bunch of little assembly files) that will land in the right places when linked back together to form the original ROM.

And, to be clear, they mostly do do this! These projects are very good at doing this!

But sometimes — especially on a larger project with many contributors — one or two things like this aren't audited properly, and fall through the cracks. Or they start out as temporary "bootstrap" approaches made during a private phase of development to get things working + compiling to a correct image; and then not all of those get cleaned up before the repo gets made public.

To me, LLMs are for asking research questions + exploring design spaces + pointing at codebases to investigate bugs. And those all benefit from the model being as "smart" (in terms of both fluid intelligence and burned-in knowledge) as possible.

I'm guessing there exist problems where "intelligence past a certain point" doesn't matter, so these medium-sized models can match the performance of the bigger models. But what problems might those be?

> The memory is not as fast as dedicated GPU memory, but it is cheap enough while delivering enough bandwidth to run AI models locally.

So, the reason "dedicated GPU memory" is fast, isn't because it's "dedicated"; it's because the types of memory built into GPU cards — GDDR and HBM — are designed for throughput over latency.

Which is to say, GDDR and HBM memory could be shared with the CPU in UMA while still being "fast" (for GPU use-cases.) In fact, the PS4/5 and Xbox 360 / One X / Series consoles have UMA architectures that use GDDR memory as their main memory, with no regular DDR memory to be found.

What I don't understand: why don't we see UMA architectures where there's both regular DDR and GDDR/HBM memory mapped into the address space of the CPU+GPU? That seems like the best of both worlds: you'd have some memory that's "tuned" for random-access CPU usage (regular DDR), and some memory that's "tuned" for streaming GPU usage (GDDR/HBM), but either type of memory can still be put to the use it wasn't "tuned" for, just with slightly-worse performance.

I guess you'd need to do a bit of software work:

1. a bit of work in the OS kernel / malloc library to get CPU workloads to "prefer" allocating DDR memory over the GDDR/HBM memory until they've exhausted DDR memory (or maybe not, if you just tell the kernel the GDDR/HBM memory is something like a zswap thinpool);

2. and a bit of work in supported ML frameworks, to teach them about a hybrid strategy between UMA "allocate anywhere, it's all the same" and NUMA "keep assets in VRAM if possible; if you spill assets to RAM, then they must stream into VRAM on access" (i.e. "at allocation time, allocate as if the system were NUMA, VRAM first then spilling to RAM; but at execution time, use the UMA codepaths, no need to copy RAM into VRAM.")

...but once that's done, it's done.

(And yes, until TI's recent move, that was true of the 5532. All the other vendors' 5532s had matching datasheet specs, including the 22V max input voltage. Because a design that was built for "a" 5532 was usually built to run it up to 100%; and that a vendor couldn't offer their part as a swap-in if it couldn't do that.)

But now, if your purchasing department (or the supplier they purchase from) happens to order TI 5532s — or if the warehouse they're sourcing from has comingled any TI 5532s into the general 5532 stock — then your product is now broken, with no real recourse except to change your entirely supply chain to one that specifically excludes TI.

- the hardware you need for a production use-case is relatively small, because production {models, bitstreams} have been heavily size-optimized, stripping out everything not needed to get a good result for the target use-cases

- but the hardware you need when tinkering/learning how to design {compute kernels, IP blocks} in the first place, must be quite a bit more powerful / higher-capacity, because your experiments will intentionally be the opposite of optimized: they'll be built for legibility / introspectability / debuggability at every level, which massively inflates and de-optimizes the resulting {model, bitstream}.

(And, to be clear here, "running someone else's finished model, which was designed and optimized to be used on something like a 4090, against your own prompt" is a kind of experimenting, which is cheap, in the same way that "deploying someone else's pre-baked FPGA bitstream, that was designed and synthesized for a $20 target FPGA, onto your own instance of that $20 FPGA, and then feeding your own input signals to it" is cheap. But that's not the kind of experimenting you'd be doing in this course while learning to design your own models!)

1. Running games that only ran at 10FPS on original hardware at a smooth 60+FPS, by calling the game's own rendering logic more frequently than the original hardware could "afford" to, but without breaking game logic (i.e. by forcibly decoupling the game's physics ticks from its presentation ticks);

2. Using out-of-viewport but in-{tile/frame}buffer data to expand the viewport to fill my screen (which can be very janky under some rendering paradigms, due to offscreen parts of tile/frame buffers being dynamically partial-updated with a loading seam; but which can work very well under other rendering paradigms, like the SNES's mode 7 where the tilemap was usually just fully populated once at mode-switch time);

3. Making games that used vector-graphics for at least part of their display, and soft- or hard-rasterized those vector graphics into the native low-resolution framebuffer, instead rasterize those graphics at my display's native resolution;

4. having the emulator recognize particular bitmap assets (tiles, sprites, 3D meshes/textures) the game is telling it to render, and swap these out for hand-crafted HiDPI / high-poly versions of those assets from an asset-pack file (as opposed to relying on the caprices of a DLSS-like upscaling model.)

Mind you, to have features like this work well, they often leave the realm of "interpreting the control-register pokes from the game differently", and enter the realm of "the game being patched to take advantage of the capabilities of the emulator." Then, as with these GOG games, you're no longer just shipping a ROM "and an emulator configured to run it well"; rather, you're shipping a co-designed product: an emulator tuned to run that ROM, and a ROM tuned to run in that emulator.

---

By doing this, you technically leave the realm that MAME-like "archival preservation" emulation usually aims for, of "faithful emulation" of both a game's logic and its presentation.

However! "Faithful emulation" folks shouldn't despair. The nice thing about this technique, is that this is all done by wrapping the original ROM in an emulator + shipping runtime-applied IPS patches.

In other words, the original game ROM is still there, unmodified, under an "isolation layer"; and everything being done to modify it is done using "reversible, conservation-grade" techniques.

Which means the emulator can provide a launcher UI to turn any of those presentation "enhancement" features on-and-off. If you're the "faithful emulation" type, you can just turn them off!

(And, under this paradigm, even with the "enhanced emulation" features on, the game logic is still preserved as-is; you're only modifying the presentation. The original game engine is still running; the original instructions are still executing cycle-accurately to how they should. So the "game feel" is preserved perfectly. If you were good at the original game, you'll still be good at playing an "enhanced emulation" of the game; nothing will be "off" about it. Even input movies recorded against the un-enhanced game should replay unmodified against the enhanced game!)

Contrast this to the average "HD remaster", where the game is at the very least recompiled for a new platform (with different timing guarantees), if not entirely rewritten atop a new engine. In that process, there's no "isolation layer"; no way to guarantee a preservation of any part of the original game logic in the remastered artifact. And like George Lucas, game developers coming back to their own works 20–40 years later, just can't help but want to tweak things. So these HD remasters end up breaking "game feel" in all sorts of ways.

I.e. nothing about this statement prevents Anthropic from running ads within Claude, as long as they run the ad-placement auctions themselves, and so aren't leaking any of the data they're using to decide which placements are relevant to which users+sessions. (This is the same thing Google does for SERP ad auctions.)

But actually, and perhaps more interestingly, nothing about this statement prevents Anthropic from building a Google AdSense competitor either. Other sites (or mobile apps, etc) could plop in an Anthropic ad iframe; and it'd be Anthropic's knowledge of your interactions with Claude that would drive what ads would show up in that iframe. The embedding site doesn't know what ads the users are seeing, so that's still not "selling users' data to third parties", per se.

But I assume that any kind of incremental linking, is mutually exclusive with link-time optimization? I.e. you'd never want to use this option for a release build?

True enough, in theory; but what exactly are you imagining would be a useful-enough signal in the OpenRouter request+response stream, that any company would want their data as training material?

Even a single OpenRouter-API-key-identified subscriber's traffic, may consist of an mixture of traffic from multiple different sessions, under potentially multiple different end-users. (Where, if the subscriber is doing security correctly, then their OpenRouter key lives on a gateway rather than in a frontend app; and so the only IP address / UA / etc OpenRouter sees is that of the gateway itself.)

And the traffic stream may also invoke multiple models, and provide multiple different system prompts for those models; which, while marked in the traffic (i.e. conveyed as part of each request), makes the resulting data much less useful in aggregate, than if it were all training data for one model with one system prompt.

Plus, there are no RLHF signals in OpenRouter data. Even if OpenRouter wanted to build a general model-neutral framework for collecting RLHF-type data, it can't force subscriber apps to do the UI-level stuff necessary to collect it (i.e. the things ChatGPT/Claude do, with "thumbs-down" buttons, A/B tested responses, etc.) Analysis would have to rely on pure transcript-level user sentiment extraction.

By the conventional use of the term, a "sucker" is always a sucker; suckers suck constitutionally.

But a professional gambler in a skill-based game (e.g. poker), is only going to lose money on net, if they happen to be playing against people with "higher ELO" than them.

And in the case of a prediction market, the "ELO" isn't absolute; people's expertise "rankings" are relative to each particular question. There's no "general factor of expertise" that makes someone able to beat the odds on every question. Each question forms its own market "niche", where only people with expertise will be interested in participating; and so each such niche is to some degree illiquid, with not enough trades to make an efficient market (i.e. the kind you wouldn't expect to find a $20 bill on the ground in.)

To be more concrete: while there are "specialists" (insiders, but also ordinary experts in hyper-specialized verticals) who might clean up by betting on the things they know a lot about, they'll generally be miscalibrated as overconfident on the things outside their specialty (see: any scientist who got famous for their research and now writes pop-science books about topics they know very little about, often making incorrect statements), and so will lose out vs "generalists" who can't successfully make the in-domain bets the specialists make, but who are better-calibrated on multiple topics (or on particular odd intersections of topics) because they spend less time hyperfocused on one niche, and more time flitting between various niches.

Which is to say: there's no "house edge" here to lose against. In a prediction market, everyone's going to be the "shark" for some questions and the "sucker" for other questions. Every question is its own game, and every game has an edge, but with that edge going to a different party. If you actually know what you know, then you can identify which questions you have the edge for (probably a finite number), answer only those, and make some (very small) amount of money. You may lose sometimes because someone knew even better than you (esp. for questions that go beyond yes-or-no, where there are 3+ mutually-exclusive prediction-categories you can buy into, such that others might "hit the bullseye" while you just "hit the ring"); but on average, if you stick to your "field of pre-eminent expertise" (presuming you have such), you would make a small positive gain over time.

That being said, anyone without a "field of pre-eminent expertise", who thinks they can place correct bets purely by being rational + doing the level of research one can accomplish using public Internet sources, is 100% a sucker, yes.

In fact, even search engines were originally optional; various people and organizations used to simply maintain huge static directory service websites, simply listing out "everything else on the web" (or more often, just the parts the author thinks are cool.)

And the people they're both making money from, are people who think they have enough expertise + exposure to function as superforecasters — and who probably could function as superforecasters, in a market with fewer "sharks" in the pool — but who lose out simply because they were slightly less well-calibrated than whoever they were trading with.

Which is to say: prediction markets can still work and be worthwhile to participate in, even if everyone in them is rational. They don't require suckers.

But, in practice, they certainly do seem to attract them.