66 E9 is not a practical compatibility problem, though: it’s not a useful encoding of a useful instruction on any CPU :)

This is not to say that a human couldn't understand a streamlined version or that the AI would not be better if it made more streamlined statements to begin with.

(I am not saying that everything mathematical that an AI produces is in any sense trivial.)

Sure, this is messy. But, critically, on x86, these are all modes, and any CPU that supports them makes them detectable and supports them in the same way. If you run long mode code outside long mode, some opcodes will be interpreted as the wrong instruction. But you will not find multiple different CPUs that decode valid instructions differently. If I run your weird old x86 code, either it will run correctly or it will fault.

Oh, and all these modes are older than RISC-V. To the extent that there are lessons to be learned, RISC-V should have learned them.

The fact that you can apparently find two RISC-V CPUs that decode some ordinary user mode instructions based on published standards as entirely different operations is bizarre, to say the least. The fact that the relevant CPU features can’t even be enumerated in user mode just makes it worse.

(There are edge cases in x86. Some invalid opcodes have different lengths on different vendors’ CPUs. This is not a problem in practice because, one way or another, they fault. There was also a little glitch in the 64-bit design where some really really old x87 FPU code that uses exceptions cannot be corrected handled by a kernel on a modern CPU.)

Also, the syslog daemon should be extremely reliable, and throwing giant table scans at it makes this more complex.

That’s quite inefficient. Get 2x RTX 6000 Pro and run DeepSeek v4 Flash for the whole company. This may or may not be cost effective compared to cheap third party providers, but it lets you own your own data, is not actually that expensive, and reports suggest that you can get 2k tok/s or so from a setup like this (at respectable batch size, but that’s exactly what you want in this situation).

I’m not sure I believe that the unquantized performance is as good as people are claiming, but even light quantization would make it fit easily.

A long time ago I had this over-optimistic idea: x86 hardware (and probably most other hardware) has these cool hardware-managed dirty page bits. So you would write to a mapped page, not even take a page fault, and the hardware would record that it's dirty. Later on the kernel would notice and flush. Excellent performance.

Hahaha. It's much much much more complex. For various reasons (maybe good, maybe bad -- see below), Linux barely uses the real hardware dirty bit. Instead, when you map a file as shared-writable, at first it might not really be mapped at all. If you read it, it gets faulted in and becomes readable. When you first try to write to it, a page fault is generated, and, on non-FRED x86, the page fault itself is very slow. The kernel will do things, including calling into the FS and updating atime [0], to make the page logically writable. It updates the page tables so that the CPU knows it's writable, and it sets the dirty bit right then (after all, this is a bit faster than letting the CPU set it immediately thereafter when you retry the faulting write).

Okay, now it's writable. Writes are essentially free until the kernel decides to write the data back to the disk. The kernel will mark the page non-writable (because it wants to get notified the next time you try to write to it) and flush the TLB (which is extremely expensive, especially on x86 systems that aren't the latest AMD CPUs). And it will write the page back, more or less as if you had used normal syscalls to write it.

There's more fun, though. Some filesystems and/or backing stores need "stable pages" -- they need the page cache pages that are being written to not be modified while being written back. btrfs, for example, wants to checksum the data and then write the data and the checksum out consistently, and if something changes the data while it's being DMAed, then this can't happen. So special locks might be taken to delay future writes to the page until writeback is done, and that includes blocking the "make writable" page fault handler. Oops, there goes performance.

Could the kernel do better? Probably. Will it? Unlikely in the near future. I've contemplated a special mechanism to map a "fast write" window onto a file that would be permanently writable and use the hardware dirty bit to tell the kernel when to transfer the data out. Even if anyone ever implemented this, it would be a very specialized thing, it would incur polling overhead, and it would be utterly silly to use it for something like syslog.

Just use pwrite or io_uring unless you have actual evidence that mmap is better.

mmap read is a different story, of course.

[0] I think that updating atime at make-writable time instead of at writeback time is both non-performant and semantically incorrect. I've never convinced the maintainers well enough, though.

With SQLite you’re looking at third party extensions that compress the data, still in row-oriented format, and might rather inefficiently recover some benefit. But WAL probably does help with the write amplification above and beyond this.

journald-style logs really want a column store IMO. It would be highly entertaining to try something like ducklake with SQLite as the catalog — the whole stack is pretty lightweight and there’s support for inlining inserts in the catalog to avoid creating silly numbers of Parquet files.

SQLite gets the last one but misses on the first two (although WAL and the improved read-only support in 3.20+ mostly gets #1). DuckDB might be decent except that you would need to connect through the daemon to read if the daemon is running. If a daemon that coordinates everything is okay, something like Clickhouse might work.

An LSM-style layer over Parquet gets all of this fairly naturally as long as readers using third party tools understand the LSM scheme. (In general there is a lack of consensus as to exactly how to correctly and efficiently use multiple Parquet files together.)

(IANAL. In the US this seems to largely be a state law issue. California’s law, to my quick non-expert skimming, is really quite clear on this point.)

And, in fact, there is a concept called an “insurable interest” that is intended to prevent this kind of thing.

If I buy an insurance contract that will pay me if your house burns down and then I burn down your house, then I’ve obviously committed arson, but I have also likely purchased that insurance contract illegally. And I don’t even need to burn down your house for that contract to be illegal.

(IANAL)

Prompt caching isn’t about caching the literal text of the prompt. It’s about caching the result of running prefill on the prompt (or, equivalently, the result of generating the prompt one token at a time by autoregressive inference, or some combination of the above in the case of speculative decoding). This is often called the “KV” cache, and it is very model-specific.