To your main argument: I absolutely agree that LLMs are terrible at producing working code, or efficient code, and that they're incapable of solving lots of important problems. I've seen LLMs make embarrassing typos, patch vulnerabilities with differently vulnerable code, and go through the motions of optimization as if blindfolded.

That is, however, not the central matter here. IMO, at present what management thinks LLMs are capable off has a larger impact than what LLMs are actually capable off, and while I'm hoping that the market will regulate itself once more and more slop projects break down, that'll take time, and I'm not confident that LLMs won't improve sufficiently by that point that the most glaring mistakes will be fixed. Of course, new and non-obvious issues may still be present, and they may cause trouble at a later point, but again, that won't happen immediately and I think we'll just end up in this infinite loop for the time being.

I will also add that AI companies' claims affect the general public's perception of the acceptable level of software quality. If Anthropic says 1000ms is great, and other companies follow the same approach, people will consider that the norm and not demand better software. This enshittification has started a long time ago -- just look at how bloated Windows and the web are -- and I don't think it's going to end just because we know things can work better.

Taking a step back, I think it's really a question of how many people will understand the value of what people like you and I can deliver. That number has been falling bit by bit before the LLMpocalypse, but now it's just becoming abysmal.

There are two steps to writing the program: building a model in your head to map understanding to algorithms, and then implementing the algorithm in code (and of course this can get recursive if the algorithm relies on other high-level mechanisms, like data structures).

I have found that most of the time, the kind of bugs that unit tests find are typos, i.e. mistakes in the second step; but the errors that actually cost time to resolve are errors in understanding, i.e. the first step.

They can't be found with testing or verification because what it means for code to be correct depends on the specification, and the error is that the specification itself is incorrect. Asking an LLM to check this one specific part of the software is thus useless, and whole-program analysis is not cheap enough to employ at this point.

So what about avoidance? When I say hand-written code is 100% correct (or at least approaches that number), I mean that with experience, I learn more about which models tend to be correct, and thus avoid bugs of the latter kind by construction. Of course typos still exist, which is why I write unit tests, and I expect LLMs to be able to find them as well; but I believe the only way to avoid incorrect models is to learn stuff by doing, failing and failing again, figuring out nitty-gritty low-level details, until at some point you become an expert in that area and know what to use.

The topic of the post is that in PHP and Lua (without LuaJIT), sometimes this inequality doesn't hold, and instead we get `log_a x > log_b x`, which is very incorrect and cannot be explained away by rounding.

Does that make more sense?

While answering a sibling comment, I found a paper analyzing various libms for their precision: https://homepages.loria.fr/pzimmermann/papers/glibc238-20230... (2023). I only skimmed it, but it looks like only LLVM's libm guarantees 0.5 ulp for every supported single-precision operation, and everyone else is wildly off. That's a pretty good result, though -- it means there's at least one reasonably compliant libm :)

Another sibling comments says that guaranteeing 0.5 ulp for double-precision operations is nigh impossible (and then another says it is after all). I don't have the knowledge to confirm which is true, but it's possible that this is the best we can get.

Node uses V8, which notably implements a ton of math manually, because it needs the results to be deterministic across devices. It seemingly uses LLVM's libm, which IIRC promises 0.5 ulp for most operations. (https://github.com/v8/v8/blob/f24c62fbc342d616032734b714116e...)

Go avoids dynamic linking, so they also have their own implementation. (https://github.com/golang/go/blob/543ead71a8e7acc2bd6f326327...) They only promise 1 ulp, but I guess in this specific case it works out better than approximations used by the default libm on their system by pure chance?

Wow, TIL! I was certain this is a mistake, but apparently PHP uses DynAsm (https://wiki.php.net/rfc/jit), which was developed for LuaJIT (https://luajit.org/dynasm.html). Cool stuff!

To answer your question, probably not. I dated the PHP change that added this "optimization" back to 2014 (https://github.com/php/php-src/commit/b547e1358d3846fad4cd0c...), while DynAsm only started being used around 2019 (https://wiki.php.net/rfc/jit). I think this is just convergent evolution.

(I'm putting "optimization" in quotes because it changes semantics.)

How small are we talking? We've had great success with bzip-style compression on ~400 KB (about 20% better than gzip), but I don't know if it scales well. It's also relatively fast to decode (something like 2x slower than DEFLATE, IIRC).

I was also considering other approaches, specifically GLZA (https://encode.su/threads/2427-GLZA) looks promising. I think it should be well-suited for code due to its design, and it seems to produce better results than bzip on LTCB (https://mattmahoney.net/dc/text.html), and with a faster decompression time.

> You can include the start delimiter in the string:

At first I had no clue why we thought that didn't work, in fact, my prototype had a fun commit specifically about adding opening brackets:

    -while b"]" + b"=" * level + b"]" in obj:
    +# Somewhat surprisingly, Lua forbids even opening brackets inside brackets.
    +while b"[" + b"=" * level + b"[" in obj or b"]" + b"=" * level + b"]" in obj:

    > print [[ a [[b c ]]
    stdin:1: nesting of [[...]] is deprecated near '['

> Another note that this doesn't cover is ending with a part of the ending terminator.

That's very useful to know, thanks!

> So you can't just use the bracketed form to encode arbitrary byte sequences, [...] if you care about the exact representation of line breaks

That's right, and the post actually covers how we resolved that closer to the end. In a nutshell, we replace CRs with an escape character, and then use a bitset to denote which symbols are supposed to be CRs and which ones are literal characters. It's not quite a string literal per se, but it's rather cheap in runtime and minimizes file size.

Thanks, that's a good explanation! I understand it better now.

> What I'd actually worry most about is poisoning the prefetchers, and speculation more generally.

I didn't know prefetchers rely on address generation instructions, I thought they only tracked accessed memory. Good to know! Do you know any relevant external resources about this, by any chance?