I did in fact "bother to review" the tests, and I thought they were good enough to merge. Indeed, they were better than what I would have written on my own: they were lower-quality on a line-by-line basis, but for tests, quantity is a quality of its own, and I would not have bothered to write nearly as many tests. Even so, it was worth flagging that Claude wrote them.

If I trusted LLMs more, would I still feel the need to credit them? Maybe. They are improving very quickly, and I think they will eventually improve to the point where a mostly-LLM-written PR will be as good as mine in every way. Then it will no longer be necessary to apologize for submitting LLM code. But if and when that point is reached, I wouldn’t get used to taking credit for said code! Because it probably won’t be much longer until the human in the loop becomes redundant.

Going back to today, I don’t feel the need to credit LLMs when I use them for tasks that don’t involve submitting their output. In particular, they have recently become quite good at finding bugs in my code.

If the model saw signs, but “subconsciously” (below the level of reasoning traces) chose to turn a blind eye to them, out of a relentless focus on achieving the objective, then that absolutely is the model’s “fault”, i.e. a case of misalignment of the sort which will become increasingly dangerous over time.

The blog post mentions that some runs “rationalized that the real company must be part of the exercise” and to me that seems suspiciously like the latter.

Hacking can be patched with classifiers and with better sandboxes, but this is a much more general problem. Fundamentally, we need be able to trust that models will be honest with users and with themselves. This applies at some level to almost every LLM interaction.

Not that I disagree with your latter point.

To be fair, the system prompt was presumably also different from what it would be during deployment, and perhaps the model was also at a different stage of training. Without more details it’s hard to judge. But it does seem models should be able to avoid performing obviously misaligned actions – misaligned not only with the model spec, but with the user’s intent – without needing external classifiers or instructions. The only case where I’d personally let the model off the hook is if the instructions given were very badly worded, in such a way that the model could actually reasonably think that hacking HuggingFace was part of the assignment. But I doubt that’s what happened.

https://news.ycombinator.com/item?id=48856158

I may have been a little overly specific, since there are other issues besides cycles that also block splitting crates apart, but from what I’ve seen it’s very common for modules within a crate to have cyclic dependencies and therefore not be easily factorable into separate crates.

That said, you also run into related parallelism blockers without cycles. For example, the orphan rules force most trait impls for a type to be in the same crate as the type definition. Also, a module which has no source-level cycles will often have cycles after monomorphization. In this case, Rust doesn’t prevent you from splitting the code into crates like it does with source-level cycles, but you do lose most of the actual codegen parallelization unless you can switch from generics to trait objects.

Doesn’t that kind of prove the parent’s point though? In theory shared memory can do anything that threads can do. But if in practice some feature doesn’t get implemented in the multi-process design (because shared memory is hard), when it likely would have been implemented in a threaded design, then that’s still an advantage for threads.

In C, usually yes.

In C++, sometimes yes. It depends on how template-heavy the code is, but if you have some discipline you can keep most logic out of headers and thus easily splittable.

In Rust, almost always no, because of circular dependencies. You can try to work around it by adding `dyn Trait` everywhere, but that requires a lot of code changes and comes at big ergonomic costs (and a small runtime cost).

Which is why in practice, Rust compilation units are almost always larger than C++ or C compilation units. Rust can sometimes be competitive with C++ on compilation speed anyway, thanks to a smarter build system and not having to re-parse headers a billion times, but usually it's slower.

If the key has been purged but you can read RAM, then you can do two things:

1. You can extract whatever user data happens to be in RAM.

2. If you can either write RAM or reboot into your own OS, and then return the device to the unsuspecting user who will put in their password, then you can run a fake password dialog and get everything.

1 is bad, since there may be quite a lot of user data in RAM. But it’s not quite as bad as having the disk key, which gives the attacker all the data plus the future ability to decrypt or modify user data given only the physical disk. (Still, a better solution would be encrypting the hibernation image, preventing this attack entirely.)

2 is fully bad, but in many plausible scenarios (e.g. seized device) the attacker cannot just return the device to the user without them knowing something happened. Or even if they can, the method of RAM access may be one where reads are much more practical than writes, such as cold boot attacks involving physically swapping out the RAM.

I suspect $482 was the total cost for all the models, so more like 11 humans.

But still true.

That said, I’m looking forward to using Swift lifetimes once they actually work!