What is all this code being used for? I'm 100% sure it's not going into their harness which has been stagnant.

  > Engineers are no less valuable, but their work is changing. All of the engineers who worked on this system would now rate Claude as better at writing code than them. But our engineers still supplied the invaluable understanding of the high-level inter-system components and the design-level tradeoffs, making informed choices around the problems we wanted to solve and the scale we knew we needed to achieve. Very often, we found that the role of the engineers was to tell Claude “no.” Agents will sometimes violate core system constraints to satisfy a pattern that should never have been allowed in the first place.

It's incredibly short-sighted to acknowledge that the baby sitters haf to tell Claude no and still not consider what subtle and nasty bugs managed to slip through. Well I guess we'll find out soon enough if that's the case. I'm betting when that's the case.

This is the most unexpected bit.

Circumstances made it so I got a new computer right around the time srht put out their anti-LLM policy. So I immediately started setting up a forgejo instance on my home network. With the latest waves of attacks on he AUR, I decided to keep that machine completely free from all AUR packages or helpers. So I also set up a package registry through the forgejo instance. With some work I have a workflow that takes a PKGBUILD and publishes an arch package that I can just `pacman - Syu` to install. Obviously I only put my own pkgbuilds and the few I have manually vetted there.

Even after manual vetting I wanted to lock things down as much as possible on the CI. So. I went with rootless podman, and some patching of arch's build infra just to get it to be able to run `pkgctl build`.

One of the main thing I looked out for was making sure no credentials were available anywhere during the jobs where the pkgbuilds (which can contain arbitrary commands) are being executed. I'm sure srht has to consider many more scenarios than I do. That will definitely make it harder to wall off credentials access. But it is still surprising that deploy keys are left laying around.

BTW this is not an attack on srht. I stopped actively using it because of their new policy. I still have a paid subscription that I do not plan to cancel anytime soon. I still have my public projects where I don't use LLMs there. I think opinionated maintainers and projects (Linus/linux, Poettering/systemd and Drew come to mind) are a net good for open source. Opinionated people make better software because they care imo. And those opinions don't have to be restricted to purely technical matters.

But anyway, it's a small team and there is a lot to cover, but they should get those leaking keys under control as much as possible.

Presumably everyone in the chain would have their own idea of what the next word/token should be. They would each try to push it in that direction using their only lever, that single token. None of that guarantees that the output is syntactically correct, nor grammatically correct. Note that LLMs at least by the way they draw their tokens have that pretty much guaranteed. So that would be a regression even from what LLMs are capable of right now. But beyond that, even if it ends up being a correct sentence, it would not be a result of intelligent thought. Even if every agent in the process was intelligent, the process is not itself a use of that intelligence. Human beings can be part of purely mechanical processes, that doesn't make the mechanisms suddenly an exhibition of intelligent thought.

This also applies to society and human history itself as processes: “History is made in such a way that the final result always arises from conflicts between many individual wills, of which each in turn has been made what it is by a host of particular conditions of life. Thus there are innumerable intersecting forces, an infinite series of parallelograms of forces which give rise to one resultant — the historical event. This may again itself be viewed as the product of a power which works as a whole unconsciously and without volition. For what each individual wills is obstructed by everyone else, and what emerges is something that no one willed. Thus history has proceeded hitherto in the manner of a natural process and is essentially subject to the same laws of motion. But from the fact that the wills of individuals — each of whom desires what he is impelled to by his physical constitution and external, in the last resort economic, circumstances (either his own personal circumstances or those of society in general) — do not attain what they want, but are merged into an aggregate mean, a common resultant, it must not be concluded that they are equal to zero. On the contrary, each contributes to the resultant and is to this extent included in it."

https://www.marxists.org/archive/marx/works/1890/letters/90_...