The agent really, really struggled to bridge the gap between the code and the actual business rules it was meant to be modelling. It also struggled to work out which functions at which layer were appropriate to write tests for. So, its tests tended to be very brittle to changes to domain logic.

Essentially, agents have always seemed to struggle with modularity and problem decomposition. Good testing is about finding the right things to test, which means working out how to subdivide the input state into an appropriate product state, and checking each behaviour independently. IMO, this is the most difficult and complicated thing about programming, so I won't say it struggled _more_ than a human would --- but humans have the advantage of being able to sleep on it?

One minor thing I observed was that it tended to get really hung up on floating point edge cases (NaNs, infinities). Maybe floating point edge cases are over-represented in the property-testing training data, but it's essentially irrelevant for my usecase, at least as far as the business rules go.

I find it incredible when the concepts in a system fall into place and start lining up and you get these unexpected relationships that "just work". It's beautiful in the same way mathematics is beautiful. It's lovely when something is made of lots of pieces at first and then suddenly collapse and simplify into a single structure.

I also enjoy other bits of programming. I like the flow state, I like the act of creation, I like feeling like I have an impact, I like being paid. So it's not like people fall into one category or another, and it's not like programming is one single thing. There's room within any discipline for creativity and for art.

I think many professional artists probably spend much of their time problem-solving, too --- authors trying to fit plot points together, sculptors working out which materials will work with which ones.

But taking it seriously, a word like "salad" is best understood, IMO, as a family resemblance [0]: a group of inconsistently overlapping properties. Or, you could understand it functionally/pragmatically (a salad is what I order when I want a salad), where the meaning is not derived from an internal property but from the social system that produced it.

For salad or sandwich, it's all rather unimportant, naturally. But I do think taking this step back and questioning what it even means for an utterance to have meaning can help dissolve some serious and otherwise contentious philosophical issues. ("Intelligence" seems to be a rather relevant example.)

Mathematical definition is enchanting to those of us with an inclination towards abstraction. But just because it's a noun doesn't mean it's a thing --- or at least, not a thing independently of the speaker and their community of speakers.

[0] https://en.wikipedia.org/wiki/Family_resemblance

This is not VCs being evil and corrupting a pristine engineer- or hacker-defined concept of true value. The VCs are all following the rules and are trying to make money off of risky investments (it's "venture" capital, after all). But the incentive structure just forces the market into either an oligopoly of largely extractive services (or into everything being free: that's why open source is also a stable point for software).

My hope is that in time, basic software services, like for communication, socialising, community hosting, and so on, will eventually become seen as core social infrastructure. I don't really think this can happen via existing institutions, even open source, because the fixed costs of making software are really high. You really need _tax_ to support this. But it's very difficult to do because the internet cuts across borders.

It's quite sad to think this experience might be on its way out, and it really highlights that "junior problem" from AI. There is just no way around diving in to get an intuition for things. Naur warned us of this 40 years ago but the lesson never seems to get learned.

By all means, put pressure on companies who are doing this. Public opinion does matter. But the fundamentals are just not going to change without regulation. It is a tough nut to crack.

If you want my opinion on that, Odin seems pretty notable to me, and I would expect there to be a Wikipedia article for it, and I'd be all for engaging with the editors on that. I wouldn't expect Wikipedia moderators to be plugged into the world of programming languages the way I am, though. Like the article says, this seems like subject-specific notability guidelines could help. It's definitely true that the programming world is missing the kind of institutional material that works well for, say, science or news.

So often, this attitude is couched as a desire for truth and objectivity. Do you know what somebody does when they _actually_ love the truth? They work hard to find it, they examine their own assumptions, they try to build systems that extract truth from an unbelievably complex world of unreliable narrators. And most importantly, they are curious: for example, curious about how other communities operate and what they can learn from them.

It is the hypocrisy that I find unbearable. The language of truth and objectivity wielded to win arguments which are fundamentally emotional in nature. There is nothing wrong with being frustrated that a community dear and significant to you, clearly notable, seems to be overlooked by a figure of (informational) authority. Let's take that frustration and work together to improve our truth-seeking institutions.

Once I started looking for it, I started noticing some of my colleagues' abilities to navigate their way through meetings and disagreements and getting their way without things even becoming arguments, the ability of some people to just put a room at ease or to communicate. And the inability of others to do so.

"Politics" is an unsavory term for somebody with engineering predilections. It evokes backhandedness and self-interest. But you can't actually both be a human being and be "above it". Every sentence, no matter how concrete and technical, is a social act with social ramifications.

I've come to have a lot of respect for people who are actually able to lead and make things happen. It is not a skill I am terribly practiced with.

It can be frustrating. The AI pretends to be a human, and so a part of my brain expects them to commit and have a "parti pris" like a human, so the exercise is a good reminder of the feedback loop. My mental model is that before the first three or four messages, the model has many finer points of its personality still underdetermined. I'd suggest that as the mechanism for "role-based prompting". And it explains the "savant sleeper agent" thing you describe. You want to get the state in the right attractor on the manifold.

These machines are pretty incredible, but for conversation-driven workflows you really have to be in the driver's seat. A human has a property that the AI does not have, at least under current architectures: we are regulated by the outside world. A bit of a tangent, but I can see how AI psychosis arises from these dynamics.

Thank you for that line --- I may steal it :)