A few years ago I started leading a team working on a product that had a lot of „legacy” quirks like that. It took me a good while to set a strong quality bar, both mentally and technically. I would frequently point out where the mindset was wrong, prioritize certain fixes over project work, fix a lot of issues myself, fight with other teams to fix their stuff, run silly contests such as „who solves the most support tickets this month gets a prize” and sometimes even „stop the world” for a few days until we got something in order.

Some people might be prone to think their team will never have the time to do that (and some may be right) but in my experience quality pays back. Often fixing a small bug takes about as long as discussing if it’s important, properly filing it in JIRA and then communicating to the outside world it’s not the priority. And eventually you reach a point where you have way more calm and focus than before because nothing is burning.

Recent example: I asked Opus for some kind of reference data for MTBF in software. It took a few minutes to do "research", ended up providing zero data, and gave me a long essay on how DORA is superior and I should be using that instead. When I clarified that we're thinking of adding MTBF next to our existing DORA metrics, it decided I need to be thought about SLOs... if it was the first time, I may have continued clarifying but since it wasn't, I just gave up on Claude for this topic.

In some cases they do. I work in a B2B vertical SaaS company and there’s both features that competitors build or rough edges around our features that make clients go „either we get X or we sign with someone else”. I agree though with the general sentiment that you don’t need SOTA models to build those - humans or humans + mid pack strong model will do.

After that it’s things I see the LLMs get wrong in practice. For almost all projects it’s when to use which build/test command + a line to use pnpm/cargo/… add instead of modifying dependency files directly + „don’t add any code comments unless specifically requested”. The rest varies a lot by project.

- a self-hosted PaaS with CI-driven deployments, managed by terraform (and possibly other IaC in the future) - think Coolify but with IaC or Komodo but easier to run full-stack apps (databases, queues and OTEL support built in)

- a benchmark for AI agents where they have to iteratively build an application - it shows how bad design decisions compound over time

There are a lot of approaches to recommendations, and I imagine the biggest players having a mix of them, so that _something_ sticks. Some of them might rely on data / features / models which are recalculated "once every X", so they don't react immediately to your input.

There's also patterns which are normal to some people but seem insane to others: "oh, I've liked this show so much I watched it 3 times!".

I think it’s actually non-trivial to determine how many repos you should have read-only access to. I frequently hop through multiple repos that I don’t contribute to, just to understand how the system is architected and what it does at different stages. We even have an internal Claude skill for finding relevant repo for a given problem which relies on personal gh access (via CLI). It _can_ be done more securely but those defaults built over many years will take time to change.