My point, I think, is still that the overwhelming focus of the tools I've seen focus on the kind of fine-tuning/setup you are describing and not the things that I find most valuable. And I think that part of the problem is that it's easy to build technology around mechanics than judgement.

I work for a startup; we have what I think is a fairly typical setup: metrics ingested from a variety of sources, fed into industry-standard metrics/dashboard solutions, triggering escalations to humans. It's fine and I'm happy we have it, but...

The highest value source of alerting right now is one of our growth marketers who pays close attention to our CRM and product analytics tool and notices when key product funnels are underperforming.

Our next highest value signals are a handful of ad hoc alerting channels, mostly in Slack, either directly from a partner telling us that something suspicious happened on their side (think: fraud) or from in-product instrumentation sent to a channel for non-engineering visibility. Members of our business/product/operations team pay attention in these places and make decisions based on their business context.

After that, our support team is increasingly able to filter customer issues and differentiate between bugs, missing features, etc.

I know someone is going to argue that these are all a sign that we haven't instrumented the right things. Fair, but also misses the point. The decision makers in these flows don't (and won't) live in traditional alerting systems and wouldn't have helped us understand breakages without these other, ad hoc processes.

My theory is that it's relatively easy to offer a technical product that moves alerts around or that manages escalation paths. It's quite hard to design a product that surfaces detail to a non-technical export and that makes it easy to build systematic rules.

Perhaps because I have a similar bio to yours, I am allergic to this view.

QA should not be forced into an engineering or automation track because the incentives are wrong. You end up with test code becoming the goal and then it usually rots due to most QA not having the experience to create a codebase that scales.

I don't think the industry today understands how to treat QA and I think that leads to a lot of assumptions that it's not useful.

I also reached a lot of similar decisions and challenges, even where we differ (ECS vs EKS) I completely understand your conclusions.

If you don't take these measures, you will lose money to fraud. You may also lose your business because you aren't meeting your AML/anti-terror obligations. (I also just had to take my annual training course).

There are a bunch of mitigations, of which identity verification is just one, and all of them are lousy for our good customers. I wish the banking systems were better and we didn't need to do any of it.

Designers have learned figma and it's the de facto tool for them; doing something else is risky for them.

Product leaders want high fidelity. They love the AI tools that let them produce high fidelity prototypes.

Some (but not all) engineers prefer it because it means less decision making for them.

As for "slapping a company on it", I agree, but also I don't think we've developed a viable alternative. Python has been limping along with one toolchain or another for my entire career (multiple decades) and it took Astral's very specific approach to create something better. It's fair to ask why they needed to be venture backed, but they clearly are and the lack of successful alternatives is telling.

It's definitely a narrow path for them to tread. Feels like the best case is something like Hashicorp, great until the founders don't want to do it anymore.

But nothing they've done in the last few years has demonstrated improvement in this area. As the person with both purchasing and final authority on these things in my org, it's hard to stomach.

If you tell me more, I might sign up. If I have to create an account first, I'm walking away.