tell me you've never run anything without telling me you've never run anything...

Comments URL: https://news.ycombinator.com/item?id=39984209

Points: 295

# Comments: 54

And totally agreed, both products take a slightly different approach and have different strengths and weaknesses, try them both and see which is a better fit!

What we're asking you to consider is how much of that is because the tooling is shutting out the roles who are naturally incentivized to care about product quality?

I think your broader thesis, while amusing, is ignoring some of the biggest and most valuable brands ever created... Apple, Rolex, Mercedes...

Regarding DOM interaction, you're missing the point. All automation that tests the front end code, regardless of how it attaches, is using a path that is different from your end user. The end user interacts with the application visually. That's why we test visually. A decrease in code-based brittleness is just a nice side effect. And as you note, this is a very high-level post outlining one key idea about quality ownership. You may be interested in this, which is one of our front end folks talking about why we believe testing visually is superior: https://www.rainforestqa.com/blog/the-downfall-of-dom-and-th...

We have been selling a QA solution for almost 10 years. In that time we've seen thousands of setups and directly worked with hundreds of teams. Your claim "weaknesses of automated test frameworks [...] are compensated for quite easily and routinely" is, quite simply, not true for the majority of engineering teams - few QA leaders, including proponents of Cypress, would agree with you.

Read the piece - it says no such thing. It's talking about 'ownership', which is distinct from participation and most certainly from interest.

We are to my knowledge the only product built for this kind of cross-functional ownership, that's why I don't recommend other products.

While I agree the post is too marketing-y - to be honest I didn't expect it to appeal so much to the HN audience - it was written in good faith. We built the product because after 7 years of building a product for one of those silos, we became convinced that the only long-term durable solution was to empower everyone to own quality. Clearly I need to figure out how to strike the balance that you outline in your last sentence, which was the goal!

I think the challenge is that, while we agree on the optimal setup, it's almost never done like this. There's lots of reasons why, but the main one (imo) is organizational scaling. The logic of scaling teams tends towards specialization, especially because developer time is extremely expensive.

There are many ways to address the problem of QA. From our perspective, if they don't broaden the ownership of QA from just developers or just QA engineers (which is where all the current products are targeted), they will exacerbate this specialization problem.

Regarding the incentive structure, I think you're right - there's no way to eliminate the friction that comes from competing incentives. Our experience has been that empowering the product organization to make that tradeoff themselves leads to the optimal outcome for the business. The goal isn't to eliminate the tradeoff between speed and quality, but surface it, and put it in the hands of the people who are the business decision makers, which tends to be product.

Re test data - we had seen this bottleneck with our previous product, which was purely about crowd testing. What we've seen since we shipped no code automation is that much of the data seeding by less mature teams can be done through the tests themselves. This is suboptimal, but with automation so cheap and fast, it works. Then over time the engineering team can seed the states that are most often created through the tests.