But the actual point is that my generative part of this pipeline is slow and expensive. So to find problems and to explain them. And this will not be replaced by a classifier. I also think that the math in the article is dependent on how much of your pipeline actually has to make decisions.

What I also noticed that there is only US hosting and I specifically need EU hosting. It's also early access without SLA which I don't personally like. The thing with the calibration is that I first believe it when I tested it on my own cases. especially how often the verifier incorrectly rejects a correct finding.

This is the point where I agree with the article that there are only benchmarks from the company itself and not independent benchmarks.