https://news.ycombinator.com/user?id=Damianf19Hacker News RSShttps://hnrss.org/hnrss v2.1.1Mon, 18 May 2026 09:35:51 +0000What's the data model that lets you compare agents that differ a lot in tools/policies? Curious if you normalize on the "what did the user actually accomplish" layer or on raw token/turn metrics, because the two paint completely different pictures of "is this agent working." We struggle with this on the eval side of our own product (email pipeline outcomes, not agents, but same shape).

]]>Tue, 12 May 2026 16:45:38 +0000https://news.ycombinator.com/item?id=48110772Damianf19https://news.ycombinator.com/item?id=48110772https://news.ycombinator.com/item?id=48110772