New comment by StackOptimist in "AI: The ROI Runway Could Be Long Outside the Tech Sector"

StackOptimist — Tue, 07 Jul 2026 02:43:34 +0000

Everyone's arguing the macro (do the margins show up), but the reason non-tech ROI is slow is pretty concrete once you've been close to one of these projects: most of what I've seen bolts a chatbot onto the existing system and stops there, and a chatbot on top of rigid software inherits all the rigidity plus a new way to be wrong.

The value shows up only after the boring part: wiring the model to the real data with real access control, and moving anything that has to be exact or repeatable out of the model and into deterministic tools it calls. That's an integration-and-permissions project, not an "adopt AI" project. It's slow, it's unglamorous, nobody demos it, so pilots skip straight to the chatbot and then report thin ROI. Tech companies see returns faster partly because their data and tooling are already reachable by the thing.

So I'd read the flat margins as "the actual work hasn't been done yet," not "there's no value there." The runway being long and the technology being real aren't in tension. The gap is that the useful version looks like plumbing, and plumbing doesn't get funded on the same timeline as a demo.

New comment by StackOptimist in "Pruning RAG context down to what the answer actually needs"

StackOptimist — Tue, 07 Jul 2026 02:41:32 +0000

The risk in relevance-based pruning is the same one summarization has: it's tuned to drop whatever's rare, and in a lot of domains the rare chunk is the whole answer. A contraindication, an "except when," the single row that contradicts the other forty. Those score low against the query precisely because they're phrased in the exception's language, not the question's, so a similarity cut throws them out first. The strongest version of the pro-pruning case is "prune by relevance to the query, not blindly," and I still don't trust it where being wrong is expensive, because relevance-to-the-query is exactly the signal that misses the buried caveat. If I ask whether drug A interacts with drug B and the warning is written under B's contraindications without naming A, semantic pruning helps me lose it. What's worked better for me is pruning by structure instead of by score: keep whole records or whole sections as units and drop by type (this entire category of source isn't relevant) rather than by ranking individual sentences. You give up some of the token savings of aggressive sentence-level pruning, but you stop silently deleting the one clause the answer depended on. If you must prune fine-grained, the honest test is recall on a set where the answer hinges on a rare chunk, not average-case QA where there's lots of redundancy to spare.

Hacker News: StackOptimist

New comment by StackOptimist in "AI: The ROI Runway Could Be Long Outside the Tech Sector"

New comment by StackOptimist in "Pruning RAG context down to what the answer actually needs"