Hacker News: tybaa

New comment by tybaa in "Achieving SOTA LongMemEval scores (80%) with RAG alone"

tybaa — Thu, 17 Jul 2025 19:59:53 +0000

At mastra.ai we ran the LongMemEval benchmark (500 questions across thousands of conversations) to systematically test our agent memory features. After seeing claims that "RAG is dead for agent memory", we decided to see what was possible.

Starting at a low 65% accuracy, we made some changes to how our memory system works and reached 80% using RAG alone. We ran the benchmark with a series of different configs (since we're a configurable framework) and saw results ranging from 63% with very conservative settings, 74% with small to medium context size, up to 80% with longer context. We accidentally spent $8k and burned 3.8B tokens figuring this out - but it proved that RAG absolutely works for agent memory when properly configured.

Achieving SOTA LongMemEval scores (80%) with RAG alone

tybaa — Thu, 17 Jul 2025 19:59:53 +0000

Article URL: https://mastra.ai/blog/use-rag-for-agent-memory

Comments URL: https://news.ycombinator.com/item?id=44597515

Points: 3

# Comments: 1

New comment by tybaa in "Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby"

tybaa — Wed, 19 Feb 2025 22:24:03 +0000

Added support in this PR https://github.com/mastra-ai/mastra/pull/1957! Isn't shipped just yet but will be soon

New comment by tybaa in "Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby"

tybaa — Wed, 19 Feb 2025 22:02:51 +0000

No worries! But I am definitely interested in chatting still - that you've tried it in multiple ways, ran into pain points, and overcame those in your own ways is super interesting and valuable. Playing around is how everyone starts and this "agents with tool use in prod" game is still very new. These APIs should work well and make sense for folks who are just getting into it as well folks who have been around the block. If you change your mind let me know! Would love to chat

New comment by tybaa in "Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby"

tybaa — Wed, 19 Feb 2025 20:18:05 +0000

Thanks for sharing! It's so helpful to hear real world experiences like this. Would you be interested in meeting up on a call sometime? I'd love to chat about how you're using MCP to help inform how we can make all of this easier for folks. We're actively thinking about our APIs for tool use and MCP right now.

New comment by tybaa in "Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby"

tybaa — Wed, 19 Feb 2025 19:27:35 +0000

Oh nice, did you write your own proxy or are you using something like https://www.npmjs.com/package/mcp-proxy ?

New comment by tybaa in "Show HN: Mastra – Open-source JS agent framework, by the developers of Gatsby"

tybaa — Wed, 19 Feb 2025 19:17:43 +0000

Hey! Glad to hear you're excited about it! Yes, we're currently working on improving our MCP support in general - we'll have more to share soon, but part of that is supporting SSE servers directly