Hacker News: stephantul

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 06:11:48 +0000

It was directed at the parent who implied that we didn’t think about this.

I agree with your point about the evals and how you can get discontinuities: good search can be worse than bad search when agents can do many searches. We’re working on it

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 04:35:50 +0000

It's not probabilistic, and exact matches will always be preferred over non-exact. So if you search for a function name this will surface it.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 04:19:25 +0000

This is a bit rude.

We didn't generate this project, we wrote it, a lot of it manually, and trained custom models. We'd been working in the real-time retrieval space for a while, and we thought coding was a good fit for this specific technology.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 04:17:08 +0000

Oh sorry that happened. Feel free to open an issue or report it here

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 04:02:19 +0000

Wow awesome, thanks for sharing! This is really useful and very much like the experiments we want to be doing in the near future

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 04:01:03 +0000

This is true, agents just don't know a lot about the things they're looking at, e.g., the number of files, file sizes, etc.

Although for small codebases it also holds that whatever you would like to find it easy to find, so search still might help you with cost

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 03:58:58 +0000

Yeah I agree. I have used semble to quickly index a large monorepo and just ask a question about it, it surfaced the right files pretty quickly. Although without an IDE, it's difficult to display them in nice way

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 03:58:08 +0000

The comparison is in the benchmarks, see the README

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 03:57:32 +0000

For chunking Semble supports all languages supported by tree-sitter-language-pack. The models we train are trained on 6 languages, but can handle way more.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 03:55:27 +0000

We hadn't found that one yet. Will do!

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 03:54:27 +0000

Yes, this is the main reason. We've released some rust stuff in the past, but Python is our main language

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Mon, 18 May 2026 03:53:57 +0000

The comparison is with ripgrep, see the benchmarks.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Sun, 17 May 2026 20:06:48 +0000

Yeah we're also interested in doing this, it's on the roadmap together with optimization of the prompt and descriptions so that models have an easier time using it.

Perhaps anecdotally: we do use this tool ourselves of course, and it's been working pretty well so far. Anthropic models call it and seem to trust the results.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Sun, 17 May 2026 18:51:41 +0000

Even so. Take a look at the NDCG numbers for grep. It's not pretty

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Sun, 17 May 2026 18:43:51 +0000

Hey, this is something we're actively investigating. We recently added a flag, `--include-text-files`, which, when set, also makes Semble index regular documents (i.e., markdown, text, json). This should also work relatively well.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Sun, 17 May 2026 18:25:04 +0000

The same holds for semble: the agent can fire off many different semble queries with different k/parameters.

I guess the point we’re trying to make is that you need fewer semble queries to achieve the same outcome, compared to grep+readfile calls.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Sun, 17 May 2026 18:23:01 +0000

You need readfile to do something with those tokens. Grep only gives you the matching lines, not the context.

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Sun, 17 May 2026 17:51:57 +0000

1) yes! It’s not accuracy, but ndcg 2) we assume that if the agent gets the correct answer in the returned snippets it does not need to read further

New comment by stephantul in "Show HN: Semble – Code search for agents that uses 98% fewer tokens than grep"

stephantul — Sun, 17 May 2026 17:32:00 +0000

Hey! Co-author here. The benchmark currently only measures retrieval accuracy.

We’re interested in measuring it end to end and also optimizing, e.g. the prompt and tools, for this, but we just haven’t gotten around to it.

New comment by stephantul in "Δ-Mem: Efficient Online Memory for Large Language Models"

stephantul — Sat, 16 May 2026 10:39:01 +0000

How would you conceptualize recall in this case? Is searching through the current version of your code and possibly git history not enough?