New comment by peer0 in "Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks"

peer0 — Wed, 20 May 2026 17:43:04 +0000

https://github.com/ggml-org/llama.cpp/blob/master/grammars/R...

llama.cpp supports grammar limiting using either GBNF or json schema (It just translate it to GBNF behind the scenes I think). So I have my harness generate a tool schema on the fly (based on what tools are possible for the current task) and pass it in at request time.

New comment by peer0 in "Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks"

peer0 — Wed, 20 May 2026 16:30:13 +0000

This seems similar to what I done using llama.cpp's "Grammar constrained generation" for my local agents. But using that instead of catching and retrying it is just literally impossible for the LLM to generate something that doesn't match a specific schema of tool choices. It is amazing how much better small models can be when you reduce the problem space to only grammatically correct answers.

Hacker News: peer0

New comment by peer0 in "Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks"

New comment by peer0 in "Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks"