With the news of OpenRouter possibly being acquired and proliferation of hosted LLM routers (i.e. Ramp Router, Vercel’s AI Gateway), I saw the need for a self hosted solution focused on cost savings, transparency, and performance. So, I built an open sourced router with a simple CLI interface that can easily sit between coding agents and GenAI workloads.

For the curious and lazy, at the moment, Millwright has the tools for,

- Providers: OpenAI-compatible APIs, Anthropic, Amazon Bedrock

- Routing: policy-controlled model roles (cheap, mid, frontier), cheapest healthy route selection

- Protocols: OpenAI Chat Completions, Anthropic Messages, text and tool translation

- Cache Affinity: role-scoped session lanes without serializing concurrent agent traffic

- Spend Tracking: per-team costs, cache usage, model/provider mix, request traces

- Cost Analysis: measured usage and modeled candidate economics (HTML, Markdown, JSON)

- Reliability: bounded failover, circuit breakers, timeouts, concurrency limits

- Setup: interactive provider, model, and pricing configuration without storing provider secrets

- Deployment: one Rust binary, Docker, SQLite or PostgreSQL

Full disclosure: parts of the codebase were built with AI coding agents. All feedback is welcome, I’d especially value feedback on the routing policy, provider coverage, and anything that would block you from self-hosting it. Feel free to open feature/request and/or contribute as well.

Comments URL: https://news.ycombinator.com/item?id=49011806

Points: 9

# Comments: 6

Comments URL: https://news.ycombinator.com/item?id=48835063

Points: 15

# Comments: 16

Comments URL: https://news.ycombinator.com/item?id=48659077

Points: 3

# Comments: 1

Comments URL: https://news.ycombinator.com/item?id=48569539

Points: 3

# Comments: 0

Centro is hiring a Senior Full Stack Engineer to join on our engineering team. We're a venture backed startup building a next-generation data layer for e-commerce to help scaling merchants manage their inventory and orders. We are currently a small team and looking for a senior developer who wants the pace and ownership required for a fast growing startup.

We're looking for an experienced engineer to join our team, take on a ton of ownership & responsibly, and make a serious impact quickly. We write Python/Django on the backend and React/Javascript on the frontend. While we don't require either to apply, having experience in either will be an added bonus. We are currently only hiring candidates that are citizens or permanent residents of Canada.

Full job post here: https://centrocommerce.notion.site/Senior-Full-Stack-Enginee.... Please email andrew [at] centrocommerce [dot] com with your CV to apply directly!

Centro is hiring a Senior Full Stack Engineer to join on our engineering team. We're a venture backed startup building a next-generation data layer for e-commerce to help scaling merchants manage their inventory and orders. We are currently a small team and looking for a senior developer who wants the pace and ownership required for a fast growing startup.

We're looking for an experienced engineer to join our team, take on a ton of ownership & responsibly, and make a serious impact quickly. We write Python/Django on the backend and React/Javascript on the frontend. While we don't require either to apply, having experience in either will be an added bonus.

Full job post here: https://centrocommerce.notion.site/Full-Stack-Engineer-08ad7.... Please email andrew [at] centrocommerce [dot] com with your CV to apply directly!