Comments URL: https://news.ycombinator.com/item?id=47857889

Points: 2

# Comments: 2

Where will browsers use be at the end of the year?

The modern orchestrator are reactive, they don’t handle spikey traffic. Your favorite retry library will cause retry storms for downstream dependencies and your public APIs. Remember EZThrottle blog posts

EZThrottle.network

Common failure modes I'm thinking about: - 429 rate limits → agents retry → hammer API worse - Partial outages → synchronized retries across customers - LangGraph workflows fail mid-execution → how to resume?

For those running agent systems at scale: - How do you handle Layer 7 failures? - Retry coordination? Circuit breakers? - How do you prevent retry storms to downstream dependencies? - Do LangGraph workflows gracefully handle API failures?

Curious what the production reality looks like.

Comments URL: https://news.ycombinator.com/item?id=47292281

Points: 1

# Comments: 2

How do you handle rate limiting across multiple workers? Do you use circuit breakers, retry libraries, or something custom? How do you prevent retry storms when 100 workers all hit the same rate limit?

Curious what's working at scale.

Comments URL: https://news.ycombinator.com/item?id=47131730

Points: 3

# Comments: 2

100 workers all retry independently → retry storm Sequential fallbacks are slow (try OpenRouter, wait 5s, try Anthropic, wait 5s) No coordination across instances

So I built a coordination layer that:

Races multiple providers simultaneously (OpenRouter + Anthropic + OpenAI) Coordinates retries across all workers (no retry storms) Resumes workflows via webhooks (idempotent keys = checkpoints)

It runs on Fly.io's anycast network + BEAM for distributed coordination. Architecture deep dive: https://www.ezthrottle.network/blog/making-failure-boring-ag... Happy to answer questions about the approach or why BEAM made this possible when other languages would struggle.

Comments URL: https://news.ycombinator.com/item?id=47043197

Points: 1

# Comments: 0

By “losing sleep” I really mean on-call fatigue during partial outages — the class of incidents where backoff, shedding, and breakers exist, but retry amplification, shared rate limits, or degraded dependencies still cause noisy pages and prolonged recovery.

I’m trying to understand how teams coordinate retries and backpressure across many independent clients/services when refactors aren’t immediately available, not replace good architecture or take ownership of someone else’s system.

If you’ve seen patterns that consistently avoid that on-call pain at scale, I’d genuinely love to learn from them.

Retry storms - API fails, your entire fleet retries independently, thundering herd makes it worse.

Partial outages - API is “up” but degraded (slow, intermittent 500s). Health checks pass, requests suffer.

What I’m curious about: ∙ What’s your current solution? (circuit breakers, queues, custom coordination, service mesh, something else?) ∙ How well does it work? What are the gaps? ∙ What scale are you at? (company size, # of instances, requests/sec)

I’d love to hear what’s working, what isn’t, and what you wish existed.

Comments URL: https://news.ycombinator.com/item?id=46866428

Points: 2

# Comments: 4

When services hit 429s or timeouts, the standard response is almost always the same: retries with backoff, sleep loops, jitter, etc. This is treated as a best practice across languages and platforms.

But in systems with high concurrency, fan-out, or shared downstream dependencies, retries often seem to amplify load instead of smoothing it. What starts as localized failure can turn into retry storms, thundering herds, and cascading outages.

It’s made me wonder whether retries are solving the wrong problem at the wrong layer — treating a coordination issue as an application-level error-handling concern.

I wrote up a longer piece exploring this idea and arguing for making failure boring again by handling it at a different layer: https://www.ezthrottle.network/blog/making-failure-boring-again

Curious how this matches others’ experience:

Have retries actually improved stability for you under sustained rate limiting?

Have you seen cases where they clearly made things worse?

If retries aren’t the right abstraction, what is?

Interested in war stories, counterexamples, and alternative approaches.

Comments URL: https://news.ycombinator.com/item?id=46841137

Points: 3

# Comments: 0

I built EZThrottle because I was tired of copy-pasting exponential backoff into every codebase.

The core idea: retries shouldn't be independent. When thousands of machines all hit a 429 and retry independently, you get retry storms that cascade into outages. EZThrottle coordinates failure in one place.

What it does: - Rate limiting per destination (default 2 RPS – conservative on purpose) - Region racing: send to multiple regions, accept first success, cancel the rest. If one region goes down, your request still completes with just a latency bump instead of a full outage - Adds event-driven architecture to your stack via webhooks – fire and forget, get results delivered

Current scale (setting expectations): Running 4 machines across 2 US regions (Dallas + Washington DC) on Fly.io. Not massive yet – we'll expand to more regions with demand. Early days.

SDKs: - Python: https://github.com/rjpruitt16/ezthrottle-python - Node: https://github.com/rjpruitt16/ezthrottle-node - Go: https://github.com/rjpruitt16/ezthrottle-go

Written in Gleam, runs on the BEAM. This is part of a larger vision (L8-OS – local-first AI stack), but EZThrottle works standalone.

Blog post with architecture diagrams: https://ezthrottle.network/blog/making-failure-boring-again

Free tier: 1M requests/month. Happy to answer questions.

@RahmiPruitt on Twitter

Comments URL: https://news.ycombinator.com/item?id=46804549

Points: 1

# Comments: 0

So I wrote Glixir, a library that wraps common OTP patterns in Gleam using Elixir’s battle-tested OTP library. It’s not as type-safe as Gleam, but I needed something production-ready today, not next year.

This lets you:

Use supervisors, GenServers, and pubsub patterns from Gleam.

Stay in Gleam where it makes sense, but drop down to Elixir OTP when you need reliability.

Ship code faster without reinventing OTP on the Gleam side.

Obviously there are trade-offs — you give up some type safety, and it’s not “pure Gleam.” But if you’re trying to build something real, you may find this a useful middle ground.

It's MIT so if you disagree with direction you can fork and do whatever you want.