In a recent project, I experimented with keeping the rate limiter entirely in-memory using ConcurrentHashMap.compute() to implement an atomic sliding window. It worked surprisingly well for smaller services where adding Redis felt like overkill.

I wrote up the approach and some of the tradeoffs here if anyone’s curious: https://github.com/ashutosh-stark/velocity-gate

I built VelocityGate because the recent surge in aggressive AI scrapers (GPTBot, Claude, Meta) is causing massive cloud bill spikes, especially for serverless/edge setups where you pay per invocation.

Most WAFs (like DataDome or Cloudflare Enterprise) that actually stop ML-driven bots are prohibitively expensive for startups. I wanted a zero-dependency, sub-millisecond filter that drops the traffic at the app layer.

The Architecture: It uses a lock-free sliding window algorithm backed by ConcurrentHashMap.compute(). I initially 'vibe-coded' parts of the IP extraction, but after some brutal (and helpful) Reddit feedback, I updated it to strictly handle X-Forwarded-For spoofing and optimized the memory footprint for the signature checks.

The Pivot: While this Java/Spring Boot version works great for my stack, the feedback made it clear that the real pain is in the Node.js/Vercel ecosystem. So, I am currently architecting a language-agnostic Go Reverse Proxy (VelocityGate Cloud) to sit in front of any origin and drop the traffic before the cloud provider bills you.

The Java source code is linked above. If you aren't on Java but want to test the Go proxy Alpha, I put a waitlist link at the top of the README.

Would love your technical teardowns on the lock-free logic or thoughts on scaling this via a sidecar proxy.

A lot of strong engineers move into EM roles expecting deeper technical impact, but end up spending most of their time on coordination, hiring, performance reviews, and cross-team alignment. That’s valuable work — just very different from building systems. More orgs should invest in strong IC tracks (Staff/Principal) so people can lead technically without managing people. Not everyone who’s good at engineering wants to optimize calendars and org charts.

Comments URL: https://news.ycombinator.com/item?id=47232382

Points: 1

# Comments: 1