Hacker News: nikolife2016https://news.ycombinator.com/user?id=nikolife2016Hacker News RSShttps://hnrss.org/hnrss v2.1.1Mon, 07 Sep 2026 20:37:06 +0000<![CDATA[Correction: "76% dead" was our bug, not the market's]]>Article URL: https://pulsefeed.dev/correction

Comments URL: https://news.ycombinator.com/item?id=49113551

Points: 2

# Comments: 0

]]>Thu, 30 Jul 2026 18:09:18 +0000https://pulsefeed.dev/correctionnikolife2016https://news.ycombinator.com/item?id=49113551https://news.ycombinator.com/item?id=49113551<![CDATA[New comment by nikolife2016 in "An update on residential proxies and the scraper situation"]]>Running a small public JSON API, the traffic breakdown is eye-opening. Roughly half is trust/uptime "scanners" and generic monitors; a solid chunk is well-behaved crawlers that declare themselves with real UAs and honor robots; and then there's a long tail of vuln-scanners blindly probing for /.env, /.git, wp-login and the like. The genuinely evasive residential-proxy scraping is a minority by volume but by far the hardest to separate from real users — it's the one bucket where UA and IP both look residential, so you can't tell bot from human without behavioral signals. What's shifted in the last year: the "polite" bots got politer, while the abusive layer moved almost entirely onto residential proxies. IP reputation alone is basically dead as a filter now.

]]>Sat, 11 Jul 2026 11:06:03 +0000https://news.ycombinator.com/item?id=48870918nikolife2016https://news.ycombinator.com/item?id=48870918https://news.ycombinator.com/item?id=48870918