This connection is shared as well. My partner relies heavily on cloud syncing. We both like to stream 4K HDR video. I like being able to get devices updated and ready to use with minimal time spent waiting for downloads.

I also live in NZ, where multi-gigabit fibre connections are often cheaper than what Americans have to pay for a fraction of the bandwidth. It’s not a notable financial burden or anything, and it’s not like we have data caps to worry about. It’s very much a situation where the use cases naturally find themselves once the option is there.

Also, 25/10 Mbps is painfully slow for a shared home connection in the modern day. There’s videos on YouTube that can push a higher bitrate than that. The absolute slowest plan that my ISP even offers is 100/20 Mbps for about $35 USD per month, while the most common/baseline plan for most households in NZ is 500/100 Mbps after the fibre carriers continued to increase speeds at the lowest tiers.

We absolutely should see more trees in many cities, but they introduce their own challenges in parking lots, especially if they’re placed retroactively.

Our peak import rate is 3x higher than our solar export rate. In other words, we’d need to sell 3 kWh hours of energy to offset the cost of using 1 kWh at peak.

We’re currently in the process of accepting a quote for home batteries. The rates here highly incentivise maximising self-use.

Coincidence?! Yeah, probably.

It's been getting worse over the past year, with the past few weeks in particular seeing a massive change literally overnight. I had to aggressively tune my WAF rules to even remotely get things under control. With Cloudflare I'm aggressively issuing browser challenges to any browser that looks remotely suspicious, and the pass rate is currently below 0.5%. For my users' sake, a successful browser challenge is "valid" for over a month, but this still feels like another thing that'll eventually be bypassed.

I'd be keen to know if you've found any other effective ways of mitigating these most recent aggressive scraping requests. Even a simple "yes" or "no" would be appreciated; I think it's fair to be apprehensive about sharing some specific details publicly since even a lot of folks here on HN seem to think it's their right to scrape content with orders of magnitude higher throughput than all users combined.

I really don't know how this is sustainable long-term. It's eaten up quite a lot of my personal time and effort just for the sake of a hobby that I otherwise greatly enjoy.

Hiding behind a residential proxy and using random user agents? Gross. Learn what consent is.

I actually do let quite a few known, "good" scrapers scrape my stuff. They identify themselves, they make it clear what they do, and they respect conventions like robots.txt.

These residential proxies have been abused by scrapers that use random legit-looking user agents and absolutely hammer websites. What is it with these scrapers just not understanding consent? It's gross.

Excluding known "good" crawlers, well over 99% of the traffic trying to hit the site has been attempting to maliciously scrape. Most of this traffic looks genuine, but has random genuine-looking user agents and comes from random residential proxies in various countries, usually the US.

For the traffic that does make it all the way to a browser challenge, the success rate is a measly 0.48%. Put another way, over 50% of traffic is already blocked by that point, and of the under 50% that makes it to a browser challenge, more than 99.5% fails that challenge.

It's been virtually no disruption to users either, since I configured successful challenges to be remembered for a long period of time. The legitimate traffic is a gentle trickle, while the WAF is holding back garbage traffic that's orders of magnitude above and beyond normal levels. The scale of it is truly insane.

There are plenty of workloads where I’d love to double the memory and halve the cores compared to what the memory-optimised R instances offer, or where I could further double the cores and halve the RAM from what the compute-optimised C instances can do.

“Serverless” options can provide that to an extent, but it’s no free lunch, especially in situations where performance is a large consideration. I’ve found some use cases where it was better to avoid AWS entirely and opt for dedicated options elsewhere. AWS is remarkably uncompetitive in some use cases.

Reddit Status used to show API response times way back in the day as well when I used to use the site, but they've really watered it down since then. Everything that goes there needs to be manually put in now AFAIK. Not to mention that one of the few sections is for "ads.reddit.com", classic.

I showed Immich to my partner and they loved it so much that we've ordered significantly more storage for the server to accommodate it. We're currently using both Google Photos and OneDrive, but with this we'll be ditching OneDrive and filling that niche with Immich (as well as expanded network storage in general).

The website and documentation is super clear about not using it as the only source of photos. This is why we'll keep using Google Photos, and why I'll also be backing up Immich and portions of the network storage to B2 via restic. I've used this snapshotting pattern for my general server data for years, and it's even saved me a couple of times. Backups are something you hope to never need to use, but boy are they satisfying when you do need to use them and have them set up properly!

I even use the network for stuff like old game ROMs and ISOs for emulation. The random access times are still magnitudes better than I’d be able to get with any HDD.

Sorry about your NBN!

What a short and sad story.