Now, should Google be mocked for their DEI? ABSOLUTELY. They are literally one of the least diverse places to work for. They publish a report and it transcends satire. It's so atrociously bad it's funny. Especially when you see a linkedin job post for working at google, and the thumbnail looks like a college marketing brochure with all walks of people represented.

But lets be real here ... DEI is a good thing when done well. How are you going to talk to the customer when they are speaking a different cultural language. Even form a purely capitalist perspective, having a diverse workforce means you can target more market segments with higher precision and accuracy.

1. use mobile phone proxies. Because of how mobile phone networks do NAT, basically it means that thousands of people share IPs and are much less like to get blocked.

2. Reverse engineer APIs if the data you want is returned in an ajax call.

3. Use a captcha solving service to defeat captchas. There's many and they are cheap.

4. Use an actual phone or get really good at convincing the server you are a mobile phone.

5. Buy 1000s of fake emails to simulate multiple accounts.

6. Experiment. Experiment. Experiment. Get some burner accounts. Figure out if they have request per min/hour/day throttling. See what behavior triggers a cloudflare captchas. Check if different variables such as email domain, useragent, voip vs non-voip sms based 2fa. your goal is to simulate a human. So if you sequentially enumerate through every document - that might be what get's you flagged.

Best of luck and happy scraping!

Let's be clear though, I'm not saying tech is bad. We'd all be doing manual labor on a a farm without it. I do think our demographic (including myself) has completely set aside any consideration for our impact in the name of optimization or a fat paycheck.

Captchas are super easy! There's a gagillion captcha bypass services for every type of captcha. Just snag the captcha token, send it in an API call, and then you get a verified captcha token.

See CGNAT for more details about mobile networks. https://en.wikipedia.org/wiki/Carrier-grade_NAT#cite_note-of...

It's pretty much impossible to stop the top 1% of the most dedicated scrapers without affecting end user experience.

But yes! especially in b2b sells. I always try to be super anal about kyc with selling to business buyers in bulk, and avoid buying from them. I straight up stick to institutional suppliers to avoid this issue :(

My real biggest risk though is when I open up my own store is chargebacks. Digital goods have a very very high chargeback risk, and I need to be clever on how I mitigate that!

Currently at 70k Euros a month. It took me about 3 years to get here. And it took a few months to get to 2k monthly revenue.. However, the first two months I was at break even and it took me about a year before I could figure out how to get this system running profitably. Currently profit margins are about 7%