Second to this are countless other areas that have a major impact on the companies bottom line that are entirely engineering driven, especially at google given they are a cloud provider and have meaningfully grown the workspace business and launched waymo in this time.

There were always other problems too, pressure on the company in both directions across many different product lines on both cost (any number of cheaper baremetal providers who are much faster at providing customers instances than they were a decade ago), and product quality (any number of startups to now bigger companies, databricks probably being the biggest success) along with a number of expensive bets that were made that didn't work out especially as interest rates began to rise (there were numbers of of different services ranging from IoT, AI, business support, robotics, groundstation, that essentially all failed).

AI infra being their latest bet, along with doubling down on custom hardware is smart, but these roles don't require the same number of SWEs and instead require a different type of high skilled professional.

Anyone with any small amount of creativity for this sort of thing could really make a big difference on improving the productivity of all sorts of team wide investigations as a running background task they have during their regular work.

However, an extra script here or there to make your life easier, adding extra UI features based on some datapoint to your internal dashboard, ect, these were things that could've taken a few days you didn't have before to get exactly right and now they can be done with only a few minutes of attention.

One example is when they stopped trying to develop a inferior product to EMR and Dataproc, and essentially just outsourced the whole effort to a deal made between them and Databricks. Because of this I assume many enterprise azure customers have better running data solutions in that space than they wouldve had they gone with just AWS or GCP.

On the other hand, having worked for Microsoft on an Azure team, there are plenty of areas that critically need a rewrite (for dozens of different reasons), and such a solution is never found (or they just release some different product and tell those with different needs to migrate to that), where they keep on building what can only really be described as hot-fixes to meet urgent customer demands that make it harder to eventually do said critical rewrite.

It's why you no longer need to speak with a person when reentering your home country in a lot of different places (israel being one of them, but also the EU, trusted travelers in the US through global entry, ect).

US tech companies are constantly under FTC audit relating to how they use user data. This is certainly not something that needs to be seriously worried about, certainly less so than say the way in which cameras placed all over cities are used to track all sorts of people or storing GPS locations attached to a specific devices UUID.

One can argue that ERP as code is higher value than whatever it is right now, but to act like this is a totally new idea is insane.

This is especially relevant for the camera, but also various other sensors and hardware modules that exist inside these phones.

That said, in recent years there are just a number of other areas that android is much better at such as deeper AI integration, which goes back to even prior to the current LLM craze.

Hell, I think with the later (since all major cloud providers deploy their own version of spark on their respective data processing cluster services), people don't even know that they aren't in fact using open source software. Hell, eventually you get to a point where companies that choose not to use these third party services eventually just open source their own improvements or abstractions as again separate open source projects that never make it into the upstream project (which are often times heavily influenced by profit making entities).

This has been the model for a very long time, going back to at least the likes of redhat. And certainly will be going forward with countless future projects. Maybe there needs to be new models of open source governance, but I have no clue how successful such a thing would even be.

If all you wanted to do was filter the dataset for certain fields, you can likely do something faster programmatically on a single machine.

I can however say that when I had a job at a major cloud provider optimizing spark core for our customers, one of the key areas where we saw rapid improvement was simply through fewer machines with vertically scaled hardware almost always outperformed any sort of distributed system (abet not always from a price performance perspective).

The real value often comes from the ability to do retries, and leverage left over underutilized hardware (i.e. spot instances, or in your own data center at times when scale is lower), handle hardware failures, ect, all with the ability for the full above suite of tools to work.

Over the past 5 years there's been significant changes and several clear winners. Databricks and Snowflake have really demonstrated ability to stay resilient despite strong competition from cloud providers themselves, often through the privatization of what previously was open source. This is especially relevant given also the articles mentioning of how cloudera and hortonworks failed to make it.

I also think the quiet execution of databases like clickhouse have shown to be extremely impressive and have filled a niche that wasn't previously filled by an obvious solution.

That said, there are a lot of other legal hurdles that would prevent Montana from ever being significant to the tech sector, despite the fact that I'm certain many skilled people would love to live there. From being the only state to not have at will employment to having a completely out of wack tax system (ratio between income and sales tax for a state entirely dependent on tourism), to countless restrictions (and often necessary because of water restrictions) on building large amounts of new housing, it just sin't happening.