From my understanding of the issue, mmap'ing the file is showing that inference is only accessing a fraction of the weight data.

Doesn't the forward pass necessitate accessing all the weights and not a fraction of them?

The point isn't that these things haven't existed in some form forever, it's that they're scaling to points where these effects are becoming increasingly dangerous, proportional to the growth of these systems.

The problem is that in 2022, google personalizes (aka re-ranks) the SERPs on enough dimensions (device type, location, user, etc) that a single mapping of search query -> ranking fails to capture any of the nuance that google is applying to their user understanding. I'm not sure how or when the SEO space will actually reckon with this, or if they'll keep just pushing poor 'visibility' metrics that can't actually be reduced to a single dimension.

For a random business that wants to advertise online, without the infrastructure and data capability to back it, they will struggle to compete unless they exist in a segment full of similar peers. When the former happens, we see articles about how PPC doesn't actually work, etc.

Reality is that it takes engineering work and infrastructure, coupled with some data capabilities to unlock real value in the online advertising space.

As noted, online advertising brought all sorts of insight and visibility over traditional 'offline' marketing channels, but with that comes more savvy competitors that will do all the data things you're not.

It's a strange intersection of needs where one wants or needs something like CUDA, but doesn't care to ensure their computation is actually running optimally. If you don't want to be bothered with control and granularity, why are you trying to write high performance CUDA code in the first place?

Would you mind elaborating as to what your hobby project was?

The harder part here will be getting access to the compute required, but again, the folks involved in this project have access to lots of resources (they've already trained models of this size). We'll likely see some trained checkpoints as soon as they're done converging.

According to his CV he's been active in the field for quite some time. The default assumption that he's an idiot and going to kill people just seems too cynical here.

Grandparent - you specifically mention having noted methodology problems, would you mind sharing where in the methodology you think he's gone wrong?