The fundamental problem is that we're trying to create a simulation model of real hardware that is (a) realistic enough to tell us something reasonable about how to expect the hardware to behave and (b) computationally efficient enough to tell us about a in a reasonable period of time.

You've got all these silicon valley guys running around "venture investing", the truth is it's more of a life style than a money making exercise. They made their money decades ago, and now they're just sort of hanging around desperately trying to tell everyone how clever they are.

It's difficult to (a) see how he can say this having given any real thought at all and (b) understand why he's going to on news interviews and winging it.

So this turns into a death march.

If you are behind, the only thing you can do is make massive capital investments to catch up. Once you're ahead you can sell tokens until someone else catches up. And, breaking the model of normal of places like chip fabrication, your billions of investment may only keep you ahead for 2 months. So you have a tiny window to sell those tokens.

But we talk about inference separately for a reason: largely inference cost is the scaling cost. Once you have a model the margin on your inference is how you get to profitability, as long as your margin is positive you can make the entire enterprise profitable by just selling more tokens. This is the same fundamental business that chip fabs work on. Yes it costs them a lot to get to the next node, but what's important is the margin they can get on the wafers they sell, because they sell tonnes of wafers.

It's pretty core to the concept of SAAS businesses that yes, you do consider all costs. But you want to focus on the margin of the bit that scales. This is why WeWork exploded, the thing they were scaling only scaled up at negative margin.

The point is that if their inference margin is positive, they can "just" scale up and become profitable. If their inference margin is negative, then scaling up the business actually causes problems.

Secondly, it's not clear that the current LLMs are a run up to AGI. That's what LeCun is betting - that the LLM labs are chasing a local maxima.

Can AI be misused? No. It will be misused. There is no possibility of anything else, we have an online culture, centered on places like Twitter where they have embraced being the absolute worst person possible, and they are being handed tools like this like handing a hand gun to a chimpanzee.

The data in the chatbots dataset about that phrase tell it a lot about how it should behave, and that data includes stuff like Elon Musk going around calling people paedophiles and deleting the accounts of people tracking his private jet.

"Social experiment" you might as well run around shouting "is jus a prank bro!".

Tim Apple is famous for very few things but

> We believe that we need to own and control the primary technologies behind the products we make

If AI is as big as we think it will be, Apple thinks they need to own it.

Of course they're going to raid their offices! They're investigating a crime! It would be quite literally insane if they tried to prosecute them for a crime and how up to court having not even attempted basic steps to gather evidence!

Is this all an effort to utilize more efficient solar panels? Are solar panels really the limiting factor for data centres?