I really don't like the hallucination rate for most models but it is improving, so that is still far in the future.

What I could see though, is if the whole unit they made would be power efficient enough to run on a robotics platform for human computer interaction.

It makes sense they would try to make repurposing their tech as much as they could since making changes is frought with a long time frame and risk.

But if we look long term and pretend that they get it to work, they just need to stay afloat until better smaller models can be made with their technology, so it becomes a waiting game for investors and a risk assessment.

If you etch the bits into silicon, you then have to accommodate the bits by physical area, which is the transistor density for whatever modern process they use. This will give you a lower bound for the size of the wafers.

This can give huge wafers for a very set model which is old by the time it is finalized.

Etching generic functions used in ML and common fused kernels would seem much more viable as they could be used as building blocks.

Ah, like a knifes edge, but would be exciting. Could have a literal bug in the code.

While rust has traits, borrowing etc, it doesn't have a lot of things with regard to types and optimization. Things like:

- A lack of GADTs, or a stronger version, dependent types, or similar type system which would allow one to encode natural relationships, recursive ones, invariants etc.

- Tail call optimization guarantees, to allow for mutual recursion and optimization since game engines are just huge state machines, and it would allow to pass functions around which could call each other via mutual recursion, while allowing it to be optimized as well.

- Efficient structural sharing of immutable state, which would be memory layout and cache friendly

- Built in profiling from the getgo which the language developers would use and refine, so you could get information about how the program behaves over time and space.

Realistically, one could argue it's not the right choice overall, but still, it's an application which would push the boundaries of what those languages have been perceived to have the greatness weakness in. An application which is mostly about handing mutable state with high performance.

The problem is when the ai makes a catastrophic prediction, and the layman can't see it.

It would be a guide rail for people at the top to align themselves with people at the bottom. To be aligned with the promises they use in fundraising from donors (of both time and money).

I'm torn with the "just don't give them money then" which a sibling commenter said, it might work short term, but what about everything people have poured into this throughout the decades? I think all that work deserves to be safeguarded, it would show that whatever resources, be it money or time, cannot just be turned on itself by a passing leadership, and that there would be a safeguard against "flushing everything down" as the only choice.

Furthermore, I just don't see a promise/company statement as being enough, after everything that has happened. There needs to be legal accountability and safeguards for not sinking a multi-generational ship.

Say, the threat of an actual litigation, would help hold them accountable in the future?

So knowledge transfer is something incredibly specific and much more narrow than what I thought. They don't transfer concepts by generalization, but they compress knowledge instead, which I assume the difference is, that generalization is much more fluid, while compression is much more static, like a dictionary where each key has a probability to be chosen, and all the relationships are frozen, and the only generalization that happens, is the generalization which is an expression of the training method used, since the training method freezes it's "model of the world" into the weights so to say? So if the training method itself cannot generalize, but only compress, why would the resulting model that the training method produces? Is that understood correctly?

Does there exist a computational model, which can be used to analyse a training method and put a bound on the expressiveness of the resulting model?

It's fascinating that the emergent ability of models disappear if you measure them differently. Guess the difference is that "emergent abilities" are kinda nonsensical, since they have no explanation of causality (i.e. it "just" happens), and just seeing the model getting linearly better with training fits into a much more sane framework. That is, like you said, when your success metric is measuring discretely, you also see the model itself as discrete, and it hides the continuous hill climbing you would otherwise see the model exhibit with a different non-discrete metric.

But the model still gets better over time, so would you expect the model to get progressively worse on a more generalized metric, or does it only relate to the spikes in the graph that they talk about? IE, they answer the question of "why" jumps in performance are not emergent, but they don't answer why the performance keeps increasing, even if it is linear, and whether it is detrimental to other less related tasks?

And if you wanted to test "emergent" wouldn't it be more interesting to test the model on tasks, which would be much more unrelated to the task at hand? That would be to test generalization, more so as we see humans see it? So it wouldn't really be emergence, but generalization of concepts?

It makes sense that it is more straightforward to refute a claim by using contradiction. Would it be good practice for papers, to try and refute their own claims by contradiction first? I guess that would save a lot of time.

It's interesting about the knowledge leakage, because I was thinking about the concept of world simulations and using models to learn about scenarios through simulations and consequence. But the act of creating a model to perceive the world, taints the model itself with bias, so the difficulty lies in creating a model which can rearrange itself to get rid of incorrect assumptions, while disconnecting its initial inherent bias. I thought about models which can create other models etc, but then how does the model itself measure success? If everything is changing, then so is the metric, so the model could decide to change what it measures as well. I thought about hard coding a metric into the model, but what if the metric I choose is bad, and we are then stuck with the same problem of bias as well. So it seems like there are only two options, it either converges towards total uncontrollability or it is inherently biased, there's doesn't seem to be any in-between?

I admit I'm trying to learn things about ML I just find general intelligence research fascinating (neuroscience as well), but the more I learn, the more I realize I should really go back to the fundamentals and build up. Because even things which seem like they make sense on a surface level, really has a lot of meaning behind them, and needs a well-built intuition not from a practical level, but from a theoretical level.

From the papers I've read which I find interesting, it's like there's always the right combination of creativity in thinking, which sometimes my intuition/curiosity about things proved right, but I lack the deeper understanding, which can lead to false confidence in results.