This is also what he means by "thrownness". Our being-in-the-world, in particular the way things have meaning to us and are intelligable, is conditional on our use and our social context. Not the other way around. We never logically arrive at our context, we were always already "thrown" into it.

I think it's actually quite a profound conceptual shift, which is why Heidegger is so popular. He's still an asshole for writing so obscurely, though.

Current LLMs are the opposite of slow and deliberative, though? With the current transformer architecture, it seems like its function is more or less "look at this text and make a guess which word comes next", which is very much a fast and intuitive task. Any reasoning comes from CoT, which is more analogous to System 2 thinking. But that System 2 thinking can't function at all without relying on the representations from System 1 thinking (which are formed in pretraining).

So you could argue System 2 thinking is some part of the LLM, but it can't be 100%.

The classic Cartesian view was that we take in raw data from the outside world and process it into more and more abstract concepts (somehow), until we say "aha! I can use this for hammering". Heidegger says that this view is wrong, and our first experience of things is in terms of a social context and in terms of use. That's what's meant by us experiencing them as "ready-to-hand".

From the starting point of the ready-to-hand, we can step back, take a scientific view, and arrive at the present-at-hand, where we examine the properties of the object seperate from its use. The argument is quite compelling, and especially convincing to me is how well this agrees with modern neuroscience. In reality, 3x as many neurons path FROM the high abstraction parts of the brain TOWARDS the visual processing parts of the brain. In the classic Cartesian view of the brain beginning with the signal and abstracting its way towards the top, this is very mysterious. In Heideggers account, where the social context and potential uses of objects are the primary constituents of our experience, this makes total sense.

I especially enjoyed this subtle joke: "We think the weak activations usually don’t matter, but the strong positives and negatives matter a lot. For instance, Santa lives on the negative pole."

I can only imagine how much joy he felt writing that.

As for serious comments on its content, I'm confused by the people who considered it difficult to read. I think the ideas presented were very elegant and fairly simple to understand. Considering neurons as gates and weights as votes was a nice way to look at it, for example. I didn't feel like the post was unnecessarily long either.

One thing that stands out to me is how ridiculous it is in the first place to try to assign a short, natural language description to the function of any particular neuron. Wittgenstein already talked about how humans seem to use words in very specific ways while being terrible at exhaustively describing why.

An example is the word "game". It's very easy for even a child to distinguish between "game" and "not a game". But it's incredibly difficult to write down a precise definition of "game" that includes everything normally considered a game and excludes everything that isn't considered a game. Writing down the function of one specific neuron seems to run into this issue. When you write down what the exact function of any neuron is, there's a good chance your clunky natural language description misses some fine nuances.

A reasonable analogy, in my view, would be that neurons could represent something like the atomic components of intuitions or "vibes". Similar to the example of "game", the feeling of knowing what is or isn't a game is composed of what Wittgenstein called "family resemblances". Your brain doesn't contain a precise description of "game", but upon encountering something, you can quickly compare it to other examples of games you've seen and intuit whether something has a "game-like vibe".

Another notorious example would be "what is a woman". For some reason, everybody has very strong feelings about what is and isn't a woman, but people have a terribly difficult time perfectly describing it. I would guess that neuron activations function like these vibes or intuitions, rather than written rules.

I very much liked the parts discussing circuit ecology. This idea sounds very promising for alignment to me.