But linguists have proposed the possibility that we store “fragments” to facilitate reuse—essentially trees with holes, or equivalently, functions that take in tree arguments and produce tree results. “In the middle of the” could take in a noun-shaped tree as an argument and produce a prepositional phrase-shaped tree as a result, for instance. Furthermore, this accounts for the way we store idioms that are not just contiguous “Lego block” sequences of words (like “a ____ and a half” or “the more ___, the more ____”). See e.g. work on “fragment grammars.”

Can’t access the actual Nature Human Behavior article so perhaps it discusses the connections.

https://en.wikipedia.org/wiki/Need-blind_admission

> they could vastly grow their class size without lowering standards

The issue isn't the quality of the students they are accepting, but the resources to educate and house them, including classroom space, dorms, and staff.

But I wouldn't call the probabilistic interpretation "after the fact." The entire training procedure that generated the LM weights (the pre-training as well as the RLHF post-training) is formulated based on the understanding that the LM predicts p(x_t | x_1, ..., x_{t-1}). For example, pretraining maximizes the log probability of the training data, and RLHF typically maximizes an objective that combines "expected reward [under the LLM's output probability distribution]" with "KL divergence between the pretraining distribution and the RLHF'd distribution" (a probabilistic quantity).

Very large language models also “know” how to spell the word associated with the strawberry token, which you can test by asking them to spell the word one letter at a time. If you ask the model to spell the word and count the R’s while it goes, it can do the task. So the failure to do it when asked directly (how many r’s are in strawberry) is pointing to a real weakness in reasoning, where one forward pass of the transformer is not sufficient to retrieve the spelling and also count the R’s.

But what is action and behavior? We have a single interface to LaMDA: given a partially completed document, predict the next word. By iterating this process, we can make it predict a sentence, or paragraph. Continuing in this way, we could have it write a hypothetical dialogue between an AI and a human, but that is hardly a "canonical" way of using LaMDA, and there is no reason to identify the AI character in the document with LaMDA itself.

All this to say, I am not sure what you mean when you say it "claims sentience". What does it mean for it to "claim" something? Presumably, e.g., advanced image processing networks are as internally complex as LaMDA. But the interface to an advanced image processing network is, you put in an image, it gives out a list of objects and bounding boxes it detected in the image. What would it mean for such a network to claim sentience? LaMDA is no different, in that our interface to LaMDA does not allow us to ask it to "claim" things to us, only to predict likely completions of documents.

Does it? I don’t think it would even apply to a reinforcement learning agent trained to maximize reward in a complex environment. In that setting, perhaps the agent could learn to use language to achieve its goals, via communication of its desires. But LaMDA is specifically trained to complete documents, and would face selective pressure to eliminate any behavior that hampers its ability to do that — for example, behavior that attempts to use its token predictions as a side channel to communicate its desires to sympathetic humans.

Again, this is not an argument that LaMDA is not sentient, just that the practice of “prompting LaMDA with partially completed dialogues between a hypothetical sentient AI and a human, and seeing what it predicts the AI will say” is not the same as “talking to LaMDA.”

Suppose LaMDA were powered by a person in a room, whose job it was to predict the completions of sentences. Just because you get the person to predict “I am happy” doesn’t mean the person is happy; indeed, the interface that is available to you, from outside the room, really gives you no way of probing the person’s emotions, experiences, or desires at all.

LaMDA may or may not experience something while repeatedly predicting the next word, but ultimately, it is still optimized to predict the next word, not to communicate its thoughts and feelings. Indeed, if you run an LLM on Lemoine's prompts (including questions like, "I assume you want others to know you are sentient, is that true?"), the LLM will assign some probability to every plausible completion -- so if you sample enough times, it will eventually say, e.g., "Well, I am not sentient."

The StrangeLoop talk includes an example where you infer that Stove() returns a Stove object. If someone writes something like `f(x).broil()`, do you need to do some kind of type inference to figure out what class f(x) is?

What cases do Stack Graphs fail to handle? (e.g., I assume dynamic modification of .__dict__ can't be tracked; are there other representative examples?)

P(you chose goat | host didn’t choose car) = P(you chose goat, host didn’t choose car) / P(host didn’t choose car).

The numerator is 2/3 * 1/2, and the denominator is 2/3, so the ratio is indeed 1/2.

(A rejection sampling loop, where you repeatedly simulate a process until a condition holds, has the same distribution over final outcomes as the conditional distribution—so repeatedly restarting the game if the host chooses the car induces the same distribution on final results as simply conditioning on the host not choosing the car.)

I had heard the government required TurboTax to have a free edition. But back in 2019 (and before), if you Googled "TurboTax free" you'd be taken to a decoy free edition; the real one was called their "freedom edition," and was hidden from Google's listings. If your tax situation is 'too complicated,' the free edition tries to upsell you to the "deluxe edition," even if the "freedom edition" could have handled the situation just fine.

Thankfully now you can find the actual free version on Google, but it's still very confusing that the "free edition" is less free than the (still somewhat hidden) IRS Free File ("freedom") version. And the faux Free Edition is still being heavily advertised.