And if you want additional assurance (due to, e.g. the computation being very long) you could probably use some cryptographic techniques to produce a certificate that the output is as it should be, and you can check that certificate as many times as you want to avoid possible random hardware errors in the process of checking the certificate.

But that doesn’t mean that it is impossible to have all our programs be formally verified. For that, if we have a formal specification for what each should do… Well, I suppose it’s possible that some program we would want is possible to implement with the desired properties, but not possible to prove that it has those properties? It is possible to enumerate (program, proof) pairs though.

But there are a lot of people, and, apparently a few of them shot up some electrical substations?

If you hold constant the (very small) fraction of people who want to cause mass harm, and increase the average ease with which a randomly selected person could cause mass harm , you would expect the amount of deliberate mass harm to increase, even if the fraction of people who would want to do that is like, 0.000001 of people?

Maybe those laws should be considered unconstitutional, idk. But those laws are currently in place.

Of course n bits can only encode for 2^n options. But to prove mathematically that we are close to the best that can be achieved in 1GB requires a mathematical definition of what we mean by better. Rather, it requires at least a choice of a proxy for what we mean by better. (Showing that some approximation of what we mean by better is close to as good as can be, would suffice. Like, if we can show that the loss can’t get much lower with a 1GB model, that would count.)

Also, ok, when I said “intelligence”, it was because you were already talking about how “smart” the model could be. So, I thought you were already on board with using the word “intelligence” to refer to the phenomenon where these kinds of models produce outputs that satisfy the kinds of tasks they are pointed at.

None of those links give an argument that the current 1GB models are the best they can be.

My understanding is that so far when training a model by distillation (using the logits of the teacher model), one can achieve better outcomes than one could if training the 1GB model from scratch on the same training data as the large model, and that so far, better models as the teacher model have yielded better results for the student model.

Like, if you output a probability distribution among n options, and then there is a continuous map from the probability distribution you describe in your output, to another probability distribution over those options…

Err, hm, maybe you need a stronger hypothesis on the continuous map? If it is contractive then it will definitely have a fixed point. I don’t remember the hypothesis needed.

I get that asking a commercial website to be as basic/supported as that website is a big ask. I don’t think the other commenter was saying that such websites should reach 100%, only that they should start from there and sacrifice only as much as is necessary.

What’s the point if you aren’t even going to try to prove a theorem? Or, heck, even really test a hypothesis?

If you include all the information the LLM uses to produce the next token as part of the state, then of course the LLM is a Markov chain.

So would be any other process for sampling continuations of a text, with finite memory.

And, my point is that the inputs it is often fed are not in the convex hull of the inputs in the training data.

When the input space is very high dimensional, this is a common outcome.

I’m not denying that the outputs are causally downstream from the training data. Of course it is.

I’m saying that the inference time inputs aren’t in the convex hull of the training time inputs. This isn’t about saying that the output isn’t because of the training data. Of course it is.

But when you have very high dimensional input space, then even with many inputs in the training data, it is still common for inference time inputs to not be in the convex hull of the train time inputs.

This has nothing to do with the complexities of how the models work after the initial embedding of the tokens as vectors. It’s just about the inputs that appear during training, and the inputs that appear at inference time.

> But an LLM can not infer a concept to which it has no information channel.

Of course! And nothing I said implies otherwise. Really, the point I’m making doesn’t even depend on what the model outputs!

If I took a best fit line from 1 parameter to a 1D output, and then provided that linear model an output that was outside the range of inputs the best fit line was obtained from, that would not be interpolation, it would be extrapolation.

It is similar here, except instead of the input being outside the convex hull due to being further away, it is outside the convex hull due to, like, the shape of the convex hull of training inputs just doesn’t include the point in question.

By "If you need new language" do you mean like, coining new words?

I don't see what would prevent them from doing this? LLMs can process text that includes newly coined terms, and respond to that text in ways that use those newly coined words in accordance with the descriptions of the meanings given for those new words in the prompt. They can also make up new words+definitions when asked to do so. Now, whether they can, without being told to do so, recognize that it would be useful to coin a new word for something, and then start using it, I don't know of any instances of this, but based on the previous two things, I don't see a reason to expect this to be fundamentally beyond what they can do?

I don't know what it would mean for a concept to be "independent of the existing language they are trained on". If there are ideas that can't be expressed in terms of the semantic primes all ideas we can express can be expressed in terms of, then I guess such an idea would be independent of our language, but I think that's a much stricter condition than what you mean (and I'm not sure if there even are any good ideas that can't be indirectly expressed in terms of semantic primes -- I kind of suspect not, unless they are like, ideas that are too big to fit in a human mind anyway).

Of course, the outputs these models produce is causally downstream from the data they are trained on, and the distribution they produce over text is largely based on the distribution over text in the training data, but altered in a number of ways (for example, to make them implement the character of the "assistant" persona).