There is a piece of knowledge you seem to be missing. Yes, a transformer will output a distribution over all possible tokens at a given step. And none of these are indeed zero, but always at least larger than epsilon.

However, we usually don't sample from that distribution at inference time!

The common approach (called nucleus sampling or also known as top-p sampling) will look at the largest probabilities that make up 95% of the probability mass. It will set all other probabilities to zero, renormalize, and then sample from the resulting probability distribution. There is another parameter `top-k`, and if k is 50, it means that you zero out any token that is not in the 50 most likely tokens.

In effect, it means that for any token that is sampled, there is usually really only a handful of candidates out of the thousands of tokens that can be selected.

So during sampling, most trajectories for the agent are literally impossible.

This comes to the core of the issue, and is where I think the disagreement comes from. Many Westerners in fact do not want "Western" values to prevail.

Why? For me those values have led to outcomes so horrendously antithetical to _my_ values, that I would not wish them for the rest of the world. Even worse, this Western centrism has led to jingoist conclusions for at least 400 years.

You totally can fly on biofuel, but it is not cheap compared to fossil fuel without externalized costs.

But as stated above, next token prediction is a misleading frame for the training process. While the sampling is indeed happening 1 token at a time, due to the training process, much more is going on in the latent space where the model has its internal stream of information.

LLMs are trained to do whole book prediction, at training time we throw in whole books at the time. It's only when sampling we do one or a few tokens at the time.

I just want to point out that this is most likely not true, and that this is cultural. The long argument you can find in the book "The Dawn of Everything".

In short, when the West came into contact with other civilizations, one of the most striking features of our culture from their point of view was how hierarchical we are.

Unless stunning has a technical meaning I'm unaware of, I like this approach of starting a technical paper with something less dry.

So the question becomes if it was done knowingly or recklessly, hence a police raid for evidence.

See also [0] for a legal discussion in the German context.

[0] https://arxiv.org/html/2601.03788v1

I would almost always take the opposite side of this bet. Once responsibility becomes diffuse enough, people would actively poison themselves as they see no alternative.

These are all stories about the facility the 5 year old toddler from last week is kept, a facility known as "baby jail".

https://www.proskauerforgood.com/2018/06/pro-bono-for-immigr... https://www.aila.org/blog/volunteering-in-family-detention-s... https://www.americanimmigrationcouncil.org/blog/stories-reve...

[1] https://www.theartnewspaper.com/2023/12/07/hampstead-heaths-...

Department of Homeland Security makes a lot more sense, but as a non-American, is not an acronym I am familiar with.

As a continental European, I do find the ick Anglo countries have with ID weird. Especially if you throw ICE and immigrants into the mix, the whole thing seems designed for collateral damage.

I don't really see how insect powder would be worse than the flour they get now. You don't even need to turn the bugs into a powder.

You absolutely can fool a lot of compilers out there! And I am not only looking at you, NVCC.

So he cannot say "OpenAI made 20B profit last year." but can say "OpenAI will make 20B revenue next year." Optimism is not a crime.

I don't understand, or do not spot the issue you are seeing. Could you expand a bit?

Quoting "The satellite-based linear trend 1993–2011 is 3.2± 0.5 mm yr−1 , which is 60% faster than the best IPCC estimate of 2.0 mm yr−1 for the same interval"

But, as the authors point out, the worst case forecasts that were within-data, are so for the wrong reasons. Quote "The model(s) defining the upper 95-percentile might not get the right answer for the right reasons, but possibly by overestimating past temperature rise."

My previous comment is regarding Figure 2, i.e. "Sea Level". I would invite you to read the whole paper. It is only 3 pages and written without jargon.

We're worse than their worst case scenario, so their models were significantly too optimistic.

In the same paper, they also note that for temperature, the models have been accurate.