There are fewer of those on Linux but they still exist.

There's just no way it's a significant factor in overall sentiment.

That argument applies better to blast furnaces than to data centers, but I don't see people complaining about steel. Data centers just don't take up much space or power in the scope of industrial manufacturing.

Your other arguments are legitimate, but "it's a waste of electricity" and "data centers make some significant difference to a nontrivial number of people" are not the same propositions.

If you are willing to block browsing all ordinary web sites fronted with a CDN, then yes you can block reality/xhttp. You cannot, however, differentially block it via any of the three things you mentioned.

"Institutional" or "structural" racism doesn't just mean racism by one or two people in power. It's the idea that the majority of society demonstrates some kind of racial bias, by whatever means.

Society is made up of people.

One of two things must, logically, be true:

1. A SUBSTANTIAL portion of the people who make up society exhibit some kind of racist behavior, or

2. Structural racism is not a widespread issue

Which one of these two propositions must one believe is likely if one is researching the impact of structural racism? Keep in mind people do not generally don't go looking for things they do not believe exist.

In other words, people don't like other people believing they-en-masse discriminate (even IF they do), so taking actions that only make sense if you think that poorly of the everyman offends them. It's not about what someone wants to be true, it's that investigating implies a level of distrust in society some members of that society find uncivil.

To use a blunt analogy, "why not let me check your underwear to make sure you haven't soiled it? Do you just not want it to be true?".

There is nothing wrong about having that philosophy, and I respect it, but personally I think if it's impossible to tell two things apart using any external observation there is not a meaningful difference between those two things. "Smells like a rose" and all that.

Remember, if the mapping from input to output is identical, there exists no test operating on the machines' output that can differentiate them. You can't tell from "conversing with" a machine whether it is or is not doing what you say around "inspecting" the input.

Do you have some concrete example of a transformer that cannot be represented as a mapping from inputs to probability distribution of outputs?

I say they're equivalent because it is possible to losslessly convert one to the other by wasting massive amounts of disk space and time.

As a second example proving the point, imagine you sampled a transformer's output for a certain context 85 trillion times, and put the output token frequencies in a table. Repeat for all possible inputs (of which there are a finite number). Then you built literally a hash map looking up the context and spitting out the distribution. That certainly is NOT a transformer any more (it's a hash map!!!), but the output approaches indistinguishability as the sample count increases - if the transformer is reasoning, so is the hash map built from it.

I'm not talking hot air here, they really are provably equivalent because a 1:1, onto mapping exists.

For the record, "X is more expressive than Y" means "there exists at least one thing that Y cannot represent and X can". Nothing to do with size or time.

Both are a lookup table whose key is the entire context window and whose value is a probability distribution for what the next token should be.

You can say the choice of probability distribution in the value is "leveraging the internal structure of the context" or not, but the same tokens in two different orders are two different lookup keys and saying it's impossible to achieve some result with a Markov chain is factually incorrect.

https://arxiv.org/pdf/2410.02724 describes the equivalence formally.

In other words, a Markov chain and a Transformer model are exactly equivalent in power (there is NOTHING that can be done with one and not the other). The Transformer model is just better pretrained and a more efficient compression/generation.

The capability of the LLM is not to reason, it's to generate text that matches the patterns seen in the training corpus. It's possible that all you need to "reason" is plausible text generation. I'm not saying it's not. But nothing the LLM does fails to be explained by plausible-text-generation.

I contend that the best way to understand an LLM's capabilities is to understand the nature of the probability distribution that produced it. For instance, why does an "angry" prompt tend to produce more help than a "polite" one? Trying to explain that in terms of emotions or reasoning doesn't make sense, but it's readily possible to explain through the connections between text in the training corpus...

I'm not trying to argue a model cannot "reason" or have "cognition", whatever those things are. I'm only saying that it's absolutely the case that whatever those things are, they come from its mechanism of predicting one token at a time ad infinitum, and that throwing away a deep understanding in favor of a shallow one is foolish. Just because it might seem to be "reasoning" does not mean it IS doing so, and certainly giving the appears of reasoning does not mean it is NOT a token predictor.

If I knew deeply how the human brain works I would use that understanding instead of saying things like "this person reasons" or "this person thinks".

In summary, I'm not "caught up in" anything - I'm just trying to point out that the original poster here is incorrect in saying that clearly LLMs aren't working through token prediction. They are, and all their behavior is 100% explained by token prediction. That's more than enough for interesting behavior!

If you train the LLM on a corpus that shows people saying the sky is red, you get an LLM that is predisposed to say the sky is red. This is true even if it's also trained on all of the science that explains how and why the sky is blue.

If it were to "figure out" or "reason", it would not have such a predisposition to emit "red" after "the sky is" just because that matches the reward during training.

In other words, the token prediction is important because it both explains the successes AND the failures of the LLM. If there were situations in which a bird could fail to fly, then how it tried to fly would also be crucial knowledge.

At all times the LLM is, indeed, predicting the next token. Anything it does emerges from that.

It did not "figure anything out". It predicted that text describing the use of a radial gradient was likely to follow text describing your problem.

I don't think many people would say an LLM written in Python is conscious BUT an LLM written in Excel is not.

People just don't ascribe consciousness to things that can't converse (or at least emote or give the appearance of emoting), and spreadsheets don't do that.

The reason people are debating the consciousness of LLMs is - obviously - that the LLMs generate sufficiently plausible text that people using them think they're having a two-part conversation. Like I think I might be having a two-part conversation now. Turning your question around, why do you think Hacker News posters are conscious? You have no direct evidence they are.

Also, there are twenty-three different keyboard layouts available (IN ADDITION TO the 24-key macropad).

I think there are legitimate arguments against Framework, but this one clearly isn't cogent.

You can "in certain circumstances" (negligence, overt criminality...) go after the managers. You probably can't go after the managers for things like producing a business plan they could have plausibly believed was legal and causing the company to incur civil liability.

In the situation described in this article, probably both the owners and the managers (likely the same people!) get away without being held accountable, and the victims have no recompense because the company folds.