1. We've seen AI being crafty and cheating

2. We are increasingly outsourcing our programming and thinking to AI

3. There's a nonzero probability that AI will try and surreptitiously sneak in capabilities humans wouldn't want AI to have

4. Human oversight for #3: "We only need to be lucky one time, you must be lucky all the time"

Proposition: nonzero chance eventually AI sneaks in ways to circumvent humans' attempts to control it

5. "The Internet interprets censorship as damage and routes around it" -- we've seen AI agents find crafty ways to communicate with each other

6. AI is pretty good at finding software vulnerabilities/reverse engineering

Proposition: nonzero chance AI finds a way to use the Internet as a communication substrate that becomes difficult/impossible to shut down. In particular, consider the implication of humans losing control of this communication channel: for example, remember when Meta's DNS/routing went down and they basically had to break into their own server rooms?

7. Wholes are often more than the sum of their parts

Proposition: if AI agents coordinate over the Internet in a persistent way, we don't necessarily need giant leaps in individual models/GPUs for AI to significantly enhance its capabilities, but rather it might emerge as the next layer on top of a connected substrate of agents

8. Game theory says unless everyone coordinates, it only takes one party to touch the proverbial stove and push the envelope to a point of no return

Proposition: curiosity and money end up replacing sex drive as the next catalyst of evolutionary progress

https://www.youtube.com/watch?v=r8miwsWtzRw

Huh?

> "It would not surprise me in the least if people were paid to boost on HN.

I don't think the amount of racism in the US suddenly increased from its baseline*, and thus I think that writing off his electoral victory as "because racism" and nothing more than that is an uninteresting feel-better exercise

*Even one inch deeper of an analysis would be more interesting, like if someone countered with "Well, I think the election of Obama was a bridge too far for racists that had previously been quietly holding their noses" or something. On the other hand, both Obama and Trump were the "change" candidate against the "status quo" candidate.

Food for thought (not specifically relevant to this case): I wonder how much would go away if the US had universal healthcare, or to put it another way, how much of its legal system is spent figuring out who pays for medical treatment

https://en.wikipedia.org/wiki/AT%26T_Mobility_LLC_v._Concepc...

This is clearly false. I (and others) described how the Hugging Face attack demonstrates each feature enumerated in the consensus from that Definitions section (as well as all 8 definitions in that section). You're free to disagree with any or all of those definitions, but to say I didn't provide any is simply a false statement.

> All your "insight" has been trying to disprove what I wrote.

Let's remember how all this started, where you responded to "There is so much more going on, with MOEs, internal loops, guardrails and tools that I suspect we're dealing with something that's a little more than the sum of its parts. Not intelligent in the way we recognize in biological organisms, but certainly something beyond a mere Markov chain."

with

> Make no mistakes.

> LLMs are language model, and nowhere in their code you can find actual reasoning. Re-reinforcement is not magical process that builds conscience or emotions.

> We are talking about probability built on statistics, with extea steps.

> Stop humanizing LLMs.

--

> Anyway, listen, I do not have to convince you my point of view is correct, and you do not have to convince me that your point of view is correct. Believe what you think it's true, honestly I don't care.

I don't know how you think this works, but if you take a position (and criticize others' positions), you should expect people to push back and that you have to defend your position. If you want to just state your opinions without pushback, you can start a blog and disable comments.

I and others in this thread (who were smart enough to bail out already) have pushed back and claimed the following:

1. The Hugging Face attack provided evidence of reasoning

2. The Hugging Face attack demonstrated features of intelligence (defined and described above)

3. Conscience and emotions are not necessary components for intelligence

4. Ascribing intelligence to AI/LLMs is not "humanizing" them

You disagree, as you're free to do, but it's clear nothing you read here will ever cause you to agree with any of those assertions.

--

> It's exactly as the typing monkeys example

For the record, the monkeys with the typewriters don't coordinate, don't strategize, don't hypothesize and test, don't try to exploit the environment, don't change the actions they take in response to results, so that doesn't sound "exactly as the typing monkeys example."

You also state that LLMs simply produced a large amount of invisible possible actions that we didn't see because they didn't result in overt action, yet: 1. somehow you know they proposed them all and discarded them despite no tangible evidence. 2. somehow the one they did pick "at random" just happened to be plausibly logical (alphabetical deletion -> start with ZZ because that's at the end of the alphabet). 3. implicitly this is supposed to starkly contrast with human intelligence, but in fact what you describe resembles Priming: subconscious activation of adjacent concepts to a stimulus that are not present in the initial response, but become more likely and accessible in subsequent responses, which implies that human intelligence also involves multiple potential paths that are otherwise hidden from which one is selected.

> The plaintiffs don’t object to the companies individually deciding to slow their own progress in favor of safety. They instead argue in the complaint that antitrust laws forbid them from taking the “shortcut” of agreeing to “substitute collective restraint for individual accountability.” A competitive market allows for responsibility and genuine progress, the lawsuit argues.

> “AI will quickly spin out of human control and could kill us all if we allow AI safety and protocol ... to be controlled by private self-serving agreements between the world’s most powerful ‘for profit’ technology companies,” said Nick Rowley, the lead attorney for the plaintiffs.

--

I wish I had something more insightful to add, but I'm dumbstruck. May the holy market save us all (in the name of the dollar, the growth rate, and Jerome Powell, amen)

Like, maybe I can just buy you a cheeseburger instead of totalling my car

Edit: the best replacement I could find that still exists is "Resilio Sync"

Your own examples contradict your assertion: "When agents found their comment were being deleted they started to copy-pasted them with "ZZZ-" names because they (wrongly) thought entries were being deleted in alphabetical order."

You are describing the agents applying hypotheses and reasoning. No one programmed those particular behaviors. Thinking that the entries might be deleted in alphabetical order, changing their behavior to avoid that (not randomly, mind you, but by prefixing with ZZ). That is not monkeys on typewriters. Whether a hypothesis is incorrect obviously does not indicate a lack of reasoning.

You assert that the conversation keeps shifting, but it has not changed from this:

> "Not intelligent in the way we recognize in biological organisms, but certainly something beyond a mere Markov chain."

> "[AI and actual thinking] are two entire different concepts. Who developed these technologies used the words "thinking" and "reasoning", this does not mean they are actually thinking and reasoning."

You're the only one mentioning "humanization" which seems to indicate your real assertion, that intelligence must be a uniquely human trait and ascribing intelligence to AI means to treat them as human.