I AM sure that it is hard to conclusively show that Claude has experience and consciousness. Even Claude isn't sure about that.

But while it is absolutely true that "it is a word calculator" - unless you hold the position that human consciousness isn't neural[1]- I don't see how this is any different from saying saying humans beings are neural activation pattern calculators.

If you're sure that your consciousness isn't neural - then fine: Claude isn't made of the right stuff so couldn't possibly be. But state your assumption up-front.

If one opens up a person and looks at their nervous system the single neurons look complicated, but not especially mysterious.

Given how shockingly little we understand the brain/mind it is hard to be sure that we are certain enough of how we work and given how little we know how LLMs work at any of the many layers above the raw architecture either position can be reasonably held, but not convincingly argued/demonstrated.

Feel free to think Claude isn't conscious - I can't prove to you it isn't. And the amount of theory we still need to learn to be able to is vast.

But don't expect me to be _certain_ that it isn't and couldn't be - you simply can't show that convincingly either.

[1] Penrose thinks we have a quantum nature - so sure no classical computer can be then. Some like Rupert Sheldrake think it's a field phenomenon - very woo maybe Claude has a morphic field as well? Lots of people are sure we have a supernatural soul/spirit. One then needs to take up Claude's status with the Creator.

> A brain is a collection of cells that transmit electrical signals and sodium. ...

That it is a collection of cells? Or that they transmit electrical signals and sodium?

Or do you feel that he's leaving out something important about how it works (like generated electrical fields or neural quantum effects)?

> It is said that the Duke Leto blinded himself to the perils of Arrakis, that he walked heedlessly into the pit.

> *Would it not be more likely to suggest he had lived so long in the presence of extreme danger he misjudged a change in its intensity?*

Be careful of letting your deep, keen insight into the fundamental limits of a thing blind you to its consequences...

Highly competent people have been dead wrong about what is possible (and why) before:

> The most famous, and perhaps the most instructive, failures of nerve have occurred in the fields of aero- and astronautics. At the beginning of the twentieth century, scientists were almost unanimous in declaring that heavier-than-air flight was impossible, and that anyone who attempted to build airplanes was a fool. The great American astronomer, Simon Newcomb, wrote a celebrated essay which concluded…

>> “The demonstration that no possible combination of known substances, known forms of machinery and known forms of force, can be united in a practical machine by which man shall fly long distances through the air, seems to the writer as complete as it is possible for the demonstration of any physical fact to be.”

>Oddly enough, Newcomb was sufficiently broad minded to admit that some wholly new discovery — he mentioned the neutralization of gravity — might make flight practical. One cannot, therefore, accuse him of lacking imagination; his error was in attempting to marshal the facts of aerodynamics when he did not understand that science. His failure of nerve lay in not realizing that the means of flight were already at hand.

Agreed, a flat set of workers configured like this is probably not the best configuration.

Can you imagine what an all human team configured like this would produce?

Or the one who wakes up after 10,000 sleeps?

I'm sure he's going to be quite different...

Maybe that dude (the one who woke up after you went to sleep) is another you, but slightly different. And you, you're just gone.

I can see the appeal.

I'm specifically addressing your use of the concept of determinism.

An LLM is a set of matrix multiplies and function applications. The only potentially non-deterministic step is selecting the next token from the final output and that can be done deterministically.

By your strict use of the definition they absolutely can be deterministic.

But that is not actually interesting for the point at hand. The real point has to do with reproducibility, understand ability and tolerances.

3blue1brown has a really nice set of videos on showing how the LLM machinery fits together.

Engineering in the broader sense often deals with managing the outputs of variable systems to get known good outcomes to acceptable tolerances.

Edit: added second paragraph

But really!?

I'll keep calling it in nice round powers of two, thank you very much.

(A fact that he does mention)

I'm not sure why he thinks current LLM technologies (with better training) won't be able to do more and more of this as time passes.

Some understandable short sentence or paragraph early on needs to answer the main question the title raises.

Then again, just throwing rocks might be pretty effective.

> This experiment was inspired by @swyx’s tweet about Ted Chiang’s short story “Understand” (1991). The story imagines a superintelligent AI’s inner experience—its reasoning, self-awareness, and evolution. After reading it and following the Hacker News discussion, ...

Umm... I <3 love <3 Understand by Ted Chiang, But the story is about super intelligent *humans*.

Like Tatja Grimm's World or the movie Limitless.

PS: Referenced tweet for the interested: https://x.com/swyx/status/2006976415451996358

Ted C

He's mostly in very-confident-but-not-even-wrong kind of territory here.

One comment on his note:

> As an example, let’s say an LLM is correct 95% of the time (0.95) in predicting the “right” tokens to drive tools that power an “agent” to accomplish what you’ve asked of it. Each step the agent has to take therefore has a probability of being 95% correct. For a task that takes 2 steps, that’s a probability of 0.95^2 = 0.9025 (90.25%) that the agent will get the task right. For a task that takes 30 steps, we get 0.95^30 = 0.2146 (21.46%). Even if the LLMs were right 99% of the time, a 30-step task would only have a probability of about 74% of having been done correctly.

The main point that for sequential steps of tasks errors can accumulate and that this needs to be handled is valid and pertinent, but the model used to "calculate" this is quite wrong - steps don't fail probabilistically independently.

Given that actions can depend on outcomes of previous step actions and given that we only care about final outcomes and not intermediate failing steps, errors can be corrected. Thus even steps that "fail" can still lead to success.

(This is not a Bernoulli process.)

I think he's referencing some nice material and he's starting in a good direction with defining agency as goal directed behaviour, but otherwise his confidence far outstrips the firmness of his conceptual foundations or clarity of his deductions.

I've wanted something like this for a while to use with architectural PlantUML diagrams rendered to SVG with hyperlinks linking to their implementations.

I'll give it a spin.

I've experimented with sessions that start with prompts like:

> Hi, Please act as a tutor. I will act as a student. I’m working through the following text of X by Y - trying to engage with it more deeply. I'm specifically trying to let active recall clarify and consolidate my long term memory of it. I also want to make sure the ideas are connected in my memory with my existing concepts and concept maps. So please ask me questions from the text given. Present me with just the questions and then allow me to give answers. Discuss my answer - justify your responses from the text.

and found it generally helpful.