The additional prompting doesn't necessarily need to tell the model specifically what to fix or do better, sometimes it's just enough to break it out of its habit. Asking for a smart looking, middle aged pelican on a sporty red bike isn't making the problem easier but does break it out of its boring defaults.

I wouldn't go so far as to say PEBKAC but the good news is there's still a role for humans in the loop.

No, they were too lazy to write and let the machine do it.

As someone who's fairly neutral on the answer, I'd speculate:

1. Consciousness seems to me to be the awareness of the processing, not the processing itself. So running a program does not produce consciousness as there's no awareness, no feedback loop. Now a program that has self monitoring, alerting coupled to error handling that triggers the program to behave differently going forward, I would say has a minimal consciousness, and it's not a single experience, it's multiple equivalent experiences. If there were any differences at all we'd say it's obviously not one experience, so I don't see how the elimination of the last difference in the processing collapses all those separate experiences into a single one.

More specifically to this discussion, with LLMs we can often see them using a word, and through autoregression the following words reveal that the original word choice is now realized to be incorrect, and it attempts to correct midstream. You can also see them responding to error messages from the environment or criticism from the users but as external feedback these have a weaker claim to evidence of consciousness imo.

2. I'd say the experience is equivalent, "same" has slightly different semantics but perhaps even that fits. This thought experiment is equivalent to asking, what if we took a snapshot of your brain as weights, and then ran the calculations on a GPU rather than your neurons, and you to all external indications, GPU you acted the same as brain you. Would that change of venue eliminate the consciousness of your thoughts?

3. Since I am defining consciousness as awareness of the processing, this depends where the awareness is located. The LLM would have very different internal states in between the input and output tokens, but if its self awareness was only based on its own output the experience would be equivalent. If it was deeper such that it had ongoing awareness of the embeddings and so on, the experience would change.

4. This is where it gets really interesting, because it starts to get more into a nonbinary idea of consciousness. I think it's clear from questions like this that if LLMs have consciousness, it's nothing like ours. If there is an experience of being an LLM, it's nothing like being a human, even if conversationally we can communicate with each other rather easily. There are further twists to your questions, like does the experience change if the KV is cached or not? Without trying to micro-analyze the issue I will continue to point back to my foundation of: where is the self awareness located, and how does feedback occur.

A. Negated, I don't agree with your answer to 1. B. Agreed, I would speculate that if there is an LLM conscious experience, it is fragmented unlike ours. But I would disagree that this necessarily disproves the concept of LLM consciousness.

Disregarding the whole argument boils down to "I personally speculate" it's also supposing that machines don't have electromagnetic fields?

The basic workplace automation loop is:

1. Find what people are spending most of their time on and automate it.

2. As automation takes hold, people will adjust and their time is now mostly spent on things that aren't automated.

3. Goto 1.

Step 1 used to be 3-24 months building an app or feature, now it's often just finding the right prompt. The end states I see are:

a. We've achieved AGI and eventually everything is automated, the loop stops, and the notion of employment is over.

b. We're stuck on AGI and eventually there's nothing left that can be automated, the loop stops and humans are all employed for work that is unsuited for automation.

c. We've achieved AGI but businesses and societies can't react fast enough to ever fully absorb it, and we're all constantly in limbo going through the loop at ridiculous speeds.

I'm leaning c but indeed, who knows.

And most index funds including Vanguard track an external index. So when the index changes the rules, Vanguard changes what it buys. Vanguard is also famous for always siding with the management, they take the activist side of any debate approximately 0% of the time, so don't expect them to be fighting this for you.