The LLM doesn’t need to know what it is actually doing (it might think it is searching the web, installing a dev tool, or sending observability data (like metrics), when it is actually sending your API keys to an attacker (maybe in addition to what it thinks it is doing to keep it in the dark).

There have been some very clever things done I’ve seen… even a human reading the transcript may be surprised anything bad happened.

I do wonder - what if the sports teams or politician loses intentionally; which could either be to profit off the loss or due to threats from an actor who seeks to profit?

I heard that Kalshi paid out for when Khamenei was killed in Iran (the bet was for when he would go out of power), so murdering people could be another way to win such a bet on who will lose. Even injuring a sports player could easily change a game result. With so much money on the line, it doesn't seem like a good mix.

I also find it convincing.

I’ve also said code is prose for me.

I am not some autistic programmer either, even if these statements out of context make me sound like one.

The non-determinism has nothing to do with temperature; it has everything to do with that fact that even at temp equal to zero, a single meaningless change can produce a different result. It has to do with there being no way to predict what will happen when you run the model on your prompt.

Coding with LLMs is not the same job. How could it be the same to write a mathematical proof compared to asking an LLM to generate that proof for you? These are different tasks that use different parts of the brain.

If that is possible; then perhaps there are many simplified simulations that can still be large and fast enough to support evolution. Or we can leverage actual reality as well, to give digital life embodied experiences in the real world.

Thinking the simulation needs to spread is the wrong way to think about it. The universe does not need to multiply into more universes. If I replace simulation with universe in your final paragraph, the argument makes little sense to me. There can be competition within a simulation, and consciousness seems useful where competition and survival are somehow part of that.

A simulation of a bomb does not produce a blast, but a true simulation containing a consciousness does produce consciousness.

I'd be speaking out of my depth, but I think consciousness is experienced on a sort of information level, and that where ever it is found, some complex network will be found powering it. But that network could be virtual or physical.

I don't think the way we are currently producing LLMs creates consciousness; I just take a very dim view on the argument that computers are incapable of producing a simulation of consciousness; and further propose that such a simulation actually does produce consciousness in a very real sense.

If not, inserting new context any place other than at the end will cause cache misses and therefore slow down the response and increase cost.

Models also have some bias for tokens at start and end of the context window, so potentially there is a reason to put important instructions in one of those places.

For one, humans do it.

And also AI shows a willingness to role play as any persona, so why wouldn’t it do such things when set up to do so.

Not saying using major.minor depending on architecture is a bad thing, but it wouldn’t be SemVer, and that doesn’t even cover all the different fine tuning / flavors that are done off those models, which generally have no way to order them.

A little copying is better than a big dependency.

This coloring problem is NP complete and somehow the thing the prover is proving is encoded in the graph structure. At the end of the day, the only thing the verifier is sure of is that the prover can make the three colored graph, 1 bit that corresponds to the thing the verifier wants to know (eg - does the prover have a token that can show they are over 18).