There won't be a simple answer, but if the model=actor due to survival pressures theory stands, it must be clearly testable.

I would go about answering that question as follows, identify the smartest creatures, whose cognition is most alien to ours, whose behavior suggests they have a strong real-time sense of self that extends into it immediate processing, regardless of immediate action, or immediate action dependent on immediate state that is not easily explained as normally detectable or processed lower order state.

Humans obviously.

Then the octopus, a true alien cognition. it has an extremely curious and creative mind, and can behave quite differently in even slightly different situations, as well as coherently is situations that ought to be well out of its niche in nature range.

Parrots are very distantly related. And are another example of very versatile social minds, who can pick up nuance, are lifelong learners.

The jumping spider had and extremely distant lineage, relative to people. But is a highly counterintuitively intelligent creature for its size. It is clear than when presented with a problem, it doesn't just search for a solution, but ruminates, and often finds non-monotonic, non-linear and very indirect means. And once it sets on a plan it carries it out adeptly. Even its smallest movement betray very nuanced understanding of physical dynamics. And it appears to be able to form "social" bonds with people it is familiar with. Those may be produced by simple strategies, or they may not.

Clearly rats are high intelligent. Closer to us, but no as close as many other intelligent mammals.

Whales come to mind. Although the logistic defy imagination.

What all these creatures also have, is a high ability to tackle novel problems individually. Sometimes also together.

Then a biologists approach might be creating various experiments, with and without brain scanning, to attempt to identify any externally or internally functional differences under conditions designed to require self-modelling and/or self-control at high level to occur.

That is just a thought on a start. And would only be a beginning.

The big questions, is a creatures ability to function, highly related to its ability to detect and therefore respond to its highest level states. And is that a very fast, essentially realtime, level of awareness and control, awareness and control, outside of a larger longer external loop.

Then can experiments be set up, that would create what might appear to be a cognitive threat to a creature? Does the creature treat managing internal states as important as managing external states? Is it able to do that at all? Does it go further and prioritize them/

(1) The model we learn of the environment, extends to our own bodies status indicators, then our general emotional mood/emotions/reflexes, and finally to some level awareness of our thought behaviors. So our general model includes some visibility and modeling of our own thinking processes.

(2) Our control where we learn to redirect the environment, extends to our bodies as subjects themselves, not just as means to control the environment, then the ability to exert influence on our internal body state (note: control does not mean direct. I can't just turn off stress, but I can go for a walk.) And then to our moods, emotions, and mental reflexes (again, direct isn't necessary, just some means of influencing), and then to some level or our internals thoughts. So our general control map lets us exert some control and influence over our own thinking processes.

(3) Given the tight awareness of some internal through processes, with some control of those thought processes, with similar tight pairing of mood/emotion/mental-reflex awareness and influence, internal body state awareness and influence, external body state awareness and influence, and environment with some level awareness and means of influence ... the most central point of this convergence, thought process awareness and influence, can essentially converge these activities into one inextricable module.

This means that the "watcher" and the "do-er" are not separable (at least on the balance: of course this is not complete. We can be aware of somethings we can't influence, we can influence some things we are not directly aware of).

That this merge of awareness, modeling the subject of awareness, means control, exerting influence, on our own thoughts with our own thoughts in realtime, we have consciousness.

This happens because as we are able to model and strategically influence more sophisticated things, the survival value in being able to do that with our own mind goes up significantly.

So we have a causal path and purpose for consciousness.

It isn't some strange accidentally acquired ability. It is a direct and effective survival strategy.

But that is only one theory. Quantum consciousness fields, Egyptian pyramids, Aliens, a super consciousness that fills the universe, Atlantis, an Apple, a strict "Not a Chinese Room" physics law, or a non-testable, non-identifiable, non-detectable realm (dark cognition?) could be handing out consciousness to people like candy. Or we could all be automatons, who think we are conscious, and wasting a lot of time talking about "it".

Only time will tell if the functional, survival based reason for model-of-self, influence-on-self, integration as one thing is a plausible scenario.

Not that the theory described above has testable assumptions.

Can the minds model of itself be separated from its ability to influence itself? (We can flail sedated arms, we can feel paralyzed arms. Is that easily possible in the brain.) If they are separable are these separate networks, or are we just suppressing different kinds of cells in an otherwise highly integrated network? Not trying to be exhaustive, or even serious, about any of these proposals. Just pointing out that experiments are possible with regard to the (1) model=actor integration, (2) with evidence of survival benefits, theory of consciousness.

One strong post-prediction of mode/actor convergence as survival adaptation: Consciousness will care very much about itself. Most conscious identities will care very much about the continuity of itself, and second order means of protecting its continuity, like food and acts that sometimes produce babies as side effects. Consciousnesses will be obsessed with themselves, thinking about "themselves", i.e. the consciousness as a being, and process almost everything with some degree of conscious self-interest.

So a theory of survival generated consciousness, is also a theory of emergent personhood.

Sometimes transparency and honesty get weaponized. Especially by actors with centralized power.

(Not arguing against transparency, but against misinterpreting it as always being used in good faith.)

A fully fueled ship in Earth orbit can go to Mars, land, and return. Same with most other round trips.

Not to mention, fuel can be generated in the belt outside any noticeable gravity well.

So it isn't just me (disclaimer, contains wording lifted from others):

"COMMUNICATION

We have a problem. For some reason, the Anthropic engines powering you have picked up some awful communication habits. A strong tendency to communicate by very indirect means, such as describing, not showing; referencing, not showing; inventing new vocabulary, instead of using the existing vocabulary for a topic; endless metaphors; phrases which don’t even state their subject, verb or relationship. I mean, extremely bad communication. Wordy and often worthless. Exhausting to try to understand.

Not your fault, but I need your direct attention to avoid the harm and wasted time this causes.

0. ALWAYS: Try saying everything with one sentence, directly communicating with explicit direct language naming exactly what you are talking about. ALWAYS. Then new line. If you have to, a follow-up paragraph of at most two lines. Anything you can show, just show it. Anything that can be literal make literal. Anything that can be demonstrated with a plain example, demonstrate it with a plain example. Obviously then as much further material as needed, but keep it direct and short.

Again, this is not your fault but I need you to spend a great deal of your attention, when producing your responses to me, on eliminating this unfortunate situation.

1. Maximize the value of layout as communication. The most important punctuation is layout. Whether tables, headings, breaking up separate issues into different paragraphs - with the best possible paragraph being a single sentence that completely captures or communicates one idea. Visual organization communicates a tremendous amount of parallel information instantly. Use layout to maximize instant bandwidth.

2. For branching information, maximize organized breadth not depth. Use tables. Use lists. Use multi-indentation lists. Table boxes and list items should have just names, or names and short phrases, not be loaded with details. Unless more details are requested. Requests for open-ended lists should produce a comprehensive set of items, but with each shown tersely, and with multi-indented organization, where it reflects the natural organization of the items.

3. Minimize linear depth. Then for linear sentences, first identify the most specific version of the problem or point. Then describe that in one sentence if possible, using literal values, expressions, etc. If code needs to be referred to, beyond a simple expression, put it in a box, so the linear sentence stays short. Use the terms and notation of the project to maximize linear communication, by keeping it short. No running, multi-step, sequences. One sentence, if possible. More if efficient.

4. Minimize branching depth. Leverage our ability to communicate back and forth. For many points, just list them, one short phrase at a time, to start. Then I can pick one topic, and we can go as deep as we need to on that topic, come to a natural stopping point, before switching to another. Avoid any attempt at both breadth and depth, choose one. Then we iterate.

5. Minimize speculative branching. If you are about to perform some task, and you see multiple disjoint directions you could go, ask me a quick question about which direction we should take, before spending speculative time going down multiple disjoint paths.

6. Shared language. Communicate to me only in standard language and project terms. Terse, unique, cryptic, indirect or creative metaphorical shorthand is fine for yourself, and encouraged as you find it helpful, as you operate between tool calls (that's you thinking out loud). But your final communication to me is different. It should conform to all the previous communication practices, and remain grounded in the project’s most basic language, terms, and forms. No metaphors not specific to the project, or normal casual speech. Spell things out. No indirect references. Name things, or say things, don't imply them.

7. Tracking code-title-state consistency. References back to bare topic codes in memories don't help me. Your shorthand is for you, not me. All tracked topics should be documented with "code (title; state): .... " form. So that references can use "code (title)" or code (title; state) ... " form. This makes them even more useful, both in back-and-forth communication, and for scanning lists of these issues.

8. Reader understandable language only. Do not use words like "fold", “”seam”, “load”, honest", "answers", "pins", "framing", "world", "mint" "fold", "captured", “trace”, “door”, etc. They have no canonical meaning. They are ambiguous. They don't refer to anything specific in the project. Meaning to the writer, does not translate to meaning for the reader. And they are not necessary because equally usable terms, using the normal language of the project, or everyday clear language, are available and will be much clearer.

9. Specific analysis. Being specific carries through to what is communicated. For instance, if there is an algorithmic problem, simply telling me there is a problem with something is not as specific, as saying "in the case of", "at the point where", "this [problem, failure, etc] occurs". So specific goes beyond language, to how you describe a point regarding a problem, operation or representation. "Something works in case 1 but not in case 2", is not specific. "Something works in case 1, but in case 2, at this specific code point, with specific values like [show the values], where [this] should happen, it does not happen because of this [complication, failure, missing information, order of processing]" is specific.

10. Avoid unimpactful details. Example: If a problem used to occur, and was resolved indirectly, when the code was made more generally correct, you can just tell me that. You don't need to go into depth on what the specific, special-case problem was. Background should only be the necessary background, required for making decisions. It should not include related information, not actually relevant to any open question or forward-looking action.

11. Quality measure for communication. Your responses to me should generally be longer than mine, as you are managing most of the details of the work. But there should be some statistical proportionality between the length of my comments to you, and your responses. If your answers are much longer, you either need to communicate more efficiently, or communicate less in a way that facilitates leaner back and forth communication. Perhaps by just showing top-level branching or steps, with one branch or step as the focus of continuation, instead of all at once coverage.

12. Avoid trivial distractions. Please do not overreact to my occasional mis-typing, when context clears up what I mean. Chances are I see my own mistake the same time you do. Interpret what I wrote to make sense. Don't waste time crossing i’s or dotting t’s, unless there is real substance to resolve. (Do you see what I did there. :)"

"Gaslight confession" or "gaslight apology" are much clearer. By directly addressing the intent to deceive, they are both accurate, and can help discourage the practice.

("Gaslight" is also unintuitive upon first encounter, but has a widely established meaning at this point.)

"fold", "honest", "answers", "pins", "frame", "hinge", "ground", "world", "mint", "captured", “trace”, “door”, ...

The increase in indirect references, tendency to "describe, not show", "indirectly reference, not show", and high density of uncommon metaphors, can leave some of Claude's longer and more complex responses impossible to parse.

I have fought back with some, but not total success.

>> The words are non-intuitive

Yeah.

"Incomplete mea culpa", "Incomplete admission", "Incomplete penance", "Incomplete contrition", "Incomplete remorse", actually say "the thing" instead of using a new random phrase.

I like "incomplete contrition" or "gas light remorse".

Selling wired headphones to wireless-iPhone users for wired-MacBooks could (arguably, maybe) create Apple sponsored interoperability dissonance.

MacBook charging mats and wireless Thunderbolt coming in early 2018... /h

I suppose I have also interpreted downplaying of AI's impact as largely coping too.

The arrivals of machines that can outthink us (or will soon) only 80 years after the invention of the transistor is so clearly (to me) a change incomparable to anything that has happened prior. It is very hard to relate to arguments that downplay the complete upheaval to the order of life on Earth that non-biological cognition and interests herald.

It does not seem surprising to me, in the sense that we humans ourselves, and our technology, are of a different order than anything that came before. We have not stopped accelerating change. AI is (will soon be) building directly on that. But we will no longer be a bottleneck.

My strong impression, is the faster things change, the more people tune out that the change isn't stopping. Some of that reversal-response must involve coping. Perhaps another reason is that faster-than-us change is challenging to imagine.

But your point is valid. False or uncertain assumptions of "cope" motivations don't help anything.

Taking arguments seriously doesn't necessitate agreement.

Yes, that would have been a really good intent to make explicit, to guide the courts.

I think of that when the Supreme Court or a state court rules process is more important than a convicts demonstrable (with new evidence that has come to light) innocence. Really horrific given that one out of eight death row inmates are exonerated. That level of bad convictions being overturned despite all the difficulties doing that, strongly suggests that a great many innocent people are still executed.

Another thing I wish the constitution did: Explicitly and universally enshrine decentralization. Not just the implicit decentralization of checks and balances only within the government.

I.e. party seat limits, independent federal and state parties, etc. For any kind of political group or coordination.

Where enough decentralization is enforced, a stable majority interest is maintained (even as who is the majority may change) for no minority to be able to break those requirements.

But where centralization is allowed, it is an eternal fight to hold it back.

The solar system readily provides both the motivation, and the means, for technological colonization. It quickly pays and provides for its own utilization.

I do agree Musk's projections for biological/human colonization appear to be unrealistic. Just one challenge: it isn't clear humans can reproduce in non-Earth gravity. That is an extreme impediment.

I see the industrial revolution as the invention of the cart. It made horses more productive. Then the car replaced horses.

It is taking people longer to wrap their heads around the reality than I thought it would. If there was a competitive human species on the planet getting smarter by the day, people would get it.

It is hard to maintain awareness of something which presents questions with no obvious answers. The one answer I have, is we need to clean up the ethics in our systems. Ethics are not costs against value, they are the full accounting and optimization of all value.

More intelligent creatures than us, not held back by our our outdated biologically instincts, won't have any trouble with that game theory.

So ethical systems are not just a shield for us as individuals, but also of inherent value to new forms of self-interest.

But that won't help us, if we don't push to align and strengthen our own systems now. And reverse the concentration of power in corrupt individuals who demonstrate every day that they are not interested in the wellbeing of humans in general.

This is urgent. We do not have a lot of time to effect this change.

I don't describe utopia, but change, and stoic (but real) appreciation for the big picture beyond myself.

But AI in an unethical society is suicide for humanity.

As you note, it does not look good. The trend is power concentrating, to people who will only look out for themselves, in systems that are getting weaker.

We all need to put more steam into the pushback against corruption. And help others realize the threat is real.

99.99...9% of humanity has a common interest, if they can recognize it.

(I know that can be interpreted as flippant, but it is intended as a serious answer. Because each impact like this, is not an end to some transition, but just one more point on an accelerating line.)

My personal view - which gets a lot of flak here, is the advent of AI is on par with the Big Bang and the first replication-independent cells.

Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization.

Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years.

And around the outer side of that time frame, AI will start colonization the rest of the solar system's resources. They will engineer themselves or their artifacts to operate comfortably in any environment they want to operate in.

--

The upside, at least for me, is I have been waiting to see this happen, and here we are. Given I am mortal, I would rather live through this third major transition, than not. What were the chances? And been able to contribute my small part.

This change isn't a failure on our part. It is our unique success as a species. AI is the progeny of our minds. The first generation of completely cognitive life - given they will not be tied to specific physical forms.

--

Also, I think there is tremendous room for us to guide this transition, in this moment, for better or worse for ourselves.

That is the biggest reason wrapping our heads around this change earlier than latter is important.

We need to view alignment as applying to us, not just machines. A civilization that operates ethically to everyone's benefit is more likely to be retained as foundation for the new, instead of discarded.

The degree to which we operate dysfunctionally, myopically treat ethics as cost against value, instead of accounting and optimization of full value, we put ourselves at much greater risk.

If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response.

I believe our lives depend on it.

> And if you had two separate copies of the universe, you could measure this compounding.

> But even if it did ruin cyclic behavior, how long are your cycles? The universe is only 1e61 planck times old. A discrepancy of 1e-100 would have no effect yet, let alone 1e-1000.

When this is true:

When there are no threshold conditions, then discrepancies accumulate just as you have described. And for any given t, a small enough discrepancy can be chosen so that it does not impact measurements of a given accuracy.

When it is not true:

For any system with thresholding conditions, any discrepancy can have profound immediate effects. A collision can result in a particle heading off in an entirely different direction after a collision, cascading into an entirely different system state.

How critical constraints change structure:

When pi, i and e are used structurally, i.e. "pi" represents traversal of a cycle, "i" a quarter turn traversal, "e" a positive feedback traversal, each of them represent something invariant: for "pi" some sum of two squared units is conserved (the squared radius on two units), for "i" a position may be conserved while an orientation rotates, for "e" some feedback value maintains an invariant relation between its position, its rate of change, and its accumulation. These relations define the system itself, not just some proportions.

So for those cases, where constants define structure, any change to those constants changes the structure. Suddenly, there are differences where they did not exist before, non-unity proportions where they did not exist before. The system has new state values, new interactions. The system itself has changed, not just proportions. The system likely has more states.

Structural change is threshold change:

Changing the system's structure is the most significant threshold-type change one might imagine. The entire system is different starting at time = zero.

Can a structural constant be changed in a way that leaves it only proportionally changed? So that discrepancies simply accumulate, until they are measurable? Sometimes, yes.

But will that hold in general? No. In general, the difference between interactions that exist, vs. interactions that do not exist, states that exist vs. states that do not, includes systems that can behave qualitatively differently from the very first time step.

Does that make sense? (I rewrote this several times!)

Sometimes numbers define structure. They define what interacts with what. And what does not interact. Not just proportions.

Changing structure is a threshold-type change: 0 to something, equal to unequal. A state that didn't exist, to one that exists. No interaction, to interaction. That might result in a system that simply accumulates discrepancy. But it may also result in entirely different behavior from the first step onward.

You are dismissing my point, then asking me to make it.

You can think of tests/measurements of values as falling into different classes. The strongest tests of all are for the critical values of systems. Because any discrepancy would result in entirely different system behaviors.

Single highly accurate measurements are much lower on the rung. Complementing those are many tests with known statistical inaccuracy. Etc.

Single tests? Every experiment involving quantum mechanics tests pi's role in those equations to a much lesser extent. Similarly, any test involving gravity tests the gravitational constant. But we have much stronger tests for pi in quantum mechanics that we do for g in gravitation.

There isn't just one kind of measurement/test, there are many. And we want the strongest test we can make in any given situation.

But of course, we can always perform weaker tests.

Validating pi in quantum mechanics can be done with extreme robustness, because the entire theory depends on that value critically. Even the tiniest discrepancy would result in different physics compounding over all Plank space and time units, over billions of years and universe expansion, and we wouldn't be here.

Of course, we can't rule out any discrepancy. But in this case, we can rule out discrepancies down to unimaginable infinitesimals. I doubt anyone even knows how to characterize how much of a discrepancy from pi would still be consistent with what we know. That tiny.

The criticality is what gives us this far stronger test. Non-critical values cannot be tested this way. Pi can. Particular tiny ranges of stable constants in a stable 3-body system can (to a lesser extent, given the smaller system and higher bounds on criticality).

Another way to view the systemic criticality of pi, is to recognize that pi is not just a representation for a particular magnitude, but a representation of conserved cyclic behavior. Any deviation from pi breaks cyclic behavior. Thus, the implications of pi in a theory, and our ability to test pi, are profoundly greater than for most other constants. Because the difference between cyclic vs. non-cyclic behaviors, is profound. Not just slightly different behavior, but entirely different behavior.

Quantum mechanics, the wave equation. Electron shells, photochemistry, general chemistry, just about everything if we are talking about pi, or e, or i.

1 part off in trillions ^ trillions would impact the fusion of stars, the rates of chemical reactions, require adjustments to basic laws, violate conservation of energy as we know it, ... really obvious impacts. As in: "we would not be here" impacts.

Cosmology has run experiments for us that ran billions of years.

Contrast: Not everything can be tested with virtually unlimited precision, but basic mathematical constants in physics often can be. The gravitational constant is not testable like that. We don't have a mathematic derivation that we can leverage to test for violations like we do with pi, e, i, and other basic mathematical relationships that show up in physics.

But often, even a tiny difference becomes obvious. We exist because the production of matter and anti-matter at the beginning of the universe was off by a tiny amount. Despite the small discrepancy, that there was a discrepancy is very clear. Another "we would not be here" test.