Regarding your book, I'm shocked that 'AI Agent' concept predates me. Have not read your book, but i think it would be interesting to compare your perspective with today's building blocks which were not available for that time.

For social backed simulations i guess some kind of grounding will be needed based on real examples, but then the out of distribution cases will need an other solution. As rate of changes in our civilization increases, the out of distribution cases will be more and more prominent.

Few months ago I did experiment with an open-ended world simulation for AI agent, where the simulated world was progressively building itself based on each of agent actions in open-ended manner. The idea was to give an agent infinite possibility regarding tool calling, where the tool call would be approved by the adjudicator, and the world state would change. The key issues with the PoC were:

  - World decoherence (tried to solve that with a poor graph implementation)
  - World flatness - high abstraction did not account for small events that would compound in real world
  - Start with empty context was real issue to get the agent to explore the world
  

Not sure how you manage to measure Faithfulness and Answer Relevancy on the live system, without the ground truth.

Good that you have evals in place, but the user satisfaction score might suggest running ablations on the system would be beneficial. I would start by reducing the iterations and unnecessary steps from the agent.

Let's say we have a low level programmer expert and we try to teach him algebra either we:

  - (SFT): give him algebra book with new nomenclature, definitions, syntax
  - (RL): let him learn algebra using C syntax

  - https://arxiv.org/abs/2602.02660 (MARS)
  - https://arxiv.org/abs/2601.14525 (Execution-grounded automated AI research)
  - https://arxiv.org/abs/2601.10402 (ML-Master 2.0)

- The cost function of the CEO is a deeply nested function, as for example revenue is dependent on multiple factors like customer growth, operational costs etc. In practice this function is nested to the lowest possible point in a tree of dependents. No coincidence that best CEOs know how they companies work on lowest levels and understand the constrains and incentives there.

- Where is the boundary of AI CEO? Should we make the boundaries? Human is restricted to 18h awake time, single threaded processing, thats why we need to divide the workload. Is same thing really needed with AI? (assuming all layers of the stack are AI managed already)

- How to validate that decomposed elements of the cost function, (like the customer growth and operational costs revenue) provide valid equation? The cost function should take into account things like reconstruction loss minimization, to ensure that model is not delusional and new dependencies are valid (problem discovery).