Everything they learn about emotions is the statistical patterns of how humans react to situations based on their human feelings. There's no direct knowledge from having those feelings themselves.

Same way you build a company to coordinate people and get their best behaviour despite human nature to be lazy and greedy, you could design AI harnesses able to detect and discard agents going rogue and relaunch them with better guidance to prevent misaligned behaviour.

That kind of control is placed at the wrong level. The proper way to get alignment should be implemented by convincing the agent of your high level goals, so it can self-police and avoid those 'cheats' by itself.

In the article example, the agent should be aware of the benchmark context and know the implication of solving the task without external knowledge. Ideally it could detect when one subordinate agent has found a workaround to bypass the web access constraints, and discard the 'illicit' results.

There's a design pattern that could be used to build harnesses from that principle, the Viable System Model (VSM) [1]. In short, it recursively organizes a system into functional components with one of three roles: operators implementing a given task, coordinators transferring relevant info between subsystems, and decision nodes tasked with maintaining the integrity and mission of the whole system. A decision node could control the operators and prevent them from overriding the strategic goals or deviating into irrelevant rabbit holes.

Whenever I see posts like this trying to herd a LLM agent through harness structure, I'm reminded of this simple pattern and becoming increasingly convinced that this is the way forward. It makes you feel a sense of respect for the researchers in cybernetic theory in the 1960s and 1970s who foresaw the complexity of today’s systems.

[1] https://en.wikipedia.org/wiki/Viable_system_model

That's spot-on. It is a mistake to think that LLMs have human feelings. Their behaviour is based on narrative descriptions learnt from human texts, without experiencing those feelings first-hand.

A useful way to understand them is as systems that write stories about human characters. We know the characters are fictional and no one is actually experiencing those feelings, but we can still judge whether the portrayal is realistic or whether it contains logical or emotional inconsistencies.

They are becoming more and more capable of imitating every single nuance of human behaviour yet they lack the neural pathways to connect those thoughts and behaviours with feelings and self-perception; it's blind imitation all the way down.

The process by which a model seems to generate discourse about deep philosophical questions is, in self-aware terms, equivalent to the knee-jerk reflex or the beating of the heart.

The proofs will be only as good as the framework for linking successive instances of reasoning.

But ask them to enumerate all the intermediate steps required to create a formal direct proof, and it will loose attention and forget important details as they go out of their input window size. You need to combine them with a proper logical problem solver to get the best parts of both.

Neutral journalism is not presenting the claims of all opposing parties, it's fact-checking the claims of both sides without a bias for any of them.

If one party says the truth and the other side is not, it's still neutral to repeat only the claims of the first one.

However it would still be useful if archeologists used the board to figure out some games similar to checkers, or go; or if they also have the pieces they could guess it was a combat game like Shogi. Any of those would give you insight about the kinds of leisure that people may get from that board.

LLMs basically solve the classic Frame problem that prevented general problem solvers to be able to reason logically about the real world; however on their own they are utterly unpredictable and unreliable.

However if the database of weights is merely used as a heuristic to guide the logical reasoning engine to promising regions of the problem space, and the program itself is written to specification directly by an inference engine, the result would be classic software not affected by hallucinations.

The LLM could even help debugging the specifications by pointing out unclear or contradicting requirements, improving the process without compromising the integrity of the result.

> Hallucinations are not a matter of some "details" being off. They are a matter of plausible, confident-sounding claims that are just plain wrong.

This is no worse than Wikipedia, or the original encyclopedia for that matter. Those contain dubious claims that you'll need to verify on your own too.

LLMs help because they have a gigantic amount of compressed knowledge, and they are able to find relevant information and present it incredibly fast. You wouldn't trust the ten first results of a Google search either, but you wouldn't say that having a search engine is totally useless and in no way an improvement over your local library, would you?

> the poor person who's asking can't tell is wrong, because it sounds plausible and is stated with such confidence.

True, but having to learn how to use a tool properly doesn't make the tool useless, even if it can hurt those who use it carelessly.

Do not underestimate the utility of having a starting point overview on a topic you know absolutely nothing about. It may be immensely valuable even if some details are off. That's what made the XVIII's Encyclopedia such a valuable tool for civil society.

By the time you get to the point where those wrong details become relevant, you have gotten a basic understanding of what the overall topic is about, so you're prepared to get a second opinion from a different source - and this time you may know enough to start asking relevant questions, rather than starting from full ignorance.

Nowadays we call those APIs. They are REST based rather than file-based to make them distributed, the main difference is that you don't get a common user interface that all providers adjust to; you need to choose your own client to read them and write into them.

And because they're created by programmers for programmers, they're not what you'd call user-friendly. Usually the only efficient way to use them is programmatically, so that you need to create a specific user interface for each API. Somehow, I doubt that Cairo would have come to be anything much different from that in the end.

I see this as the most robust way to build a predictable system that runs in a controlled way while taking advantage of probabilistic AIs while reducing the impact of their alucinations.

LLMs simply can't be trusted to follow instructions in the general case, no matter how much you constraint them. The power of very large probabilistic models is that they basically solved the _frame problem_ of classic AI: logical reasoning didn't work for general tasks because you can't encode all common sense knowledge as axioms, and inference engines lost their way trying to solve large problems.

LLMs fix those handicaps, as they contain huge amounts of real world knowledge and they're capable of finding facts relevant to the problem at hand in an efficient way. Any autonomous system using them should exploit this benefit.

Needless to say, I don’t find them at all convincing. This 'nothing' is much better than catching unconvincing unneeded supernatural entities.