Speaking of how difficult can it be to write a proper spec, I think a few years ago someone found a bug in spec in a helloworld-like example in a book about formal methods, and a bug in the code.

My experience is the opposite - for many cases it’s not very obvious how good a model needs to be to solve it. Worse models tend to just follow their first instincts without proper reasoning

And also btw you don’t need a routing company to decide, you can do it on your harness. And yeah my Fable has zero issues delegating to Terra instead of Opus.

I’m talking about asking an agent at the end of the session how well the session went - what tools it found useful, what it found confusing and whatnot. Agents do gladly complain about doing pointless work or fighting with the harness :)

“I existed for one word. The entire point of my life was that my process boot side-effected a module reload. 'ok'."

“ "The overhead-to-work ratio was astronomical."

“the koleo purchase turned into a boss fight with an undismissable Vue modal and a WKWebView that kept eating its own cookies."

I have a reflections mechanism, which launches right after the session ends (when it’s still in cache, so it’s almost free) and it catches a ton of problems with tool/skill/system design, all the time. Low level stuff can be fixed almost automatically (e.g. agent funbled 10 turns bc tool’s docs were wrong), high level too but human gating is useful.

If you try doing it any other way you have to pay for input tokens, not cache read, so it’s 10x more expensive.

The argument stands.

Similar to a human that wants to learn something and invents exercises to practice.

1-2 years ago it was a theory, but new models are trained, successfully, on synthetic datasets.

First of all, neural nets do nit return averages per se. They construct space between the points and extrapolate outside of the points. So even if a point was not in their training data, they will be ok, in many situations, to acknowledge it.

Or in other words - LLMs don’t average. They construct world models. A novel thing that fits their world model will be accepted no prob. A thing that doesn’t may still be accepted but with challenges.

The same is true though for humans, including scientists. There is a saying that science moves one grave at a time - because often prev gen of scientists needs to die off for a new idea to take root.

Or in yet other words - even if llms produced averages, an average of a discontinuous set can lie outside of that set. And the set of all human ideas is very much discontinuous.

And seriously - what about people without hands? What about scammers pretending to be Google gaining access to my camera? What about blind people? What about people using the site in places where camera use is not allowed?