Next was discovering "create a mathematical model of the problem and derive the solution as a result" type prompts.

But, the real "oh s**" was a longer process of spec'ing a compiler/runtime for real-time DSP (with a lot of novel ideas) and it actually working.

My sequence was: (1) if helps me understand myself, (2) if helps me put together good ideas, (3) it can generate novel ideas given the right inputs, (4) it can build useful tools on my machine, (5) it can compound good ideas into better and better ideas with repeated passes, (6) it can build significant, ambitious machinery that's way beyond my ordinary capacity.

Current frontier: it can compound large codebases into better and better machinery with repeated passes.

The key thing I track is whether I'm running a process that converges and compounds or whether I'm spinning in place / diverging.

- Durable, 'enterprise grade' software patterns are baked into the runtime and into common, stable libraries that everyone uses

- You can use Ash, which pretty much entirely solves architectural considerations for many types of backends

- The tooling for inspecting and enforcing style (tidewave, credo, dialyzer, Dan's "vibe" ecosystem tools) is far beyond what I see in other ecosystems

- Ecosystem coverage for pretty much everything you need, including numerical software

- Excellent performance escape hatches (NIFs)

And, as has been shown in various benchmarks, agents are quite good at it.

My one problem in practice has been that getting tests right is hard. LLMs need a lot of cajoling to not build flaky tests with all the concurrency, and I find myself spending hours rewriting parts of the test suite once or twice a week.

I found an HTML version here: https://www.jsoftware.com/papers/J1990.htm

You can also use Rectangle or Spectacle or others in place of Raycast.

Foolproof with zero magic.

[1] https://github.com/wojciech-kulik/FlashSpace

- A well-structured folder of markdown files that I constantly garden. Every sub-folder has a README. Every files has metadata in front-matter. I point new sessions at the entry point to this documentation. Constantly run agents that clean up dead references, update out of date information, etc. Build scripts that deterministically find broken links. It's an ongoing battle.

- A "continuation prompt" skill, that prompts the agent to collect all relevant context for another agent to continue

- Judicious usage of "memory"

- Structured systems made out of skills like GSD (Get Shit Done)

- Systems of "quality gate" hooks and test harnesses

For all of these, I have the agent set them up and manage them, but I've yet to find a context-management system that just works. I don't think we understand the "physics" of context management yet.

Maybe that's another way of saying: I was trained as a designer, and now the distinction between design (read: architecture, service-design, product, ux, cx) and programming is blurring.

As a test, I've been attempting an incredibly complex project that goes far beyond my abilities as a kind of deliberate worst-case-scenario. It's more or less a programming language for a very specific purpose that compiles to a custom bytecode and runs on a custom runtime with specific performance guarantees.

I've spent part of the last month iterating on a formal model of the system and various specifications. Along the way, I teach myself how to understand and critique the part of the system I'm working on, however I also deliberately keep things just beyond my understanding by opportunistically pulling in concepts from various sources ... algebraic topology, obscure corners of PL, concepts plucked from similar systems. It's a complete monstrosity with, now, hundreds of supporting documents, research spikes, processed references, critique passes, etc.

If I'm able to complete this project and have it work as expected, I think I'll have learned a lot about what is or isn't possible. If the current design does in fact work, I'm fairly confident I'll have advanced the state of the art in the niche field I'm working in.

As for the point about complexity, I will meditate on it.

My first cut is that what you're getting at is something like: you're using "complexity" in the technical sense of the word, in which case we're in the realm of so-called "wicked problems" and such. There is some kind of 'making sense' that goes beyond 'mere ideas,' or at the very least something like "throw more minds and hands at the problem," in this context, is the wrong attitude.

In such a world, a machine that amplifies human effort (while obscuring it) is not the right tool for the job—more than likely you find yourself spinning around an attractor basin.

I personally have a tentative conclusion that I specifically am able to avoid this by amplifying certain strategies I've developed over my life, but really, I don't have solid confirmation yet.

In any event, I'm enjoying the experiment, but I'll reflect on why you seem to be certain about it.