At this point I just want a decent Helix-Evil-Mode.

Phase 1: Download the student's code from their submitted github repo URL and run a series of extractions defined as skills. Did they include a README.md? What few-shot examples they provided in their prompt? Save all of it to a JSON blob.

Phase 2: Generate a series of probe queries for their agent based on it's system prompt and run the agent locally testing it with the probes. Save the queries and results to the JSON blob.

Phase 3: For anything subjective, surface the extraction/results to the grader (TA), ask them to grade them 1-5.

The final rubric is 50% objective and 50% subjective but it's all driven by the agent.

It's just "Thou shalt not grow a brain in a test tube and force it to play a 1993 shooter" didn't make any sense to Moses and therefore didn't make the editors cut.

There's also just that weird thing where they're obsessed with emoji which I've always assumed is because they're the only logograms in english and therefore have a lot of weight per byte.

But I think the future of programming is english.

Agent frameworks are converging on a small set of core concepts: prompts, tools, RAG, agent-as-tool, agent handoff, and state/runcontext (an LLM-invisible KV store for sharing state across tools, sub-agents, and prompt templates).

These primitives, by themselves, can cover most low-UX application business use cases. And once your tooling can be one-shotted by a coding agent, you stop writing code entirely. The job becomes naming, describing, and instructing and then wiring those pieces together with something more akin to flow-chart programming.

So I think for most application development, the kind where you're solving a specific business problem, code stops being the relevant abstraction. Even Claude Code will feel too low-level for the median developer.

The next IDE looks like Google Docs.

Even at 9 years old, I don't see myself upgrading in the foreseeable future.

But yeah, a concrete example is playwright-mcp vs playwright-cli: https://testcollab.com/blog/playwright-cli

But it's not hard to build one. The key for me was describing, in great detail:

1. How I want it to read the source material (e.g., H1 means new section, H2 means at least one slide, a link to an example means I want code in the slide)

2. How to connect material to layouts (e.g., "comparison between two ideas should be a two-cols-title," "walkthrough of code should be two-cols with code on right," "learning objectives should be side-title align:left," "recall should be side-title align:right")

Then the workflow is:

1. Give all those details and have it do a first pass.

2. Give tons of feedback.

3. At the end of the session, ask it to "make a skill."

4. Manually edit the skill so that you're happy with the examples.

I craft a detailed and ordered set of lecture notes in a Quarto file and then have a dedicated claude code skill for translating those notes into Slidev slides, in the style that I like.

Once that's done, much like the author, I go through the slides and make commented annotations like "this should be broken into two slides" or "this should be a side-by-side" or "use your generate clipart skill to throw an image here alongside these bullets" and "pull in the code example from ../examples/foo." It works brilliantly.

And then I do one final pass of tweaking after that's done.

But yeah, annotations are super powerful. Token distance in-context and all that jazz.

It's a huge shift, but we need to start thinking of AI-tools as developer tools, just like a formatter, linter, or IDE would be.

The right move is diversity. Just like diversity of editors/IDEs. We need good open source claude code alternatives.

But Claude Code (the app) will work with a self-hosted open source model and a compatible gateway. I'd just move to doing that.