I wrote my own agent, and it sends data to LLMs in this order: "General Prompts (How to write good code)" + "The Code" + "The Feature Request". This means the KV cache will be used even when the feature request changes.

And output tokens are usually way less than the input tokens.

So I think that my approach is very lightweight on token usage compared to an interactive session.

It would be interesting to measure it for the other agents out there. Sending a feature request two times vs an interactive session.

In an interactive session, adding "Fine, but make the button red" after the model generated a first solution more than doubles the tokens used. As the model now not only gets the original code and the feature request but also the updated code plus the change request as input tokens.

Sending a feature request to an LLM and then sending the feature request again with "The button shall be red" only doubles the tokens used.

Personally, I already see LLMs and agents as blackboxes. I give each feature request to multiple LLMs and then compare the results. I don't manually use "sessions" at all. I just look at the outcome. When I dislike it, I "git reset --hard", change my prompts and restart the feature request.

To have an ongoing sense of which agents perform best, I keep a log and calculate an ELO score of which agents meet my demands best. This score is imporant to me, not so much how the agent achieves it.

In fact, I usually let multiple LLMs implement the same feature, and then I compare them. I even run my own arena in which I calculate Elo scores for LLMs from my perspective of which one implemented features better.

Having the ability to control code agents via voice would not take away my ability to do that. But I think in the future, that will become less and less necessary. If we look back at this conversation in five years, it will look very archaic, and we will be used to having superhuman AI do everything for us. In 10 years, it will sound like a strange idea that humans were once fiddling with code to improve the quality.

    How would you detect the
    presence of bugs in this
    scenario?

    How would you make sure the LLM
    isn't adding yet another
    useless, redundant function to
    the code base?

"It changed the plot function so it takes another parameter called linewidth. It also added an input field in the stylecontrols section where the user can ...".

    you will be surrounded by an ecosystem of
    devices, none of which stand alone, but are
    more like portals to interact with your agents

And then a minute later I could ask it "Is copilot done yet?" and it replies "No, looks like it is still working on it". And then a minute later I ask again. It replies "Yes, it finished. It changed chart.py and styles.css. Do you want me to tell you what specific changes it made to the files?".

But it looks like none of the chat agents with voice interface have such a connector at the moment? An SSH connector would be the most useful. But a "GitHub Codespace connector" or something like that would also do.

I wonder if that will be a missing piece for long. If so, I would build an agent with voice mode and ssh connector myself. But I guess it should come out from the big guys any moment now?

I can tick files in Vim, those get concatenated into a prompt. Along with a feature request. Plus an instructions file that tells the LLM how to reply. Plus my general "rules for good code" file, plus one "rules for good code" file per language involved, plus a project specific overview file. The LLM then answers with a list of changes it wants to make to the code. My tooling then applies those changes and I look at them via "git diff". If I like it, I commit. If not, I change one of the prompts and start the process again.

Instead of replying with code changes, the LLM can also decide to request more files. I wrote a little DSL for that.

I described the beginnings of this workflow last July:

https://www.gibney.org/prompt_coding

Feels like an eternity ago. I think I will write a new blog post this July and describe how the workflow has evolved over the past year.

Comments URL: https://news.ycombinator.com/item?id=47816487

Points: 1

# Comments: 1

    You can’t import or require() a module
    that only exists in memory.

If there is no money to pay them, they should get shares in the company. So if their R&D is successful, they participate in the outcome.

You could put the example code on the filesystem of that VM too.

Couldn't it write them in a web based dev environment?

    how would claude code work from a browser environment?

Maybe a browser plugin that lets the agent use websites is enough?

What would be a task that an agent cannot do on the web?

    German chancellor Friedrich Merz ... 
    lashed out at German workers to
    “simply do a little more,”

A friend of mine, an engineer who works in the German car industry, recently told me that nowadays he has a lot of free time. Because the company he works for has so few orders that the company is granted "Kurzarbeitergeld" - the government pays 60% of the salary if the employees work less.

That blew my mind. If I had fewer orders, I would work more to increase the quality of my product and my efficiency. Working less as a reaction to losing market share seems completely counterproductive to me.

The last time I was excited about the performance of local computers was in the 90s I think.

Modern laptops are so insanely fast. Not sure if they are 2x, 10x or 100x faster than I need them to be. But I never hear fans. I never have to wait for the machine these days.

But ...

The 13 inch version is heavier than a ThinkPad X1 Carbon. Which has a 14 inch screen and can run Linux.