For people who have some expertise, the models accelerate the grunt work, but you’re the one validating it.

I’ve been using agents to do the code for me, but I explicitly tells it to iterate in small changes that I can review and approve as it happens, not after a 2k lines diff. It makes me understand what’s being made, and test every little change before we move on to change other parts of the code. Also helps you correct any misunderstanding right from the start.

As an example, if you chat with an LLM it can sort of explain concepts it learned in its training, because it created some internal representation of them. However it’ll never be able to explain things about your project unless you tell it (then it can simply repeat it, which is the same RAG does, the LLM is just spitting it out without any deep understanding of it).

You can try to cram as much context to it as possible and overload it with extra information, or you can understand this difference and separate a task that you know the LLM can handle because you’ll be able to leverage its knowledge and judge the final work yourself.

But this distinction needs to be more clear, I think. The LLM doesn’t have any deep understanding of anything you’re creaming into the context window the same way it does with concepts it learned during training from millions of examples and building some internal model of the concepts and their relationships.

I kinda [wrote about this before](https://dielsonsales.github.io/2023/05/11/starting-a-new-blo...) but the takeaway is that writing forces you to structure your thoughts, giving them a final shape and enables you to notice flaws or gaps in your own thinking process. It’s impossible to do this without writing.

Every time you work in a complex software project, you have all the context in your head about what you must take into consideration. LLMs don’t, you have to explain every little detail to them, but there’s no telling where it stops: do you have to explain X to it or is it in the training already?

When you try to shape how a LLM should behave and what it should know, you end up writing a Bible of relevant context that’s increasingly difficult to maintain as well, often with outdated or contradictory information, and it is still free to ignore things you have written and proceed however it wants.

That’s why so many people only feel comfortable giving the LLM some limited task to do, because you can judge if that specific task needs just enough context that it can handle by itself.

As you said, building a script that only you use personally or a very simple thing that just accomplishes one task and it’s easy to test require almost no engineering, and an LLM can often build those with very little downsides.