[0] https://github.com/NVIDIA/personaplex

[1] https://arxiv.org/abs/2410.00037

Congratulations on getting this out!

I think this is an interesting question because we have not fully figured out the best way to onboard people to our codebases. Each person is responsible for multiple codebases (yay microservices!), and no one else commits to a repository while they have dibs. We also have conventions for how agents write documentation around deployments and validations.

In theory, when a new person joins the team or is handed a repository, they can throw some tokens at the codebase, interrogate it, and ask questions about how things are implemented.

> But what is the result of your work?

The end result is a final, working codebase. The specs and sprint plans are also committed to the repository for posterity, so agents in a fresh session can see what work has been completed and the trajectory we are moving toward.

In practice, we try to close the loop with agents: plan -> generate -> run tests/validators -> fix -> repeat. What I mainly contribute is taste and deciding what to do next: what to build, what "done" means, and how to decompose the work so models can execute. With a strong definition of done and a good harness, the system can often converge with minimal human input. For debugging, we also have a system that ingests app logs plus agent traces (via CXDB).

The more reps you get, the better your intuition for where models work and where you need tighter specs. You also have to keep updating your priors with each new model release or harness change.

This might not have been a clear answer, but I am happy to keep clarifying as needed!

Even though all three of us have very different working styles, we all seem to be very happy with the arrangement.

You definitely need to keep an open mind, though, and be ready to unlearn some things. I guess I haven’t spent enough time in the industry yet to develop habits that might hinder adopting these tools.

Jay single-handedly developed the digital twin universe. Only one person commits to a codebase :-)

We’ve been working on this since July, and we shared the techniques and principles that have been working for us because we thought others might find them useful. We’ve also open-sourced the nlspec so people can build their own versions of the software factory.

We’re not selling a product or service here. This also isn’t about positioning for an acquisition: we’ve already been in a definitive agreement to be acquired since last month.

It’s completely fair to have opinions and to not like what we’re putting out, but your comment reads as snarky without adding anything to the conversation.

Your intuition/thinking definitely lines up with how we're thinking about this problem. If you have a good definition of done and a good validation harness, these agents can hill climb their way to a solution.

But you still need human taste/judgment to decide what you want to build (unless your solution is to just brute force the entire problem space).

For maximal leverage, you should follow the mantra "Why am I doing this?" If you use this enough times, you'll come across the bottleneck that can only be solved by you for now. As a human, your job is to set the higher-level requirements for what you're trying to build. Coming up with these requirements and then using agents to shape them up is acceptable, but human judgment is definitely where we have to answer what needs to be built. At the same time, I never want to be doing something the models are better at. Until we crack the proactiveness part, we'll be required to figure out what to do next.

Also, it looks like you and Danvers are working in the same space, and we love trading notes with other teams working in this area. We'd love to connect. You can either find my personal email or shoot me an email at my work email: navan.chauhan [at] strongdm.com

Funnily enough, the marketing department even ran a campaign asking, “What does DM stand for?!”, and the answer was “Digital Metropolis,” because we did a design refresh.

I just linked the website because that’s what the actual company does, and we are just the “AI Lab”

No local model out there is as good as the SOTA right now.

At home on my personal setup, I haven't even had to move past the cheapest codex/claude code subscription because it fulfills my needs ¯\_(ツ)_/¯. You can also get a lot of mileage out of the higher tiers of these subscriptions before you need to start paying the APIs directly.

* DRYing/Refactoring if needed

* Documentation compaction

* Security reviews

I’m happy to answer any questions!

Web because there is also gopher://gopher.navan.dev

In an ideal world, this workaround would not be needed

¯\_(ツ)_/¯