https://tsz.dev/benchmarks/micro

Fully agentic coding is working well for projects like this since no matter how you write the code, the only way to truly know "it's working" is if it passes the test.

With the right skills you can make well designed software with agentic coding too. It's not as easy as a simple "convert this to rust" prompt, at least today.

Doing this sort of project is giving me a glimpse of what it is going to be like managing software projects. As software engineers we have to learn how to manage much bigger changes and in a much higher level of abstraction. I personally don't think models are good enough for this level of automation yet but in a weekend that I had access to Fable I could see how things are going to change soon. Most of criticism towards LLM coding was not applicable to Fable. I'm not hyping anything, just an observation.

The DJ analogy is useful actually. I live in Berlin and essentially everyone is a DJ but only a few get to make money from it. The difference is of course taste but also grit and how well those people leverage available tools to them. A good DJ knows how to use the tools and has a good understanding of the market. Different skill that a musician but nevertheless a valuable skill

https://github.com/tsz-org/tsz

https://github.com/user-attachments/assets/0b80b07b-d88f-414...

tsz is my main side project. Trying to learn from this for how to make software in fully automated fashion. tsz's goal is to match tsc (tsgo) but perform better. I am not passing all tsc's own test cases and working towards making it work on complex type packages.

https://github.com/tsz-org/tsz

I'm not against using LLMs in any ways. https://tsz.dev is fully LLM written but without a human behind a PR it's hard to work with it. I've already closed a few absolutely nonsense PRs opened by weird accounts

If I had time and making a polished web page was my goal I could probably do better but this was not the point!

Feel free to steal! This was one shot with Claude Code. You can take it and adopt it to your need

https://azimi.me/axial-flux-motor-explainer/

https://github.com/tsz-org/tsz

But overall, this is pretty normal for compilers to have this sort of "unexpected" tests failing due to some work in an area. It happened to me when I was coding everything manually back in the day too

In a project like mine (https://github.com/tsz-org/tsz) I am constantly frustrated that models were not doing enough research and were not taking into account other situations. Again and again models would produce code that would fix one thing and break 2 other tests that were "unrelated".

With Fable it seems like tasks are taking much longer (I have not seen a pull request from Fable sessions yet) but reading the transcription of those sessions I can see how it is doing the right thing by not leaving any stone unturned.

As the article says, it's hard to communicate this "feeling" about models because it is very project specific but I thought I share

From the model card:

In light of the ability of recent models to accelerate their own development, we've implemented new interventions that limit Claude's effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design. Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms. Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user.