Thats why it is so amazing for speed runs and prototypes. Here it is legitimately > 10X faster.

So if you / team are going to implement a new feature, what does that look like? Do you write Gherkin or similar, unit tests or both? Can you provide an example of what that might look like? How much of this has changed for you since the pre-AI days?

Maybe I misunderstand but this seems like a fairly low bar in the test suite only covers existing bugs.

I'd argue that if you aren't going to look at the code you actually need a fully comprehensive test suite - in the sense that if the tests pass, the code is correct and you don't have to look at it at all. The problem is, that isn't very quick to create it seems. Of course, if there is a way to do it quickly in a way that is reproducible by others I'd love to hear about it.

One thing however is a test suite is not usually exhaustive in the sense that any code that passes the tests is valid. Usually tests are more complimentary in nature. Therefore you could still possibly get code degradation, potentially.

For this example, there seems to be no explanation for the LLM to know when to use this curl command, etc. Is the idea that the linear API is known in the LLM weights already and therefore there is no need to include "the manual" in the context window? If so, it's a pretty narrow win.