As I was reading this article, a similar thought occurred to me: "I wonder if that's better or worse than a human?" Unfortunately, there was no human baseline in this study. That said, there are studies that compare LLM to human performance. Usually, humans perform much better (like 5-7x better) at long-running tasks.

In other words, a human would probably do better than an LLM on this task.

Humans lose to LLMs in narrow, well-specified text/symbolic reasoning tasks where the model can exploit breadth, speed, and search. Usually, the LLM performed ~15% better than humans, but I saw studies that were as high as 80%. To my surprise, these studies were usually about "soft skills" like creativity and persuasion.

I'd say you've at maximum doubled the code. The test ensures you write only what you need to get the test pass. Without them, devs get distracted and wander until the feature works. Usually distracting themselves with tons of YAGNI violations along the way. In my experience, untested code bases have a ton of unnecessary code.

I don't understand how it would double the bugs. The article has references saying it reduces them. But, even thinking about it, I don't see why you'd say that.

Tests shaping your code is bad, because? Because you already have "enough constraints"? Sorry, I don't get it.

But, I dunno, I haven't looked at the source code. It might be very difficult to maintain.

Besides, there's too many variables in your anecdote. Is it a laptop from 1995? Is the OOM from a bug that could/should be fixed?

Thing is, this doesn't usually matter. I have never gotten an out of memory error from a leak in Java. Now compare that to all the development time I've saved by not having to deal with pointer arithmetic. I consider it a huge win. It's all about the type of apps you're making.

Java background is probably biasing me, but I don't like this. It interferes with my ability to find code because the file list in every directory is twice as large.

What are the benefits of putting it together?

Everyone is interpreting this as, "write 10 tests then try to get them all to pass at once". That is not how you TDD. You write one, then get it to pass, then write another.

Maybe you mean, "write the test before the code", but when you say "write all tests before the code", it's not interpreted the same way.

Never heard anyone say this. Kent Beck has said the opposite on podcasts.

Unfortunately, the project isn't "finished" after you achieve initial success. You have to maintain it for years after that. Without good tests, you get yourself into a state where one change breaks two things. You need teams of people manually checking it still works. It takes them an order of magnitude longer to check than it would take a computer. Your feedback loop that answers the question, "does it still work?" is in weeks instead of minutes.

You stop changing the bad code because it's too scary and dangerous. The bad code reduces the frequency at which you release new features. You slow to a crawl. The team of people testing gets expensive and eats into your profits. Some competitor eats your lunch.

In my experience, these problems occur earlier than you expect. Usually before you even attain success. I'm too lazy to comb over my app checking for bugs every time I develop a new feature. I get bored too easily to check that the same bug doesn't crop up over and over again. I'm also too impatient to wait for another person to do it. In these conditions, TDD speeds you up.

I've never worked on a manually tested project that avoided these problems. It usually happens around month 2 and gets even worse as time goes on.

Here's why I think TDD fails: The consultants say, "it's so easy! Just red-green-refactor!" No, it's extremely complex and difficult, actually. There's a ton of depth that nobody talks about because they're trying to get you to try it. The typical dev learns about "Just red-green-refactor" and fails. Now "TDD did not live up to expectations".

For example, did you know there's different styles of TDD? The author of this article doesn't seem to (I may be wrong, but he gives no indication). He's talking about Detroit style TDD (also known as classicist). This is the style Ron Jeffries, Uncle Bob, Martin Fowler, Kent Beck, etc. practice and advocate for. There's another style called London style (AKA mockist) that Gary Bernhardt, etc. practice. And then there's Discovery Testing from Justin Searls. There's probably others I haven't heard of yet.

They all have different pros and cons and a very common pitfall is to use a style in a way it's not really meant for. e.g., Using mockist to gain confidence your high level features work. Expecting classicist to help you drive out good design for your code.

To wrap this up, there's so much unknown unknowns about TDD it's usually premature to judge the technique. If you haven't tried all these styles, one might click with you. A lot of people are concluding they hate all sports when they've only tried baseball.

Comments URL: https://news.ycombinator.com/item?id=12047745

Points: 1

# Comments: 0

http://www.npr.org/blogs/money/2014/10/28/359419934/who-stud...

http://www.randalolson.com/2014/06/15/the-double-edged-sword...

What makes you say that is unlikely?

I hear this a lot. What's the reasoning?

Comments URL: https://news.ycombinator.com/item?id=9195645

Points: 1

# Comments: 0