These tools are nothing alike and the reductionism of this metaphor isn’t helpful.

I highly doubt that.

Empirical studies show that humans have very little effect on error rates when reviewing code. That effect disappears quickly the more code you read.

Most programmers are bad at detecting UB and memory ownership and lifetime errors.

A piece of wood comes off the table it’s cut or it’s not.

Code is far more complex.

How long until people get exhausted with the new volume of code review and start "trusting" the LLMs more without sufficient review, I wonder?

I don't envy Linus in his position... hopefully this approach will work out well for the team.

A table saw isn't a probabilistic device.

It’s not LLMs replacing workers, it’s people. People who have a lot of money and don’t sell their labour for a paycheque. And the systems that compel them to such actions.

It’s already bad. I’m not looking forward to the future. These systems are terrible. It’s a future without people that they want for some reason. I’d rather deal with people incompetent, tired, annoyed people than an LLM.

To those reading this thread though, be wary of the answers LLMs generate: they're plausible sounding and the LLM's are designed to be sycophants. Be wary, double check their answers to your queries against credible sources.

And read the source!

How many bytes is a pointer in C? How many bytes is a shared pointer in C++? What does sysctl do? What about fsync?

What is a mutex lock? How is it different from a spin lock?

You want to find the n nearest points to a given point on a 2-D Cartesian plane. Could you write the code to solve that on your own?

Can you answer any of these questions without searching for the answer?

I don't use LLMs and I learn things fine. Always have. For several decades. I care deeply about the underlying code and systems. It annoys me when people say they do and they cannot even understand how the computer works. I'm fine with people having domain-specific knowledge of programming: maybe you've only been interested in web development and scripting DOM elements. But don't pretend that your expertise in that area means you understand how to write an operating system.

Or worse: that it prevents you from learning how to write an operating system.

You can do that without an LLM. There's no royal road. You have to understand the theory, read the books, read the code, write the code, make mistakes, fix mistakes, read papers, talk to other people with more experience than you... and just write code. And rewrite it. And do it all again.

I find the opposite is true: those who use LLM coding exclusively never enjoyed programming to begin with, only learned as much as they needed to, and want the end results.

But I will demand my money back or sue you if your crappy code leaks my personal information, destroys my property, performs worse than advertised, or otherwise harms me in some way.

There was sloppy code before LLMs. It’s what they were trained on. And it’s why they generate slop.

All that code that was rushed out to meet an arbitrary deadline made up by the sales team, written by junior and lazy senior developers, pushed by the, “code doesn’t matter,” folks. Code written by the enterprise architecture astronauts with a class hierarchy deeper than the Mariana Trench. A few years down the line you get bloated, slow, hard to maintain spaghetti piles of dung. Windows rendering text that stutter when you scroll them. Virtual keyboards that take seconds to pop up. Browser tabs that take more available memory than was available to send astronauts to the moon and back.

When humans write it you generally have a few people on a team who are concerned with these things. They try to reign in the slop generating, “always be shipping,” people. You need a mix of both. Because each line of code is a liability as much as it’s a new feature.

If it does all go down in flames, even floor value is not going to be that valuable.

I can't predict the future but it's smelling a lot like a recession already under way that is bigger than the sub-prime crash.

I don’t know what SaaS has to do with FOSS. The point of FOSS was to allow me to modify the software I run on my system. If the device drivers for some hardware I depend on are no longer supported by the company I bought it from, if it’s open source, I can modify and extend the software myself.

The Copy Left licenses ensure that I share my modifications back if I distribute them. It’s a thing for the public good.

Agent-based software development walls people off from that. Mostly by ensuring that the provenance of the code it generates is not known and by deskilling people so that they don’t know what to prompt or how to fix their code.

It’s a good thought and I tend to think that this is the way I would feel more productive: better languages that give us the ability to write better abstractions. Abstractions should provide us with new semantic layers that lose no precision and encapsulate lots of detail.

They shouldn’t require us to follow patterns in our code and religiously generate boilerplate and configuration. That’s indirection and slop. It’s wasted code, wasted effort, and is why I find frameworks like React to be… not pleasant to use. I would rather generate the code that adds a button. It should be a single expression but for many reasons, in React, it isn’t.

The C compiler Anthropic got excited about was not a “working” compiler in the sense that you could replace GCC with it and compile the Linux kernel for all of the target platforms it supports. Their definition of, “works,” was that it passed some very basic tests.

Same with SQLite translation from C to Rust. Gaping, poorly specified English prose is insufficient. Even with a human in the loop iterating on it. The Rust version is orders of magnitude slower and uses tons more memory. It’s not a drop in Rust-native replacement for SQLite. It’s something else if you want to try that.

What mechanism in these systems is responsible for guessing the requirements and constraints missing in the prompts? If we improve that mechanism will we get it to generate a slightly more plausible C compiler or will it tell us that our specifications are insufficient and that we should learn more about compilers first?

I’m sure its possible that there are cases where these tools can be useful. I’m not sure this is it though. AGI is purely hypothetical. We don’t simulate a black hole inside a computer and expect gravity to come out of it. We don’t simulate the weather systems on Earth and expect hurricanes to manifest from the computer. Whatever bar the people selling AI system have for AGI is a moving goalpost, a gimmick, a dream of potential to keep us hooked on what they’re selling right now.

It’s unfortunate that the author nearly hits on why but just misses it. The quotes they chose to use nail it. The blog post they reference nearly gets it too. But they both end up giving AI too much credit.

Generating a whole React application is probably a breath of fresh air. I don’t doubt anyone would enjoy that and marvel at it. Writing React code is very tedious. There’s just no reason to believe that it is anything more than it is or that we will see anything more than incremental and small improvements from here. If we see any more at all. It’s possible we’re near the limits of what we can do with LLMs.

Nothing will save you from a bad specification. And there's no royal road to knowing how to write good ones.

It might be illuminating to see what a mathematically precise specification can and cannot do when it comes to generating programs. A major challenge in formal methods is proving that the program implements the specification faithfully, known as the specification gap. If you have a very high level and flexible specification language, such as TLA+, there is a lot of work to do to verify that the program you write meets the specification you wrote. For something like Synquid that is closer to the code there are more constraints on expressivity.

The point is that spoken language is not sufficiently precise to define a program.

Just because an LLM can fill in plausible details where sufficient detail is lacking doesn’t indicate that it’s solving the specification gap. If the program happens to implement the specification faithfully you got lucky. You still don’t actually know that’s true until you verify it.

It’s different with a narrow interface though: you can be very precise and very abstract with a good mathematical system for expressing specifications. It’s a lot more work and requires more training to do than filling in a markdown file and trying to coax the algorithm into outputting what you want through prose and fiction.

No side effects is a hefty constraint.

Systems tend to have multiple processes all using side effects. There are global properties of the system that need specification and tests are hard to write for these situations. Especially when they are temporal properties that you care about (eg: if we enter the A state then eventually we must enter the B state).

When such guarantees involve multiple processes, even property tests aren’t going to cover you sufficiently.

Worse, when it falls over at 3am and you’ve never read the code… is the plan to vibe code a big fix right there? Will you also remember to modify the specifications first?

Good on the author for trying. Correctness is hard.

I can’t even convince most developers to use model checkers. Far more informal than a full proof in Lean. Still highly useful in many engineering tasks. People prefer boxes and arrows and waving their hands.

Anyway, I don’t know that I’d want to have a system vibe code a proof. These types of proofs, I suspect, aren’t going to be generated to be readable, elegant, and be well understood by people. Like programs they generate it will look plausible.

And besides, you will still need a human to review the proof and make sure it’s specifying the right things. This doesn’t solve that requirement.

Although I have thought that it would be useful to have a system that could prove trivial lemmas in the proof. That would be very neat.

I wish there was a tag or something we could put on headlines to avoid giving views to slop.

So it is with software.

You might not need a blueprint if you’re building a shed in your back yard. This is the kind of software that and user might write or script themselves. If it’s kind of off nobody is going to get hurt.

In many cities in North America you can’t construct a dwelling with plumbing connections to a sewer and a permanent foundation without an engineer. And you need an engineer and a blueprint to get the license to go ahead with your construction.

Because if you get it wrong you can make people in the dwelling sick and damage the surrounding environment and infrastructure.

Software-wise this is where you’re handling other people’s sensitive data. You probably have more than one component that needs to interact with others. If you get it wrong people could lose money, assets could get damaged, etc.

This is where I think the software industry needs to figure out liability and maybe professionalize a bit. Right now the liability is with the company and the deterrents are basically no worse than a speeding ticket in most cases. It’s more profitable to keep speeding and pay off the ticket than to prevent harm from throwing out sloppy code and seeing what sticks.

Then if you are building a sky scraper… well yeah, that’s the large scale stuff you build at big tech.

There are different degrees of software with different requirements. While not engineers by professional accreditation, in practice I would say most software developers are doing engineering… or trying.

What I agree with in the article is that AI tools make bad engineering easier. That is for people building houses and skyscrapers who should be thinking about blueprints: they are working under assumptions that the AI is “smart” and will build sky-scrapers for them. They’re not thinking about the things they ought to be and less about things that will pass on to the customers: cost and a product that isn’t fit for use.

A bridge that falls down if you drive too fast over it isn’t a useful bridge even though it looks like a bridge.