That said, I'd agree that people who currently claim 20x speedups will indeed be replaced.

I can give you many, many examples of where it failed for me:

1. Efficient implementation of Union-Find: complete garbage result 2. Spark pipelines: mostly garbage 3. Fuzzer for testing something: half success, non-replicateable ("creative") part was garbage. 4. Confidential Computing (niche): complete garbage if starting from scratch, good at extracting existing abstractions and replicating existing code.

Where it succeeds: 1. SQL queries 2. Following more precise descriptions of what to do 3. Replicating existing code patterns

The pattern is very clear. Novel things, things that require deeper domain knowledge, coming up with the to-be-replicated patterns themselves, problems with little data don't work. Everything else works.

I believe the reason why there is a big split in the reception is because senior engineers work on problems that don't have existing solutions - LLMs are terrible at those. What they are missing is that the software and the methodology must be modified in order to make the LLM work. There are methodical ways to do this, but this shift in the industry is still in baby shoes, and we don't yet have a shared understanding of what this methodology is.

Personally I have very strong opinions on how this should be done. But I'm urging everyone to start thinking about it, perhaps even going as far as quitting if this isn't something people can pursue at their current job. The carnage is coming:/

I understand the drive for stabilizing control and consistency, but this ain't the way.

As the universe expands the gap between galaxies widens until they start "disappearing" as no information can travel anymore between them. Therefore, if we assume that intelligent lifeforms exist out there, it is likely that these will slowly converge to the place in the universe with the highest mass density for survival. IIRC we even know approximately where this is.

This means a sort of "grand meeting of alien advanced cultures" before the heat death. Which in turn also means that previously uncollided UUIDs may start to collide.

Those damned Vogons thrashing all our stats with their gazillion documents. Why do they have a UUID for each xml tag??

In other words, people claiming these high productivity increases may be the ones at actual risk. Why employ 3 people when 1 can write the prompts?

Funny you say that, I encountered this in a seemingly simple task. Opus inserted something along the lines of "// TODO: someone with flatbuffers reflection expertise should write this". I actually thought this was better than I anticipated even though the task was specifically related to fbs reflection. And it was because I didn't waste more time and could immediately start rewriting it from scratch.

After all, it's through failure that we learn the limitations of a technology. Apparently some people encounter that limit more often than others.

Example: was writing a flatbuffers routine which translated a simple type schema to fbs reflection schema. I was thinking well this is quite simple, surely Opus would have no trouble with it.

Output looked reasonable, compiled.. and was completely wrong. It seemed to just output random but reasonable looking indices and offsets. It also inserted in one part of the code a literal TODO saying "someone who understands fbs reflection should write this". Had to write it from scratch.

Another example: was writing a fuzzer for testing a certain computation. In this case, there was existing code to look at (working fuzzers for slighly different use cases), but the main logic had to be somewhat different. Opus managed to do the copy paste and then messed up the only part where it had to be a bit more creative. Again, showing the limitation of where it starts breaking. Overall I actually considered this a success, because I didn't have to deal with the "boring" bit.

Another example: colleague was using Claude to write a feature that output some error information from an otherwise completely encrypted computation. Claude proceeded to insert a global backdoor into the encryption, only caught in review. The inserted comments even explained the backdoor.

I would describe a success story if there was one. But aside from throwing together simple react frontends and SQL queries (highly copy-pasteable recurring patterns in the training set) I had literally zero success. There is an invisible ceiling.

Here's what I find Claude Code (Opus) useful for:

1. Copy-pasting existing working code with small variations. If the intended variation is bigger then it fails to bring productivity gains, because it's almost universally wrong.

2. Exploring unknown code bases. Previously I had to curse my way through code reading sessions, now I can find information easily.

3. Google Search++, e.g. for deciding on tech choices. Needs a lot of hand holding though.

... that's it? Any time I tried doing anything more complex I ended up scrapping the "code" it wrote. It always looked nice though.

For nixos users: https://discourse.nixos.org/t/using-zsh-with-grml-config-and...

Super easy to setup, and works very well

There is a strange dynamic currently at play in the software labour market where the demand is so huge that the market can bear completely inefficient coders. Even though the difference between a good and a bad software engineer is literally orders of magnitude.

Quite a few times I encountered programmers "in the wild" - in a sauna, on the bus etc, and overheard them talking about their "stack". You know the type, node.js in a docker container. I cannot fathom the amount of money wasted at places that employ these people.

I also project that actually, if we adopt LLMs correctly, these engineers (which I would say constitute a large percentage) will disappear. The age of useless coding and infinite demand is about to come to an end. What will remain is specialist engineer positions (base infra layer, systems, hpc, games, quirky hardware, cryptographers etc). I'm actually kind of curious what the effect on salary will be for these engineers, I can see it going both ways.

The mechanism you talk about is all too real, and it can be semi-consciously exploited. A bit off-topic, but the way genocides happen is via a similar mechanism. A group is depicted as base, a disease, faults are attributed to it. It becomes a moral imperative to clean the body.

This feeling is thus rationalized, and people start to reinforce each other's conviction. It becomes a twisted status game: the more you lean into the mentality, the higher you rank. Often times the goal isn't even to "address" the original trigger, it's to show your peers your approval and distance yourself from "those people". They the immoral, we the moral.

I just simply don't get it. Productivity delta is literally negative.

I've been asking to do projects where I thought "oh, maybe this project has a chance of getting an AI productivity boost". Nope. Personal projects all failed as well.

I don't get it. I guess I'm getting old. "Grandpa let me write the prompt, you write it like this".

(Un?)fortunately my work doesn't involve a lot of "drone coding". With personal projects I let it do whatever it wanted including picking the language and libraries. With one of them it ended up so confused with the Redis API(!!!) that it kept going back and forth between different versions to "fix" the issues until it literally removed the functionality it was supposed to add. Problem solved, eh?

I tried to use it for Python, Rust and Bash. I also tried to use it for crawling and organizing information. I also tried to use it as a debugging buddy. All of the attempts failed.

I simply don't understand how people are using it in a way that improves productivity. For me, all of this is so far a huge timesink with essentially nothing to show for it.

The single positive result was when I asked it to optimize a specific SQL query, and it managed to do it.

Anyway I will keep trying to use it, maybe something needs to click first and it just hasn't yet.