If LLMs were truly able to replace me, I'd be disappointed, sure - I've spent 30 years developing mastery of a craft, it's sad to see it go. But I'd resign myself and move on.

But that's not what makes me mad.

I get angry because I simply don't believe it is true. I have a reasonable math and tech background to where I grok how this stuff works at a fundamental level, and I'm utterly unconvinced that it is performing any sort of reasoning.

Call it a stochastic parrot, a token extrusion machine, whatever, but these things are not thinking or reasoning.

That doesn't mean they aren't useful- they clearly are very useful for many tasks.

My anger comes from the global attempt to replace things that require human reasoning with LLMs.

There's this push to use this stuff way past the point of "helpful accelerator" to "why do we need programmers/doctors/lawyers/etc..."

I think it's incredibly dangerous.

So while the tool is useful, I don't think we've figured out how to use it appropriately. For all the short term appearance of productivity boost it provides (whether this is real in a total-cost sense is still an active question), I think the risks to skill development and quality outweigh those benefits in many cases, and are being overlooked.

One day a few years ago my dad came by and was admiring it (it was a QS8) and asked to borrow it so he could play piano again.

I, of course, said sure, but was feeling a little salty about it inside, because I wanted it to play, that's why I had it all set up.

Anyway, about a year went by and I asked him about it to see if he was done with it.

He said "oh that thing? I gave that away, was just taking up space"

-.-

I don't know. I used to agree with this, but after the umpteenth time of Claude recommending some obsolete or dead old library, old version, getting major version breaking changes dead wrong, writing code for it that's not even API compatible with the published docs, etc... I started to question whether it was actually faster. I end up pouring over the original documentation anyway.

I have learned some new things, been exposed to some new techniques, and learned about some new libraries, so it's hard to tell.

The problem is made worse by so much of the internet being AI slop now, traditional searching is a huge time waste too.

Looking forward to the next chapter of tech where we're able to use these tools appropriately and not destroy everything of value with them.

She'd heard that if you made a video game and sold it, you would make a lot of money, so she'd decided to take her life savings, $200k, and hire someone to make a game. She didn't know what kind of game or anything, just "make a game".

I was really worried about her and spent two hours on the phone with her trying to educate her and help her protect herself and her savings.

At the end, she just got sort of mad at me and I could tell she was just going to get someone else to do it.

Was so sad. Wish I could have helped her more.

The companies that agree with you will be at an interesting place when they have piles of AI slop code and no talented developers.

He told me they are massively pulling back on the AI stuff.

Right now the lashback is about cost, because that's the most easily measured pain point.

Soon, we'll start seeing a deeper understanding of the quality issues. At that point, it's likely this whole experiment gets firmly put in a bin of the toolbox where it belongs.

One idea is that maybe it could figure out how many L's are in the word "google" [1]

Or, maybe which days of the week have a "d" in their spelling [2].

[1] https://x.com/FatherPhi/status/2059659658428912040?s=20

[2] https://x.com/FatherPhi/status/2054212816069132461?s=20

In the meantime, these [1] are pretty funny.

[1] https://x.com/huskirl

For an example, look at some of Julia Mossbridge's work.

If even a small part of her work is true and valid, it points to something far outside our current framework.

You don't need to go as far afield as Mossbridge, though - that's an extreme example. Pretty much any modern neuroscience will make you question a lot of assumptions, at least it did for me.

I don't think they include areas where correctness, determinism or human reasoning are important.

At least, not in isolation.

Decision making can be done by trained machines following rules, but that's different that reasoning. A thermostat isn't reasoning when it decides to turn on the air conditioner, to argue otherwise expands the definition of "reason" to be so broad that it becomes useless.

LLMs are trained on human knowledge and reasoning that results from human cognition, and they are excellent at stochastic mimicry - if the argument is that they are actually reasoning, then some sort of equivalent to human cognition must be present for that to be true. Lacking that, they are nothing more than "token extrusion machines" with some potentially useful characteristics.

No, they don't.

They are token predictors that use statistical techniques to emit the randomly weighted next most likely token given the previous token list.

The result is a strange mimic of human reasoning, because the tokens it predicts are trained on strings that were produced by humans that were reasoning, but that's not the same thing.

Human cognition is complex and poorly understood, and the nature of the mind is an area of study almost as old as consciousness itself. We don't know exactly how it works, or what its exact relationship to the brain is, but we do know that it is not a simple token predictor.

LLMs, by their very nature are constrained to the concept of language and the relationship between existing words in a corpus. This is a box they can not escape.

Modern neuroscience suggests that the human brain is much more vast than that, and in many ways looks like it is constrained by language, but certainly not limited to it.

But they didn't ask humans, they asked a machine. We expect our machines to behave in predictable ways.

> If the prompt already contained answers to all the decision points that come up when writing the spec then the prompt would already be the spec itself.

This is one of the best arguments against using LLMs I've seen.

It reduces to the classic argument- at the point where you've described a problem and solution in sufficient detail to be confident in the results, you've invented a programming language.

Privacy first, opt-in AI, total control over site blocking, zero ads.

You're the customer, not the product.

There are certain types of AI that will, and they are amazing: weed identification and laser zapping as a replacement for toxic pesticides, for example.

LLMs? I'm skeptical. I think we're in the middle of a mass delusion that stochastic parrot token extruding machine slop somehow equals "productivity".

From what I've seen it's just making the age old "activity over achievement" problem worse, while degrading skills.

One of two things is going to happen. Either we collectively find ways to recognize the limits of these things and use them in appropriate, limited ways or we devolve to Idocracy.

There's a third option that everyone seems to be breathlessly betting on, that the models improve to the point of human reasoning, but that seems like the most improbable outcome to me.

Are you aware of any reputable study that supports this? Everything I've seen, coding included, has productivity at a net neutral at best, with large cost increases due to LLMs.