Anyhow, I think you're (absolutely! ugh) right about the politics and I try to make the same point to people: whether you love or hate LLMs, accepting the "inevitabilism" framing is just ceding control of the Overton window. For better or worse, technology adoption can be and has been slowed by politics. We don't have nuclear plants everywhere. We don't have Project Orion starships colonizing Mars. We still have very strong social stigmas against genetic selection for human embryos, etc. This all can change in a heartbeat, and I'm not sure that policing the hardware rather than holding specific humans accountable for bad LLM outcomes is productive, but fundamentally: yes, we can stop it.

Comments URL: https://news.ycombinator.com/item?id=48149803

Points: 3

# Comments: 0

I am not trying to be snarky; I used to think that intelligence was intrinsically tied to or perhaps identical with language, and found deep and esoteric meaning in religious texts related to this (i.e. "in the beginning was the Word"; logos as soul as language-virus riding on meat substrate).

The last ~three years of LLM deployment have disabused me of this notion almost entirely, and I don't mean in a "God of the gaps" last-resort sort of way. I mean: I see the output of a purely-language-based "intelligence", and while I agree humans can make similar mistakes/confabulations, I overwhelmingly feel that there is no "there" there. Even the dumbest human has a continuity, a theory of the world, an "object permanence"... I'm struggling to find the right description, but I believe there is more than language manipulation to intelligence.

(I know this is tangential to the article, which is excellent as the author's usually are; I admire his restraint. However, I see exemplars of this take all over the thread so: why not here?)

(People are fuzzy and while game theory is fun and even sometimes useful, this kind of stuff skeeves me out and I think it lures “smart” people in and gets them to empower the SBFs of the world. My altruism is ineffective and I’m happy that way.)

You were either a very talented baby or we’re justified in questioning your ability to assess the correctness of nitpicky formalisms.

But yes, I think the best rebuttal to Dijkstra-style griping is Perlis' "one can't proceed from the informal to the formal by formal means". That said I also believe kind of like Chesterton's quote about Christianity, they've also mostly not been tried and found wanting but rather found hard and left untried. By myself included, although I do enjoy a spot of the old dependent types (or at least their approximations). There's an economic argument lurking there about how robust most software really needs to be.

- Edsger Dijkstra, 1988

I think, unfortunately, he may have had us all dead to rights on this one.

But there was nobody there, and I'm only disappointed in myself for not noticing.

Without the inferred writer, it's much less interesting to me, except as a reminder that models change and I can't rely on the old tics to spot LLM prose consistently any more.

That said, I _did_ read your comments history (only because you asked!) and - well, I don't know, you seem very reasonable, but I notice you're upset with people talking about "hallucinations" in code generation from Opus 4.6. Now, I have actually spent some time trying to understand these models (as tool or threat) and that means using them in realistic circumstances. I don't like the "H word" very much, because I am an orthodox Dijkstraist and I hold that anthropomorphizing computers and algorithms is always a mistake. But I will say that like you, I have found that in appropriate context (types, tests) I don't get calls to non-existent functions, etc. However, I have seen: incorrect descriptions of numerical algorithms or their parameters, gaslighting and "failed fix loops" due to missing a "copy the compiled artifact to the testing directory" step, and other things which I consider at least "hallucination-adjacent". I am personally much more concerned about "hallucinations" and bad assumptions smuggled in the explanations provided, choice of algorithms and modeling strategies, etc. because I deal with some fairly subtle domain-specific calculations and (mathematical) models. The should-be domain experts a) aren't always and b) tend to be "enthusiasts" who will implicitly trust the talking genius computer.

For what it's worth, my personal concerns don't entirely overlap the questions I raised way above. I think there are a whole host of reasons people might be reluctant or skeptical, especially given the level of vitriol and FUD being thrown around and the fairly explicit push to automate jobs away. I have a lot of aesthetic objections to the entire LLM-generated corpus, but de gustibus...

Humanity has possessed nuclear weapons for 80 years and has used them exactly twice in anger, at the very beginning of that span. We can in fact just NOT do things! Not every world-beating technology takes off, for one reason or another. Supersonic airliners. Eugenics. Betamax.

The best time to air concerns was yesterday. The next best time is today. I think we technologists wildly overestimate public understand and underestimate public distrust of our work and of “AI” specifically. We’ve got CEOs stating that LLMs are a bigger deal than nuclear weapons or fire(!) and yet getting upset that the government wants control of their use. We’ve got giddy thinkpieces from people (real example from LinkedIn!) who believe we’ll hit 100% white collar unemployment in 5 years and wrap up by saying they’re “5% nervous and 95% excited”. If that’s what they really think, and how they really feel, it’s psychopathic! Those numbers get you a social scene that’ll make the French Revolution look like a tea party. (“And honestly? I’m here for it.”)

So no, while I _think_ you’re correct, I don’t accept the inevitability of it all. There are possibilities I don’t want to see closed off (maybe data finally really is the new oil, and that’s the basis for a planetary sovereign wealth fund. Maybe every man, woman, and child who ever wrote a book or a program or an internet comment deserves a royalty check in the mail each month!) just yet.

But if you are genuinely confused by the attitudes of your peers, try asking not "what do I have that they lack" ("curiosity"?) but "what do they see that I don't" or "what do they care about that I don't"? Is it possible that they are not enthusiastic for the change in the nature of the work? Is it possible they are concerned about "automation complacency" setting in, precisely _because_ of the ratio of "hundreds of times" writing decent code to the one time writing "something stupid", and fear that every once in a while that "something stupid" will slip past them in a way that wipes the entire net gain of AI use? Is it possible that they _don't_ feel that the typical code is "better than most engineers can write"? Is it possible they feel that the "learning" is mostly ephemera - how much "prompt engineering" advice from a year ago still holds today?

You have a choice, and it's easy to label them (us?) as Luddites clinging to the old ways out of fear, stupidity, or "incuriosity". If you really want to understand, or even change some minds, though, please try to ask these people what they're really thinking, and listen.

This isn't what many of us probably would have wanted, but I think the public blowback when "AI-coded" systems start failing is going to drive us there. (Note to passing hype-men: I did not say they will fail at higher rates than human-coded systems! I happen to believe this, but it is not germane to the argument - only the public perception matters here.)