Why would you want to replace your lawyer with a set of tensors that does not actually think and makes mistakes like this? Lawyers tend to get hired in high stakes situations. Why wouldn't you instead say that this would be a great tool for lawyers to use judiciously in researching precedents, etc?

I don't understand what people are doing with models that makes them assign agency or intelligence to them. When I manage to forget the financial fuckery of the AI buildout and its implications, when I manage to forget scaremongering by loathsome CEOs, I still have the same fascination and excitement at the idea of LLMs as I did when I was playing with the GPT API prior to the release of ChatGPT.

LLMs are, to me, truly amazing tech. It's so fascinating to me that they now DO have emergent properties that look at face value like reasoning and intelligence. But every day that I work with them, I am repeatedly clobbered over the head with the fact that they do NOT reason and are NOT intelligent.

Why can't we be fascinated by emergent properties of intelligence without immediately jumping 10 steps into the future and, like a limit in calculus, assume that "this is it-- we're on the cusp of AGI"? To me, the fact that LLMs can combine existing ideas that people hadn't thought of combining in solving a novel problem is extremely cool. But my first thought is-- this is an amazing new tool for mathematicians and researchers. Instead, most everyone seems to jump the gun to the "humans are obsolete next year" conclusion.

Comments URL: https://news.ycombinator.com/item?id=49261868

Points: 2

# Comments: 1

I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.

It's not even about votes, I don't even need votes. Whenever I write something that I am happy with, I read and reread it imagining I am reading it as a third person. Sometimes it forces me to rework my arguments. I wouldn't write to convey my ideas through a chatbot. And that's what this post is about-- killing the internet and with it decimating any audience you might have accrued if you had something to say and you published it on a website.

I do not believe this to be the case; model improvements to me do not appear to converge towards that outcome. But at this point normal behavior from these entities would be perceived as a bearish signal I suppose, and so the show must go on.

I definitely resent how this state of affairs has made me feel derision towards so much in this field that I used to love.

Actually engaging my brain to solve the lower-level, on-the-ground code allows me to think of better ways to do things while I'm writing them. It's like writing anything. You start with something you want to convey, a thesis, and then it evolves and becomes better as you write it. An agent will just write it with no thought, as in it will reflect one of the LLM 'ghosts' as Andrej Karpathy puts it, doing something in the same way that someone in the training data has done it on a similar or different problem. This is why I get conniptions now when I am sent generated text or am expected to read it on a public forum. It's disrespectful of the time of every person expected to read it.

That said, I wouldn't like to go back to the before times without having the agentic option available. Ideally, businesses should not mandate how LLM's are to be used at their company, and just let the devs find their own flow. That is, if quality is even a factor that any company optimizes for anymore.

On another note, I am continuously flabbergasted that even Google cannot get an agentic LLM assistant right. I used to use the Google Assistant ALL the time when it did not make use of LLM's. It was AMAZING. I could just say "wake me up at 7" and it would understand that I meant 7AM the next day, or call "mom", or "set a timer for 10 minutes" while I'm cooking. It was genuinely useful to me.

Google Assistant last week sent me to the Google Search page with my prompt "wake me up at 7am", and Gemini read out loud "setting an alarm for 7am tomorrow morning". But it was bullshit, it doesn't have the power to set alarms, it was just gaslighting me.