(Of course there is also scope for debate about how much world model today's LLMs have; it seems like it's more than none even though it has to be built out of token-shuffling parts. But that's not relevant here.)

(I do agree that Mr Trump is a shockingly bad president in oh so many ways. But the malaise being described here doesn't seem to have started in 2016. Not every bad thing is his fault.)

(In practice, they presumably couldn't do that, at least not effectively, because the code is open source and someone else could fork it. But let's imagine that somehow they could require all Thunderbird users to pay them.)

That doesn't, of course, mean that it would be better overall. Thunderbird users would go from getting Thunderbird for free and maybe having reason to donate some money, to having to pay some money just to keep the ability to use Thunderbird: obviously worse for them. There'd probably be more money available for Thunderbird development, which would be good. The overall result might be either good or bad. But it would, indeed, no longer be unclear whether and why a Thunderbird user might choose to pay money to the Thunderbird project.

If instead I donate to an open-source project, I'm not doing it in order to get access to the product; I already have that. I'm doing it because I hope they will do something with the money that I value. (Possible examples: Developing new features I like. Rewarding people who already developed features I liked. Activism for causes I approve of. Continuing to provide something that benefits everyone and not just me.)

And so I care a lot what they're going to do with the money, in a way I don't if I (say) pay money to Microsoft in exchange for the right to use Microsoft Office. Because what they're going to do with the money determines what point there is in my giving it.

Sometimes, everything the project does is stuff I think is valuable (for me or for the world). In that case I don't need to ask exactly what they're doing. Sometimes, it's obvious that what happens to the money is that it goes into the developer's pockets and they get to do what they like with it. In that case, I'll donate if the point of my donation is to reward someone who is doing something I'm glad they're doing, and probably not otherwise.

In the case of Thunderbird, it's maybe not so obvious. Probably the money will go toward implementing Thunderbird features and bug fixes, but looking at the history of Firefox I might worry that that's going to mean "AI integrations that actual users mostly don't want" or "implementing advertising to help raise funds", and I might have a variety of attitudes to those things. Or it might go toward some sort of internet activism, and again I might have a variety of attitudes to that depending on exactly what they're agitating for. Or maybe I might worry that the money will mostly end up helping to pay the salary of the CEO of Mozilla. (I don't think that's actually possible, but I can imagine situations where Mozilla wants some things done, and if they can pay for them via donations rather than using the company's money they'll do so, so that the net effect of donating is simply to increase Mozilla's profits.)

And I don't think anyone's asking for anything very burdensome in the way of transparency. Just more than, well, nothing at all which is what we have at the moment. The text on the actual page says literally nothing beyond "help keep Thunderbird alive". The FAQ says "Thunderbird is the leading open source email and productivity app that is free for business and personal use. Your gift helps ensure it stays that way, and supports ongoing development." which tells us almost nothing. And "MZLA Technologies Corporation is a wholly owned for-profit subsidiary of the Mozilla Foundation and the home of Thunderbird." which tells us that donations go to a for-profit subsidiary of the Mozilla Foundation (which I believe is the same entity that owns the Mozilla Corporation, but like most people I am not an expert on this stuff and don't know what that means in practice about how the Mozilla Foundation, the Mozilla Corporation and MZLA Technologies Corporation actually work together).

Maybe donated money will lead to MZLA Technologies Corporation hiring more developers or paying existing developers more? Maybe it'll be used to buy equipment, or licences for patented stuff? Maybe it'll be used to advertise Thunderbird and get it more users? Maybe it'll be used to agitate for the use of open email standards or something like that? Maybe. Maybe some other thing entirely. There's no way to get any inkling.

Anyway, thanks for giving some indication of what you didn't like.

I vigorously encourage anyone who thinks something I wrote is bad to downvote it as they see fit, but it would be nice if some of those people would tell me what about my comment they found so objectionable. (It all seems pretty well reasoned to me -- but it would, wouldn't it?)

[EDITED to fix an inconsequential typo]

The structured thing you mention is the one that bugs me most. I genuinely think that most human writing would be improved by having more of the "signposts" that LLMs overuse. Headings, context-setting sentences, bullet points where appropriate, etc. I was doing "list of bullet points with boldfaced intro for each one" before the LLMs were. But because the LLMs are saturating their writing with it, we'll all learn to take it as a sign of glib superficiality and inauthenticity, and typical good human writing will start avoiding everything of that kind, and therefore get that little bit harder to read. Alas.

I can't speak for anyone else, but what I feel when I read yet another glib "it's just a stochastic parrot, of course it isn't doing anything that deserves to be called reasoning" take is much more like bored than it is like upset.

Today's LLMs are in some sense "just predicting tokens" in some sense. Likewise, human brains are in some sense "just shuttling neurotransmitters and electrical impulses around" in some sense. Neither of those tells you what the thing can actually do. To figure that out, you have to look at what it can do.

Today's best LLMs can do about as well as the best humans on problems from the International Mathematical Olympiad and occasionally solve easyish actual mathematical research problems. They write code about as well as a junior software developer (better in some ways, worse in others) but much faster. They write prose about as well as an average educated person (but with some annoying quirks that are annoying mostly because they are the same quirks over and over again).

If it pleases you to call those things "thinking" then you can. If it pleases you to call them "stochastic parroting" then you can. They are the same things either way. They are not, on the face of it, very much like "just repeating things the machine has already seen", or at least not more like that than a lot of things intelligent human beings do that we don't usually describe that way.

If you want to know whether an LLM can do some particular thing -- do your job well enough for your boss to fire you, write advertising copy that will successfully sell products, exterminate the human race, whatever -- then it's not enough to say "it's just remixing what it's seen on the internet, therefore it can't do X" unless you also have good reason to believe that that thing can't be done by just "remixing what's on the internet" (in whatever sense of "remixing" the LLM is doing that). And it's turning out that lots of things can be done that way that you absolutely wouldn't have predicted five years ago could be done that way.

It seems to me that this should make us very cautious about saying "they can't do X because all they can do is regurgitate a combination of things they've seen in training".

(My own view, not that there's any reason why anyone should care what I-in-particular think, is a combination of "what they're doing is less parroting than you might have thought" and "you can do more by parroting than you might have thought".)

So, anyway, this particular instance of the stochastic-parrot argument started when someone said: of course the AIs are yes-men, because figuring out when to agree and when not to requires actual logic and thought and the LLMs don't have either of those things.

Is it really clear that deciding whether or not to agree when someone says "I think maybe I should break up with my girlfriend" or "I've got this amazing new theory of physics that the establishment is stupidly dismissing" requires more logic and thought than, say, gold-medal performance on IMO problems? It certainly isn't clear to me. Having done a couple of International Mathematical Olympiads myself in my tragically unmisspent youth, I can assure you that solving their problems requires quite a bit of logic and thought, at least for humans. It may well be harder to give a good answer to "should I leave my job?", but it's not exactly "logic and thought" that it needs more of.

Someone reported that Claude is much less yes-man-ish than Gemini and ChatGPT. I don't know whether that's true (though it wouldn't surprise me) but: suppose it is; do you want that to oblige you to say that yes, actually, Claude really thinks logically, unlike Gemini and ChatGPT? I don't think you do. And if not, you want to avoid saying "duh, of course, you can't avoid being a yes-man without actually thinking and reasoning, and we all know that LLMs can't do those things".

If you spend $35k on a nice computer, and then earn $35k from doing some work using it, that doesn't mean that buying the computer has paid for itself unless the computer is solely responsible for that income. It probably isn't.

It's not necessarily even true that after doing that work it's "paid for", in the sense that getting the $35k income means that you were able to afford the $35k computer: that only follows if you didn't need any of that income for other luxuries, such as food and shelter.

If you're earning $50/hour, 40hr/week then what you've done after 17.5 weeks is earned enough to buy that $35k computer. Assuming you don't need any of that money for anything else, like food and shelter.

If the fancy computer helps you get that income then of course it's perfectly legit to estimate how much difference it makes and decide it pays for itself, but it's not as simple as comparing the price of the computer with your total income.

Regardless of how much it contributes, if you have plenty of money then it's also perfectly legit to say "I can comfortably afford this and I want it so I'll but it" but, again, it's not as simple as comparing the price of the computer with your total income.

Which kinda makes sense: it feels natural to do this with (1) things that are trivial or offensive, so that the pretentious LinkedIn versions are as different as possible, or (2) things with some sort of aura of greatness about them, so that the bathos of the sterile LinkedInSpeak is as great a contrast as possible.

But for #2 it feels to me like great writing rather than culturally important and revered writing is more the point (though of course there's some tendency for those to go together). I tried a few famous bits of writing to see what happened. Here are some outputs; guessing what the inputs were is left as an exercise for the reader. In most cases this is just the first bit of the LinkedIn translation. I think some are much easier than others.

I recently connected with a global traveler from an emerging market who shared some powerful insights on legacy and leadership.

It’s a global industry standard: any high-net-worth individual with a solid portfolio is actively looking to scale their personal life and onboard a long-term partner.

April is officially the most challenging month for growth.

Reflecting on the current landscape: it’s a time of massive disruption and unparalleled opportunity. We’re seeing a true dichotomy—the age of wisdom vs. the age of foolishness.

Reflecting on my career journey and personal ROI. When I look at my bandwidth and how my energy is spent before reaching my mid-career milestones in this vast, competitive landscape, I realize that sitting on my unique value proposition is a major career risk.

I was out there solo-navigating my journey, scaling high-level peaks and valleys like a cloud, when I suddenly hit a major milestone: I encountered a massive network, a high-performing host of golden daffodils.

Big news from Xanadu! Kubla Khan just greenlit a massive, high-end pleasure-dome project.

Team, it’s time to lean in and go again! [rocket emoji] We either hit our KPIs or we leave it all on the field. [100% emoji] In a steady market, there’s nothing better than a growth mindset and staying humble. But when the industry gets disrupted, it’s time to activate beast mode. [lion emoji]

(Cf. https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal....)

(Note 1: For all I know it may well be true that the reporting is 100% accurate and Palantir's claim to deserve a reply is 100% bullshit. I'm not saying they're in the right here! But I think the actual story is a bit less horrible than "Palantir is taking these guys to court because they didn't like their reporting" sounds without the relevant context. They're not, e.g., trying to get damages from the newspaper, or trying to get what they wrote retracted, or anything like that.)

(Note 2: I am not an expert on Swiss law or on this case, and I am accordingly not 100% confident of any of the above. In the unlikely event that whether I'm right about this matters to anyone reading, they should check it for themselves :-).)

"I suppose that, on a practical level, a take-home for the practicing mathematician is that if you use ChatGPT, don’t trust it to generate valid proofs, and even when it finds a valid proof, don’t be so sure it’s a good proof. And whatever you do, don’t have ChatGPT create a bibliography for you."

In isolation, that's all very good advice. But however is a take-home from what goes before? You used ChatGPT, it did generate a valid proof, it was "a solid by-the-book argument that employed a method I’ve used myself" which may or may not imply "a good proof" but suggests it was at least OK, and nothing in the story you told involves ChatGPT generating bibliographies at all.

https://www.youtube.com/watch?v=CR2A6FDiGEA is about Lindsey Graham in 2020 defending the Republicans' insistence on pushing through a nomination to the Supreme Court in the last year of Trump's presidency. It also includes a clip of Lindsay Graham in 2018 saying that if a Supreme Court vacancy opens once the primaries have started, "we'll wait till the next election".