But even if you find them persuasive, there is something unpleasant about AI alarmism as a cultural phenomenon that should make us hesitate to take it seriously.

First, let me engage the substance. Here are the arguments I have against Bostrom-style superintelligence as a risk to humanity"

--

The framing here seems to me to equate "AI risk" and "AI alarmism" with buying in to belief in "Bostom-style superintellgence".

I'm not sure if the author meant to put anyone who is alarmed by developments in what we're calling "AI" into the same bucket as "AI obsessives want to make it into a programming problem, by designing a God-like machine", but I think this conflation is unfair and, frankly, dangerous.

I don't know what superintelligence is. I don't even know what intelligence is. And I don't really know what either "artificial" or "general" mean either when talking about "AGI".

You can believe, as I do, that these things can be, and will inevitably will be if we don't radically correct course, used to do very bad things independent and short of being "God-like". When you have systems which can hypothesize, synthesize, and test thousands if not millions of potential infectious agents in bulk [0], and can then order the ingredients for you from dodgy websites via some "claw", and then when you put these systems under the unsupervised control of millions of people with varying levels of stability and altruism, something extremely bad is exceedingly likely to happen.

I understand that 2016 is ages ago and things change, but I came away from the article with the impression that if I'm worried about AI risk then I'm a clown like the three pictured in the "Outside Argument" section (you're a Google-Glass-wearing cringe nerd if you're alarmed). Maybe that's my fault and I'm not smart enough to understand the actual point of the article. If I have misinterpreted, I welcome the correction.

[0] https://www.nature.com/articles/s41467-024-53759-4

Sounds like a Watchmen Comedian logo t-shirt. It could be construed as a bold choice but was probably just what was on the top of his t-shirt stack that day.

You're not the only one.

The current Pope Leo XIV explicitly named himself after the the previous Leo, Pope Leo XIII, who was pope during the Industrial Revolution (1878-1903) and issued the influential Encyclical Rerum novarum (Rights and Duties of Capital and Labor) in response to the upheaval.

“Pope Leo XIII, with the historic Encyclical Rerum novarum, addressed the social question in the context of the first great industrial revolution,” Pope Leo recalled. “Today, the Church offers to all her treasure of social teaching in response to another industrial revolution and the developments of artificial intelligence.” A name, then, not only rooted in tradition, but one that looks firmly ahead to the challenges of a rapidly changing world and the perennial call to protect those most vulnerable within it.”

https://www.vatican.va/content/leo-xiii/en/encyclicals/docum...

https://www.vaticannews.va/en/pope/news/2025-05/pope-leo-xiv...

Imagine hearing pre-attention-is-all-you-need that "AI" could do something that Donald Knuth could not (quickly solve the stated problem in collaboration with his friend).

The idea that this (Putnam perfect, IMO gold, etc) is all just "statistical parrot" stuff is wearing a little thin.

“The laws of nature should be expressed in beautiful equations.”

- Paul Dirac

“It is, indeed, an incredible fact that what the human mind, at its deepest and most profound, perceives as beautiful finds its realisation in external nature. What is intelligible is also beautiful. We may well ask: how does it happen that beauty in the exact sciences becomes recognizable even before it is understood in detail and before it can be rationally demonstrated? In what does this power of illumination consist?”

- Subrahmanyan Chandrasekhar

“I often follow Plato’s strategy, proposing objects of mathematical beauty as models for Nature.”

“It was beauty and symmetry that guided Maxwell and his followers.”

- Frank Wilczek

“Beauty, is bound up with symmetry.”

- Herman Weyl

"Still twice in the history of exact natural science has this shining-up of the great interconnection become the decisive signal for significant progress. I am thinking here of two events in the physics of our century: the rise of the theory of relativity and that of the quantum theory. In both cases, after yearlong unsuccessful striving for understanding, a bewildering abundance of details was almost suddenly ordered. This took place when an interconnection emerged which, thought largely unvisualizable, was finally simple in its substance. It convinced through its compactness and abstract beauty – it convinced all those who can understand and speak such an abstract language."

- Werner Heisenberg

Maybe (just maybe) these things (whatever you want to call them) will (somehow) gain access to some "compact", beautiful, "largely unvisualizable" "interconnection" which will be the self-evident solution. And if they do, many will be sure to label it a statistical accident from a stochastic parrot. And they'll right, for some definitions of "statistical", "accident", "stochastic", and "parrot".

Donald Knuth is an extremal outlier human and the problem is squarely in his field of expertise.

Claude, guided by Filip Stappers, a friend of Knuth, solved a problem that Knuth and Stappers had been working on for several weeks. Unfortunately, it doesn't seem (from my quick scan) to have been stated how long (or how many tokens or $) it took for Claude + Stappers to complete the proof.

In response, Knuth said: "It seems that I’ll have to revise my opinions about “generative AI” one of these days."

Seems like good advice. From reading elsewhere in this comment section, the goalposts seem to be approaching the infrared and will soon disappear from the extreme redshift due to rate at which they are receding with each new achievement.

Update:

Rebecca Heineman - Organizer

It’s time. According to my doctors. All further treatments are pointless. So, please donate so my kids can create a funeral worthy of my keyboard, Pixelbreaker! So I can make a worthy entrance for reuniting with my one true love, Jennell Jaquays.

My daughter Cynthia Elizabeth Heineman, will be making the arrangements

> Says who?

https://en.wikipedia.org/wiki/John_von_Neumann#Mathematical_...

Von Neumann's mathematical fluency, calculation speed, and general problem-solving ability were widely noted by his peers. Paul Halmos called his speed "awe-inspiring." Lothar Wolfgang Nordheim described him as the "fastest mind I ever met". Enrico Fermi told physicist Herbert L. Anderson: "You know, Herb, Johnny can do calculations in his head ten times as fast as I can! And I can do them ten times as fast as you can, Herb, so you can see how impressive Johnny is!" Edward Teller admitted that he "never could keep up with him", and Israel Halperin described trying to keep up as like riding a "tricycle chasing a racing car."

He had an unusual ability to solve novel problems quickly. George Pólya, whose lectures at ETH Zürich von Neumann attended as a student, said, "Johnny was the only student I was ever afraid of. If in the course of a lecture I stated an unsolved problem, the chances were he'd come to me at the end of the lecture with the complete solution scribbled on a slip of paper." When George Dantzig brought von Neumann an unsolved problem in linear programming "as I would to an ordinary mortal", on which there had been no published literature, he was astonished when von Neumann said "Oh, that!", before offhandedly giving a lecture of over an hour, explaining how to solve the problem using the hitherto unconceived theory of duality.

A story about von Neumann's encounter with the famous fly puzzle has entered mathematical folklore. In this puzzle, two bicycles begin 20 miles apart, and each travels toward the other at 10 miles per hour until they collide; meanwhile, a fly travels continuously back and forth between the bicycles at 15 miles per hour until it is squashed in the collision. The questioner asks how far the fly traveled in total; the "trick" for a quick answer is to realize that the fly's individual transits do not matter, only that it has been traveling at 15 miles per hour for one hour. As Eugene Wigner tells it, Max Born posed the riddle to von Neumann. The other scientists to whom he had posed it had laboriously computed the distance, so when von Neumann was immediately ready with the correct answer of 15 miles, Born observed that he must have guessed the trick. "What trick?" von Neumann replied. "All I did was sum the geometric series."

(My son loves to play.)

BTW, the old Windows installer runs great under Crossover on Apple Silicon.

I'm going to try running this native. I'll file issues for what I can't readily fix.

It's appropriate that even the chief of the purveyors of advertisements uses the same language ("load") that clinicians use for viral infections.

https://www.pbs.org/faithandreason/theogloss/imago-body.html

But pushing aside those outmoded ways of thinking did make it easier to sell people useless goods and services which can't fill the hole where dignity and self-respect could instead reside.

And on top we've built the artifice of "social" media (the most shameless oxymoron imaginable) producing sufficient anxiety ensuring that the hole remains unfilled despite all effort.

This is a frightening confluence.

https://www.axios.com/2022/11/15/ftz-crypto-bankman-fried-de...

"Last spring, Bankman-Fried pledged to spend upwards of $1 billion on the 2024 election, particularly if Donald Trump were to run."

https://www.opensecrets.org/elections-overview/biggest-donor...

Sam Bankman-Fried

  "Solidly Democrat/Liberal"
  Rank 6
  Contributor FTX.US, Washington, DC
  Total Contributions $39,884,256
  Total Hard Money $1,047,256
  Total Outside Money $38,837,000
  To Democrats $36,846,356
  To Republicans $240,200

https://www.foxbusiness.com/politics/ftx-founder-sam-bankman...

"In addition, Bankman-Fried wired maximum individual donations to Boozman and incumbent Republican Sens. Susan Collins of Maine, Richard Burr of North Carolina, Bill Cassidy of Louisiana, John Hoeven of North Dakota and Lisa Murkowski of Alaska throughout the election cycle."

Collins, Burr, Cassidy, and Murkowski are strongly aligned against Trump.

Boozman received his contribution(s) during his primary against Jake Bequette, a strongly pro-Trump Republican.

Hoeven likewise during his primary race against Rick Becker, another strongly pro-Trump Republican.

Also Boozman:

https://www.benzinga.com/markets/cryptocurrency/22/11/296766...

"Stabenow noted that she is working with her Republican counterpart on the committee, Sen. John Boozman (R-Ark), and regulators to finalize the Digital Commodities Consumer Protection Act (DCCPA) bill in preparation for a committee vote. DCCPA gives power to the Commodity Futures Trading Commission (CFTC) to regulate the trading of digital commodities. This bill was backed by Sam Bankman-Fried, co-founder of the crypto exchange FTX."

'Mr Herapath, I understand that you wish to visit the whaler, and you have my permission to go. You are no doubt aware that there is a great deal of ill-feeling between the United States and England, and that most unhappily the Leopard was the cause of some of it: that is why I thought best to forbid the usual ship-visiting, to prevent quarrelling of any kind. You also know the Leopard's condition: one day's use of a forge and the proper tools would enable her to put to sea rather than winter here. The whaler certainly possesses a forge, but as a gentleman you will understand that I am extremely reluctant to ask a favour of the American skipper, extremely reluctant to expose the service or myself to a rebuff. I may add that he is equally reluctant to come a-begging to me, and I honour him for it. However, on reflection he may feel inclined to exchange the use of his forge for our medical services. You may give him a view of the situation, but without committing us to any specific request - harkee, Mr Herapath, don't you expose us to an affront, whatever you do. And if it should turn out that he would like the exchange, why, I should be very much obliged to you. Very much obliged indeed, for I should be even more reluctant to use force.'

It's almost a cliche at this point, but the prefrontal cortex isn't mature until between 25 and 30 on average.

"One key part of that trajectory is the development of the prefrontal cortex, a significant part of the brain, in terms of social interactions, that affects how we regulate emotions, control impulsive behavior, assess risk and make long-term plans. Also important are the brain’s reward systems, which are especially excitable during adolescence. But these parts of the brain don’t stop growing at age 18. In fact, research shows that it can take more than 25 years for them to reach maturity."

So, yes, teach your children how to avoid predators. That is excellent. But this is the last line of defense. Since children have major impulse control and emotional regulation deficits and the predators have a major asymmetrical advantage in behavioral engineering, it is overwhelmingly the job of the parents to the extent possible to just keep the predators away.

> "Internet is scary"

Damn right it is. Children are uniquely impressionable and imprintable for a long time. Seeing or being forced to do gnarly stuff at the wrong time is permanently disfiguring.

> Kids will find a way to hang out with their friends.

Yes, the traditional way that would happen is at someone's house. Together. In person. Which provides some level of protection against predation and a fuller/richer/healthier social experience. Where the venue is virtual those protections are lost and more vigilance is required.

> If you get in the way of it, you'll quickly find yourself on the losing end of a years-long battle.

There are wolves in the world. There always have been and always will be (as you say). It's a never ending and virtually thankless job (in fact, you will regularly be abused for doing it), but keeping the wolves at bay is parenting job #1. Get them to maturity whole, healthy, intact, and self-sufficient.

I'm not going to share experiences to the extent of the OP, but I have kids and I've met some wolves.

Get your credit card from a local credit union with at least one physical presence near you.

Do your own backups or contract to an independent company whose primary occupation is doing backups.

Manage your own email either directly or through an independent email provider. Buy a domain and use that for your email address to increase flexibility to move from one provider to another.

Diversify.

Anything else increases your risk of getting caught in a personal and financial wood chipper.

Tech Company: At long last, we have created the Torment Nexus from classic sci-fi novel Don't Create The Torment Nexus"

Three months, available to all adults.

"All adults in the United Kingdom will be able to get their booster Covid-19 vaccine doses three months after their second shot, the government indicated on Monday, in a dramatic acceleration of the country's inoculation drive that comes amid fears over the Omicron variant."

Steps:

* Allow (and encourage) private sector monopolies to form

* Use its leverage and influence to "encourage" those monopolies to do things which the US government is not allowed to do (examples: censorship, data collection)

The list of First Amendment precedents, for example, with respect to the government is long and detailed. On the other hand, private companies operate under fewer restrictions and can act much more freely.

So if we have a situation where there are a small number of monopolies and they, through personal relationships (marriage), lobbying, and the revolving door have a cozy relationship with the US government, then the government can effectively censor (for example) by more or less declaring "will no one rid me of these turbulent tweets" and have the monopolies jump to comply (or face increased risk of antitrust action for example).

This is a win-win for both: the monopolies avoid antitrust and other unwanted regulation by playing ball, and the government gets to ignore the Bill of Rights by "outsourcing" actions to what are in effect de facto quasi-governmental entities. It's only a loss for the people for whom the Bill of Rights was written to protect.