Hacker News: mikestylz

New comment by mikestylz in "Sharing AI progress in mathematics"

mikestylz — Wed, 07 Oct 2026 04:56:07 +0000

We can't stop it. But you can use your voice to buy time and resources for alignment and safety research. A few additional months may make a world of difference.

New comment by mikestylz in "Sharing AI progress in mathematics"

mikestylz — Wed, 07 Oct 2026 01:21:19 +0000

> When AI becomes more capable of navigating practical problems without human intervention, I will start to be concerned.

Given the past rate of progress, why not start being concerned now? It's a bit like the economist saying that the optimal number of flights to miss is not zero. If you keep landing short on your estimations for how far this technology will go, next time you should err on the other side.

And regarding Vending-Bench 2 (https://andonlabs.com/evals/vending-bench-2) my understanding is that models do pretty well on it now.

New comment by mikestylz in "Responsible Release of AI-Generated Mathematics"

mikestylz — Thu, 01 Oct 2026 03:04:28 +0000

In the past we might have said horses, stone, and bronze were three things necessary to advance civilization. In the future I'm sure the AIs will prompt themselves with nice problems. But for now I agree with AGMAI that we should be careful not to disrupt the field too severely. It's reasonable to keep it around even if it's just functioning as a temporary "strategic reserve of expertise"

New comment by mikestylz in "I'm sorry, you're not going to die from an AI-engineered supervirus"

mikestylz — Fri, 11 Sep 2026 07:07:16 +0000

You seem like a smart person. I'm glad you're pushing back but cool the tone, and let's see if we can get to an agreement. There is not much evidence for these trade-offs put forward in the article, just on the order of five instances in nature which form a kind of curve if you squint. Should this be convincing? Well there are even definite outliers in history. Take smallpox on the native americans, or the bubonic plague. Ten years ago a mysterious virus wiped out 90% of the world's Bellinger River turtles. Why aren't these points in the article? Before the invention of the airplane there were many biologists saying such a thing was impossible, because they were extrapolating from the example of avian flight.

As for complete satisfaction with arguments, I'll admit you're right. I am often reasonably convinced but I don't know if I would ever give 100% confidence to anything. There can still be loads of zeros or nines after the decimal though. What's concerning here is that the article speaks in absolutes. I'm inclined to thinking in expected values. Even if we believed there were only a 1% chance such a virus could ever be designed by an AI, if the effect were to be a global plague of biblical proportion, we should still take that possibility extremely seriously. There's no humility shown in the writing, and that's the single most problematic part to me.

New comment by mikestylz in "I'm sorry, you're not going to die from an AI-engineered supervirus"

mikestylz — Thu, 10 Sep 2026 02:44:39 +0000

It is not, because we should not expect a manufactured technology to behave exactly as an evolved one. My understanding is there is not much incentive for a virus to kill off all its hosts. When they do it is largely an accident, and it is not uncommon for a severe virus to eventually get replaced by a less virulent and thus more successful strain. It's pretty hard to hill-climb evolution to such a point since the virus could starve itself in the process. And induction on evolution feels misguided to me. Consider our species. A million years ago it would have seemed completely reasonable to state that the absence of advanced tool use in the kingdom of life suggests it is biologically impossible. After all, such a thing had not occurred in hundreds of millions of years!

To be clear, I am not saying that such a virus will exist eventually. I am just noting that the arguments displayed here are not completely tight, and there are unknown unknowns at play. Some things are unfalsifiable. This is common.

New comment by mikestylz in "I'm sorry, you're not going to die from an AI-engineered supervirus"

mikestylz — Thu, 10 Sep 2026 01:40:16 +0000

The magical level of intelligence required is not really a flaw of the argument's deductions, it is its very premise. If you believe that there is a possibility of an AI working on itself, you get the approximate differential equation d/dt f(t) >= C*f(t). In plain English, at least some proportion of development's progress goes back into future development. The (least case growth, assuming equality) solution to this is an exponential function, and exponential blowup leads to stupendous and magical seeming results. I then ask - what is evolution if not a dumb optimization process? How are we supposed to believe that it devised miracles such as the human mind, insect flight, echolocation and so on while denying the possibility of a future EXTREMELY power simulation to do the same? I went in really wanting to be convinced by this article for the sake of my own mental health, but I am disappointed. What we really need for such a statement is concrete physical evidence that a virus is simply incapable of having such facilities. Or perhaps an equivalence established showing biological defense is no harder than offense. Until it exists, I will continue to worry.