As I argued in another comment here, I think even with unlimited budget, humans wouldn't be able to reproduce these results within a few years. I think AI has crossed a superhuman theshhold in math problem solving skills.

There are lots of reasons I believe this. One is that frontier AIs are very clearly superhuman on shorter problems (as can be seen on benchmarks like FrontierMath, or just by playing with them yourself if you have a source for difficult problems)--so these achievements are "just" scaling that advantage to research-sized problems. Second, AI is also clearly superhuman in its knowledge across fields, so it can make connections and combine techniques that humans won't be able to access. Third, these problems are famous and have withstood efforts of many of the best humans, in many cases being considered unapproachable.

Altogether it seems like AI is not only smarter but also more cost efficient that humans for math-adjacemt problem solving. I'm not saying that AI is superior at everything in math, eg theory building, but I think it's really unlikely you could get any group of humans that could compete with OpenAI's latest results.

In either case, no practical algorithm is expected to come from it, because the asymptotic gain isn't realized for practical input sizes--sometimes called a "galactic algorithm". But it's theoretically very interesting for breaking through a conjectured barrier.

It's like getting scooped. If you just founded a startup based on tech XYZ, should you be happy when someone releases an open source XYZ? Should a news reporter be happy when another network breaks the story they were working on? On the one hand, society got the value of the thing you wanted to do. On the other hand, now you need to find something else to do, which might be really annoying.

Real-world applications are far off, but developing mathematical understanding does tend to leak over into applied physics and CS.

A cynic might say this is all just intellectual games, and though there's a grain of truth, it's too cynical imho. This isn't like 8 queens where there's no hope for applications or generalizations. A lot of this stuff fundamentally affects our understanding of how numbers and systems behave, what are the limits of computation, etc.

Even if someone doesn't care about theoretical results, it's still exciting that AI has become superhuman in a domain as broad as math. That shows there's potential to be superhuman in other domains as well.

There are also many plausible arguments why our ability to train them to be helpful/trusting/aligned can fail. The smarter AIs get, the harder it is to be sure they're trained correctly. There are already reports that AIs are able to detect whether they're in a training environment and change their behavior accordingly.

Even if these are low probability scenarios, the risk-reward is terrible, so I think it's rational to be extremely cautious about AI risk.

The chess-math analogy would imply AI could bring us into a golden era of math competitions for humans. But I don't think it says anything good about prospects for humans in research math.

Now LLMs have produced multi-thousand line Lean proofs. This is impossible by simply "try everything and see what sticks". LLMs are able to target their efforts to only promising proof strategies. Yes it helps that they work at superhuman speed, so they can try thousands of strategies where a human might try a dozen. But their results cannot be explained only by compute increases; they need genuine mathematical insight.

But I agree LLMs have a lot of potential for checking proofs--both informally (they can read quickly and find gaps) and formally (by attempting to formalize).

But with that in mind, I mostly agree with the article: if it's not a violation of "single source of truth", then abstractions are just a convenience. If it starts being inconvenient, then it's not doing its job and there's no reason to use it. It's a serious code smell if a function needs several flags for custom behavior; that means it's probably the wrong abstraction or violating the single responsibility principle. If there is a legit need for lots of customization, an often-good way to handle is to take a function/functor as an argument for the customization. E.g., rather than `solve(f:double -> double, max_iters = 99, x_abs_tol = 1e-15, x_rel_tol = 1e-15, ...)` you can do `solve(f:double -> double, stopping_criteria: StoppingCriteriaClass)`

I thought code golf would take longer for AIs because there's so little training data (it's more niche), but we're seeing AIs starting to match expert humans there too. Sucks because golf has been my favorite type of programming puzzle.

It's crazy how far AIs have come in problem solving ability.