Funny when it’s someone else eh?

Just learn a new trade Terry.

(I am kidding if it wasn't obvious)

Thats a fair point, but Ive seen LLMs modify the surrounding environment to “pass the test”

A human still has to be responsible at some point.

Tasks, oh yeah 100%.

It really depends on the context. A react button background color - meh - the critical part of the backend where a new 300ms delay will cost the company millions then yeah.

Not saying it doesn’t exist, just saying this sounds dangerously close to a boomer talking about the 1950s

If you prompt an LLM to build a web game (or something) it iterates over and over until the game works. And these days the output does indeed work. If you look at the code, it can often be a jumbled mess, but it still does work. It will take some sleuthing to understand what it is doing, and, if you care, clean up the code.

I think OpenAI did the same thing with Lean. The implications are far more interesting, but from what I understand the proofs are “slop”. Correct, but not elegant in any way.

It is still a wildly amazing technique to find a solution (or possible one) to maths problems.

I have used LLMs to do things I didn't know how to do, then reviewed what it did and learned a new technique.

No reason it cant work like that for maths.

Not everything is a product.

It’s like competitive knitting, or competitive architecture.

This focus on the inane is what has really damaged this industries perception in engineering IMO.

And the solution in the article is: how do best make them conform, and “do a bunch of work on themselves” so they are not like themselves, but more like the average.