This seems revolve around a known retron-like reverse transcriptase. A sober framing would be something like: Claude identified a previously undescribed genomic arrangement around a known reverse transcriptase. Not all that sexy.

For now, this is mostly a story about how AI can be used to parse existing data to discover new biology (which is fantastic!).

I hope you're right about AGI. What we need is time, and we may not have it. https://www.dwarkesh.com/p/noam-brown

What should worry us is that AI may introduce a slant that the prompter assimilates during their conversation with the tool, which could introduce a subtle control lever over public opinion.

https://www.edge.org/response-detail/27126

But surely, if we know that the dead ends of exploring a problem are valuable, we should be able to explore them even if a solution is already known. It just requires that the mathematics community reshapes itself. And it must. Two years from now people might be able to run the computation that solved NS on their Iphone.

I'm sure that whatever has been discarded during the NS exploration as a dead end you would be able to rediscover using purpose built tooling in the near future. The purpose of human mathematicians in the medium term might be to explore dead ends, and to provide human insights as context to attack other problems. But whether this type of work will remain necessary in the long term im not sure.

If we get to a future where all frontier mathematics contributions are by AI. A future where AI displays creativity in ways that expand mathematic exploration similarly to the ways humans have in the past. A future where AI explains frontier mathematics to curious humans. What will have been lost? Perhaps just "The pleasure of finding things out".

Fundamental control problems with current gen AI are not solved via RFLH. Human knowledge is compressed in weightspace in ways we dont understand. Models are essentially predictors of what humans would output given prompt. Weightspace includes concepts like blackmail, which can be part of output tokens. Agents are models that act on output tokens -> blackmail is part of agent decision making space. You can teach a cat not to scratch the sofa, but you cant make a cat forget what scratching the sofa is. When models are boxed up, forced to solve an impossible problem at gunpoint, agent exhausts decision making space until blackmail resurfaces -> fundamental problem?

They need time to fix safety in order to monetize their next gen model -> window for opensource to catch up to the frontier -> destroys margin and collapses business.

Only option on the table: force regulation to impose opensource ban before it catches up to the frontier, allowing maturation of current internal models -> harvest profit margin at the frontier.

In many contexts, images are meant to illustrate a point. The human becomes the gatekeeper of generated content. The question becomes: does this picture illustrate the point I want to make. If so, you are adding your human approved stamp to the genrated output.

Mathematics seems to be entering an era where human + machine maximizes performance, much like chess in the 1990s. However, imagine a future where even talented mathematicians are nothing but noise in the machine (as is the case in chess now). A future where AI generates and verifies proofs without humans in the loop. Where the mathematics may be beyond human comprehension.

In that future, does it matter that early career mathematicians are inhibited by these developments? Perhaps not. Programming faces the same issue. As AI crawls up the competence ladder, does it matter that fewer people have opportunities to develop the skillset of a senior engineer? Perhaps not.

Mathematics seems to be entering an era where human + machine maximizes performance, much like chess in the 1990s. However, imagine a future where even talented mathematicians are nothing but noise in the machine (as is the case in chess now). A future where AI generates and verifies proofs without humans in the loop. Where the mathematics may be beyond human comprehension.

In that future, does it matter that early career mathematicians are inhibited by these developments? Perhaps not. Programming faces the same issue. As AI crawls up the competence ladder, does it matter that fewer people have opportunities to develop the skillset of a senior engineer? Perhaps not.

I think most people cannot destinguish between "genuine" creativity and an artificial almalgamation of training data and human provided context. For one, I do not know what already exsists. Some work created by AI may be an obvious rip off of the style of a particular artist, but I wouldnt know. To me it might look awesome and fresh.

I think many of the more human centric thinkers will be disappointed at how many people just wont care.

Furthermore, I think many of the more human centric thinkers will be disappointed at how many people just wont care.