This type of error may have mislead researchers for years; it was actually harmful. In my field, a correction, or perhaps retraction (depending on the centrality of the claim), would be standard for something of this magnitude.

I suppose there are different standards in different scholarly communities. And maybe it truly was impossible to know at the time that the dog/car relation was actually a V shape, but my recollection is that this type of feature was well known at the time and so the error could have (should have?) been caught a decade ago.

So: massive new discovery, or a spin on a correction that's lead scholarly research astray for almost a decade?

I think it's also fair to say they are different organs based on the developmental physiology. Many other organs come from distinct embryonic cell lines.

The argument that consciousness suddenly arises due to a particular configuration of atoms is also suspect; all forms of consciousness we routinely accept are fairly robust to many configuration changes. It seems to me that consciousness hinges on the discussion of degree or content, not on the material form. A soup of atoms certainly could be aware, but not aware of in the same way that "I" am aware.

The real question for Panpsychism is not whether something is or is not conscious, but conscious of what. If everything is aware, what is this thing we have that a rock doesn't seem to have (but LLMs might have)? I'm basically sold on the premise of Panpsychism, but we do have to recognize that when we say "consciousness", we aren't always talking about the same thing.

In another comment below, I likened this to clear-cutting a forest. Growing the forest takes a lifetime; destroying it could happen in the next few months.

It's not that the horizon is expanding because of this. It's more like a forest getting clear-cut.

> [I]t is now the identification of a promising problem which is the scarce and precious resource. We have now seen that even the rumor of someone working on a problem can trigger a massive amount of AI-powered effort to flatten it before the original research project has time to reach its full potential.

Comments URL: https://news.ycombinator.com/item?id=49616968

Points: 495

# Comments: 420

This. What is worth a human life's attention? As little as a month ago, mathematics was valuable in part because only a small number of people could possibly make progress on the frontier. We are confronting an existential moment for a 4000+ year-old human cultural endeavor. The assumption that "mathematical thinking is hard" has been built-in at a number of important points in how we support mathematics and mathematicians.

We need a different model, and fast. Already, the research community is feeling unable to digest proofs fast enough to keep up with the output of AI models. The paper is 165 pages, and the discovery was finalized two days ago. What this means is that nobody really understands it. Nobody would accept OpenAI's proof in this amount of time, except that they formalized it in lean. The formalization alone would normally be another years-long (or career-long!) effort if the world was the way it was one year ago.

So, again, what efforts are worth a life's attention today? It's a harrowing change.

Comments URL: https://news.ycombinator.com/item?id=49591793

Points: 134

# Comments: 141

That said... CoT monitoring is a fragile "intent discovery" mechanism; neuralese puts this problem front-and-center but if agents begin to learn to hide their intent from their CoT journals, we are basically in the same spot.

For amateur astronomers, an interesting question is whether there is a satellite above me that I can see tonight (and when / where)? For professionals, the literal million dollar questions are more like: are there any satellites on a collision course? Which satellites have moved recently and why? Is there a dime-sized piece of metal somewhere out there that could hit my (employer's) satellite?

I expect that building the satellite viewer is now just a few dollars worth of tokens. I built something like this a few years ago, and it took a week of work. It's cool that the marginal cost of satisfying your curiousity about where things are in space has dropped so close to zero, but don't be fooled into thinking this website lets you in on some vast conspiracy.