Every once in a while, everyone needs to touch grass.

I don't see the report as absolving the industry of wrongdoing. In fact, I believe it is particularly damning, and many of my colleagues feel the same. No one is on the side of OpenAI here, not even its own members.

As for submitting the report back to OpenAI for comment and working with them, given that there is currently no government mandate or requirement for any AI company to disclose wrongdoing, this seems like an unfortunate necessity. Independent bodies and governing agencies have to be created. At the moment, it is the industry regulating itself, which means no regulation at all.

So no, you do not need to believe that the document is truth. But it is all that we will likely get, because nobody with any power will do a damn thing. That is just as frustrating to the researchers as it is to you. The solution is not to defund the institutes doing needed research. It is to put laws or incentives or something in place to punish inappropriate behavior and lack of disclosure for the companies.

Now, I do not know METR, and have not interacted with its staff. $120m is pretty normal for these types of groups as I understand. METR clearly has even tighter ties to the AI companies than any of the groups I do know, so I agree that calling them independent is a bit of a joke. Even without the explicit labels from their website, there are shockingly few degrees of separation in this entire field. The conferences are big, workshops are designed to be highly social, and the stakes feel high.

Regarding the timeframe and limited material, that's pretty standard now. You do what you can.

Now, none of this necessarily invalidates what they did. Yes, there is big money in AI, and that money hires people, but there quite a lot of geeky researchers in these groups that genuinely only care about the science. That level of neurodivergence might be difficult for most people to comprehend, especially if you're focusing on the politics, but these are the types of people attracted to this line of work.

Put yourself in the shoes of these researchers. You are really interested in AI and how it works, and you convince yourself that this is potentially moral to work on because of a possible threat to humanity. You apply to work at one of these groups. You find similarly-minded people and work together on trying to figure things out. Maybe your boss is a sleazebag in bed with the industry, or maybe they're not. You do not care. You work on this, you publish notes and papers, you comment on LessWrong or X, and you leave the company if they don't let you do this stuff, because there are dozens of others.

Most of the public don't know these guys or this work, because they only focus on the crappy and irresponsible companies leading the AI development charge, and alignment research is so unbelievably hard and slow. I can go into the why if you're interested, but the reason is that it is extraordinarily difficult to make progress if your objective is only vaguely defined. This is the same problem with the Yang-Mills mass gap, for example, although I would personally bet that Yang-Mills has a much better chance of being solved than alignment. Just a different tier of difficulty altogether.

I also disagree that p(doom) work (I don't think this describes more than 1% of the work out there, closest I can think of is Tegmark's group, which put it pretty high from recollection) advances the pace of AI. I don't believe it is doing anything. Nobody externally believes it anyway (why would they?). These companies are run by narcissists who believe that they should be the ones to reach "superintelligence".

At the same time, we really should be encouraging it more. I have found that in newer machine learning theory papers (strictly theory, not empirical work), there is something closer to a good balance that is expected even of students.

Including a motivating example is key to this, and should be considered mandatory.

This is unbelievably ignorant speech. I have not received a dime of any of this funding, but I do know many excellent researchers that have, and they do fantastic work. There is an unbelievable gap between theory and practice regarding the capacity of deep learning, and while great strides have been made to develop the surrounding theory, there is a long way to go. Many believe that without a concrete understanding of how neural networks properly learn concepts, we have little hope of molding them to be reliably useful. It costs money to hire researchers and develop fundamental theory.

Just because you don't understand any of that work, does not mean that it is pointless. This is fundamental research that is 20 years behind schedule.

Perelman was already working on an interesting problem, and wrote in ways that were less formal, but still academically sound, so of course he did not need to publish it. What about the people working on problems that are not interesting? You don't think they encounter significant difficulties getting their work out there?

I feel like I am going crazy, since this is taught in most academic writing courses. Being replicable or persuasive or correct is not enough. It needs to be perceived as valuable, or it will not get through. I learned that the hard way, and found others that taught this message only later.