Because it’s being pushed by the c-suite and board, so most people aren’t in a position to push back. And it’s a quasi-religion now, so even pushing back at all can result in being branded as a “skeptic”.

People do raise concerns, though. And the responses aren’t very satisfactory. For example, the biggest concern that has been raised thus far is open PRs, and the answer has always been that all we need is AI code review.

The sad part is that this technology is incredible. It’s us choosing to turn it into a slop cannon (and the labs sure seem to encourage this).

I want to leave the industry as soon as I can.

Nobody wants to think anymore. Coworkers are now just intermediaries for their LLMs. Talking to them is just talking to the LLM - sometimes directly copied and pasted, sometimes minimal effort to conceal what they’re doing. It is so disheartening.

And the sad part is, LLMs are incredible and can enable you to do much better work if you can stay in the loop, and stop focusing only on shipping speed. But from what I have observed, very few people care to do this. Who cares about substance when middle management thinks your productivity is 10x?

This is what I am still grappling with. Agents make more productive, but also probably worse at my job.

I think there are three broad scenarios to consider:

- Super-intelligence is achieved. In this scenario the economics totally break down, but even ignoring that, it’s hard to imagine that there are any winners except for the the singular lab that gets here first.

- Scaling laws hold up and models continue to get better, but we never see any sort of “takeoff”. In this scenario, models continue to become stale after mere months and labs have to spend enormous amounts of money to stay competitive.

- Model raw capabilities plateau. In this scenario open source will catch up, but labs will have the opportunity to invest in specific verticals.

I believe that we’re already seeing the third scenario play out, but time will tell.

It also goes beyond just the total number of fatalities. Just like we don’t accept DUIs, we shouldn’t accept negligence or laziness from autonomous vehicle developers even if their product is safer than human drivers.

A group of ex frontier lab employees? You could declare AGI today. A more diverse group across academia and industry might actually have some backbone and be able to stand up to OpenAI.

The key difference is that they claim to have not used any verifiers.

I think that the number of students who are even aware of the competition is way lower than the total number of students.

I mean, I don’t think I’d have been a great competitor even if I tried. But I’m pretty sure there are a lot of students that could do well if given the opportunity.

My proving skills are extremely rusty so I can’t look at these and validate them. They certainly are not traditional proofs though.