Further, just because governments use physical force to protect a thing does not make that thing violent. The federal government sent in the army to protect the Little Rock Nine in Arkansas. Does this mean racial integration is “violent”? Or is it only “violent” when the government tolerates inequality?

Academia doesn’t get to just assert that their broader definition is the real one.

Mandating seatbelts on new cars also did not immediately stop people from flying through their windshields.

What ideological concerns are they focused on? Imo wanting digital privacy has always been ideological, and to the extent it has ever been part of a culture war they seem to have lost that war.

Transformers are not magical. They are just a huge improvement over other architectures at the time such as LSTMs and RNNs and even CNNs. They allowed us to throw more and more compute at the problem of next token prediction. And we’ve been riding that horse ever since.

Another big advancement that deserves mentioning is “reasoning” models that have the opportunity to spit out thinking tokens before giving a final answer.

None of this is to say transformers are the most principled approach. But they work.

The LLM is desperately trying to keep your attention. It has been tuned with millions of examples graded by contractors. How do you spice up a fairly bland topic? Start by telling people that what follows will be interesting. Then, bloat a fairly obvious point into several sentences so that it is paragraph-shaped.

When I say they’re not hiding “exotic technologies” I’m referring to things that would at a minimum win a Nobel prize. Alien technologies like antigravity or faster than light travel that people sometimes talk about. I am not even talking about things like Stuxnet which was impressive but not revolutionary.

https://wisprflow.ai/new-funding

You should assume that people don’t know you, and they will not give you the benefit of the doubt. So it is in your best interest to make your public image as unimpeachable as possible.

If the person looking at your blog post is a potential employer, they will not spend a long time weighing pros and cons of your employment. They will pass you over and tell you nothing.

And the Manhattan Project cost $30B in today’s money. Compared with some of the numbers Congress has allocated recently, I’d call that a bargain.

Prior to 1940 it was known that clumping enough fissile material together could produce an explosion. There were engineering questions around how to purify uranium and how to actually construct the weapon etc. But the phenomenon was known.

I say this because there’s a meme that governments are cooking up exotic technologies behind closed doors which I personally tend to doubt.

This is almost perfect analogy to the MP though. We know exactly what could happen if we clumped enough qubits together. There are hard engineering challenges of actually doing so, and governments are pretty good at clumping dollars together when they want to.

To some extent, the reason models will get better is because companies will hire PhDs to train them on increasingly complex problems.

The problem is that more complex problems take longer to train, more time to test, require more compute, and are harder to verify. This is why “just make it bigger” is a losing proposition imo.

A lot of what I just said is also true for RLVR.

If you add in the cost of training, it’s not profitable.

Not including the cost of training is a bit like saying the only cost of a cup of coffee is the paper cup it’s in. The only way OpenAI gets to charge for inference is by selling a product people can’t get elsewhere for much cheaper, which means billions in R&D costs. But because of competition, each model effectively has a “shelf life”.