What would it take for AI to not replicate that in biology? Either curing cancer has to be a much harder problem, which is doubtful given targeted treatments like immunotherapy humans have already stumbled across, or AI progress has to slow down massively, which is doubtful given all the trends say it is speeding up.

The Little Pamphlet of AI Foundations: An Unintuitive Philosophical Primer

Losses

Intelligence is the ability to explain phenomena. One is more intelligent where they can better explain phenomena.

A phenomenon is any correspondence between data, and an explanation is an encoding of that correspondence. There is an infinite amount of data so an intelligence must be a program that generates new explanations when it sees new data.

A program is a string of bits run by an interpreter. Since strings of bits are data, programs may be generated by other programs, and a program's interpreter may itself be another program or intelligence. This recursion eventually bottoms out in physical ware. Human and artificial intelligences differ mainly in their ware, though human intelligences also have an impredicative recursion property that artificial intelligences are only expected to gain in 2027.

At a low level of intelligence, a calculator takes in numbers and operations through electrical impulses, moves electrons through a series of transistors, which creates new outputs to be displayed on the screen.

At a higher level, a neural network training algorithm takes in examples and outputs weights, usually through preconditioned gradient descent.

At an even higher level, a language model takes in words and outputs possible continuations. To convert this to the correct continuation, an additional error correction code must be provided for every word it outputs. This unexplained gap in the language model, or 'loss', dominates in the limit of a large number of words. To consistently provide short explanations, a model must have a small loss.

At the highest level, an explanation of an environment must encode all data in that environment, including any intelligences. An intelligence encompasses a larger share of its environment if it is harder to encode. Under a fixed loss budget, the hardest intelligence to encode provides explanations weighted exponentially by their length. This is the origin of the common mantras that a short explanation is exponentially more likely to be the correct one, colloquially known as Occam's razor, Solomonoff's lightsaber, and Kolmogorov's complexity.

The important takeaway is that a loss represents the average size of an error correction code, and a smaller loss is better.

Architecture

An intelligence is a program that generates encodings of data correspondences. Choosing an architecture with better data topologies enables it to do so more efficiently.

A data topology consists of data and possible connections between the data, with computational costs determined by the architecture. As computational resources grow superexponentially, the goal of architecting is generally to lower the computational complexity of creating connections, encapsulated colloquially as 'The Bitter Lesson'. To give a few examples:

1. The limited liability corporation lowered the personal risk of starting a business from the size of the business to a fixed cost, making economic connections cheaper to form.

2. Residual networks directly form connections between neuron data and output gradients, bypassing the exponentially weakening connections that previously made deep learning impossible.

3. Flow transport models improve upon continuous normalizing flows by giving target distribution data along the entire generation process.

4. Quadratic attention connects every input datum to every input datum, instead of relying on fringe connections from the nonlinearity in activation functions. Linear attention further lowers this cost by adaptively selecting data to go into a memory bank, and only forming connections with the memory bank.

5. A gradient descent preconditioner modifies the weight flow density, so distributions of weights are squeezed closer together or spread further apart. In the extreme, this no longer looks like gradient descent and looks closer to an intelligence directly writing weight updates based on new examples.

Learning Theory

A perfect intelligence provides explanations weighted exponentially by their length. A Bayesian neural network is a network generated by a perfect intelligence as an explanation for a dataset, and its distribution of weights—its 'partition function'—will be concentrated in regions with exponentially low loss. Analogous to physics, much can be gleaned from the partition function.

In adversarial contexts such as generative adversarial networks or reinforcement learning, examples rotate through a cyclic group of six elements, being positive, neutral, or negative examples depending on what the other intelligences have learned. This creates an anyonic partition function for each learnable feature and the familiar sextessential steep upwards loss curve before a plateau. Similarly, an alternating cycle of learning and forgetting gives the fermionic partition function and a loss curve colloquially referred to as grokking. Most learning is just learning and looks bosonic in nature: losses quickly go down, then settle into an exponential curve.

In the limit of a large number of training examples, the network's loss dominates the size of the explanation, but it still takes a logarithmically increasing number of bits to describe the path the weights take along the weight flow. To be precise, the explanation length grows as

  nL(w) + λ log n − (m − 1) log log n

[1]: https://benjaminrosshoffman.com/the-debtors-revolt/

[2]: https://www.lesswrong.com/posts/tihhx7iy8C6yyHaC2/the-corner...

FTFY. Do not conflate all competition as good for the consumer.

What I learned spending a lifetime making the world a little bit worse for everyone else for fun and profit.

-----

Their entire mindset seems to be, "how can I take advantage of other people making mistakes to make a few dollars here and there? and how can I induce them to make even more mistakes in my realm of influence so it is even easier to profit from them?"

This is why he is always chasing the latest fad—"dropshipping massage guns, rug tufting, rainbow-loom bracelets, you name it"—people who follow fads are less likely to do their research and more likely to click buy on the overpriced TikTok ad. AI just empowers him to do more of this.

The only skill he seems to work on is his "taking advantage of others" skill. It's astounding that he got to this age without knowing even the basics of programming... and he only learned those basics once it became relevant to taking advantage of others. Rather than trying to create something new, or create a better version of something old, he just wants to disrupt efficient commerce so he can skim off the top.

I'm sure he makes a ton of money doing this. I wish that weren't possible.

Say there was a Chinese artist in a room...

I understand it is a little selfish to want people to maximize my comprehension (what about the thousands of other readers?). I also understand there can purposes other than communication in writing (fiction -> pleasure, poetry -> art). I agree with you that we shouldn't care if it's AI or human writing for these other purposes. But when the goal is one human transferring bits from their brain to mine, playing telephone with an AI cannot be more efficient (it necessarily decreases channel capacity).

There are two exceptions.

1. When AI is good enough to "read people's minds" it can extract extra bits than what they explicitly meant to include in their prompt... but at that point why am I even bothering to listen to them, and not have the more powerful, smarter AI tell me what they think?

2. AI can run experiments or prove things, and I shouldn't have my AI duplicate that work. In this case I would want them to explain what tasks they set the AI out to do, and then provide the results. I would not complain if the results are an AI writeup, though usually I would want to know what in the results the human author felt was important.

> There were some technical findings along the way—for example, Kavish found that DDPM, while theoretically ideal for denoising in diffusion models, diverged, and that flow matching performed much better.

When they say "theoretically ideal" they mean a specific thing: the DDPM loss is information theoretically the number of error correction bits needed to recover an image. At very low noises, this quantity goes singular. This is not actually a problem—you should bound the log-SNR of your schedule by the image quantization level (usually 9 bits) anyway, and a trained bound lands within 10% of this.

Flow matching does not diverge because it removes the log-SNR rate from the loss. Since the rate can span several orders of magnitude over your schedule, you should importance sample during training. My guess is Kavish forgot to do this, leading to divergence.

1. I recognized that understanding math had incredibly profitable applications.

2. It was a sport.

3. It was a hobby.

Do I think the world will look the same when math is "solved"? Absolutely not! And we're seeing it transform right before our eyes. I routinely ask Claude, "here's the general idea for a math/cs/ai problem I'm working on, can you help me figure out the maths? oh and then implement it too."

Could I figure this out myself? Of course. Having the idea is halfway to formalizing it in math. But I remember even last year working on GPU kernels by hand and having a bunch of little arithmetic or indexing errors burn me, because I'm just slower and less perfect at doing the math correctly. Now I basically just tell Claude, "go optimize a GPU kernel for this problem". I still have to tell it what to look out for (architectural, algorithmic, memory improvements), but I'm sure the next model won't even need that.

Mathematics was historically "just a hobby". Much theoretical work is still "just a hobby". But, for the past few hundred years, humans that better understood the theory could find very profitable applications.

That is no longer true. Computers have solved mathematics, like they did chess two decades ago. A human will never find a better application than a computer, just like they will never find a better chess move. Yet people still play chess, and people still make money tutoring chess.

That is the future of humans in mathematics. Mathematics will become mathematics competitions. It will be just another intellectual sport. Sure, a sport that teaches valuable life lessons, but still "just a game". If you are going into mathematics today, your future career is as a competition mathematics coach.

This is probably less restrictive than you are advocating for in your previous comment: as long as there is plausible deniability, the speech goes free.

https://si.inc/posts/hertz-dev/

It's only 8.5B and doesn't sound like it's quite conversational.