The real magic in generative modeling comes from the post training process that comes after, which usually (e.g., RLHF) approximates Reverse KL (given limited capacity, try to perfectly cover what you can, but it's fine to drop the rest entirely). This gives amazing results, but is also the cause of AI oddities like the "AI Image Pixar Look", many of the verbal tics of LLMs, and all AI music using the same small set of voices. Jensen-Shannon Divergence sits right in the middle of Forward and Reverse KL and is what many GANs are claimed to approximate. Ideally, it is a better trade-off between diversity and fidelity.

This isn't like text classification, the signal many orders of magnitude higher bitrate and so many more corners need to be cut. It's likely going to be nearly impossible or at least not remotely worth it to generate an audio signal that is truly undetectable in the foreseeable future.

Comments URL: https://news.ycombinator.com/item?id=46829965

Points: 93

# Comments: 37

Statistically, if those in the control group had gotten the treatment, then in expectation 9 of those people wouldn't have had their cancer return or died. It must be exciting to run these sorts of trials with super promising drugs, but also a little bittersweet/dark.

Thankfully, most of it doesn't reach your Spotify feed. I think most of it is garbage, but I'd fight for the right of people to continue posting it. All things algorithmic have this exploration/exploitation, diversity/fidelity tradeoff and Spotify has theirs tuned very heavily toward exploitation/fidelity. I think there is a cool opportunity for someone to put the tradeoff dial into users hands.