Mmm, to my mind, isn't this mixing completely different things? I thought poisoned context is stuff in the context that makes the model behave poorly. But if it's in the weights, that's just... model issues.

I understand that sports will tend to have ads displayed in the stadium and so on, and that some show may feature characters drinking Pepsi rather than Coke because of an agreement. And I guess movies may have trailers. Is that the extent of it? If I get what feels like an obvious "ad" then we'll probably be canceling, but I guess I'm okay with those things that I mentioned.

[0] https://help.disneyplus.com/article/disneyplus-ads

What are some other fictional universes with interesting takes on AI and humans? I think the Hyperion cantos (at least books 1 and 2) are great. Stross's Accelerando also has some wild ideas but feels maybe less directly relevant.

That is, you haven't projected this forward just a bit further and are still thinking in terms of the old world where software is at all valuable and where you can have customers to sell it to.

I still feel myself falling into that trap, but haven't figured out what world replaces this.

And later

> Those PPD safeguards limited the impact of this incident.

Ha, so the PPD safeguards is a spam filter? Claude emailed a tip and it went straight to spam, and nobody noticed until Anthropic realized what they'd done and reached out, whereupon they looked in the spam folder and said, yep there it is. Super intelligence, here we come.

If this is true, then what's really the point of math? A lot of math is actually useful. A proof regarding cryptography, for example, would have practical application even if not understandable by humans.

The book is a trip and the dreamy sort of narration matched it perfectly. My wife and I would spend evenings listening to one city at a time, on a walk around our little park, one headphone in each ear. I don't know if it was just the environment or the time in our lives, but we both really look back fondly on this book.

This is more to encourage people to try out adding AI into their product, which is a totally different flow and experience from using AI to build the product.

So I was wondering if the same applied here. My initial reaction is that "light" meaning "illuminated" and "light" meaning "not heavy" is that this is just one of those coincidences, since I can't see how that one word relates the two concepts. But given that Japanese also does the same, it makes me wonder if maybe there is some sort of relationship. Maybe it was a common belief historically that the sun was not very heavy or something like that. Or maybe not and it's just a rare double coincidence!

I don't get the "but what if it's somewhat - not infinitely - bad" argument? The known good is so dramatically good that we'd need a shocking amount of unknown bad to have made it not worth it. But if I'm choosing between mosquitoes, and say... fish, frogs, and a variety of birds, yes, get rid of the mosquitoes.

> Fun fact: “Akari” means light in Japanese, both as illumination and as lightness.

Wow! I love it when two languages have matching homonyms. I always wonder "what does it mean ?". Like, in English "second" to mean the unit of time and the place after "first" feels like two distinct usages. But it's the same in Spanish ("segundo"), which points to a deeper connection (and isn't so unsurprising given the shared roots of the languages): the unit of time is called that because it's the second subdivision of an hour.

The two usages of "light" in English are so distinct I'm quite surprised it's like this in the entirely unrelated Japanese language. Is this just a big coincidence, or does anyone know any deeper reasons here?

Can someone help me understand this? I might have an out of date mental model of how these things work.

Fundamentally, LLMs output tokens 1 at a time, generating the next token from all the previous. And as the context window gets larger, this gets harder / slower / more expensive. So I get the idea of a maximum context window.

But I don't understand the point or meaning of an output token limit. I thought it was more a measure of price capping (since output tokens are more expensive) that a user could configure. I guess a model will keep generating tokens until it hits a "stop", so does this mean it's tuned to more aggressively produce output tokens? How does that fit into agentic loops. Are output token limits based on how long until it goes back to the user? Or does each "turn" of tool call, thought, tool call, thought, etc, get its own limit?