Note: I work for sentry.

Note: I haven’t had a chance to test it out in anger personally yet.

#!/usr/bin/env -S uv run --script # /// script # requires-python = ">=3.12" # dependencies = [ # "ffmpeg-normalize", # ] # ///

> Interestingly, Llama 1 65B, a similar-sized model released in February 2023, had memorized only 4.4 percent of Harry Potter and the Sorcerer's Stone. This suggests that despite the potential legal liability, Meta did not do much to prevent memorization as it trained Llama 3. At least for this book, the problem got much worse between Llama 1 and Llama 3.

> Harry Potter and the Sorcerer's Stone was one of dozens of books tested by the researchers. They found that Llama 3.1 70B was far more likely to reproduce popular books—such as The Hobbit and George Orwell’s 1984—than obscure ones. And for most books, Llama 3.1 70B memorized more than any of the other models.

- https://gregoryszorc.com/blog/2024/12/03/transferring-python...

- https://astral.sh/blog/python-build-standalone

Comments URL: https://news.ycombinator.com/item?id=39363303

Points: 1

# Comments: 0

Although it’s also hollowed out the group of people using StackOverflow (and perhaps stack overflow has restricted open access to its data for further scraping), so future iterations of LLMs will have less up to date training data to use.