Thunderbird is no longer owned by the Mozilla corporation, so now you can donate directly to them.

If you think that writing down everything you want to teach sounds like a lot of work, well, that's how you benefit from relying on a textbook to supply the content for you instead.

EDIT: Perhaps I should've read TFA first, considering that it describes a textbook grown out of the author's lecture notes for a course taught without textbook.

It's in Dortmund: https://www.dasa-dortmund.de/en/

The author also has a list of his favorite tech museums: https://aresluna.org/fav-tech-museums/

Comments URL: https://news.ycombinator.com/item?id=48474234

Points: 2

# Comments: 0

In this case, the claim is that both are happening: companies aren't making decisions independently and they're doing so in a way that discriminates against certain demographics. But the evidence needed for each half of the claim is different.

Though there are things like looped transformers that reuse the same parameters multiple times even for a single token, so maybe those will finally give us AGI if scaled up to a trillion parameters and looped hundreds of times. (Sounds expensive!)

Comments URL: https://news.ycombinator.com/item?id=48433305

Points: 52

# Comments: 0

- Human brains are estimated to have a few hundred trillion synapses. If you tried to replicate this in a neural network model with one parameter per synapse, it would be much larger than the largest models in use today.

- Conventional wisdom in form of the Chinchilla scaling law suggests that to train such a gargantuan model, you would need an even more gargantuan training corpus.

- But no human has read anywhere near as much as even relatively small Chinchilla-optimal models. In fact, rather than acquiring as much data as possible as efficiently as possible, children might rather rewatch the exact same video for the umpteenth time. When they learn arithmetic, it's from just a paltry few examples provided by the teacher in school.

- Large neural networks trained on such little training data would quickly memorize it perfectly and overfit horribly.

- Individuals with photographic memory demonstrate that human brains indeed have the memorization capacity you would expect based on synapse count, and appear to show difficulties with generalization as a side-effect.

- Speculatively, typical humans forget and generalize instead of memorizing because synaptic strengths are reduced during sleep in an analogue to regularization by weight decay.

- Therefore, maybe we should train extremely large models on little data with extremely strong weight decay to counteract memorization, and hope a large learning rate will quickly "catapult" it to a generalizing solution.

What I'm missing is a discussion of how much this would cost, even if you handle deployment by distillation into smaller, faster, less data-efficient models.

It's a bit weird though that they have a graph of tons of uranium hexafluoride shipped that shows the last shipment in 2018 and nothing since then.

i.e. the blowup is only exponential in one direction.