https://en.wikipedia.org/wiki/IKEA_effect

Readme doesn't mention that LLM providers do context caching — your costs aren't quadratic unless you frequently invalidate the cache.

Edit: here's a paper on it from 2006, https://link.springer.com/chapter/10.1007/978-3-540-76928-6_...

[1] https://www.bbc.co.uk/news/live/cly147rky81t

Actually yes there is; I have come across many people who believe this, specifically saying that IQ is fixed.

> that intelligence is an innate fixed quality

I would also disagree with this — intelligence can be increased, (e.g. through education, training, and practice), and also decreased, (e.g. by lifestyle / environment).

I.e. the idea that IQ is some innate fixed quality has evidence against it. It seems obvious that this is the case, given that people get their children tutors so they can do better at IQ tests to get into schools...

[1] https://www.psypost.org/major-iq-differences-in-identical-tw...

[2] https://www.sciencedirect.com/science/article/pii/S000169182...

> As a developer, you just pick the best one and find good hparams for it

It would be more correct to say: "As a developer, (not researcher), whose main goal is to get a good model working — just pick a proven architecture, hyperparameters, and training loop for it."

Because just picking the best optimizer isn't enough. Some of the issues in the article come from the model design, e.g. sigmoids, relu, RNNs. And some of the issues need to be addressed in the training loop, e.g. gradient clipping isn't enabled by default in most DL frameworks.

And it should be noted that the article is addressing people on the academic / research side, who would benefit from a deeper understanding.

While there is some quality content on there, the amount of terrible content getting vast amount of views is pretty high.

I guess one question is whether TV is much better.. I would say on average it probably is less bad, although there have also been / are questionable unethical tv shows. But at least with TV shows there's more likely to be a few more layers of questioning / analysing / looking at the ethics, with responsible people involved.

We've got a model out there now that we've just seen has put someone's life at risk... Does anyone apart from that company actually know how accurate it is? What it's been trained on? Its false positive rate? If we are going to start rolling out stuff like this, should it not be mandatory for stats / figures to be published? For us to know more about the model, and what it was trained on?

[1] https://arxiv.org/abs/1506.02640

It's not a uniform distribution after the first measurement, t_obs. That enables us to update the distribution, and it becomes a decaying one.

I think you mistakenly believe the distribution is still uniform after that measurement.

The best guess, that it will last for as long as it already survived for, is actually the "median" of that distribution. The median isn't the highest point on the probability curve, but the point where half the area under the curve is before it, and half the area under the curve is after it.

And the above equation is consistent with that.

Wrong. There's "model state", (I assume you mean hidden layers), not just in the generated text, but also in the initial prompt given to the model. I.e. the model can start its planning from the moment it's given the instruction, without even having predicted a token yet. That's actually what they show in the paper above...

> It is only after the model has found itself in a poetry generating context and has also selected the first line-ending word, that a rhyme scheme "emerges" as a variable

This is an assertion based on flawed reasoning.

(Also, these ideas should really be backed up by evidence and experimentation before asserting them so definitively.)