Hacker News: linearlayerhttps://news.ycombinator.com/user?id=linearlayerHacker News RSShttps://hnrss.org/hnrss v2.1.1Sat, 13 Jun 2026 10:10:24 +0000<![CDATA[New comment by linearlayer in "The unbearable slowness of being: Why do we live at 10 bits/s?"]]>You seem quite upset. Can you explain exactly which quantities don't make sense?

]]>Wed, 18 Dec 2024 16:44:21 +0000https://news.ycombinator.com/item?id=42452142linearlayerhttps://news.ycombinator.com/item?id=42452142https://news.ycombinator.com/item?id=42452142<![CDATA[New comment by linearlayer in "The unbearable slowness of being: Why do we live at 10 bits/s?"]]>No it's not

]]>Wed, 18 Dec 2024 16:36:28 +0000https://news.ycombinator.com/item?id=42452019linearlayerhttps://news.ycombinator.com/item?id=42452019https://news.ycombinator.com/item?id=42452019<![CDATA[New comment by linearlayer in ""Attention", "Transformers", in Neural Network "Large Language Models""]]>There is a mistake in the article, right? Multi-headed attention doesn't average together multiple attention heads. Rather it concatenates them and then right-multiplies by a matrix such that the output dimension matches the input dimension. That matrix might learn averaging, but it's not built in a-priori.

]]>Tue, 26 Dec 2023 16:30:42 +0000https://news.ycombinator.com/item?id=38773185linearlayerhttps://news.ycombinator.com/item?id=38773185https://news.ycombinator.com/item?id=38773185