Hacker News: lucidrains

New comment by lucidrains in "The Waymo Ojai Will Soon Offer Autonomous Rides Around the U.S."

lucidrains — Thu, 08 Jan 2026 17:14:54 +0000

They need to have a service that allows for pets. It is the only reason I still use Ubers!

New comment by lucidrains in "BERT is just a single text diffusion step"

lucidrains — Mon, 20 Oct 2025 21:58:16 +0000

there has been some movement on that front, in the form of adding expand / delete tokens! https://hkunlp.github.io/blog/2025/dreamon/

New comment by lucidrains in "From multi-head to latent attention: The evolution of attention mechanisms"

lucidrains — Sat, 30 Aug 2025 13:33:15 +0000

It is a reference to the beatles song, mainly because Noam Shazeer is a music lover

New comment by lucidrains in "TREAD: Token Routing for Efficient Architecture-Agnostic Diffusion Training"

lucidrains — Mon, 18 Aug 2025 19:45:50 +0000

very nice, will have to try it out! this is the same research group from which Robin Rombach (of stable diffusion fame) originated from

New comment by lucidrains in "Hand: open-source Robot Hand"

lucidrains — Thu, 17 Jul 2025 17:20:47 +0000

https://theopenexo.nau.edu/

New comment by lucidrains in "Schizophrenia is the price we pay for minds poised near the edge of a cliff"

lucidrains — Sun, 29 Jun 2025 17:43:31 +0000

I am sorry to hear that.

New comment by lucidrains in "Schizophrenia is the price we pay for minds poised near the edge of a cliff"

lucidrains — Sun, 29 Jun 2025 17:43:13 +0000

indeed, it just becomes less likely

New comment by lucidrains in "Schizophrenia is the price we pay for minds poised near the edge of a cliff"

lucidrains — Sun, 29 Jun 2025 14:10:21 +0000

if you are a man and make it past age of 29 without starting to hear voices, you can breathe a sigh of relief (I did)

New comment by lucidrains in "Launch HN: Vassar Robotics (YC X25) – $219 robot arm that learns new skills"

lucidrains — Tue, 10 Jun 2025 19:43:14 +0000

Love to know as well!

New comment by lucidrains in "Surprisingly fast AI-generated kernels we didn't mean to publish yet"

lucidrains — Sat, 31 May 2025 00:48:43 +0000

oh wow, I was looking for use of islands or map-elites that I missed this.. thought it was the blandest mimetic evolution possible

New comment by lucidrains in "Differential Transformer"

lucidrains — Tue, 08 Oct 2024 15:22:38 +0000

does this not mean we should explore usage of talking heads (Shazeer et al) a bit more? https://arxiv.org/abs/2003.02436

New comment by lucidrains in "Psilocybin desynchronizes the human brain"

lucidrains — Wed, 17 Jul 2024 21:14:52 +0000

Indeed, the best setting is in nature on a beautiful day

New comment by lucidrains in "GPUs Go Brrr"

lucidrains — Tue, 14 May 2024 00:02:06 +0000

amazing work! thank you!

New comment by lucidrains in "GPUs Go Brrr"

lucidrains — Mon, 13 May 2024 15:10:02 +0000

would be interested to see thunderkittens (great name!) tackle the flash attention backwards pass, which is an order of magnitude harder than the forward

New comment by lucidrains in "xLSTM: Extended Long Short-Term Memory"

lucidrains — Wed, 08 May 2024 14:37:58 +0000

https://arxiv.org/abs/2404.08819

New comment by lucidrains in "Better and Faster Large Language Models via Multi-Token Prediction"

lucidrains — Wed, 01 May 2024 14:15:27 +0000

wow, so prophet net does work! i spent so much time experimenting with it back in the day, but just lacked the scale to see a positive result.

New comment by lucidrains in "Self-reasoning tokens: teaching models to think ahead"

lucidrains — Sun, 21 Apr 2024 19:57:52 +0000

indeed, simsiam is a great example of the effectiveness of using stop gradient

New comment by lucidrains in "Self-reasoning tokens: teaching models to think ahead"

lucidrains — Sun, 21 Apr 2024 00:50:51 +0000

could even try it with a fraction of the attention heads, instead of introducing new tokens

New comment by lucidrains in "Self-reasoning tokens: teaching models to think ahead"

lucidrains — Sun, 21 Apr 2024 00:18:43 +0000

yes, it is a stop gradient mask on the attention matrix, iiuc. worth trying

New comment by lucidrains in "But what is a GPT? Visual intro to Transformers [video]"

lucidrains — Mon, 01 Apr 2024 19:47:34 +0000

I can't think of anyone better to teach attention mechanism to the masses. This is a dream come true