Hacker News: viktor_von

New comment by viktor_von in "We replaced RAG with a virtual filesystem for our AI documentation assistant"

viktor_von — Sat, 04 Apr 2026 03:12:10 +0000

> yet somehow people married the acronym to one very particular implementation of the idea.

Likely due to the rise in popularity of semantic search via LLM embeddings, which for some reason became the main selling point for RAG. Meanwhile keyword search has existed for decades.

New comment by viktor_von in "The average American spent 2.5 months on their phone in 2024"

viktor_von — Mon, 30 Dec 2024 09:57:06 +0000

Ad-free and fast, Wikipedia is imo the best learning resource on-the-go, at home, or at work.

New comment by viktor_von in "Were RNNs all we needed?"

viktor_von — Sat, 05 Oct 2024 20:08:45 +0000

> The information cost of making the RNN state way bigger is high when done naively, but maybe someone can figure out a clever way to avoid storing full hidden states in memory during training or big improvements in hardware could make memory use less of a bottleneck.

Isn't this essentially what Mamba [1] does via its 'Hardware-aware Algorithm'?

[1] https://arxiv.org/pdf/2312.00752

New comment by viktor_von in "Were RNNs all we needed?"

viktor_von — Fri, 04 Oct 2024 04:20:34 +0000

> It's astounding to me (and everyone else who's being honest) that LLMs can accomplish what they do when it's only linear "factors" (i.e. weights) that are all that's required to be adjusted during training, to achieve genuine reasoning.

When such basic perceptrons are scaled enormously, it becomes less surprising that they can achieve some level of 'genuine reasoning' (e.g., accurate next-word prediction), since the goal with such networks at the end of the day is just function approximation. What is more surprising to me is how we found ways to train such models i.e., advances in hardware accelerators, combined with massive data, which are factors just as significant in my opinion.

New comment by viktor_von in "Were RNNs all we needed?"

viktor_von — Fri, 04 Oct 2024 02:37:41 +0000

> I remember one of the initial transformer people saying in an interview that they didn't think this was the "one true architecture" but a lot of the performance came from people rallying around it and pushing in the one direction.

You may be referring to Aidan Gomez (CEO of Cohere and contributor to the transformer architecture) during his Machine Learning Street Talk podcast interview. I agree, if as much attention had been put towards the RNN during the initial transformer hype, we may have very well seen these advancements earlier.