Hacker News: marmaduke

New comment by marmaduke in "Titans: Learning to Memorize at Test Time"

marmaduke — Thu, 16 Jan 2025 10:35:21 +0000

similar to RWKV7’s new (sub quadratic) attention mechanism which models key values as v≈kS’ and does an in-context descent on ||v - kS’||^2/2 (where the state matrix S is one attentional head) , explained more by the author here https://raw.githubusercontent.com/BlinkDL/RWKV-LM/main/RWKV-...

and i tried to unpack it a bit here https://wdmn.fr/rank-1-take-on-rwkv7s-in-context-learning/

New comment by marmaduke in "An Introduction to Neural Ordinary Differential Equations [pdf]"

marmaduke — Sun, 12 Jan 2025 08:11:46 +0000

ha, yeah, thanks.

New comment by marmaduke in "An Introduction to Neural Ordinary Differential Equations [pdf]"

marmaduke — Sat, 11 Jan 2025 21:41:48 +0000

no, wrote it by hand for use with my own Heun implementation, since it’s for use within stochastic delayed systems.

jax is fun but as effective as i’d like for CPU

New comment by marmaduke in "An Introduction to Neural Ordinary Differential Equations [pdf]"

marmaduke — Sat, 11 Jan 2025 18:33:30 +0000

looks like a nice overview. i’ve implemented neural ODEs in Jax for low dimensional problems and it works well, but I keep looking for a good, fast, CPU-first implementation that is good for models that fit in cache and don’t require a GPU or big Torch/TF machinery.

New comment by marmaduke in "C: Simple Defer, Ready to Use"

marmaduke — Mon, 06 Jan 2025 19:25:22 +0000

i like how the contraction it’s and abbreviation T’is are anagrams

New comment by marmaduke in "Zildjian, a 400-year-old cymbal-making company in Massachusetts"

marmaduke — Thu, 02 Jan 2025 23:43:00 +0000

I didn’t look it up, but at first glance, it reminded me of discussions like this one

https://youtu.be/qnT48wO0UL0

New comment by marmaduke in "An Unreasonable Amount of Time"

marmaduke — Wed, 01 Jan 2025 13:11:18 +0000

that’s not what the story says. in any case, the point is to explain, in terms of dualistic if-then logic, that the if (you practice now) and then (you will wake up) are a single non-dual thing. but to communicate in in terms which make sense to the dual, if-then mind, one needs to use dualistic language.

New comment by marmaduke in "Ask HN: Are you unable to find employment?"

marmaduke — Tue, 31 Dec 2024 00:37:54 +0000

do you have a link for your game? how was the release process?

New comment by marmaduke in "Ask HN: Are you unable to find employment?"

marmaduke — Sat, 28 Dec 2024 19:55:16 +0000

how far from your previous experience is the game work?

New comment by marmaduke in "A Practitioner's Guide to Wide Events"

marmaduke — Wed, 25 Dec 2024 09:24:17 +0000

i wonder if there are any semi automated approaches to finding outliers or “things worth investigating” in these traces, or is it just eyeballs all the way down?

New comment by marmaduke in "Cerebrum: Simulate and infer synaptic connectivity in large-scale brain networks"

marmaduke — Wed, 25 Dec 2024 09:14:35 +0000

https://www.biorxiv.org/content/10.1101/2024.10.25.620245v1....

covers some of the recent perspectives on this modeling approach if you’re interested.

New comment by marmaduke in "Cerebrum: Simulate and infer synaptic connectivity in large-scale brain networks"

marmaduke — Tue, 24 Dec 2024 20:15:59 +0000

having worked on whole brain modeling the last 15 years and european infra for supporting this kinda research, this is a terrible buzzword salad. the pdf is on par with a typical masters project.

New comment by marmaduke in "Cerebrum: Simulate and infer synaptic connectivity in large-scale brain networks"

marmaduke — Tue, 24 Dec 2024 20:13:32 +0000

HH is kinda the opposite of LIF on the abstraction spectrum.

New comment by marmaduke in "The Opposite of Documentation is Superstition (2020)"

marmaduke — Sun, 01 Dec 2024 09:43:42 +0000

The pigeon experiment is a great one to learn from not just about programming or software, but about life in general. Where are you getting your next dopamine hit? Is it random? Maybe that’s where our idiosyncrasies come from.

New comment by marmaduke in "Understanding SIMD: Infinite complexity of trivial problems"

marmaduke — Sat, 30 Nov 2024 15:08:10 +0000

> CPUs have the memory capacity advantage

perhaps also more precisely they also have quite an advantage on anything that needs and plays nicely with caches? when I sliced my problem to maximize cache usage, I saw pretty clear scalability with cores: L1/L2 cache bandwidth is ~30GB/s, so e.g. a 32 core system starts to compete with the big consumer GPUs.

New comment by marmaduke in "Understanding SIMD: Infinite complexity of trivial problems"

marmaduke — Sat, 30 Nov 2024 13:56:03 +0000

My approach to this is to write a bunch of tiny “kernels” which are obvious to SIMD and then inline them all, and it does a pretty good job on x86 and arm

https://github.com/maedoc/tvbk/blob/nb-again/src/util.h

New comment by marmaduke in "In Praise of Print: Reading Is Essential in an Era of Epistemological Collapse"

marmaduke — Thu, 28 Nov 2024 20:16:27 +0000

> People won't even pay for most of these pocket distractions

If someone is paying, the transaction, by construction reinforces the psychological boundaries that obliteration eliminates. So I think not paying is part of it, just like addicts ignore the (perhaps partially non monetary) price of their behavior.

New comment by marmaduke in "A career-ending mistake"

marmaduke — Sun, 24 Nov 2024 19:26:18 +0000

I think “a cloud never dies” is more apt for this sentiment

New comment by marmaduke in "MomBoard: E-ink display for a parent with amnesia"

marmaduke — Fri, 15 Nov 2024 08:08:24 +0000

How would you want to host or update the markdown file? Sending an email for instance? Or run your own host?

New comment by marmaduke in "OpenCoder: Open Cookbook for Top-Tier Code Large Language Models"

marmaduke — Sat, 09 Nov 2024 19:02:13 +0000

> Unlike most prior efforts, we release not only model weights and inference code, but also the reproducible training data, complete data processing pipeline, rigorous experimental ablation results, and detailed training protocols for open scientific research.

Regardless of the specific performance of this model versus another model, I think it’s good to keep in mind that everyone benefits from this kind of work