Hacker News: jdeaton

New comment by jdeaton in "Attention Wasn't All We Needed"

jdeaton — Fri, 23 May 2025 20:24:43 +0000

First four things on the list are attention

New comment by jdeaton in "Programming languages should have a tree traversal primitive"

jdeaton — Tue, 29 Apr 2025 21:28:45 +0000

Its called recursion

> Doesn't for_tree(...) look a lot nicer and simpler and less error prone than needing to implement a recursive function for each operation you would want to do on a tree?

No it does not

New comment by jdeaton in "Sapphire: Rust based package manager for macOS"

jdeaton — Tue, 22 Apr 2025 20:51:28 +0000

Yeah i vote it should be rebranded “why”

New comment by jdeaton in "The effect of deactivating Facebook and Instagram on users' emotional state"

jdeaton — Mon, 21 Apr 2025 15:38:13 +0000

0.061 standard deviations? Thats like almost nothing?

New comment by jdeaton in "Nvidia adds native Python support to CUDA"

jdeaton — Sat, 05 Apr 2025 00:54:29 +0000

Pytorch???

New comment by jdeaton in "Are noise-cancelling headphones to blame for young people's hearing problems?"

jdeaton — Mon, 17 Feb 2025 21:49:04 +0000

Maybe shes wearing those noise canceling headphones because of her auditory processing condition and not the other way around??

This seems like basic speculative attribution error- no research here.

New comment by jdeaton in "How to scale your model: A systems view of LLMs on TPUs"

jdeaton — Thu, 06 Feb 2025 17:28:08 +0000

Sounds limited

New comment by jdeaton in "How to scale your model: A systems view of LLMs on TPUs"

jdeaton — Wed, 05 Feb 2025 01:52:44 +0000

yeah it looks exactly like that file but replace "import numpy as np" with "import jax.numpy as np" :)

New comment by jdeaton in "How to scale your model: A systems view of LLMs on TPUs"

jdeaton — Wed, 05 Feb 2025 01:46:13 +0000

Something nice about this guide is that it generally transfers to GPU directly thanks to JAX/XLA.

New comment by jdeaton in "How to scale your model: A systems view of LLMs on TPUs"

jdeaton — Wed, 05 Feb 2025 01:32:57 +0000

if you're using tpu why are you using pytorch

New comment by jdeaton in "How to scale your model: A systems view of LLMs on TPUs"

jdeaton — Wed, 05 Feb 2025 01:23:40 +0000

The interesting thing about this comment is that JAX is actually higher-level even than pytorch generally. Since everything is compiled you just express a logcial program and let the compiler (XLA) worry about the rest.

Are you suggesting that XLA would be where this "lower level" approach would reside since it can do more automatic optimization?

New comment by jdeaton in "We fine-tuned Llama 405B on AMD GPUs"

jdeaton — Tue, 24 Sep 2024 16:47:32 +0000

JAX has a sub-system called Pallas[1] with a Triton-like programming model and an example implementation of Flash Attention [2]. It is quite fast. On TPUs I've heard that the XLA compiler already emits a flash-attention-like computation graph for a regular JAX implementation of attention so there's no need to have some specialized kernel in that case.

1. https://jax.readthedocs.io/en/latest/pallas/index.html

2. https://github.com/jax-ml/jax/blob/main/jax/experimental/pal...

New comment by jdeaton in "Kids who use ChatGPT as a study assistant do worse on tests"

jdeaton — Thu, 05 Sep 2024 04:22:33 +0000

Why is the study specifically of Turkish students

New comment by jdeaton in "The Art of Finishing"

jdeaton — Mon, 02 Sep 2024 21:12:31 +0000

Its not overwhelming clear that finishing is an important piece of a side project. Real work at your job is the place where you can prioritize finishing over other outcomes. By not forcing yourself to finish everything in your side projects you open yourself up to greater levels of exploration and learning which is, after all, the point of side projects to begin with.

New comment by jdeaton in "Pie doesn't need to be original unless you claim it so"

jdeaton — Fri, 30 Aug 2024 16:34:39 +0000

Software isnt pie. It isnt even food

New comment by jdeaton in "Poor Foundations in Geometric Algebra"

jdeaton — Mon, 26 Aug 2024 17:19:10 +0000

> very real toxicity within the geometric algebra community. I can’t do much about

I was hoping the article would be about this instead. OP wondering if you have any elaborations for us to hear.

New comment by jdeaton in "Zen, a Arc-like open-source browser based on the Firefox engine"

jdeaton — Wed, 21 Aug 2024 15:45:45 +0000

The macos disk images are "broken" according to my os.

New comment by jdeaton in "Leaving Neovim for Zed"

jdeaton — Mon, 19 Aug 2024 00:20:10 +0000

Helix has been trying to implement a plugins system for like 2 years that should tell you everything you need to know about its future prospects

New comment by jdeaton in "Good programmers worry about data structures and their relationships"

jdeaton — Fri, 16 Aug 2024 19:48:36 +0000

I dont understand what it means to “move complexity into data”

New comment by jdeaton in "Wet-lab innovations will lead the AI revolution in biology"

jdeaton — Tue, 13 Aug 2024 22:02:01 +0000

> people unacquainted with biology have a false perception of how low-throughput biology experimentation is. In many ways, it can be. But the underlying physics of microbiology lends itself very well to experiments that could allow one to collect tens-of-thousands, if not millions, of measurements in a singular experiment. It just needs to be cleverly set up.

I think this passage gets to the fundamental rift of disagreement in perspective between those focused purely on computational advances versus innovating in wet lab techniques.

Why? Because years of peoples' careers have been wasted waiting on promises from molecular biologists claiming they will make these "clever" high-throughput experiments work. In my experience, they'll spend months to years concocting a Rube Goldberg machine of chained molecular biology steps, each of which has (at best) a 90% success rate. You don't have to chain many of these together before your "clever" setup has a ~0% probability of successfully gathering data.