Hacker News: jballanc

Adaptive Low-Rank Transformer with Dynamic Expert Routing for Continual Learning

jballanc — Sat, 06 Jun 2026 06:57:05 +0000

Article URL: https://zenodo.org/records/20064618

Comments URL: https://news.ycombinator.com/item?id=48422164

Points: 3

# Comments: 0

New comment by jballanc in "Ask HN: What are you working on? (May 2026)"

jballanc — Sun, 10 May 2026 21:39:56 +0000

I've been working on RVW, my adaptation of the standard transformer model that is capable of online continual learning without catastrophic forgetting. I finally published the first pre-print of my early experiments: https://doi.org/10.5281/zenodo.20064617

Now I'm working on expanding the work into more parameters and improving performance. I just finished an extremely harsh test of a Nemotron-flavored RVW that consisted of stretches of a random assortment of domains interspersed with long runs of single domains. Across all of it the model didn't forget (and actually improved on some of the more challenging domains). PPL on SmolTalk is still in the ~18 range, which I'd like to get lower, but this is all with only 4B params.

Currently, I'm training a Llama 3.2-flavored RVW with only about 2B params to see how that turns out. Depending on results of that, I may take it to Gemma 4 next.

Show HN: RVW – A transformer model capable of online continual learning

jballanc — Thu, 07 May 2026 15:06:01 +0000

Article URL: https://zenodo.org/records/20064618

Comments URL: https://news.ycombinator.com/item?id=48050336

Points: 1

# Comments: 0

New comment by jballanc in "NetHack 5.0.0"

jballanc — Sun, 03 May 2026 01:24:09 +0000

IIRC, there was always a way to filter out certain messages (or that may be an alt.org customization, but it's been a part of my config file for a while now).

New comment by jballanc in "NetHack 5.0.0"

jballanc — Sun, 03 May 2026 01:22:06 +0000

For real! Valkyrie is the perfect "just bash things while only half paying attention" class. Great for when I'm playing to unwind (as opposed to playing as a challenge to myself).

At least there's still Samurai.

New comment by jballanc in "What can we gain by losing infinity?"

jballanc — Thu, 30 Apr 2026 19:23:05 +0000

I think Douglas Adams had one of the best quotes regarding observing infinity:

"Infinity itself looks flat and uninteresting. Looking up into the night sky is looking into infinity – distance is incomprehensible and therefore meaningless."

New comment by jballanc in "Networking changes coming in macOS 27"

jballanc — Tue, 28 Apr 2026 02:05:41 +0000

It's been a while since I worked at Apple, but back in the day the entire OS X Server team made extensive use of kerberized NFS shares for moving around large files...

...the last version of Server shipped in 2021 (and the last real version shipped almost a decade before that).

New comment by jballanc in "United Wizards of the Coast"

jballanc — Mon, 27 Apr 2026 19:17:17 +0000

My first job after finishing my undergrad degree was performing quality analysis on corn starch. As a condition of employment, I had to sign a paper saying anything I invented related to corn was property of my employer.

New comment by jballanc in "Meta tells staff it will cut 10% of jobs"

jballanc — Thu, 23 Apr 2026 22:31:19 +0000

It's been more than a few years since I worked at Apple, but they were always unique in the tech space in that their retail division dwarfed headcount. If I recall correctly all of OS X Lion was produced by around 3,000 engineers (and probably less, since I think that count included iLife and iWork).

New comment by jballanc in "Ask HN: What Are You Working On? (April 2026)"

jballanc — Wed, 22 Apr 2026 00:45:48 +0000

No public repo yet, but coming soon. Just filed for a patent on the technique and am preparing a paper. Posted the first figure I have for the paper here: https://dev.to/jballanc/what-would-you-do-with-an-ai-model-c...

New comment by jballanc in "Ask HN: What Are You Working On? (April 2026)"

jballanc — Mon, 13 Apr 2026 00:24:20 +0000

I've been working on an ML model capable of robust continuous learning, resistant to catastrophic forgetting without relying on replay, an external memory system, or unbounded parameter growth. Last week I confirmed the first non-toy, 580M parameter version soundly beat LoRA, EWC, and full fine tuning. This week I'm scaling up to 4.4B parameters...

New comment by jballanc in "We're running out of benchmarks to upper bound AI capabilities"

jballanc — Fri, 10 Apr 2026 22:09:17 +0000

We need benchmarks that can distinguish between continuous learning and long-context extrapolation.

Ask HN: What would you do with an AI model capable of continuous learning?

jballanc — Thu, 09 Apr 2026 22:57:15 +0000

Let's say, hypothetically, that you had a model that did not need to re-train in order to incorporate new information in its weights. What would you do with such a model?

Comments URL: https://news.ycombinator.com/item?id=47711381

Points: 4

# Comments: 6

Ask HN: Why don't frontier AI model providers continuously improve their models?

jballanc — Thu, 09 Apr 2026 01:11:26 +0000

Just what the title says: I'm wondering why we're still, years after ChatGPT, having to wait weeks or months for "the next version" of a model when so much else in the software world has moved toward continuous improvement?

Comments URL: https://news.ycombinator.com/item?id=47698187

Points: 1

# Comments: 1

New comment by jballanc in "Music for Programming"

jballanc — Mon, 06 Apr 2026 17:42:55 +0000

Based on what you've already mentioned, there's a good chance you're familiar, but on the off chance you're not: "Funkungfusion" (or, really, anything off the Ninja Tune label) might be right up your alley.

New comment by jballanc in "Arm AGI CPU"

jballanc — Tue, 24 Mar 2026 20:31:59 +0000

Eh, I'm not so sure it'll be that big a deal. The whole supply chain is so twisted and tangled all the way up and down. Shuffling out one piece doesn't seem like it will, on its own, be so major. Samsung made the chips for the iPhone, then made their own phone, then Apple designed their own chips made by TSMC, now Apple is exploring the possibility of having Samsung make those chips again.

Also, it takes a willful ignorance of history for ARM to claim this is the first time they've manufactured hardware. I mean, maaaaybe, teeeeechnically that's true, but ARM was the Acorn RISC Machine, and Acorn was in the hardware business...at least as much as Apple was for the first iPhone.

New comment by jballanc in "Theodosian Land Walls of Constantinople (2025)"

jballanc — Sun, 22 Mar 2026 23:27:34 +0000

Fun fact about that cannon: it took so long for the cannon to cool off between shots that the Byzantines were able to patch each hole it caused before the next shot.

The Singularity Is Coming

jballanc — Sat, 21 Mar 2026 14:14:06 +0000

Article URL: https://www.manhattanmetric.com/blog/2026/03/the-singularity-is-coming

Comments URL: https://news.ycombinator.com/item?id=47467223

Points: 2

# Comments: 0

New comment by jballanc in "Ask HN: Where should an independent researcher publish work on ML?"

jballanc — Fri, 20 Mar 2026 15:47:20 +0000

Thanks for the pointer! Definitely looks interesting...

Ask HN: Where should an independent researcher publish work on ML?

jballanc — Fri, 20 Mar 2026 15:29:42 +0000

A quick bit of background: I have a Ph.D. in Evolutionary Biology and published a peer-reviewed paper on my thesis topic when I was in grad school. Then, I went to work as a software engineer.

Now, since leaving academia, I have by no means lost interest in science. If anything, I've followed the world of research with as much interest and attention as ever and, because I no longer have to play the perpetual game of one-upmanship that pervades academic departments, I have been free to spread my interest around to more diverse topics. It's been rather freeing.

At the same time, my paycheck depends on delivering code, and so I've delivered code and not academic papers. Then, something interesting happened: I splurged on a Claude subscription and suddenly I have the most attentive research assistant I could have imagined all without the need for an academic department or drawn-out grant proposal process.

The only hurdle remaining is: where should I publish? Unfortunately, as I don't have an academic affiliation, I cannot get automatic access to publish to arXiv (and anyone I know who could endorse me is focused in on the biological sciences subjects from my time in grad school, not cs.LG). I've thought about simply posting to GitHub and linking from my personal site, but I worry if that's enough to establish priority and/or garner real critique and feedback.

I'm contemplating PLOS One, but I don't know the ML community well enough to know if that's an appropriate venue? Any help on how to re-enter the world of scientific publishing (in a new area) would be much appreciated!

Comments URL: https://news.ycombinator.com/item?id=47456005

Points: 2

# Comments: 2