Hacker News: vatsachak

New comment by vatsachak in "A global workspace in language models"

vatsachak — Mon, 06 Jul 2026 21:23:55 +0000

Yeah, the end paragraph about recurrent neurons in humans being replaced with layers in an LLM is a good one.

The mammalian brain uses recurrence extensively, which backpropagation isn't good at. Recurrence is essential because it lets us have a "dynamic architecture", swapping layers for "clock cycles".

We currently do recurrence extremely inefficiently through "thinking" whereby the model feeds it's end output into it's beginning input. But recurrence is abound in the brain.

My guess is that in 10 years we will have the inklings of an analog computer which can perform Neural Predictive Coding.

New comment by vatsachak in "Jim Keller's startup is building a factory to mass-produce small chip fabs"

vatsachak — Sun, 05 Jul 2026 20:33:23 +0000

Great! Hopefully we can get 10 year behind technology from small fabs. There's so much you can do with a laptop from 2016

New comment by vatsachak in "AI has torched the market for junior programmers"

vatsachak — Sat, 04 Jul 2026 20:14:50 +0000

I've just learned to impress VCs what can I say

New comment by vatsachak in "AI has torched the market for junior programmers"

vatsachak — Sat, 04 Jul 2026 20:14:01 +0000

Nah I've actually gotten really good at sounding like an AI to bait lol

New comment by vatsachak in "Hunting a 16-year-old SQLite WAL bug with TLA+"

vatsachak — Fri, 03 Jul 2026 19:07:29 +0000

Gotta love TLA+

I wonder if anyone has worked on porting it to Lean and making tactics for it

New comment by vatsachak in "Local Reasoning for Global Properties"

vatsachak — Fri, 03 Jul 2026 14:39:17 +0000

Type checking is the same as proof verification. It is called the Curry-Howard correspondence

New comment by vatsachak in "The fall of the theorem economy"

vatsachak — Fri, 03 Jul 2026 03:54:17 +0000

That's fair. Two things that are heavily underrated are architecture and encoders

New comment by vatsachak in "The fall of the theorem economy"

vatsachak — Thu, 02 Jul 2026 23:01:25 +0000

You get a fields medalist to email the reviewer to say "wait that paper is really good actually"

New comment by vatsachak in "Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train"

vatsachak — Thu, 02 Jul 2026 18:48:10 +0000

Yes, my comment was kind of a non sequitur

New comment by vatsachak in "The fall of the theorem economy"

vatsachak — Thu, 02 Jul 2026 16:31:47 +0000

Everyone, including blind people use their visual cortex to play chess

New comment by vatsachak in "Is One Layer Enough? A Single Transformer Layer Matches Full-Parameter RL Train"

vatsachak — Thu, 02 Jul 2026 15:05:49 +0000

I still can't believe that LLM encoders aren't unsupervised learned.

So much left on the table

New comment by vatsachak in "The fall of the theorem economy"

vatsachak — Thu, 02 Jul 2026 14:09:15 +0000

LLMs sure, but AlphaZero had no visual cortex yet can smash Magnus Carlsen easily.

I think that we're not that far away from AI that can be superhuman at all facets of theorem proving.

I think that we're far away from an AI that can create good abstractions and construct a theory to prove theorems.

New comment by vatsachak in "The fall of the theorem economy"

vatsachak — Thu, 02 Jul 2026 14:05:39 +0000

I've had a paper unrejected from Duke. The publication process sucks

New comment by vatsachak in "Opening up 'Zero-Knowledge Proof' technology to promote privacy in age assurance"

vatsachak — Thu, 02 Jul 2026 05:32:09 +0000

Even if you had to submit a picture of your driver's license, you can send someone else's

New comment by vatsachak in "Professor denounces mass AI fraud on an exam at Brown"

vatsachak — Sun, 28 Jun 2026 22:04:19 +0000

The other day some sucker told me, "don't throw that trash over there it's littering" and I told that sucker "there's no way anyone could enforce it" lol.

Some people get up in their jimmies about this but if they don't want me doing something they should make it impossible to do it

New comment by vatsachak in "AI learns the “dark art” of RFIC design"

vatsachak — Sat, 27 Jun 2026 20:16:58 +0000

Yeah it's a hype slop piece

New comment by vatsachak in "Previewing GPT‑5.6 Sol: a next-generation model"

vatsachak — Fri, 26 Jun 2026 23:00:29 +0000

JEPA in chess leads to interpretable chess boards:

https://arxiv.org/abs/2606.11860

JEPA in image classification leads to interpretable image latents

https://arxiv.org/abs/2508.10104

Easy intro to JEPA, demonstrating that interpretability is as easy as running a PCA on latents

https://youtu.be/kYkIdXwW2AE?is=CfCBcy1jLt-FfI2E

New comment by vatsachak in "Previewing GPT‑5.6 Sol: a next-generation model"

vatsachak — Fri, 26 Jun 2026 19:57:43 +0000

Look at VLM mechanistic interpretability papers vs just pca on JEPA trained weights.

JEPA gives you interpretability for free.

I have not personally inspected them and my view is maybe a more exaggerated/dramatic claim of those working in the JEPA sphere

New comment by vatsachak in "Previewing GPT‑5.6 Sol: a next-generation model"

vatsachak — Fri, 26 Jun 2026 17:32:26 +0000

All of these LLMs are getting better at being at an LLM

But GPT-5.5 is as useful an LLM can be; it has solved lemmas I've thought about for a year, it can implement typed STLCs in Rust when I give it a formal grammar, it can help me analyze Postgres planner dumps.

It's great at tasks that have short solutions but

- they cannot learn based on a project

- their long term planning capabilities are worse than worms

- they are unconfident in decision making

- their internal representations are disgusting compared to JEPA

- they don't have any "system clock" like humans and computers do

- LLM architecture is not modular like computer architecture or human brain architecture

There's so many issues with LLMs. I wish that companies can start working on the next generation of architectures before the bubble pops

New comment by vatsachak in "Record type inference for dummies"

vatsachak — Fri, 26 Jun 2026 05:34:05 +0000

An anonymous record type in a language whose type system is an enum LangType in Rust is just HashMap>

My pet project requires an STLC with anonymous record types and type inference for anonymous records wasn't too bad