Hacker News: riemannzeta

Transformers and Evolution

riemannzeta — Tue, 03 Mar 2026 13:12:48 +0000

Article URL: https://www.symmetrybroken.com/asymmetric-evolution/

Comments URL: https://news.ycombinator.com/item?id=47231804

Points: 1

# Comments: 0

What Art Is Doing

riemannzeta — Sat, 28 Feb 2026 20:24:10 +0000

Article URL: https://www.symmetrybroken.com/what-art-is-doing/

Comments URL: https://news.ycombinator.com/item?id=47199835

Points: 1

# Comments: 0

Coherence at 300K

riemannzeta — Fri, 27 Feb 2026 19:54:24 +0000

Article URL: https://www.symmetrybroken.com/coherence-at-300-kelvin/

Comments URL: https://news.ycombinator.com/item?id=47184766

Points: 1

# Comments: 0

New comment by riemannzeta in "Maintaining Divergence"

riemannzeta — Sun, 15 Feb 2026 22:40:57 +0000

The synchronization tax is the price of building the bridge. Maintenance is the price of keeping the bridge standing. Transmission is the price of crossing it.

Maintaining Divergence

riemannzeta — Sun, 15 Feb 2026 22:40:57 +0000

Article URL: https://www.symmetrybroken.com/maintaining-divergence/

Comments URL: https://news.ycombinator.com/item?id=47028474

Points: 1

# Comments: 1

New comment by riemannzeta in "Mathematicians disagree on the essential structure of the complex numbers (2024)"

riemannzeta — Tue, 10 Feb 2026 22:29:47 +0000

I really know almost nothing about complex analysis, but this sure feels like what physicists call observational entropy applied to mathematics: what counts as "order" in ℂ depends on the resolution of your observational apparatus.

The algebraic conception, with its wild automorphisms, exhibits a kind of multiplicative chaos — small changes in perspective (which automorphism you apply) cascade into radically different views of the structure. Transcendental numbers are all automorphic with each other; the structure cannot distinguish e from π. Meanwhile, the analytic/smooth conception, by fixing the topology, tames this chaos into something with only two symmetries. The topology acts as a damping mechanism, converting multiplicative sensitivity into additive stability.

I'll just add to that that if transformers are implementing a renormalization group flow, than the models' failure on the automorphism question is predictable: systems trained on compressed representations of mathematical knowledge will default to the conception with the lowest "synchronization" cost — the one most commonly used in practice.

https://www.symmetrybroken.com/transformer-as-renormalizatio...

Your Agent's Memory Is Broken. Here's Why

riemannzeta — Fri, 06 Feb 2026 22:40:18 +0000

Article URL: https://ramsriharsha.substack.com/p/your-agents-memory-is-broken-heres

Comments URL: https://news.ycombinator.com/item?id=46919134

Points: 4

# Comments: 1

New comment by riemannzeta in "Attention at Constant Cost per Token via Symmetry-Aware Taylor Approximation"

riemannzeta — Wed, 04 Feb 2026 17:15:35 +0000

Neat result. The symmetry exploitation here reminds me of recent work connecting neural network training dynamics to renormalization group theory. Charles Martin's SETOL paper https://arxiv.org/abs/2507.17912 shows that well-trained layers converge to something like an RG fixed point—the eigenvalue spectrum of the weight matrix develops power-law tails with exponent α ≈ 2, which is the signature of scale invariance. At this fixed point, the "effective correlation space" is low-dimensional: you can truncate the SVD aggressively and recover nearly identical test accuracy.

I wonder if there's a connection to your Taylor truncation order. In RG terms, higher-order polynomial interactions are "irrelevant operators"—they get suppressed as you flow toward the fixed point. If trained attention heads are sitting near this fixed point, that might explain why modest truncation orders work: the network has already learned to concentrate its computation in the lower-order terms. A testable prediction: layers with α closer to 2 (measurable via weightwatcher https://github.com/CalculatedContent/WeightWatcher) might need fewer Taylor terms for accurate approximation than layers with α far from 2. If true, you could potentially use the spectral statistics to adaptively choose truncation order per-head.

New comment by riemannzeta in "Once Thought to Support Neurons, Astrocytes Turn Out to Be in Charge"

riemannzeta — Tue, 03 Feb 2026 23:51:50 +0000

Not an expert in neurobiology, but... heck this is Hacker News:

Let's assume that the brain implements a renormalization group flow in a way similar to transformers, as described here: https://arxiv.org/abs/2507.17912

In biological neural networks, synaptic weight distributions presumably have some effective α. Learning perturbs α away from optimal — overfitting would correspond to α drifting toward values that capture training noise rather than generalizable structure. The question becomes: what mechanism pushes α back toward 2?

Astrocytes are positioned to do exactly this. They don't modify individual synaptic weights (that's Hebbian learning), but they modulate the gain of synaptic transmission across vast swaths of tissue simultaneously. This is a different kind of operation — more like rescaling the entire weight matrix or reshaping its spectral density than updating individual entries.

Consider what happens during sleep. The synaptic homeostasis hypothesis (Tononi & Cirelli) proposes that sleep globally downscales synaptic strengths, renormalizing the system after a day of potentiation. This is literally a renormalization operation — a rescaling that preserves relative structure while adjusting the overall scale. If astrocytes mediate this process (and the Quanta article suggests they track sleep debt via calcium accumulation), they might be implementing something like the ERG step that SETOL identifies as optimal.

The scale-invariance condition is key. SETOL states that the ideal layer is governed by a Scale-Invariant transformation equivalent to a single step of an Exact Renormalization Group transformation. Astrocytes, by modulating transmission globally rather than synapse-by-synapse, naturally implement transformations that preserve relative structure while adjusting scale — exactly the kind of operation that could maintain or restore scale invariance.

The zebrafish "giving up" experiment becomes interesting through this lens. When the fish swims futilely, neural circuits are active in a regime that's metabolically expensive but informationally unproductive — the current model (swimming should work) is wrong, and the system is far from any useful attractor. The accumulated evidence in astrocytes might track not just "futility" in behavioral terms but deviation from optimal α — a signature that the current operating regime violates the ERG condition.

The state switch, then, isn't just behavioral resignation. It's the system recognizing that its current configuration is far from scale-invariant equilibrium and forcing a transition to a different basin where better spectral properties can be achieved. The adenosine release that triggers the switch would be the biological mechanism implementing what mathematically amounts to a large step in parameter space toward a more ERG-compliant regime.

The deepest connection might be to criticality itself. Systems at critical points exhibit scale invariance — correlations extend across all scales, and the system looks statistically similar under coarse-graining. The α = 2 condition in SETOL might be a signature of criticality in the weight space, just as power-law correlations are signatures of criticality in physical systems.

If brains operate near criticality, maintaining that critical state requires feedback — something must detect drift toward subcritical or supercritical regimes and push back. Astrocytes, with their slow timescales and broad spatial integration, are ideally positioned to monitor deviations from criticality and implement corrective homeostasis.

The Kadanoff-Wilson RG originated in the theory of phase transitions. SETOL adapts it to neural networks. The biological realization might be: neurons implement fast inference (the "flow" of representations through layers), while astrocytes implement slow RG steps that maintain the spectral conditions (α ≈ 2) under which that inference remains optimal.

Testable Predictions This framework suggests several predictions: * Synaptic weight distributions should drift away from α = 2 during extended waking and return toward it during sleep. * Astrocyte-deficient or astrocyte-disrupted animals should show abnormal weight spectral properties — perhaps more variance in α across brain regions, or systematic drift away from optimal values. * The giving-up threshold might correlate with cumulative deviation of local circuit α from 2 — not just behavioral futility, but a kind of "spectral stress" that astrocytes track. * Interventions that artificially maintain α ≈ 2 (if such were possible) might reduce the need for sleep or astrocytic modulation.

Any experts here? Astrocytes as implementers of biological RG steps that maintain the spectral conditions for optimal inference?