Hacker News: liamdgray

New comment by liamdgray in "Agentic Coding Is a Trap"

liamdgray — Mon, 04 May 2026 00:52:25 +0000

Great idea. I would love to see your transpiler Mind sharing it?

New comment by liamdgray in "Chrome DevTools MCP"

liamdgray — Sun, 15 Mar 2026 23:03:55 +0000

Please do!

New comment by liamdgray in "Denmark's government aims to ban access to social media for children under 15"

liamdgray — Fri, 07 Nov 2025 23:25:17 +0000

Some students even wish for a ban to reduce the pressure to keep up with social media.

That reminded me of Warren Buffet asking for his kind and to be taxed more.

New comment by liamdgray in "ISR: Invertible Symbolic Regression (2024)"

liamdgray — Sun, 17 Aug 2025 03:05:34 +0000

Abstract: "We introduce an Invertible Symbolic Regression (ISR) method. It is a machine learning technique that generates analytical relationships between inputs and outputs of a given dataset via invertible maps (or architectures). The proposed ISR method naturally combines the principles of Invertible Neural Networks (INNs) and Equation Learner (EQL), a neural network-based symbolic architecture for function learning. In particular, we transform the affine coupling blocks of INNs into a symbolic framework, resulting in an end-to-end differentiable symbolic invertible architecture that allows for efficient gradient-based learning. The proposed ISR framework also relies on sparsity promoting regularization, allowing the discovery of concise and interpretable invertible expressions. We show that ISR can serve as a (symbolic) normalizing flow for density estimation tasks. Furthermore, we highlight its practical applicability in solving inverse problems, including a benchmark inverse kinematics problem, and notably, a geoacoustic inversion problem in oceanography aimed at inferring posterior distributions of underlying seabed parameters from acoustic signals."

ISR: Invertible Symbolic Regression (2024)

liamdgray — Sun, 17 Aug 2025 03:05:18 +0000

Article URL: https://arxiv.org/abs/2405.06848

Comments URL: https://news.ycombinator.com/item?id=44928543

Points: 7

# Comments: 1

New comment by liamdgray in "Neural Symbolic Regression that scales (2021)"

liamdgray — Sun, 17 Aug 2025 02:12:33 +0000

Abstract: "Symbolic equations are at the core of scientific discovery. The task of discovering the underlying equation from a set of input-output pairs is called symbolic regression. Traditionally, symbolic regression methods use hand-designed strategies that do not improve with experience. In this paper, we introduce the first symbolic regression method that leverages large scale pre-training. We procedurally generate an unbounded set of equations, and simultaneously pre-train a Transformer to predict the symbolic equation from a corresponding set of input-output-pairs. At test time, we query the model on a new set of points and use its output to guide the search for the equation. We show empirically that this approach can re-discover a set of well-known physical equations, and that it improves over time with more data and compute."

Neural Symbolic Regression that scales (2021)

liamdgray — Sun, 17 Aug 2025 02:12:10 +0000

Article URL: https://proceedings.mlr.press/v139/biggio21a.html

Comments URL: https://news.ycombinator.com/item?id=44928360

Points: 1

# Comments: 1

New comment by liamdgray in "Recasting Self-Attention with Holographic Reduced Representations (2023)"

liamdgray — Sun, 17 Aug 2025 01:59:00 +0000

Abstract: "In recent years, self-attention has become the dominant paradigm for sequence modeling in a variety of domains. However, in domains with very long sequence lengths the O(T^2) memory and O(T^2H) compute costs can make using transformers infeasible. Motivated by problems in malware detection, where sequence lengths of T≥100,000 are a roadblock to deep learning, we re-cast self-attention using the neuro-symbolic approach of Holographic Reduced Representations (HRR). In doing so we perform the same high-level strategy of the standard self-attention: a set of queries matching against a set of keys, and returning a weighted response of the values for each key. Implemented as a “Hrrformer” we obtain several benefits including O(THlogH) time complexity, O(TH) space complexity, and convergence in 10× fewer epochs. Nevertheless, the Hrrformer achieves near state-of-the-art accuracy on LRA benchmarks and we are able to learn with just a single layer. Combined, these benefits make our Hrrformer the first viable Transformer for such long malware classification sequences and up to 280× faster to train on the Long Range Arena benchmark."

Recasting Self-Attention with Holographic Reduced Representations (2023)

liamdgray — Sun, 17 Aug 2025 01:58:20 +0000

Article URL: https://proceedings.mlr.press/v202/alam23a.html

Comments URL: https://news.ycombinator.com/item?id=44928315

Points: 2

# Comments: 1

New comment by liamdgray in "Composing Linear Layers from Irreducibles"

liamdgray — Sat, 16 Aug 2025 22:19:52 +0000

Abstract: "Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental building blocks remain poorly understood. We investigate this compositional structure in linear layers by asking: can we identify/synthesize linear transformations from a minimal set of geometric primitives? Using Clifford algebra, we show that linear layers can be expressed as compositions of bivectors -- geometric objects encoding oriented planes -- and introduce a differentiable algorithm that decomposes them into products of rotors. This construction uses only O(log^2 d) parameters, versus O(d^2) required by dense matrices. Applied to the key, query, and value projections in LLM attention layers, our rotor-based layers match the performance of strong baselines such as block-Hadamard and low-rank approximations. Our findings provide an algebraic perspective on how these geometric primitives can compose into higher-level functions within deep models."

Composing Linear Layers from Irreducibles

liamdgray — Sat, 16 Aug 2025 22:19:34 +0000

Article URL: https://arxiv.org/abs/2507.11688

Comments URL: https://news.ycombinator.com/item?id=44927323

Points: 2

# Comments: 1

New comment by liamdgray in "Introduction to latent variable energy-based models (2024)"

liamdgray — Mon, 11 Aug 2025 00:41:57 +0000

Abstract: "Current automated systems have crucial limitations that need to be addressed before artificial intelligence can reach human-like levels and bring new technological revolutions. Among others, our societies still lack level-5 self driving cars, domestic robots, and virtual assistants that learn reliable world models, reason, and plan complex action sequences. In these notes, we summarize the main ideas behind the architecture of autonomous intelligence of the future proposed by Yann LeCun. In particular, we introduce energy-based and latent variable models and combine their advantages in the building block of LeCun’s proposal, that is, in the hierarchical joint-embedding predictive architecture."

Introduction to latent variable energy-based models (2024)

liamdgray — Mon, 11 Aug 2025 00:41:11 +0000

Article URL: https://iopscience.iop.org/article/10.1088/1742-5468/ad292b/pdf

Comments URL: https://news.ycombinator.com/item?id=44859739

Points: 2

# Comments: 1

New comment by liamdgray in "Modern Methods in Associative Memory"

liamdgray — Sun, 10 Aug 2025 23:59:42 +0000

Abstract: "Associative Memories like the famous Hopfield Networks are elegant models for describing fully recurrent neural networks whose fundamental job is to store and retrieve information. In the past few years they experienced a surge of interest due to novel theoretical results pertaining to their information storage capabilities, and their relationship with SOTA AI architectures, such as Transformers and Diffusion Models. These connections open up possibilities for interpreting the computation of traditional AI networks through the theoretical lens of Associative Memories. Additionally, novel Lagrangian formulations of these networks make it possible to design powerful distributed models that learn useful representations and inform the design of novel architectures. This tutorial provides an approachable introduction to Associative Memories, emphasizing the modern language and methods used in this area of research, with practical hands-on mathematical derivations and coding notebooks."

Modern Methods in Associative Memory

liamdgray — Sun, 10 Aug 2025 23:58:49 +0000

Article URL: https://arxiv.org/abs/2507.06211

Comments URL: https://news.ycombinator.com/item?id=44859559

Points: 5

# Comments: 1

New comment by liamdgray in "Dense Associative Memory for Pattern Recognition (2016)"

liamdgray — Sun, 10 Aug 2025 23:55:57 +0000

"Abstract A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of models that smoothly interpolates between two limiting cases can be constructed. One limit is referred to as the feature-matching mode of pattern recognition, and the other one as the prototype regime. On the deep learning side of the duality, this family corresponds to feedforward neural networks with one hidden layer and various activation functions, which transmit the activities of the visible neurons to the hidden layer. This family of activation functions includes logistics, rectified linear units, and rectified polynomials of higher degrees. The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions - the higher rectified polynomials which until now have not been used in deep learning. The utility of the dense memories is illustrated for two test cases: the logical gate XOR and the recognition of handwritten digits from the MNIST data set."

Dense Associative Memory for Pattern Recognition (2016)

liamdgray — Sun, 10 Aug 2025 23:55:38 +0000

Article URL: https://proceedings.neurips.cc/paper_files/paper/2016/hash/eaae339c4d89fc102edd9dbdb6a28915-Abstract.html

Comments URL: https://news.ycombinator.com/item?id=44859538

Points: 2

# Comments: 1

New comment by liamdgray in "Hypertokens: Holographic Associative Memory in Tokenized LLMs"

liamdgray — Tue, 05 Aug 2025 01:21:11 +0000

Is this intended to run on a quantum computer? You mention Grover's search algorithm, for example.

New comment by liamdgray in "How to put algorithms into neural networks? (2019) [video]"

liamdgray — Mon, 04 Aug 2025 22:42:01 +0000

Recorded at the ML in PL 2019 Conference, the University of Warsaw, 22-24 November 2019.

Anton Osokin (Higher School of Economics, Moscow), https://aosokin.github.io/

Slides available at https://docs.mlinpl.org/conference/2019/slides/anton_osokin_...

Abstract: Recently, deep neural nets have shown amazing results in such fields as computer vision, natural language processing, etc. To build such networks, we usually use layers from a relatively small dictionary of available modules (fully-connected, convolutional, recurrent, etc.). Being restricted with this set of modules complicates transferring technology to new tasks. On the other hand, many important applications already have a long history and successful algorithmic solutions. Is it possible to use existing methods to construct better networks? In this talk, we will cover several ways of putting algorithms into networks and discuss their pros and cons. Specifically, we will touch using optimization algorithms as structured pooling, unrolling of algorithm iterations into network layers and direct differentiation of the output w.r.t. the input. We will illustrate these approaches on applications from structured-output prediction and computer vision.

How to put algorithms into neural networks? (2019) [video]

liamdgray — Mon, 04 Aug 2025 22:40:33 +0000

Article URL: https://www.youtube.com/watch?v=hP4fLzbeMXU

Comments URL: https://news.ycombinator.com/item?id=44792140

Points: 3

# Comments: 1