Hacker News: pjankiewicz

New comment by pjankiewicz in "We found a division by zero bug in FFmpeg with a vibecoded fuzzer"

pjankiewicz — Thu, 27 Aug 2026 19:09:11 +0000

Or it used something called an "analogy" which is a valid way to solve new problems.

New comment by pjankiewicz in "Stripe to Buy OpenRouter for $7B"

pjankiewicz — Mon, 17 Aug 2026 14:24:08 +0000

It is changing so often that to keep the costs and quality at a pareto level you need to experiment with a lot of different providers. And the pain you are describing should be a normal process in AI. I use model pools where the tasks are optimized for multiple AI providers. For example I was experimenting with GPT Luna and it turned out that the model is good but tool shy so I had to improve the instructions. Now this model is my main model for the chat in my app. Next week it can be a different model.

New comment by pjankiewicz in "Compression Is Prediction"

pjankiewicz — Tue, 11 Aug 2026 20:54:13 +0000

That was my thinking as well mainly to increase the speed of training. But it may turn out that the simple statistics that you can capture like this may account for 1% of the training and are likely to be captured as the first thing during the training.

But actually these techniques are used but they are hidden as speculative decoding with increasing complexity of approximations. For example you can have a part of the network that predicts the next word based on the markov chain, the next approximation is more complex etc.

This paper proposes something similar where you can inject memory without training https://arxiv.org/abs/2605.16893

New comment by pjankiewicz in "Compression is prediction"

pjankiewicz — Tue, 11 Aug 2026 20:39:04 +0000

I agree that technically they are doing the same thing but in practice LLMs are better compressors than PNGs (learned this while I was researching this topic). That was quite surprising to me.

New comment by pjankiewicz in "Compression is prediction"

pjankiewicz — Tue, 11 Aug 2026 20:19:14 +0000

I was thinking about the same topic and the conclusion can be wrong. LLMs are compressors, but compressors are not LLMs. Mixing this can let you believe that you can use a compressor to do the same thing as LLMs, which you cannot.

Specifically I was thinking about a way to inject knowledge into LLMs training by using statistical properties of text in such a way that you don't have to train the LLM to achieve some level of predictions. There are actually some papers that inject n-grams statistics as a part of the neural network weights.