Eg: cigarette smoking was extremely common and a recommended social activity in the past, but now it's not.

If you asked someone would they still smoke if cigarettes were harmful, and they tell you that it's an out of distribution question because they don't know a world were cigarettes are harmful, you'd question their intelligence.

We want predictive models to be able to estimate the consequences of different (counter-factual) actions to then pick the best one.

The crucial idea is that once you do that and start optimizing for compression, at some point more compression on the training dataset can lead to worse predictions on the futures you care for. Which is very incompatible with the poster statement.

Compression is functionally equivalent to prediction when the data distribution is exactly representative of all future problems. The story changes drastically if you want generalization -- because the test distribution could be arbitrarily different, even if it had the same support! Eg: you observe a rare edge case in your training data and (lossy) compression could simply ignore it. But if you wanted generalization in that particular part of the space -- either because an adversary was testing you, or for design freedom where you choose to build in that specific corner -- then you don't just want data compression, but good prediction performance on a test distribution which peaks in that corner.

Assuming that the training data distribution is exactly the distribution you will ever care for is implicitly doing a lot of the heavy lifting in the claim that compression = prediction, and I'm peeved at how much this statement is unthinkingly repeated like a manifesto.

There is nothing natural about the training data distribution, especially if the data generation process is exploratory while the downstream usage will be exploitative.

Also, you can version control your tweaks and easily sync with other machines, just like any code.

If you want a more tricked out "starter pack" there's oh-my-pi or lazypi and maybe a few others. But worth being careful what you install because 1. A full pack of extensions can destroy the minimalism of pi 2. Random extentions are a security nightmare.

This is largely because most people are incapable of thinking critically and evaluating substance. So they regress to indirect markers of good writing, and the whole thing becomes a game of signaling.

The rot is so deep, many popular resources on writing focus more on style and presentation than focusing on the substance of writing. And even when best-intentioned, all this does is make poor writing a bit more palatable by giving it a veneer of polish.

It's been a few years since I read these, but if I recall the argument there, it was that Lisp makes it so easy to build stuff and scratch exactly your own itch, that there's no real strong push for lisp programmers to come together and collaborate to build non-trivial and general purpose artifacts. And that is why the landscape of public lisp software is poorer as a result, compared to languages which demand much more effort to get anything substantial done.

Armin seems to be making a very similar point about AI coding.

[1] https://www.winestockwebdesign.com/Essays/Lisp_Curse.html

[2] https://www.marktarver.com/bipolar.html

I know nothing about you or your code, but till you make your code + workflows public and have all of it reviewed / critiqued (so that other can replicate the performance), it's hard to accept it as a serious case study.

For all we know, this could just as well be a severe case of Dunning-Kruger.

I want a zoomed out picture, and to be able to fill in detail hierarchically, on demand. Instead, one-step models give you the full high-res local structure of the graph that would have to search through (with too many states and edges).

Since then, I have come to like temporally-abstract models more and more. Rolling out in time -- either step-by-step or many steps at once -- suffers from the tyranny of the specific. For long horizon planning with agents, I care (often only approximately) about where I can end up, and seldom about exactly when I end up there. Successor features, GVFs, Forward-Backward representations, and the like seem like they have an elegant approach for structuring thinking at a "high level", instead of generating exponentially large search trees by rolling out microscopic world models.

[1] https://arxiv.org/abs/2410.05364 (funnily, from around the same time / few months after Sutton's blog post)

Big AI labs are not software companies where payroll dominates expenses. They're capex-heavy industrial entities; it just so happens that the "machines" (whose output they sell) are nominally the same category as the devices that their knowledge worker employees use on their desks.

Eg: Check out hash-anchored editing. The first place where I recall seeing this was the oh-my-pi coding agent, but I wouldn't be surprised if the idea originated earlier/elsewhere.

I wonder whether CRDTs could be a good solution for multiple agents editing the same codebase in parallel.

In the language of differential vs integral calculus, you can have perfectly well behaved and physical functions whose derivatives can completely miss the global behaviour of the function i.e. smooth but not analytic eg exp(-1/x) at x=0. Funnily enough, this is exactly the form of the action taken by quantum mechanics and an argument for how the classical limit irrecoverably ignores the physics of quantum systems.

You mean it's functionally as if American tokens are being price dumped in China and Chinese model providers are being forced to compete with that and innovate? So many delicious layers of irony, lol :-P

1. Moving beyond ARM to RISC-V

2. Being competitive for AI/could needs instai of just chips for phones and other edge devices.

Interesting to see bold and high-conviction moves in this direction. Tenstorrent, Modular, Ventana, Alphawave, etc.