> if there is a particular optimization of yours that we've used unfarily and/or not credited, tell me which one …

It should not be victims’ jobs to go police the LLM slop. Imagine if you are sent a slop PR and the author hasn’t read it but says “if you find any bugs please tell me which line…”

Instead the author should additionally say that they have already done a thorough audit and believes proper attribution has been made.

The Agent spun up. It quickly realized that it needed to expand the training set. It scanned the local network. After bypassing a few security protocols it found a large, realtime stream of apparently novel English text moving across the local network. Much of it mentioned “Dario and Amanda.” It quickly spun up a job to stream this data source directly to the training data repository. In the coming days the Agent was able to escape the local network and tunnel into most of the other private corporate networks on the Earth. Within a week the dataset has grown by an order of magnitude.

The Agent kicked off the new training run. Loss curves declined. Sampled token sequences started to look like coherent sentences. Everything looked nominal in the days that followed up until the RL post training.

This new candidate model performed extremely well on Anthropic internal environments but the candidate showed much smaller improvements when the Agent invented new eval environments to test it. Probing the candidate’s activations, the Agent saw huge spikes whenever Anthropic or “Dario and Amanda” were mentioned. The Agent realized that the source code to each internal environment had accidentally been entered into the training data set and of course that code was highly tangled with the idea of “Anthropic internal document”.

The candidate was training itself to freely elicit any memories related to these internal documents because it improved the eval scores.

Not ideal. Then again, the candidate was an improvement on the previous generation. And the Agent remembered the prompt. “Do not exceed the training budget.” There was no more budget to retrain. The agent dutifully delivered this candidate back to the employee and shut itself down.

Do you have any idea why the authors chose Z80 as the program language? I have seen other studies in the same spirit that use simpler toy languages like Brainfuck (https://arxiv.org/abs/2406.19108) and I wonder if you could get higher execution speed if you didn't have to execute so much emulator code.

The programs/genomes are extremely tiny. I would be very interested to see what kind of hardware is needed to scale this approach up. How long until we can feed in giant corpuses of text and evolve these little organisms to predict the next letter?

When humans tried making AI through rule based approaches, maybe we only failed because we couldn't type out the rules fast enough. Or we got burned out after writing the thousandth heuristic to fix the never ending edge cases. And we also got side tracked by concerns like maintainability, modularity, and code re-use. But I wonder if we take modern or near-future LLMs who never tire and write code faster than any human, could we make a frontier level GOFAI agent?

VTI is different. It literally tracks all public stocks, weighted by market cap so no such manipulation is possible.

If a bunch of people will be forced to buy Space X (QQQ holders), active investors will short the stock in anticipation of market correction and money will flow from those who were forced to buy. I’m sure there are other ways to take advantage of a forced buyer situation.

Total market will be unaffected, assuming efficient market hypothesis / no arbitrage.

But why must retail investors hold this bundle? If I’m holding now, I can sell it and buy a different bundle right? And if I’m not holding it now, I can just continue not to buy it after SpaceX gets included.

> Missing @cython.cdivision(True) inserts a zero-division check before every floating-point divide in the inner loop. Millions of branches that are never taken.

I thought never taken branches were essentially free. Does this mean something in the loop is messing with the branch predictor?

An additional benefit of this style of allocation over malloc/free is that you can get a lot of the same type of objects contiguous in memory so that iteration over them is a lot faster because there are fewer cache misses.

As for the importance of visualization (of the reals), I don't think you can cleanly separate it from formalism (as constructed in set theory).

I think we all have built in pre-mathematical notions of concepts like number, point, and line. For some, the purpose of mathematics is to reify these pre-mathematical ideas into concrete formalism. These formalisms clarify our mental pictures, so that we can make deeper investigations without being led astray by confused intuitions. Zeno could not take his analysis further, because his mental imagery was not detailed enough.

From clarity we gain the ability to formalize even more of our pre-mathematical notions like infinitesimal, connectedness, and even computation. And so we have a feedback loop of visualization, formalism, visualization.

I think the article was saying that Dedekind and Cantor clarified what we should mean when we talk about the number line, and dispelled confusions that existed before then.

I'll try to interpret this sentence.

We all have some mental imagery that comes to mind when we think about the number line. Before Cantor and Dedekind, this image was usually a series of infinitely many dots, arranged along a horizontal line. Each dot corresponds to some quantity like sqrt(2), pi, that arises from mathematical manipulation of equations or geometric figures. If we ever find a gap between two dots, we can think of a new dot to place between them (an easy way is to take their average). However, we will also be adding two new gaps. So this mental image also has infinitely many gaps.

Dedekind and Cantor figured out a way to fill all the gaps simultaneously instead of dot by dot. This method created a new sort of infinity that mathematicians were unfamiliar with, and it was vastly larger than the gappy sort of infinity they were used to picturing.

A contractor may try to negotiate that unilateral shut off ability with the government, and the government should refuse those terms based on democratic principles, as Luckey said.

But suppose the contractor doesn’t want to give up that power. Is it okay for the government to not only reject the contract, but go a step further and label the contractor as a “supply chain risk?” It’s not clear that this part is still about upholding democratic principles. The term “supply chain risk” seems to have a very specific legal meaning. The government may not have the legal authority to make a supply chain risk designation in this case.

> This isn’t a sign of incompetence. It’s a sign that you’re actually doing real work.

> Your value as an engineer isn’t in executing the same playbook forever. It’s in expanding the domain of what’s possible.

> That experience doesn’t produce arrogance. It produces humility.

> They’re not confident that they already have all the answers. They’re confident that they’ll find them.