> Model SOTA moves faster than chips can be designed or produced.

Here's my analysis of how to etch relatively big LM into silicon: https://news.ycombinator.com/item?id=47109252

Given some amount of work with the fab before main pipeline set (I think a year long process), one can then spew LM-on-a-chip in six months or less and much more than 2 per year, because there can be several LMs in pipeline.

  > Matrix multiplication is after all nothing but an aggregate sum over a cross join.

They converted linear algebra expressions into relational algebra expressions, optimized them using equality saturation and converted back into linear algebra. With some noticeable speedups.

They used relational algebra for optimization because relational algebra has canonical form - there is only one most optimal expression.

And, equality saturation is most efficiently implemented using generic join algorithm.

  > 186 bytes of machine code

(the link is a translation to Russian, I was unable to find the original text in English)

Also consider that toxoplasma changes rat's dopamine serum level to such extent that they stop fearing cats. The same happens with humans, changes in dopamine serum level are so profound that toxoplasma-infected people become more appealing than regular people.

What makes one socialize is dopamine [1]: "Hypomania, manifesting with feelings of euphoria, omnipotence, or grandiosity are prone to appear in those moments when medication (dopaminergic medication - thesz) effects are at their maximum."

[1] https://en.wikipedia.org/wiki/Dopamine_dysregulation_syndrom...

This dopamine dysregulation occurs naturally in people with toxoplasmosis.

This kind of mistake - mistaking dopamine and endorphine, - strongly suggests a fake. I know of one such fake from the former employee of my friend, you can read his account here [2]. My friend was a lead of the "secret" laboratory (next to Kitay Gorod metro station in the center of Moscow, in an institute that needed no documents to enter) that researched how "psychics" and "healers" achieve their results.

[2] https://lit.lib.ru/t/torin_a/text_0100.shtml (it is in Russian, but language is not a barrier at all nowadays)

The laboratory researched what these "extrasensing people" do and found nothing - given very little training an ordinary men can do the same and even better. At the end of USSR, there was a string of visitors from USA there in the lab, and one of those visitors successfully persuaded US government to open similar research program, according to [2].

The fearmongering of "endorphin-producing bugs that make one socialize and spread" looks like, well, fearmongering to obtain funds for Big Pharma's government contracts, etc. It messes up two very different neurotransmitters, a mistake that real scientist would not make.

My son got caught trying to fix Windows installation on my wife work computer with bootable Linux Mint USB stick.

"Digital Naive," yes, of course.

[1] https://en.wikipedia.org/wiki/Byte-pair_encoding

It naturally takes care of common prefixes and suffixes.

It is easy and fast to apply using radix tree or with finite automata. Even without radix tree, it is possible to have processing speed in the range of hundredths of thousands of bytes per second.

Thus, it appears, that ~585 GPU years can factor 1024 bit RSA. 2.2^((1024-896)/34)=19.5, expected growth of resources' usage compared to 896 bits factorization, multiplying it by 30 GPU years for 896 bits gives about 585 GPU-years.

This will cost about $20M with Cognition AI setup.

  >  I have met some if the coolest people in CCCs but also some of the worst people.

The in-place property can be utilized to make it very close to cache-oblivious algorithm.

If they write 2 (two) lines of code, the combined output of them with LLM, as I read it now, is 20K SLOC. This is yearly output of human programmer.

It is very much possible to write 100 lines of (debugged, reviewed) code per day for human. And these 100 lines of code would result, if we apply their LLM multiplication factor, in 50 (fifty) man-years of work!

Even if they write very little code, say, one line per month, after a year of work they will get result of about 6 (six) years of human programmer. SQLite is 155K SLOC, about 8 man-years.

And that what raised my question: what is the problem domain that requires so much code?

Because 100 lines of (debugged, reviewed) code per day is a good speed for seasoned software engineer. I assume that you can produce more than 100 lines of something per day as a prompt.

So, what is the problem domain that requires one to write several thousands of lines of code per day?

  > all were able to find every problem planted there, with fairly little steering, and no spoilers.

[1] https://arxiv.org/abs/2404.13076

  > no escaping

https://en.wikipedia.org/wiki/Royal_Road#A_metaphorical_%22R...

[1] https://leodemoura.github.io/blog/2026-8-1-postmortem-for-ke...

If tvtropes or any other material related to The Talos Principle was used to train models, we don't need much else to have agents-with-forum discussing and reverse engineering "puzzles" and human culture.

  > I'm excited by the option in Astra to route PCBs.

Eurisko was used for VLSI chip design, then used rules discovered there to design TCS Traveler winning fleet.

It is unbelievable how artificial intelligence walk in circles.

  > the whole message board thing

There it is an important part of the plot and makes these robots appear conscious.

[1] https://tvtropes.org/pmwiki/pmwiki.php/VideoGame/TheTalosPri...

[1] https://en.wikipedia.org/wiki/Solomon_Shereshevsky

"His memory was so powerful that he could still recall decades-old events and experiences in the smallest details. After he discovered his own abilities, he performed as a mnemonist; but this created confusion in his mind. He went as far as writing things down on paper and burning it, so that he could see the words in cinders, in a desperate attempt to forget them. Some later mnemonists have speculated that this was a mentalist's technique for writing things down to later commit to long-term memory. Reportedly, in his late years, he realized that he could forget facts with just a conscious desire to remove them from his memory, although Luria did not test this directly."

I believe that one need to have superhuman memorization abilities to have definite confusion due to too much remembered. More trivial explanation of this effect in normal ageing persons is age-related brain shrinkage.

  > Obviously the brain is not a computer,

  [2] https://www.nature.com/articles/s41598-024-62539-5
  [3] https://pmc.ncbi.nlm.nih.gov/articles/PMC11655932/
  [4] https://pmc.ncbi.nlm.nih.gov/articles/PMC12230014/
  [5] https://pubmed.ncbi.nlm.nih.gov/6204761/

  > "Second, there is no guarantee that a given neural network can be approximated by DISCOVER"

They train what appears as embeddings for outer product of roles and fillers. The role for language model can be a position in text, the filler can be an embedding of a word at that position. Then that matrix of a sum of these outer products is linearly mapped into NN encodings and then decoded by NN decoder.

The embeddings learned by this process are not necessarily smaller than original ones. Given that they participate in an outer product computation gives me impression that the resulting sum is much bigger than actual NN encoding, that is why it needs to be linearly mapped into NN encoding.

So, this paper will not necessarily lead to any computation savings.

But I am at page 6. ;)