This argument is cute and all, but ... does a data-point of 1 from 200 years ago really give us much confidence? We replaced physical labor with a massive service sector.

Now we're automating the service sector so now people can go to... eeh... the 3rd category of jobs? Seems like physical labor is the most stable career at the moment; what machines have not already automated is pretty difficult to replace it turns out. But we outsourced most of that to low cost countries except plumbers and electricians.

But will a population of plumbers really be able to maintain a population of plumbers employed?

Sure, the transformer is great for making larger neural networks with better learning potential, which are improving protein folding models a fair bit. But do we need the combined budget of the Apollo program or interstate highway system (adjusted for inflation) per year, to develop better molecular simulation models? (no, the most advanced ones run on mundane hardware and trained just fine on pre 2020 infrastructure).

So while it's true that; "AI" ((primarily) Neural network based deep learning techniques) are wonderful tools to make society better; slop generators absorbing the entire energy budget of a few small nations to generate infinite propaganda, linked-in posts and shrimp Jesus is only tangentially helping in that goal while destabilization civilization in the process.

The biggest reason large models are un-attainable for local applications is the lack hardware with large amount of unified/graphics memory (and the cost of the platforms that do). Once the memory slog goes back to normal and hardware manufacturers adapt to demand, we may see consumer hardware with large memory capacity effectively opening the door for slow but usable frontier model inference (assuming improvements in model efficiency and compute capacity)

At that point, inference becomes a race to the bottom. The large labs hope they can attain a leap in capability (which is increasingly looking bleak, with a average catch-up of just a few months) or market dominance through integration (integration in platforms and OS, exclusive deals with companies or governments).

For coding agents, i suspect no player will manage lock in enough market to enforce pricing much higher than the true inference cost, and catering to programmers becomes an unsustainable proposition. We will instead be further hit with a lot of AI integrated into our other tooling costs, such as GitHub, Microsoft suite, G-suite, forcing in AI functions as a value-ad into the total cost without giving the option to exclude them. (using their market position)

In a contrived example of a pbuf {length:int, payload:byte[1]}

LEB128 can trick you into reading the payload as part of the length, but then hopefully trigger a code check against invalid buffer read. (or one byte outside the struct if the payload is also malicious)

Binou64 can trick you to read 7 bytes into other memory, before any buffer size validation is done.

It's then not uncommon to log with a helpful; "buffer with length: 26624894573377(7 bytes of stolen data) is invalid", or just crash.

It's to the point that Bijou64_decode should perhaps take "end_adress" or "max_read" to catch this kind of attack.

(If you dont validate a malicious pbuf, you're in for a bad time regardless of integer format, but these int formats add their own way to trigger a buffer overrun despite a proper check.)

An adveserial package can claim to have a 255 tagged integer but not actually have any followup, tricking the payload parser into an incorrect offset and reading straight off into followup memory.

It's a classic thing to check for when dealing with variable length strings or binary, but it may not cross the mind when it's hiding in the Bijou64_decode(*buff, *cr) function.

So Temu should be sued if a house burns down from a generic-brand e-bike that they imported and took money for.

We also have laws making the store selling the thing that burnt down your house liable for what they sold, which make them think twice about selling a random off-brand fire-starter with unknown manufacturer. This worked great until Temu, Amazon, and Alibaba entered the market claiming to be "marketplaces" connecting "importers to suppliers" while clearly behaving like a store.

The core issue is that, if the producer cannot be sued, the seller cannot be sued, then there is no reason to follow any safety what-so-ever. So fine the distributor until they put some quality control or standards on the producers they give market to, may solve the issue.

The US has this issue as well, though more focus on individuals suing for each case rather than broad-spectrum compliance regulation. The outcome is the same; with nobody to sue, there is no reason to make things safe for human use.

https://youtu.be/B90_SNNbcoU

In practice, the compiler is an often an omitted dependency of any c code.

The up-front investment of a memory fab is measured in billions, and takes years to construct and get running. The margin on the chips themselves is terrible, so without scale its not worth even trying. DDR5 is a industry standard that takes some effort to conform to, but the licence fees is a drop in the bucket to the cost of creating a fab.

The fabricators were cautious about increasing production, and slow to start planning. It takes further time to build up capacity, and if the demand drops down, they may end up producing dram at a loss when the market flips over to oversupply. The demand whiplash could kill any company that dared betting on increasing production. See the "bullwhip effect" https://en.wikipedia.org/wiki/Bullwhip_effect which has killed semiconductor fabricators before.

There is a discussion to be had about how to maintain national semiconductor production in Europe and US as a strategic industry, but historic attempts have all failed.

The age where the web was usable at all without JavaScript is long gone. No scraper would get much scraping done without JavaScript these days.

using 1886 or 1888 makes Google correctly identify that no such sensus exist.

asking about 1887 specifically makes Google refer to some supposed great effort to track passenger pigeon population mids of the species decline.

I've gotten better at phrasing myself adequately in one go. Rute mechanical memorization has also made writing itself cheaper. (read my username)

I can now yap quite adequately over text, yet i regularly find AIs at a minimum 2x as verbose as my preferred phrasing after manual word mashing.

Those "tolkens" humans "count" are translated to a ~2048 (depends on model) floating point vector.

bird => {mamal, english, noun, Vertebrate, aviant} has one r but what if you make it 20% more "french". Is is still 1 r? That could be the word "bird" in french, or it could be a french speaking bird or a bird species common in france.

If nearest neibour distance to the vocabulary of every language makes the vector no longer map to "bird"; then the amount of rs' must change, using a series of trained conditional checks (with some efficiency where languages have some general spelling patterns).

That is such an unreasonable amount of compute, that it is likley faar cheaper, easier and more reliable to train the model to memorise the output:

{"MCP":"python", "content":"len((c for c in 'strawberry' if c='r'))"}

The attention mechanism allow LLMs to learn this kind of absurdly inefficient calculations. But we really shouldn't use LLMs where they're outperformed by trivial existing solutions.

The models now whaste a vast amount of useless neurons memorising the character count the entire English language so that people can ask how many r's are in strawberry and check a tickbox in a benchmark.

The architecture cannot efficiently or consistently represent counting letters in words. We should never have forced trained them to do it.

This goes for other more important "skills" that are unsuited to tranformer models.

Most models can now do decent arithmetics. But if you knew how it has encoded that ability in its neurons then you would never ever ever ever trust any arithmetic it ever outputs, even in seems to "know" it (unless it called a calculator MCP to achieve it).

There are fundamental limitations, but we're currently brute forcing ourselves through problems we could trivially solve with a different tool.

it's indeed a mess.

I want competent and skilled coworkers. I care about our hiring process, and the hiring process of where I apply. Many modern screening processes are abysmal, and a abysmal screening process is reflected in the company and culture over time.

My experience of university exams makes it very clear that studying for test and studying to understand a topic are two different goals that collide or even contradict.

I dont want to hire anyone that studdied for the test instead of the topic. Placing any higher stakes on the test result encurrage the wrong behaviour and filters the wrong people.

I have friend who failed physics because they spent all their time writing their own kernel for mips assembly. And plenty of classmates who aced the exam by memorising prior year question examples.

who would you hire?

Its a sign that the developer didn't pay attention to what they committed. Like a spelling error, or forgetting to run the linter.

If the IDE added "written with vscode" i would be equally furious.