I find it amazing how robust the current deep learning models are. A simple linear combination of every weight did not degrade the performance of the model, but enhanced it.

Don't get me wrong. I hate war. And never-ending wars like the Iraq War anger me to no end (and for that matter, I think G.W. Bush and his cabinet were truly evil). Of course, the danger is real; a military built for defense can easily become an instrument of tyranny or empire if left unchecked. That is why we must maintain rigorous civilian oversight and strict checks and balances over its power. But that does not mean the military, by default, is always evil, right?

I wonder if any tech company managed to thrive long in history by betting so violently on fear mongering and regulatory capture.

Not to be sarcastic but just to offer an observation: in a sufficiently large or bureaucratic organization, preventing an incident from happening can rarely get you any credit or visibility. Such achievement falls into the bucket of "what you're supposed to do". So, those who navigate company dynamics well would rather let the incident happen and then be loud on the follow-up action items. The trick is not to turn an incident into a diaster, so it's a dedicate act.

People think otherwise with AI partly because Anthropic kept telling us that they didn't have to write code or review code any more for most of their work. Their agent swarms just comb through their github, slack and wikis to figure out what to do next, and another swarm of agents just review, test, merge, deploy, A/B test, and revert the code. Boris alone merged nearly 300 PRs in the past week (or two?). So the top research labs seem have broken the productivity seal.

And then they talk about this recursively self-improving AI that is so powerful, so autonomous that they advocate that every company should be prepared to "pause" the effort. And their Fable/Mythos has this specific restriction as mentioned in their model card[1] that they are going to reject requests to tune and train models because, well you guess it, the models are too powerful to be used by mere mortals.

[1] We’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). Using Claude to develop competing models already violates our Terms of Service, but enforcing this restriction through our safeguards avoids accelerating the actors most willing to violate these terms. Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT).

If we don't consider the potential loss of our jobs, on the other hand, isn't it great that we don't have to repeatedly do what we already know how to do? I mean, how many times can we feel the thrill by writing the same CRUD applications? How many times do we have to design the same idempotent APIs? It's also a relief that we could spend way less time figuring out mitigations or root causes when there is a production incident.

This reminds me of the scribes before Gutenberg's moveable-type printing press. They spent their life in scriptoriums copying the manuscripts by hand. They earned three times of the average income of their times. They were highly skilled labor. It required years of training, deep literacy, and a high level of domain expertise. Yet, history showed that even highly specialized expertise can be mechanically reproduced.

That appears to be exactly what LLMs are doing for us: automating the digital equivalent of manual transcription, such as setting up the repetitive boilerplate, sketching out the standard APIs, finding predictable bug fixes.

I'm not sure about others, but I have to face the same existential question today: as software engineers, where does our true value lie? Is it merely in learning, memorizing, and, reproducing patterns that others have already built. More often than not, patterns that an LLM can now piece together better and faster? Or is it in taking everything we’ve learned and applying it to solve entirely new, messy, and uniquely human problems? If our worth is tied to how well we copy the past, we are already obsolete. Our value has to shift from being human repositories of known solutions to being creators who venture into the unknown.

It is, of course, easier said than done. Hence I have likely the same level of stress as other software engineers.

I suspected it was more likely that the intellectual work had already been done in a similar way by a number of other people, and GPT-4 picked up that work.

I realized it long ago: one needs output to make meaning. Input can only be the cherry on a cake in one's life. That, actually, makes FIRE or Fat FIRE not so sustainable unless one has other hobbies.

Just realized something: if one worries about losing jobs to AI, token's high unit cost is good news. To say the least, high cost would delay the displacement, if any, right?

In the meantime, someone shared the below on X. I guess the moral of the story is that "good enough" does not just displace software engineers, but also models.

   > I Went From $3,000/Month on Claude to $5/Week on DeepSeek

   > And honestly?80% of my work is identical.

   > For the past two months, I was burning $3-5K monthly on Claude Code. Every idea from design to development to testing - full end-to-end automation, even simulating users to test my products and provide feedback.

   > Extremely token-intensive. But Claude's caching sucked, making it insanely expensive.

   > Then I discovered DeepSeek V4.

In contrast, imagine if we had the same AI 20 years or so ago. Could AI really write Jersey? I guess not as people were still trying to understand JAX-RS. Could AI really answer all the questions about React? I guess not as React was just invented. Would we use 10x fewer people to build out infra on the public cloud or the entire so-called Big Data platforms? I guess not, as they were still rapidly evolving and we'd need so many engineers to explore so many different possibilities? Could we use AI to build our ML ecosystem with 10X fewer people? I highly doubt so. Heck, 20 years ago R was all the rage and Python's ecosystem was not mature at all. Oh, and mobile computing, could AI lead to 10X fewer people to build all the mobile apps and the underlying infra?