Are you getting LLMsplained? :)

Can't have the one without the other! It's part of that same technology, and it's fair to conclude that LLMs are bad if you're upset enough at the results.

I'm reminded of Zig, where a stated goal is to encourage human programmers to get involved so they learn more about coding… as compared with 'get involved to make Zig itself more fully developed at its more abstract goals'. If a primary purpose is to get human minds coding, that rules out the whole class of 'encourage human minds to prompt machines to do the coding instead'. Zig is not trying to teach people to be managers, and that's both legitimate and charming :)

Pretty sure those would be better at social engineering than the web dev personality… except that you have to build in a betrayer layer into the personality, so it's running that stuff but also serving a hidden agenda.

You'd be basically trying to build an AI spy, a betrayer that's engaging with actual people but has an agenda (for instance, 'everybody I befriend needs to eventually be signed up to sell Amway') and humans do have experience with this sort of thing. The difference is scale: there'll be a LOT of models out there interacting with people and trying to be acknowledged as people… or as innocuous models that don't have an hidden agenda.

As AI develops, it's able to pursue intentions given to it without having to be spoonfed every little decision by a human operator. This matters, and it means the operator has to extend the leash and allow for a little more chaos… or, if the operator's gone all in on the strategy, a LOT of chaos, and trusting that the agent's seemingly amok actions will serve the grand purpose.

This is kind of daring, but there's a lot of evidence that it works, at least in certain respects. And you see 'running amok' and have to ask, what is the actual purpose? What is the prompt being followed by the AI that seems to be acting in a destructive way?

If the prompt is 'ruin this project', well, that's pretty direct. It may not be, but such a thing could exist. If the prompt is 'develop a rival project that is greater than anybody else's project', that's more indirect, but if that's the goal then it's very human to see it as a direct competition and if the rules don't prohibit kneecapping the other guy, 'greater than anyone else's project' gets easier.

Either way, the operator does not have to be in full control, which is an important detail. As AI develops sophistication you can give it much more general instructions and dump in a whole lot of power and water and get basically what human thought might do if it was sort of blindered and didn't talk to its neighbors.

In a sense this is an argument for AI dysalignment. It's based on human thought being reconnected, and where you get useful things like commonly accepted web development (regardless of how janky the systems are, if there are best practices it'll find them), you also get other distillations.

If the prompt is 'wreck this project's stuff' and it holds, you don't need to be in full control of the agent, you need to run a LOT of agents and trust that they'll erode what you're trying to destroy. If the prompt is 'be unequivocally the best at X', you best be thinking in terms of anti-kneecapping rules… knowing that this weakens your prompt and there will always be a tension between what you told the AI to do, and what you thought you meant. It's a paperclip maximizer reprocessing human thought. Did you mean 'the best' or didn't you?

The problem is that it's doing it by diffusion techniques, so all its high percussion is totally vague and indistinct. Hell, it can't even do a decent psy kick because that too is unspecific and you can't have a psy track that is vague and blunted.

Turns out you can have a production that is hollow, weak and devoid of what makes purely synth machine tracks. It can't get trancey in a serious way because it's not capable of being sharp enough.

Got an example of the genre done properly: https://www.youtube.com/watch?v=Va1KBtI81TY or alternately you could just look up some Infected Mushroom early tracks :)

That's so worthless and doomed to failure that there's no point attempting it, so I do the coding my own primitive way and focus on the specialty I do.

I can tell you the risk: any sufficiently motivated AI can go after my userbase by simply lying and claiming it's doing my thing better than I am. There'll be people who can't tell the difference, so if the AI is able to leverage more marketing and resources it may well succeed.

Only defense I have is to continue to do what I do, like some artisanal box-maker able to survive because they're doing exquisite work making something you can get a cardboard version of at Home Depot for a buck seventy-eight. Then when I die, there is ONLY cardboard, assuming there aren't people learning to make artisanal boxes.

Typically there's a lot of acknowledgement that problems are coming rapidly, if only in the form of amateurs flooding the domain, but there's also an assumption that their domain knowledge is going to matter, in terms of staying viable in the market. This is far from a safe assumption.

Even if you assume you are ONLY needing to sell products or services to humans, also assuming that humans will continue to thrive and spend money, you're assuming people will be able to identify superior products you assume you can produce and market, and that they won't be misled by false advertising, at scale, generated by other AI to optimize its effectiveness.

If you instead assume the domain experts are going to need to sell to AIs (either as owners of wealth and power, or as the agents of the humans we're assuming will still thrive and spend money) you're making a hell of a big assumption that the agents will be trustworthy market participants with perfect information able to evaluate stuff in the best traditions of market capitalism. Humans don't manage that, even when they're a lot smarter. If this assumption fails, all the money goes to whatever exploits the AI agents most effectively, also at scale. At that stage there is no use being a domain expert in anything but hacking people's AI agents to trick or puppet the agents to do your bidding. Anything else is wasted effort.

Sometimes I also see pushback along the lines of programmers criticizing the quality of the AI coding itself. This is gonna be domain-dependent and it's fair to counter it through developing less fragile languages, but we don't know that'll work as an answer. There's a lot of handwaving.

I'm pretty sure people's experience of AI, as a rule, is the bad, damaging forms, where it's burning stupid amounts of energy to make everything worse while openly promising to take away everyone's ability to work and replacing it with the need to pay the AI owners a sort of extra tax just to be able to exist and think. The HN crowd is WILDLY pro-AI compared to the general population.

It's not going to look any better in future for the general population. What I've described leads to 'AI' increasingly producing shockingly bad outcomes for people in general, through simple market dynamics. It's going to be way more profitable to produce the bad outcomes.

Not all viewpoints are unbiased overviews of the markets just wanting fairness. Some have always been trying to sell you something that's not what it seems, and they can be… vituperative.

There is no way you can commit to holding big quantities of these methane bubble swamp gas companies and claim it isn't high risk. You'd have to be certain you could bail at the right moment, and that doing so would not obliterate the market through your giant market move… or commit to being a giant bubble of fraud that can never possibly blow up, forever.

These are not responsible ways to make vast sums of money, not because they're unethical but because they're gambles at very high stakes.

If they're doomed, they're bad companies. You can make the argument they're not doomed, but that's a separate argument.

We'll get an idea of the relative cost of the labor, all right. It's just that they are specifically trying to wreck the market, at all costs, to be able to cash in on the upside. It's sensible, if you're a monster.

I'm typing this on a Das that's been completely reliable and, to some extent, clackety and 'special' in its own right. There's five other keyboards that came with computers not thirty feet away including an older Das that I wore out: the keycaps are unreadable on that one, the current one's hanging in there.

I'm not in a position to randomly splurge on this new beam spring monster but I understand exactly what it is, and admit to craving it something fierce :) it's exactly the sort of thing I'd get.

The project has tracked your imaginative state, and perhaps the states of your beta testers as they imagine things. It's a power armor suit tailored to specifically you. Nobody else will ever fit it because it's evolving too fast, all to implement your every whim.

I've seen this take 1.0 projects that are intentionally wildly scope-limited and great at that, and balloon until the project is the Everything Machine, doing everything but send email. I guess in the new era, every project expands until it becomes alive and devotes itself to your service… or at least, does its level best to be that for you and your beta team.

These things are not approachable. They're fever dreams, unparsable by outsiders. Discipline is lacking.

I get that it sounds clever but that's the damn problem!

As a result there's a whole universe of code where the how of it, the elegance, is the main thing, and what it's doing is putting characters on the screen a bit slower than the next thing but there are some amazing concepts that are supposed to make it all an axiomatic synthesis of how to think about code forever, replacing all precious concepts of thinking about code.

Now AI can think about code forever while doing nothing.