V4-pro in particular seems very capable, but will just dramatically completely misunderstand user intent, it seems almost like it wasn't trained at all on non LLM generated instructions mid conversation.

No really, that's not particularly accurate, they use so much gig work because no-one else wants to work for them not because they would be unwilling to pay a little extra, or only want the absolute cheapest labor they can get on the planet.

They want senior white collar professionals and scientists and researchers especially since these companies already on some level believe their models are as good as any senior employee in any field (it's probably the models generating text saying that, but that's besides the point). But who's going to work on contract for a company that wants to automate them out of a job? Realistically no-one unless they get some shares in the thing that will destroy their future earnings potential and ability to control their own destiny if it works out.

But they can find enough educated white collar professionals on unemployment or in unstable academic employment that will take an extra job on even if it's only 50 $/h or 70 $/h and compromise on any solitary they might have but the work output you get from that is only going to be as good as what you ask for, if they had better respect for the professions they want to automate, it would be better.

Like is that an acceptable wage in the US for difficult skilled work, not particularly but it's not rock bottom exactly, and it's not bad for other English speaking countries, working conditions and stated mission are more of an issue than being cheap.

Training pipeline on a modern LLM is also going to be quite indirect during the long tail of post training, and heavy on automated RL, the human feedback might end up getting used in the form of automated grading guidelines like what you did for research, with the same issues as that, compounded by the input being LLM generated and models being biased towards model output by default. It's more of a feedback on the loop rather than in the loop.

I think if changes will happen in this direction it will be around formal verification, it's more difficult to trust LLM generated code if you aren't completely brainroted on press releases, and traditionally formal verification was seen as more effort than it's worth, but a tool that's great that generating insane amounts of text is quite well suited to formally verifying code, and assuming you can figure out how to make the specification readable to people, you just don't (in theory) need to ever look at the code or the proof, even if it might be helpful to anyways in practice. I'm quite excited the future of software might be fully unhackable software being the standard, with hacking becoming something that's talked about as a brief criminal fad in the beginning of the 21st century, like how you might hear about coin clipping from before fiat currency.

Of course we should really be talking about using the state or otherwise to make training larger and larger models impossible. It's not in the public good if LLMs actually get good enough to replace a lot of human labor, only a small handful of billionaires and their cronies will ever benefit from that. The Luddites were not wrong after all.

On your right you can see the CEOs justifying longer hours and lower pay because AI will replace your job one day anyways, and then asking you why you aren't 10x more productive with Claude. On the left you can see the AI companies deciding who will be in charge of the fascist regime once they no-longer need workers other than for the coal mines. They reckon they can get 120 good years before they biosphere is uninhabitable, which they are worried about because what if the next LLM figures out immortally for them, maybe they will have to close the coal mines too after all.

The realistic scenarios are still pretty bad but less so.

Just assuming LLMs will scale to the point of being more useful as all these companies are banking on is not unfounded but it's also just an educated guess because of their own optimistic prediction they will be able to bring about a new fascist world they are in charge of before they run out of money. It's very likely we are all screwed but it's also very likely LLMs will not scale up fast enough so they can replace the majority of human labor.