We all interact with systems through mental models, but if many devs are just prompting claude when something doesn't work, they might read what claude found, but lose out on the exploration, debugging, and work that builds and reinforces the correct mental model and discourages the wrong one. And if devs are missing out on the mental models, will they actually be capable of driving efficient solutions to problems as the mental models get worse.

Disclosure: I have a small investment in them.

And maybe this is just a part that the article dodges, but for something described as non-catastrophic, I would suggest that there shouldn't be a presumption of changes. In my view there should be an assessment of the risk of re-occurrence, and if it was a near miss, what is the risk had it not been a near miss. And then even should changes be part of the outcomes, how expensive are those to implement compared to whatever the issue was, and if they're too expensive compared to the risk, don't put resources into it.

In my case it was for mobile wireless signaling traffic (all the coordination for creating a mobile internet connection, handing the connection off between towers, etc), and I'd credit it as one of the reasons you're mobile internet connection is so stable. When LTE first came out, myself and many others solved all sorts of bugs in the equipment and protocols by using or building these sorts of tools.

So just to expand on why that's the case. As a net producer of energy right now, the US produces more than it consumes. This is why it get's pointed out, that as a net producer, the trade deficit with Canada is also a surplus (or reduction in deficit) on the other side with other countries. Yes it may not be the same physical barrel that is exported vs consumed, but in a fungible way they're commodities. If you separate out that portion, we (Canada) then buy more goods from the US then we sell. And that's also just the goods trade, not including services which the US also sells more of to Canada than the other way around.

Don't forget taxes (tariffs) have also increased substantially. Even if a supplier isn't paying tariff's, if their competitors are they can often just raise their prices, or their prices go up due to the supply/demand shift.

This is my understanding as well, and as I recall the results of the research were also more nuanced than most people seem to indicate. So not only were the novices with a wider range of self-estimates, the average result of those perceived scores was still below the experts. So it was never that low performers thought they were experts (although this may show up in the raw results with some portion of the population tests), just that on average they perceived their performance to be better than it actually was. And the high competence group while overestimating their results, still thought they'd do better than the low performers.

As an interviewer, I much prefer the signals generated through a work-sample interview. I'm much more confident in the hiring recommendation than I get from a 1 hour zoom session. However, if I look at teams that were built through the work-sample and zoom interviews, I'm not sure the outcomes were that noticeably better.

As an interviewee, I think I understand the frustration being on the other side has. With an in-person interview, I often have a good sense that I bombed the interview or something to improve on or replay in my head, less surprising outcomes. On the work samples it's harder to know whether you're making mistakes, or are being out-competed by someone putting in 4 times the effort to polish the solution beyond what their regular work product would be. Although I had one really good outcome where the work-sample interview really flagged the internal dysfunction of a company.

And then with both interview processes, I still think there is a really big unknown on what the false no-hire rate is, how much effort is getting wasted rejecting candidates that would actually fit the team.

So having to choose a process as an interviewer, I'm with you and would always choose a work-sample interview. On whether it should be considered the "gold standard", I'm much more hesitant, I think there are some limitations that are still hard to control for.

I do wish Starfighter/Stockfighter model had gained more traction, would've been interesting to see a recruiting company specialize in this and then seeding the interview results to multiple companies model work out.