lolwut. The open-weight models are inscrutable black boxes for which we can't possibly get real token counts? Typical lazy clanker, BSing their way out of doing the whole job.

I would have been really interested to see someone explore whether greater exposure to non-poetry AI text generalizes to greater ability to sniff out AI poetry as well, but sadly this was not that sort of study.

> rule of three, em dashes, etc

You appear to be misinformed about how Pangram specifically works, it is not based on pattern detection of that sort. I recommend reading their whitepaper, it's a pretty understandable explanation of exactly how they trained their classifier.

Even more specifically, the very fact that people would prefer, if they had the option, to use other harnesses with roughly equivalent feature sets strongly implies that the harness is not bringing them any value they couldn't get from a bunch of other places, including open-source equivalents.

Anthropic might want you to use their harness for their own reasons (control over caching, logging your interactions for training data, et cetera), but the idea that the Claude Code harness itself is bringing significant value which would help to lock users into the Anthropic ecosystem more than the Claude models alone do is kind of laughable. So _of course_ it seems like a baffling and arbitrary restriction to many users.

I'm not a huge fan of their methodology for the AI-degraded cases either (ideally one would set up the mirror pairs by taking some real repositories and rewinding history a month or so and then having a succession of independent agents reimplement each bit of feature work and bugfixes over that period of time), but it's at least a coarse approximation whereas I just don't trust the cleanup methodology to resemble anything real in the first place.

This feels like a terrible approach, sufficient to condemn the entire study.

Apparently half of the "minimal pairs" in this work were constructed in this way. I simply am not going to trust any conclusion that requires assuming these AI "cleaned" repos are in any way representative of actually-good codebases.

To be entirely honest I haven't noticed much of a capability gap between the two for the sorts of things I ask of an AI agent. Maybe Opus is _slightly_ smarter or slightly better at long-running tasks but the difference is slim enough it could just be a placebo from the Claude branding / hype.

I'm looking forward to giving GLM-5.2 a spin sometime soon and seeing how it stacks up. If nothing else 1M context is a great improvement, feels like between DeepSeek v4, then MiniMax M3, and now GLM-5.2 adding it 1M is rapidly becoming "table stakes" for agentic models.

In any case, the Pangram link was just meant as objective corroboration of what's pretty blatantly obvious if you just read the text.

"Not cloud credits, not a managed platform, not a serverless function bobbing in someone else's abstraction."

"I used to work at Heroku. That sentence still does a lot of load-bearing work in how I think about computing."

"Here's the part that would have sounded like science fiction during my Heroku years: I didn't do most of the migration."

If you read these chunks of text and don't immediately feel the AI slop alarms blaring in the back of your head, you are perhaps underprepared for the modern internet.