Comments URL: https://news.ycombinator.com/item?id=48587146

Points: 5

# Comments: 1

Comments URL: https://news.ycombinator.com/item?id=48582061

Points: 2

# Comments: 0

What are you using Claude models for? Coding only? Computer use? Which harness?

It avoided answering 2/21 tests in this specific benchmark mark, that's already 90% max score already.

[0]: https://aibenchy.com/?q=glm

This means, that models are losing more and more general and domain-specific knowledge.

Look at those graphs on ARtificialAnalysis, GLM-5.1 still performs similarly or better:

AA-Omnisicence Accuracy: https://i.snipboard.io/5DYmpx.jpg

IFBench: https://i.snipboard.io/74kg0R.jpg

I still feel like models are not getting any smarter for a few months already, they just changed their training to be focused more on some areas than others, so shifting the intelligence from one place to another, not necessarily increasing the overall intelligence or "AGI" score.

GLM-5.2 is already close to Opus-4.7 level:

https://aibenchy.com/compare/anthropic-claude-opus-4-7-mediu...

[0]: https://aibenchy.com/compare/deepseek-deepseek-v4-flash-high...

Sure, you don't need to read an entire book to get the idea to do something, but that's way in which ideas get into your head and consciously or not, guide your steps in certain directions.

Of course, everything in self-help books should be taken with a grain of salt and also applied differently for each individual. It's just a way to get more ideas of what's possible, similar to travelling: it doesn't force you to do things in a certain way, it just shows you what's out there.

I thought that was the code harness simply minifying the outputs. Many models now no longer return the entire chain-of-thought (to avoid distillation attacks). So yes, we don't get the raw LLM output, but I think it's just the thinking summarized, not a complex orchestration or different models.

I do agree though that now cloud models are kind of a black box, that's not only obfuscated but also changes over time. Companies seem to be changing model capabilities without notifying users, or even hiddenly serving completely different models. This is even worse via OpenRouter, with providers serving open-source models, some of them serve heavily quantized versions or even completely different models.

I don't understand, why does it make you think this is the case?

> how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself

Can you give an example?

https://aibenchy.com/compare/moonshotai-kimi-k2-6-medium/moo...