Hacker News: pyentropy

New comment by pyentropy in "Ask HN: How are thinking efforts implemented?"

pyentropy — Sun, 07 Jun 2026 21:40:15 +0000

I'm considering the possibility that it's good to break the prefix and cache because the LLM itself was rewarded (during post-training) with different prefixes/system prompts, each containing reasoning traces of the correct size.

I might be very very wrong though and LLMs disagree with me, insisting that cache is preserved and the system message doesn't have to change (even though it often contains effort level in context) if effort level changes across turns, and that all you have to do is tell the inference lib that parses think tags to early-close think tags that are too long.

New comment by pyentropy in "Ask HN: How are thinking efforts implemented?"

pyentropy — Sun, 07 Jun 2026 21:09:55 +0000

Examples with inference of different reasoning effort levels is in the OpenAI docs as well - https://developers.openai.com/cookbook/articles/openai-harmo...

https://docs.vllm.ai/en/latest/features/reasoning_outputs/#a...

https://developers.openai.com/api/docs/guides/reasoning

New comment by pyentropy in "Ask HN: How are thinking efforts implemented?"

pyentropy — Sun, 07 Jun 2026 21:09:02 +0000

LLM-judge/parallel branching ≠ multi-token prediction ≠ reasoning effort.

See https://developers.openai.com/cookbook/articles/openai-harmo... and src/openai/types/shared/reasoning_effort.py

New comment by pyentropy in "Ask HN: How are thinking efforts implemented?"

pyentropy — Sun, 07 Jun 2026 16:38:10 +0000

The number of tokens you predict at time (multi or not) has nothing to do with whether the model wants to emit any, some or a lot of reasoning tokens in reasoning tag -- similar to how branch prediction will not really change the for loop iteration count.

New comment by pyentropy in "Ask HN: How are thinking efforts implemented?"

pyentropy — Sun, 07 Jun 2026 15:48:15 +0000

Take a look at the harmony repo which specifies the internal OpenAI format - the effort level is specified in the context after the <|start|> tag - https://github.com/openai/harmony

Note that inference libs also have parsers that put hard limits on reasoning tokens with separate counters (similar to how you can put a limit on token generation per completion versus waiting for an ). For that, take a look at vllm reasoning docs.

Dancing to the State of the Art? How candidate lists influence LKH TSP solvers

pyentropy — Thu, 28 Aug 2025 10:40:21 +0000

Article URL: https://arxiv.org/abs/2407.03927

Comments URL: https://news.ycombinator.com/item?id=45050543

Points: 1

# Comments: 0

New comment by pyentropy in "Nvidia’s $589B DeepSeek rout"

pyentropy — Tue, 28 Jan 2025 09:10:59 +0000

If H800 is a memory-constrained model that NVIDIA built to avoid the Chinese export ban on H100 with equivalent fp8 performance, it makes zero sense to believe Elon Musk, Dario Armodei and Alexandr Wang's claims that DeepSeek smuggled H100s.

The only reason why a team would allocate time on memory optimizations and writing NVPTX code rather than focusing on posttraining is if they severely struggled with memory during training.

I mean, take a look at the numbers:

https://www.fibermall.com/blog/nvidia-ai-chip.htm#A100_vs_A8...

This is a massive trick pulled by Jensen, take the H100 design whose sales are regulated by the government, make it look 40x weaker and call it H800, while conveniently leaving 8-bit computation as fast as H100. Then bring it to China and let companies stockpile without disclosing production or sales numbers, and have no export controls.

Eventually, after 7 months, US govt starts noticing the H800 sales and introduces new export controls, but it's too late. By this point, DeepSeek has started research using fp8. They slowly build bigger and bigger models, work on the bandwidth and memory consumptions, until they make r1 - their reasoning model.

Why do I have a blog (and has it ever paid off?)

pyentropy — Fri, 02 Aug 2024 01:08:47 +0000

Article URL: https://fikisipi.substack.com/p/why-do-i-have-a-blog-and-has-it-ever

Comments URL: https://news.ycombinator.com/item?id=41135233

Points: 1

# Comments: 1

New comment by pyentropy in "Why haven't biologists cured cancer?"

pyentropy — Sun, 07 Jul 2024 11:26:24 +0000

You should start a blog... or maybe not - pursue the battle in academia/work and occasionally drop nuggets of wisdom like this somewhere. But do not delete them.

Busy Beaver, the current BB(5) conjecture and bbchallenge.org

pyentropy — Mon, 24 Jun 2024 20:10:43 +0000

Article URL: https://fikisipi.substack.com/p/busy-beaver-the-current-bb5-conjecture

Comments URL: https://news.ycombinator.com/item?id=40780146

Points: 1

# Comments: 0

New comment by pyentropy in "Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?"

pyentropy — Thu, 13 Jun 2024 22:23:03 +0000

I updated the post with a a link to counter-argument from Sabine Hossenfelder, the arguments from Zvi and three points from my side.

New comment by pyentropy in "Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?"

pyentropy — Thu, 13 Jun 2024 22:22:09 +0000

I updated the post with a a link to counter-argument from Sabine Hossenfelder, the arguments from Zvi and three points from my side.

New comment by pyentropy in "Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?"

pyentropy — Thu, 13 Jun 2024 16:18:45 +0000

Scott worked at OpenAI Safety and he likes it: https://scottaaronson.blog/?p=8047

But is the "-ed" in worked a problem?

New comment by pyentropy in "Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?"

pyentropy — Thu, 13 Jun 2024 16:17:52 +0000

Thank you.

New comment by pyentropy in "Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?"

pyentropy — Thu, 13 Jun 2024 16:17:33 +0000

It is a question. I tried to put what my opinion is on a few statements but I absolutely cannot summarize 160 pages (Business Insider did using GPT, which I find insulting and funny) nor have a 100% opinion on something that involves national security, secrets and other stuff that I don't have access to.

Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?

pyentropy — Thu, 13 Jun 2024 09:20:52 +0000

Article URL: https://fikisipi.substack.com/p/is-aschenbrenners-165-page-paper

Comments URL: https://news.ycombinator.com/item?id=40667545

Points: 44

# Comments: 68

Short post: A look at Devin, the AI-powered software engineer

pyentropy — Wed, 13 Mar 2024 01:10:25 +0000

Article URL: https://fikisipi.substack.com/p/short-post-a-look-at-devin-the-ai

Comments URL: https://news.ycombinator.com/item?id=39686936

Points: 4

# Comments: 0

My 2023 prediction mistakes and the new Metaculus scoring function

pyentropy — Mon, 08 Jan 2024 16:26:26 +0000

Article URL: https://fikisipi.substack.com/p/my-2023-prediction-mistakes-and-the

Comments URL: https://news.ycombinator.com/item?id=38914346

Points: 2

# Comments: 0

New comment by pyentropy in "In the long run, we're all Dad"

pyentropy — Fri, 22 Dec 2023 13:40:44 +0000

You haven't read Scott's blog enough :)

He's an atheist psychiatrist. However, he enjoys how natural selection, social dynamics and reputation can also be modeled by the moral rules of most religions. For example, going to therapy isn't that different from practicing confessions in a church.

Time.mk: disrupting the Macedonian online media using clustering algorithms

pyentropy — Tue, 19 Dec 2023 23:11:19 +0000

Article URL: https://fikisipi.substack.com/p/the-man-that-disrupted-macedonian

Comments URL: https://news.ycombinator.com/item?id=38703249

Points: 2

# Comments: 0