New comment by yashchimata in "Kimi K3, and what we can still learn from the pelican benchmark"

yashchimata — Fri, 17 Jul 2026 18:48:34 +0000

One thing i keep thinking: you only run the pelican once per model. Run the same model a few times and you get some different pelicans, so some of "this one is better" might just be which run you picked for it. Would love to see 8 runs per model side by side. I bet for two close models, the gap between runs is about as big as the gap between the models.

Hacker News: yashchimata

New comment by yashchimata in "Kimi K3, and what we can still learn from the pelican benchmark"