It's really tiring to have to tweak everything with each model release and then watch those changes mess up cheaper models in the process.

Now I can't wait for someone to distill K3 into a Qwen 3.6 27b or Poolside S 2.1 sized models for a proper fast local Composer 2.5 replacement.

I'll never get why he thinks China would just sit there and let the US dominate them in AI when all it would take is a few of their boats blockading Taiwan to put a stop to it all.

We had a bunch of things that we never published that ended up being major research findings years later at top conferences.

There's no chance K3 is a distill of Fable, it came out way too soon after the limited fable release to be feasbile.

If you look at all of the top ML conferences, chinese labs contribute way more to advances in ML than "Open"AI and Anthropic: https://www.reddit.com/r/TheMachineGod/comments/1pi4q7f/pape...

This K3 release just helped every other lab on the planet stay in the race by making it possible for them to build on top of it, placing them at the frontier starting line instead of having to spend billions of their own dollars and risking it all to attempt to catch up.

The open source contributions I linked to above will move the whole field forward and reduce the costs of training and inference for everyone.

Open science compounds on it self, every new advancement pushes the field forwards and opens up new grounds for future improvements.

It is impossible for a single closed lab to consistently stay ahead of the rest of the field, especially in a huge growing research area like machine learning. The only only advantage the big labs have is money, but the naive scaling game is not sustainable long term when you have to pay 10-100x more then the fast followers and we start getting more and more open models or use case specific models that can handle 90% of high volume use cases.

Research is a high variance, low expected value activity, meaning that the few large concentrated labs have to be conservative with their bets and double down on proven things when scaling up. The rest of the field is like a diversified portfolio, with thousands of players making smaller riskier bets that only require a few of them to succeed (like K3 did here, and DeepSeek a year ago)

EDIT: also if you look at most of the work from OpenAI, it's mostly taking existing promising open research work and scaling it up. (except for things like CLIP and etc from Alec Radford)

Anyone who claims open source and open weights models are "decel" needs to get their head checked

https://github.com/MoonshotAI/MoonEP

https://github.com/kvcache-ai/AgentEnv

https://github.com/MoonshotAI/FlashKDA

They definitely used closed private saas products to train their own models, to prove that just drop random small screenshots of any popular product behind a login screen and see how well it's able to identify all of them. ex: https://x.com/michalwols/status/2079968211865330165

or other similar "AI" startups https://x.com/envconfig/status/2079613455296827402

They're very well equipped to be the volume discount store of inference.

we'll get new quants, dspark speculators, distills and optimized kernels

as long as there are near frontier models available there will be inference providers selling them at or below cost of inference in attempt to get market share.

Even now on the $200 plan I use up my Fable credits in a single day and had to start using codex and openrouter for more usage because Fable burns $100s an hour when billed on usage.

Even their hail mary of being first to "AGI" will never happen because all it takes is China blockading Taiwan or Nvidia cutting them off to stop them from eating up a large chunk of the economy.

There is no scenario where the rest of the world will sit on their toes and let OpenAI or Anthropic monopolize "AI". Too many countries, large well capitalized players and partners / suppliers who could never let that happen.

Kimi K3 allows all existing players to restart at the frontier and keep competing with OpenAI/Ant. It also gives employees at these labs a better more lucrative path of starting new labs with fresh books and clean cap tables, building on top of K3 without needing to spend all the capex on pretraining their own models. Plenty of them already vested their stock and would have 0 problems raising 100s of millions of dollars for new labs, making them paper billionaires over night.

main problem is that the harness files get loaded into context early in the session and slowly wash away as new information comes in.

My second point is that the models are way better at things like Rust or Lua than Python or JS, because the average person producing code in those languages has way more programming experience, so the code quality of training data online in those languages is higher than the programming 101 medium blogspam type content you see in more popular intro languages.

So to make best use of the models steer them down familiar paths, mention common pattern and frameworks, use popular packages and languages that have the high median quality online.

I started my project with a few simple interface definitions and a short design / architecture doc that I include in the AGENTS.md file, but no matter how hard I try all of the models just end up ignoring it and sprinkled new seeds of variants of the same stuff all over my code base, that with each new session grow new branches.

Databricks actually just posted some of their own benchmarks on how harness alone impacts costs https://www.databricks.com/blog/benchmarking-coding-agents-d...

simple things like passing more file context, model having to explore the code base at start of each session, writing comments or markdown docs ends up increasing, running into test / build issues can 3-10x your costs.

PS: my code is still mostly TS and rust but I'm considering moving some of my annotations into .d.ts files and having them generated from runtime types (ala MonkeyType).