Hacker News: woctordho

New comment by woctordho in "CUDA for AMD on Windows"

woctordho — Mon, 14 Sep 2026 06:10:10 +0000

No, 10.1 is current

New comment by woctordho in "Detecting and countering misuse of AI: September 2026"

woctordho — Fri, 11 Sep 2026 05:39:08 +0000

Let me put my two cents: In China we've got accustomed to the fact that every word we say will be seen by the surveillance, so it's not a big problem that Anthropic also see it. Also we know that they can see it but they can't stop it. There are all kinds of ways to work around account blocking.

As the old saying goes, communists disdain to conceal their views and aims.

New comment by woctordho in "Qwen 3.8 follows GPT-5.5 Pro reasoning prefills"

woctordho — Wed, 09 Sep 2026 21:35:19 +0000

Reasoning works as long as there is a consistent latent space representation. Any kind of poison will just become part of the representation. There's evidence that even directly training on encrypted reasoning traces works, because the length is already a strong signal.

New comment by woctordho in "Path to Astra: critical capabilities and frontier safeguards"

woctordho — Wed, 02 Sep 2026 23:44:01 +0000

There are 'transfer stations' and that's how exactly I use GPT and Claude in China. OpenAI and Anthropic do not sell in China, so we use their AI with a much lower price like 1% of the official API price. The largest transfer stations have TBs of traffic every day, and the traffic is eventually possessed by the open source community.

Subscription engineering is a deep field. Neither OpenAI nor Anthropic have any technical advantage in this field.

New comment by woctordho in "Path to Astra: critical capabilities and frontier safeguards"

woctordho — Wed, 02 Sep 2026 11:02:48 +0000

All the RL data are exactly public. There are huge amount of distilled data freely available, and that amount is more than enough to train a ~10T model.

New comment by woctordho in "Seedance 2.5"

woctordho — Sun, 02 Aug 2026 00:58:50 +0000

MiniMax H3 is going to release weights. You can locally run it with definitely less than $10k (and possibly faster than Seedance's queue), and it's fun to train it for whatever you need.

New comment by woctordho in "Kimi-K3 on HuggingFace"

woctordho — Mon, 27 Jul 2026 14:59:37 +0000

GGUF is at least better than bnb. From what I know, bnb does not yet find a way to quantize MoE with enough accuracy, and maintain the dequant-MoE kernels. In the age of Qwen 3.0, people tried to make some bnb '4-bit' quants of MoE models, but actually the MoE part is not quantized. It's a pity that even Unsloth gave up low-VRAM finetuning with MoE (although they're making their GGUFs for inference), and the world of local training looks stagnated for months.

GGUF is maintained by all the llama.cpp developers. There are many quantization formats and algorithms under this container format, some are optimized for MoE (such as APEX quant), some for CPU and some for GPU, some work surprisingly well below 4-bit (and even near 1-bit). It also supports recent architectures like linear attentions and mHC.

New comment by woctordho in "Kimi-K3 on HuggingFace"

woctordho — Mon, 27 Jul 2026 07:45:44 +0000

Speaking of finetune, currently a common practice is LoRA over bnb 4-bit base model, but I think it's time to replace bnb with GGUF as the base model format. GGUF is actively supporting new model architectures and more aggressive quantizations.

I've made some proof of concept in https://github.com/woct0rdho/transformers5-qwen3.5-recipe . We can finetune Qwen3.5-35B-A3B in 16 GiB VRAM, and DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, without CPU offload. This works well on unified memory machines like Strix Halo.

Even so, larger models like Kimi-K3 still require multiple GPUs and nodes, and there are a lot more to do compare to single-GPU training.

New comment by woctordho in "Petals: Run LLMs at home, BitTorrent-style"

woctordho — Thu, 23 Jul 2026 04:23:33 +0000

Distributed training is much harder than distributed inference but not impossible. See the recent development of DiLoCo at Nous Research and Prime Intellect.

New comment by woctordho in "Petals: Run LLMs at home, BitTorrent-style"

woctordho — Thu, 23 Jul 2026 02:36:11 +0000

Relevant: Why Switzerland has 25 Gbit internet and America doesn't https://news.ycombinator.com/item?id=47652400

New comment by woctordho in "Petals: Run LLMs at home, BitTorrent-style"

woctordho — Thu, 23 Jul 2026 02:34:00 +0000

AI Horde has some measures to prevent Sybil attack that returns wrong results, but not enforce zero data retention. Prompts belong to the whole open source community. For example https://huggingface.co/datasets/la-ji/sd-prompt-in-the-wild

New comment by woctordho in "Petals: Run LLMs at home, BitTorrent-style"

woctordho — Thu, 23 Jul 2026 02:31:30 +0000

Petals is from 2022. Nowadays intelligence of smaller models, quantization techs, and optimizations to run models faster on consumer GPUs have improved a lot.

For distributed inference of smaller LLMs and diffusion models that fits in one consumer GPU rather than splits on multiple machines, there are already pretty good solutions such as AI Horde (formerly Stable Horde) [0]. Notably, it's the default provider that powers SillyTavern. It also has an interesting economy model of kudos.

[0] https://stablehorde.net/

New comment by woctordho in "China’s open-weights AI strategy is winning"

woctordho — Wed, 22 Jul 2026 09:26:19 +0000

So is making a PR different from making the whole software. This is what an open source community is good for.

New comment by woctordho in "China’s open-weights AI strategy is winning"

woctordho — Tue, 21 Jul 2026 11:17:03 +0000

See the recent development of DiLoCo at Nous Research and Prime Intellect.

New comment by woctordho in "China’s open-weights AI strategy is winning"

woctordho — Tue, 21 Jul 2026 11:15:32 +0000

There's a lot of individual effort of improving the models. See how many finetuned models and LoRAs are there on Hugging Face.

New comment by woctordho in "Qwen 3.8"

woctordho — Tue, 21 Jul 2026 10:55:24 +0000

There is a forum named Zhihu. AI translation works mostly well to translate contents there into English.

New comment by woctordho in "Kimi K3: Open Frontier Intelligence"

woctordho — Fri, 17 Jul 2026 04:10:33 +0000

Yes in a mid-sized company. I'm exactly doing this, and what I'm competing against is the OpenAI API priced 0.2 CNY = 1 USD in China.

New comment by woctordho in "Alternative(s) to run CUDA on non-Nvidia hardware"

woctordho — Tue, 14 Jul 2026 09:41:43 +0000

There's nothing wrong to run CUDA on non-Nvidia hardware. CUDA has an interface that is reasonably well-designed, well-documented/reverse-engineered, and battle-tested for decades. What we need is not to invent another interface just under the name of 'open standard', but to implement the same interface. ROCm is exactly doing this, and so are other hardware SDKs such as MooreThread and Alibaba T-Head.

New comment by woctordho in "DSpark: Speculative decoding accelerates LLM inference [pdf]"

woctordho — Sat, 27 Jun 2026 12:06:19 +0000

And humans don't run on markets.

New comment by woctordho in "The gap between open weights LLMs and closed source LLMs"

woctordho — Sat, 27 Jun 2026 06:54:07 +0000

Fun fact: Hacker News is canonically banned in China, but I'm still talking here. There are plenty of techs to work around region block. The incentive to report somebody is comically called '50w' (500k CNY) and no one gives a shit about it in real life.