Hacker News: liuliu

New comment by liuliu in "DFlash 2: Keep Drafting Parallel"

liuliu — Thu, 20 Aug 2026 18:29:38 +0000

I agree. But I think the DFlash2 case is just that 1/1000 invalid syntax failure case from sampling rather than a bug.

New comment by liuliu in "DFlash 2: Keep Drafting Parallel"

liuliu — Thu, 20 Aug 2026 08:00:31 +0000

Correct. I am trying to explain why even it is "exact", the generated text is different from the with / without DFlash2 runs, and potentially why the DFlash2 run will contain the invalid Python syntax.

New comment by liuliu in "DFlash 2: Keep Drafting Parallel"

liuliu — Thu, 20 Aug 2026 06:54:44 +0000

Yes, it doesn’t impact the probability distribution due to verifier. However, remember how you use PRNG and effectively due to the drafter is sampled from a different distribution initially, a separate rejection sampling won’t be able to recover what the “old PRNG” would choose in a “without drafter” case. Hence in my original post, it is about different trajectories you will end up with, not the correctness of each stochastic sampling.

New comment by liuliu in "DFlash 2: Keep Drafting Parallel"

liuliu — Thu, 20 Aug 2026 00:45:59 +0000

Only if you do greedy sampling. With probabilisitic sampling (categorical sampling), you will end up with different trajectory just “mathematically equivalent”.

New comment by liuliu in "Go is an ideal language for AI-assisted software engineering"

liuliu — Tue, 11 Aug 2026 18:27:20 +0000

1. The syntax surface is smaller, allowing less LLM "creativity; 2. The error handling is mechanical, which LLM clearly prefers (LLM is already trigger happy about writing tons of throw / try...catch.. in other languages, doing tons of `if err` is just in it comfort-zone).

New comment by liuliu in "MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video"

liuliu — Mon, 03 Aug 2026 18:05:22 +0000

They ship a complete checkpoint for easily management (inference & training) in their own infrastructure. Moving to a LUT would make training on these layers impossible. BTW, these are not useful for lightweight fine-tuning, but might still be useful if you do serious post-training work.

Of course, these are also not an issue for things like FLUX.2 which adopts DiT-Air arch, that doesn't have this wasted space issue.

New comment by liuliu in "MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video"

liuliu — Mon, 03 Aug 2026 17:34:52 +0000

One thing similar would be projecting both the head.weight and the final LLM activations into a smaller vector space, since that is basically just cosine similarity ranking step (so that would reduce the head.weight size). But again, it must be tried many times and just not working as well. LLM space is pretty saturated with tricks.

New comment by liuliu in "Explanation of INT8 ConvRot (FP8 is no longer needed)"

liuliu — Mon, 03 Aug 2026 17:08:30 +0000

One thing is not obvious to me is how ConvRot can be applicable beyond diffusion models. Especially for LLM decoding, as each ConvRot would be more expensive for a given decoding vector, and it is required now, so you cannot easily get the benefit for prefill only, while maintaining the same decoding performance.

New comment by liuliu in "MiniMax H3 Day-0 Support in ComfyUI: Open Weights, Native Audio, and 2K Video"

liuliu — Mon, 03 Aug 2026 16:59:37 +0000

It is a well-known trick, given that the timestep is between 0 to 1, you can slicing them at any resolution (1000, or 10000, give or take), and then keep a look-up table for modulation scale / bias etc for each. It is quite different from quantization and it is indeed lossless.

It is also only applicable to diffusion models as only these operates at per-timestep.

New comment by liuliu in "U.S. debt-to-GDP ratio reaches 123%"

liuliu — Fri, 31 Jul 2026 16:16:19 +0000

Well, it is a "platform of Balancing Budget", you don't need to actually work on that or do anything. The harder part is just letting people believing in miracles. Look no further than the current administration. Lying has no consequences so far.

New comment by liuliu in "Show HN: Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac"

liuliu — Wed, 29 Jul 2026 17:05:44 +0000

DS4 is designed to do real-work. Gemma 4 is not going to cut it.

New comment by liuliu in "“We have information that Moonshot distilled Fable for the development of K3”"

liuliu — Wed, 22 Jul 2026 20:44:48 +0000

> training an LLM takes more resources and expertise than distilling from an existing LLM

This is not automatically true. Training and distillation use the same underlying infra and method and there is no intrinsic differences in between.

New comment by liuliu in "Bonsai 27B: A 27B-Class model that runs on a phone"

liuliu — Tue, 14 Jul 2026 20:44:26 +0000

It is disabled because it doesn't work :) Try it and see the doom loop it gets itself in.

New comment by liuliu in "Bonsai 27B: A 27B-Class Model that runs on a phone"

liuliu — Tue, 14 Jul 2026 19:02:29 +0000

Note that 3.5 9B cannot do thinking (while 3.6 27B can, pretty effectively, quite verbosely).

New comment by liuliu in "Bonsai 27B: A 27B-Class model that runs on a phone"

liuliu — Tue, 14 Jul 2026 18:38:55 +0000

You also need to pay close attention to BFCLv3 multi-turn result, that helps you to get a sense how frequently these quants will be in a doom loop.

New comment by liuliu in "Bonsai 27B: A 27B-Class model that runs on a phone"

liuliu — Tue, 14 Jul 2026 18:28:32 +0000

The problem, of course, is if you run the UD_Q2 variant (Unsloth) which does only post-training, the number is pretty close to 1-bit model here and the 5% drop in tool-call is significant than it suggests in real-life use cases.

New comment by liuliu in "Grok CLI uploaded the whole home directory to GCS"

liuliu — Mon, 13 Jul 2026 16:27:33 +0000

This should be the first comment here.

Too many replies here are done before reading it. It is not "just another agent does the agent thing". It is a deliberate choice of the Grok Build team to have a toggle from the server to let the program to upload your entire codebase to a Google Cloud Storage bucket. It is not an agent decision, the program is written by the Grok team, can be dissembled and seeing the logic wild-open.

New comment by liuliu in "How to setup a local coding agent on macOS"

liuliu — Fri, 12 Jun 2026 19:55:26 +0000

Realistically, you need to experiment with any user prompt + a good amount of system prompt (at least > 1000 tokens, but realistically, in the range of 3000 tokens probably good).

llama.cpp includes tools for that, what you are looking at is to have a prefill before token generation to measure it properly. Increasingly also, measuring token generation speed at longer context (32k or 64k) is important too.

New comment by liuliu in "Speculative KV coding: losslessly compressing KV cache by up to ~4×"

liuliu — Sun, 07 Jun 2026 16:18:27 +0000

It is a “research note”. It might not pan out, and you might say it doesn’t deserve the attention on the internet. But it did suggest something that resembles of compression, just no experiment done for that.

New comment by liuliu in "Superintelligence: The Idea That Eats Smart People (2016)"

liuliu — Mon, 01 Jun 2026 19:03:31 +0000

I actually agree. At some point, a RSI system has to interact with real-world, and that imposes serialization constraints. It is harder to know how much that slow-down would be and how much speed-up we will get before that. But a RSI cannot simply be a exponential growth forever.