Together with bf16 related deliberate hardward degrades on consumer-level nvidia gpus, i.e., gtx 10, rtx 20, 30, 40, 50 series, things gets sour really quickly.

Gemma family (gen 1 to gen 4) is consistent with extreme range of activations, i.e., 600000, essentially forcing people to use bf16 kv cache and accept a short context window, e.g., 31b, iq4_xs quantization, 100k context window on 32gb memory. Or, people use q8 kv cache, 200k context window, and accept a large performance penalty.

In contrast, for qwen 3.5 family, the largest activation is below 2000, making q8 or even lower-precision kv cache essentially free estates. Together with linear attention, which doesn't require kv cache, full 262k context window can be easily reached.

Qat training with w4a16 target, while improving performance on inference with low-precision weighs, doesn't solve kv cache problem at all.

In the end, a qat is a qat, and there are unseen efforts behind qat checkpoints. Thank you gemma team for releasing qat checkpoints.

Wait, *Excluding Chinese language.

This is ... curious.

P.S. Where is gemma 4 124b?

If you don't like be treated like anything but human, you should seriously consider replacing chrome with ungoogled chromium or other browsers.

In addition, using the word microslop instead of microsoft is again justified, too.

[1] https://github.com/Lightning-AI/pytorch-lightning/issues/216...

[2] https://github.com/Lightning-AI/pytorch-lightning/issues/216...

[3] https://github.com/Lightning-AI/pytorch-lightning/issues/216...

[4] https://github.com/Lightning-AI/pytorch-lightning/issues/216...

[5] https://socket.dev/blog/lightning-pypi-package-compromised

When Qwen 3.5 27b released, I didn't really understand why linear attention is used instead of full attention because of the performance degradation and problems introduced with extra (linear) operators. After doing some tests, I found that with llama.cpp and IQ4_XS quant, the model and BF16 cache of the whole 262k context just fit on 32GB vram, which is impossible with full attention. In contrast, with gemma 4 31b IQ4_XS quant I have to use Q8_0 cache to fit 262k context on the vram, which is a little annoying (no offenses, thank you gemma team, too).

From benchmarks, 3.5->3.6 upgrade is about agent things. I hope future upgrades fix some problems I found, e.g., output repetitiveness in long conversations and knowledge broadness.

Besides, I'm old enough to recall that META has trained a version of LLAMA 4 specifically for LM arena elo benchmaxxing and PR things, and proceeded to release a different version of LLAMA 4.

The elo ranking [1] is too good to be true. I don't know why gemma-4-26b-a4b performs better than gemma-4-31b.

Also waiting for more bugfixes in llama.cpp, sglang and vllm to do proper evaluations.

[1] https://arena.ai/leaderboard/text/expert?license=open-source

If you want to install APKs directly on Android phones selling in China, you'll face even more draconian restrictions imposed by both Chinese OEMs and Chinese government, e.g., cannot install telegram [1], cannot install VPNs [2], called by local police station after installing VPNs [3], and so on. And you do not have the freedom to even talk about these restrictions freely without getting sued or censored.

[1] https://xcancel.com/whyyoutouzhele/status/168915238841261670...

[2] https://xcancel.com/whyyoutouzhele/status/197843066556268971...

[3] https://xcancel.com/whyyoutouzhele/status/170299205759627676...

1.5M records of PRs affected. Does Microsoft copilot ask users for the permission of adding ads inside their PRs before actually doing the thing? Do users show their consents on this matter?

Now EVERYONE can see ads disguised as PRs on GitHub. Does Microsoft asks everyone for the permission of showing ads before actually doing the thing? Do users show their consents on this matter?

Good taste Microslop.

The experience of utilizing tiered storage (gpu vram, ram, and ssd) is generally poor for a lot of LLM inference engines out there, e.g., llama.cpp, sglang, vllm, etc..

My own experience shows that both weight and KV cache offload to ram on sglang and vllm is unavailable or unusable. Copying extra parameters from documents and adding them to already working commands results in errors. Llama.cpp does support weight offload, but the experience is not pleasant, low pcie (gpu <-> ram) utilization, low gpu utilization, and really low tokens per second.

This is sad for local LLM community. First we lost wizardLM, Yi and others, then we lost Llama and others, now we lost Qwen...