You don't have to go into historical events. This is still happening now.

Jews are still fighting for their survival and the moment Israel stops fighting, millions of Jews will die.

And to be fair 2: The other package repos also suck.

I do plan to also develop an interactive guide that breaks down post-rotation quantization fundamentals in a more educational, hands-on way.

Comments URL: https://news.ycombinator.com/item?id=47998261

Points: 3

# Comments: 3

Comments URL: https://news.ycombinator.com/item?id=47989262

Points: 2

# Comments: 0

https://towardsdatascience.com/how-a-2021-quantization-algor...

tl;dr

TurboQuant is a recent paper from Google and NYU that has gained massive traction in mainstream media and the AI community. As implementations of TurboQuant are integrated into various popular projects, it is important to note its relation to EDEN quantization.

TurboQuant is essentially a partial implementation of EDEN quantization (first work published in NeurIPS 2021, extention published on ICML 2022). The few differences that do exist make EDEN significantly better.

We have also published a detailed comparative report here: https://arxiv.org/abs/2604.18555

In any case, the quantizer is indeed an extension, regardless of whether it was explicitly framed that way in the paper. I say this not to diminish their contribution at all, but just to clarify the relationship, as it was also stated in the vLLM doc.

While I probably overstated things by using the very general phrase "taking advantage," I want to be very precise about the claim, as I believe these works are foundational to quantization, beyond the scope of deep learning. The mechanism of applying a deterministic biased quantizer, such as Lloyd-Max, to the induced post-rotation distribution, alongside mathematically correcting its inherent bias, is a distinct contribution (which asymptotically improves the worst-case error).

If there is a classical paper that utilizes such a combination, I would genuinely be very eager to review it. But to my knowledge, this was not introduced prior to DRIVE and EDEN.

It is worth noting that taking advantage of the post-rotation distribution was not actually done until DRIVE (2021), which was made possible via our proper scaling. Furthermore, applying a Lloyd-Max codebook post-rotation was introduced EDEN.

We consider these to be the foundational works in this regard.

HIGGS is an extension of EDEN (using the well known method for blockwise Lloyd-Max).

The proper framing of this "TurboQuant" layer in vllm (which does not include JQL) is precisely EDEN 22 without the scale correction.

The quantizer in TurboQuant is EDEN quantization (2021) applied to the KV-cache. It is neither a novel quantizer nor an improvement in quantization techniques.

In DRIVE/EDEN, we already introduced the version used in "TurboQuant"'s paper and suggested an optimal scale configurations which are better in both mse-minimizing and unbiased scenarios.

Both EDEN and its 1-bit variant have been implemented in PyTorch, JAX, and TensorFlow across numerous open-source libraries and are used in various applications. I am currently writing a blog post that will document these in detail.

EDEN defines a scale parameter, S, for which we suggest specific optimal values for both biased and unbiased versions. As shown in the note I shared, these values lead to clear empirical improvements. Consequently, users who rely on the less optimal S value and the unbiasing method popularized by TurboQuant will generally see inferior results compared to those using EDEN with the optimal scale values suggested in our original papers.

(In any case, I want to emphasize that TurboQuant quantizer is a private case of EDEN)

We were the first to introduce post-rotation distribution-aware quantization in 2021. This was later implemented in many fields, including federated learning, vector retrieval, databases, inference engines, and KV-cache.

It would be appropriate to receive credit for this. Furthermore, it is baffling to see the name "TurboQuant" repeated in this context, considering the many works published from 2021 onwards.

The blog post mentioned above essentially guides you through EDEN quantization but ultimately settles on a sub-optimal MSE-minimizing version and an unbiasing trick. This trick often costs a full bit more than DRIVE/EDEN requires to achieve the same results using the unbiasing scale shown in the original 2021 paper.