TurboQuant (Zandieh et al. 2025) has been all the rage in the past two days, and I've seen lots of comments here attempting to explain the magic behind it. Many of those comments boil down to "dude, it's polar coordinates!!!", and that's really misleading. The most important part has nothing to do with polar coordinates (although they are emphasized in Google's blog post, so the confusion is understandable).
TurboQuant is a vector quantization algorithm. It turns a vector of numbers into another vector of numbers that takes up less memory.
Quantization is a fairly basic operation. If you have an n-dimensional vector that looks like this:
0.2374623 0.7237428 0.5434738 0.1001233 ... Then a quantized version of that vector may look like this:
0.237 0.723 0.543 0.100 ... Notice how I simply shaved off the last four digits of each number? That's already an example of a crude quantization process. Obviously, there are far more sophisticated schemes, including grouping coefficients in blocks, adaptive thresholds, calibrated precision based on experimental data etc., but at its core, quantization always involves reducing coefficient precision.
Here is the key idea behind TurboQuant: Before quantizing a vector, we randomly rotate it in the n-dimensional space it resides in. The corresponding counter-rotation is applied during dequantization.
That's it.
Now you probably feel that I must have left out an important detail. Surely the rotation can't be completely random? Maybe it's sampled from a particular distribution, or somehow input-dependent? Or perhaps there is another operation that goes hand in hand with it?
Nope. I didn't leave anything out. Just applying a random rotation to the vector dramatically improves quantization performance.
But why?
Because the magnitudes of the coefficients of state vectors in language models aren't distributed uniformly among the vector dimensions. It's very common to see vectors that look like this:
0.0000023 0.9999428 <-- !!! 0.0000738 0.0000003 ... This phenomenon has many names, and it shows up everywhere in transformer research. You can read about "massive activations" (Sun et al. 2024) and "attention sinks" (e.g. Gu et al. 2024) for a deeper analysis.
What matters for the purposes of this explanation is: Vectors with this type of quasi-sparse structure are terrible targets for component quantization. Reducing precision in such a vector effectively turns the massive component into 1 (assuming the vector is normalized), and all other components into 0. That is, quantization "snaps" the vector to its nearest cardinal direction. This collapses the information content of the vector, as identifying a cardinal direction takes only log2(2n) bits, whereas the quantized vector can hold kn bits (assuming k bits per component).
And that's where the random rotation comes in! Since most directions aren't near a cardinal direction (and this only becomes more true as the number of dimensions increases), a random rotation almost surely results in a vector that distributes the coefficient weight evenly across all components, meaning that quantization doesn't cause information loss beyond that expected from precision reduction.
The TurboQuant paper proves this mathematically, and gives an exact description of the distribution behavior, but the intuitive understanding is much more straightforward than that.
This idea isn't new in principle (QuIP is another quantization method that employs a similar trick), but TurboQuant combines it with a second step that eliminates biases that arise when quantized vectors that are optimal in a certain sense (MSE) are used to compute inner products, which is what happens in attention blocks. See the paper if you're interested in the details.
[link] [comments]
