Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization

Preserve the direction, not the magnitude.

The Muon optimizer is a compelling alternative to Adam for training LLMs, achieving large compute savings through gradient orthogonalization. But Muon's state is unusually sensitive to quantization: orthogonalization discards singular-value magnitudes and keeps only directional information, so tiny errors in singular-vector directions get amplified into the update.

MuonQ is a low-bit Muon framework built on directional fidelity optimization — three complementary techniques that together enable stable pure 4-bit quantization of Muon's optimizer states, recovering most of full-precision Muon's loss and downstream accuracy while cutting optimizer-state memory by up to 7.3×.

Directional fidelity optimization

Muon's polar step maps every singular value to one, so only the error component that rotates singular vectors survives. MuonQ therefore targets the cosine similarity of the momentum, not its reconstruction error — attacking directional error at three stages of a single training step.

1

Pre-quantization normalization

Gradient and momentum norms swing across steps, so per-step quantization error swings too — and accumulates into an anisotropic drift the polar step is sensitive to. Normalizing both to unit Frobenius norm before quantizing makes every step inject equal-magnitude, isotropic error that never develops a preferred direction.

Gradient → Normalize → Add momentum → Normalize → Unit momentum → Decompose · Quantize · Update

2

Structural decomposition for orthogonalization stability

Orthogonalization amplifies directional error. MuonQ decomposes momentum via truncated top-k SVD (warm-started power iteration) and quantizes each factor separately — left singular vectors column-wise, scaled right vectors row-wise. Each singular direction becomes its own quantization group, so errors only rescale singular components rather than rotating them, and the polar step absorbs them.

before orthogonalization

after orthogonalization

no decomp. (k=0)

with decomp. (k=512)

3

μ-law companding quantization

Because Muon weights all singular directions equally, resolution near zero — where momentum values pack densely — matters more than preserving outliers. MuonQ applies μ-law companding (μ = 255) to reallocate quantization bins toward this dense central region, shifting the objective from outlier preservation to dense-region distinguishability.

Gran.	RE↓ Uni.	RE↓ Comp.	CS↑ Uni.	CS↑ Comp.
Tensor	0.509	0.238	0.879	0.973
Row	0.127	0.111	0.992	0.994
Column	0.254	0.159	0.969	0.988

4-bit quantization of k_proj. Companding (Comp.) lowers reconstruction error and raises cosine similarity at every granularity.

4-bit training that tracks full precision

Pre-training on GPT-2 (Medium / Large) and LLaMA (350M / 1.1B) over FineWeb, all under the standard Muon protocol on A100s. MuonQ4 hugs the Muon32 curve from the first steps, while naive Muon4 never closes its gap.

GPT-2 Medium

LLaMA-350M

LLaMA-1.1B

Zero-shot downstream accuracy

Head-to-head on LLaMA 1.1B

Every component earns its place

This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Artificial Intelligence for Science program, under contract DE-SC0025390. This research used resources of the National Energy Research Scientific Computing Center (NERSC), a DOE Office of Science User Facility, under Contract No. DE-AC02-05CH11231, using NERSC awards ASCR-ERCAP0030039 and ALCC-ERCAP0031379.

BibTeX

@inproceedings{su2026muonq,
  title     = {MuonQ: Enhancing Low-Bit Muon Quantization
               via Directional Fidelity Optimization},
  author    = {Su, Yupeng and Zhang, Ruijie and Liu, Ziyue
               and Zhao, Yequan and Zhang, Zheng},
  booktitle = {Conference on Language Modeling (CoLM)},
  year      = {2026},
  url       = {https://arxiv.org/abs/2605.11396}
}