Analyze MX error
The method starts from the observation that MX quantization depends on both feature statistics and fixed block structure.
Overview
Abstract
Post-training quantization is central to deploying large language models under tight memory, bandwidth, and energy budgets. LATMiX targets one of the hardest settings: low-bit microscaling formats, where each small block shares scale information and activation outliers can dominate quantization error.
LATMiX learns invertible affine transformations that redistribute activation mass before MX quantization. The transforms are optimized with a small calibration set, then folded into model weights so the evaluated model remains a standard Hugging Face/PyTorch causal language model.
The method supports MXFP4, MXINT4, and NVFP4 workflows, and is evaluated with RTN/GPTQ baselines, WikiText-2 perplexity, and zero-shot tasks from the LM Evaluation Harness.
How it works
Method
LATMiX keeps the deployment path practical: learn transformations during calibration, fold them into the network where possible, then apply standard quantization and evaluation flows.
Learn affine transforms
LATMiX expands beyond rotations by learning invertible affine transformations parameterized with LU or QR decompositions.
Use distillation
A KL distillation loss keeps the transformed quantized model close to the floating-point teacher on calibration examples.
Fold into weights
Learned transformations can be absorbed into linear operators, preserving an efficient inference graph.
Quantize and evaluate
The transformed model is quantized with RTN or GPTQ and evaluated on WikiText-2 plus seven zero-shot reasoning benchmarks.
Evaluation
Results
Up to 4% improvement in average accuracy recovery
Across seven zero-shot benchmarks and WikiText-2 perplexity, LATMiX consistently improves low-bit MX quantization over strong baselines.
- Experiments cover Llama 3.2, Llama 3.1, and Qwen3 models from 1B to 32B parameters.
- Both MXFP4 and MXINT4 settings quantize weights and activations, matching the extreme low-bit deployment regime targeted by the method.
- The LU variant is often strongest in practice, while the QR variant provides a complementary full-rank parameterization.
| Format | Model | Best baseline | LATMiX-LU | LATMiX-QR |
|---|---|---|---|---|
| MXFP4 | Llama-3.2-1B | 93.07 | 93.88 | 93.42 |
| MXFP4 | Qwen3-1.7B | 92.35 | 95.62 | 93.74 |
| MXFP4 | Qwen3-4B | 93.10 | 96.64 | 95.64 |
| MXFP4 | Qwen3-32B | 98.82 | 99.04 | 97.01 |
| MXINT4 | Llama-3.2-1B | 88.21 | 90.34 | 91.17 |
| MXINT4 | Qwen3-1.7B | 90.47 | 92.24 | 92.96 |
| MXINT4 | Qwen3-8B | 95.82 | 97.16 | 96.00 |
| MXINT4 | Qwen3-32B | 98.32 | 99.49 | 96.16 |
The full paper reports all models, methods, average accuracies, and recovery values across ARC Easy/Challenge, HellaSwag, WinoGrande, PIQA, BoolQ, and OpenBookQA.
Citation
BibTeX
If you use this codebase, please cite the ICML 2026 LATMiX paper:
@inproceedings{
gordon2026latmix,
title={{LATM}i{X}: Learnable Affine Transformations for Microscaling Quantization of {LLM}s},
author={Ofir Gordon and Lior Dikstein and Arnon Netzer and Idan Achituve and Hai Victor Habi},
booktitle={Forty-third International Conference on Machine Learning},
year={2026},
url={https://openreview.net/forum?id=dayZnm0dgh}
}