Huawei Open-Sources MACaQ, Quantizes 122B Model in 64 Minutes

Huawei's Cloud & AI lab says it can quantize a 122B-parameter model in 64 minutes using 32.5 GB of device memory. The claim opens the README of MACaQ, a new Apache 2.0 quantization framework the group published on GitHub under the huawei-csl handle.

MACaQ stands for Model Agnostic Calibrated Quantization, and the stated reach is wide for one release. It targets llama.cpp with 2/3/4-bit GGUF outputs for text, vLLM with W4A4 or weight-only serving, Diffusers with W4A4 for image, video and audio generation, and PyTorch with W4A4 for speech. The repo lists Ascend CANN kernel work shipping via pull requests. Supported models named in the README include Qwen3 variants at 4B, 27B, 30B, 35B and 80B, OLMoE-1B-7B, FLUX.2-klein-4B, Wan2.2-TI2V and Stable Audio 1.0. The project's tagline reads "High-quality GGUF for local inference. Model-agnostic quantization beyond text."

On accuracy, the authors are blunt. They claim MACaQ reaches "lower perplexity than MR-GPTQ and DuQuant++, or match their best Llama-3-8B perplexity with 11× less quantization time." For Qwen3.8-27B under W4A4, the project reports 2.33x prefill and 2.58x decode speedups against a BF16 baseline, with perplexity shifting from 2.8463 to 2.8994. A pre-quantized Qwen3.6-35B-A3B-MACaQ-R GGUF ships alongside the code.

The drop lands in a busy week for low-precision tooling, days after llama.cpp merged FP8 KV-cache support for OCP-FP8 models. The arXiv preprint cited in the repo lists Muller, Bich, Boretti, Chang, Zhuang and Cavigelli as authors.