Sub-3-Bit Models Finally Learn to Reason

🔥 What happened Researchers from Chinese institutions (CAS and others) dropped On-Policy Distillation (OPD): instead of training on fixed corpus prefixes like standard QAD, OPD lets the quantized student generate through its own broken trajectory and corrects it there with a frozen full-precision teacher. At 1.88 effective bits, MATH-500 retention jumps from 35% to 70%, HumanEval from 66% to 91%. 💡 Why it matters Sub-3-bit models used to collapse into repetitive loops on long reasoning chains — burning the decoding budget before finishing a solution. OPD roughly doubles reasoning retention overnight and beats matched-budget QAD by a wide margin. That makes 2-bit inference on edge devices actually viable. ⚡ Our take If you're still burning FP16 on GPUs in 2026, you missed the memo. Caveat: the teacher has to run during training — this is no free lunch for fine-tuning pipelines.