Stanford's 31x Faster Text Generation

🔥 What happened Stanford researchers led by Hermann Kumbong dropped Next-Concept Modeling (NCM): text gets compressed into a hierarchy of discrete codes, then a transformer generates one level at a time — all codes within a level in parallel. On XSum it matches the best diffusion model's ROUGE-L while running 31x faster. 💡 Why it matters Diffusion language models promised parallel sampling but collapse with few steps and burn compute on full sequences every pass. NCM's coarse-to-fine trick sidesteps that — and even beats autoregressive and diffusion baselines on Sudoku reasoning with fewer function evaluations. ⚡ Our take If inference is the dominant workload, a 31x speedup isn't a benchmark flex — it's a P&L line item. The open question: does the VQ-VAE middle layer hold up on real long-context tasks, or is this a short-sequence party trick?