Majority Vote's Hidden Flaw in TTRL

🔥 What happened Researchers at the University of Melbourne show that majority-vote pseudo-labels in Test-Time Reinforcement Learning are far less reliable than assumed. They split the error into population-majority bias and prompt-dependent finite-rollout variance, then propose Prompt-Conditional Distributionally Robust DPO to down-weight sampling-sensitive prompts. 💡 Why it matters Their method matches or beats the strongest TTRL baseline in 24 of 30 settings across six reasoning benchmarks and five model backbones. A cache-based mechanism also curbs self-supervised forgetting during online adaptation. If you run TTRL without external labels, the majority isn't always right. ⚡ Our take Finally, someone calls out the myth that majority voting is a free lunch in self-supervision. If you ship TTRL in production, bake in this robust reweighting — or keep guessing.