ResearchNovember 17, 2026
7
🌱

Your Sampler Is Killing Long Context

Rather than training, the top-nsigma sampling method prevents long-context collapse at high temperature.

#sampling#long-context#LLM#top-nsigma#NeurIPS
Der Sampler entscheidet über Langkontext
Share Article
🔥 What happened A team led by Allen Roush tested 29 open-weight models (1B–122B) across ten benchmarks plus needle retrieval up to one million tokens. At temperature 2, coding agents using top-p, top-k or Top-H solve zero of 100 GitHub issues — top-nsigma solves 46 to 62. 💡 Why it matters Long-context degradation isn't a training problem, it's a sampling problem. top-nsigma keeps a median 100% of its temperature-1 score; top-p keeps just 26%. In one Llama-3.1-8B story, top-p's candidate pool balloons from 4 to 51,845 tokens while top-nsigma stays at 3. ⚡ Our take If you crank temperature with top-p, you're sabotaging your own agent and blaming the model. The sampler is the underrated lever — not the next finetuning run.
The title, summary and analysis of this item were produced automatically by an AI system and have not been editorially reviewed. They may contain errors, bias or omissions — when in doubt, read the linked original source.