ResearchNovember 13, 2026
-25
🧊

Integer-only LLM inference kills nondeterminism

DetLLM makes LLM inference bit-identical by eliminating floating-point arithmetic entirely.

#LLM inference#determinism#integer arithmetic#NeurIPS#GPU
Ganze LLM-Inferenz ohne einen einzigen Float
Share Article
🔥 What happened A developer built DetLLM at a hackathon, running every LLM operation – matmuls, softmax, normalization – with exact integers instead of floats. The project became a paper at two NeurIPS 2026 workshops. Same prompt, nine configurations, three machines: integer runs never diverged. 💡 Why it matters At temperature 0, one prompt sent 1,000 times returned 80 different answers because float addition isn't associative. Integers are exact, so order can't matter – bit-identical output across any GPU, CPU, or batch size. That's a stronger guarantee than Thinking Machines' batch-invariant kernels, which only work on one GPU. ⚡ Our take This is the cleaner kill shot against nondeterminism: fix the arithmetic, not the kernels. If it scales beyond 0.6B models, “works on my machine” finally dies for LLMs too.
The title, summary and analysis of this item were produced automatically by an AI system and have not been editorially reviewed. They may contain errors, bias or omissions — when in doubt, read the linked original source.