Stanford: LLM Research Expires in 16 Months

🔥 What happened Stanford researchers Owen Queen and James Zou tested 50 widely cited LLM claims against 20 models – 44 percent have already "expired." Average shelf life: just 15.7 months. 💡 Why it matters After two years, 39 percent of findings are dead. Claims about what models can't do decay fastest: all nine reasoning weaknesses – from strawberry letter-counting to lost-in-the-middle – are gone. Only 16 percent still hold. ⚡ Our take Anyone selling today's benchmarks as truth is shipping tomorrow's landfill. Papers without model version, date, and prompts are scientific garbage.