ResearchNovember 15, 2026
-48
🧊

Attribution only works in short windows

A contribution argues that data attribution should steer pretraining, noting trajectory attribution only holds within short, validated checkpoint windows.

#Data Attribution#Influence Functions#Pretraining#LLM#arXiv
Attribution nur im kurzen Fenster verlässlich
Share Article
🔥 What happened Researchers evaluated MAGIC, a trajectory data attribution method, on GPT-2-style models (86M and 152M parameters) learning multistep arithmetic. Full-run attribution scores barely correlated with actual retraining (LDS 0.17–0.22). 💡 Why it matters Only short checkpoint windows (100 steps) produced accurate predictions (LDS 0.86–0.99 at 152M). This proves attribution is a local tool, not a global guarantee – anyone claiming data responsibility across entire training runs is fooling themselves. ⚡ Our take The AI world treats influence functions as a silver bullet for data quality. This paper is a reality check: without retraining validation, attribution is a shot in the dark.
The title, summary and analysis of this item were produced automatically by an AI system and have not been editorially reviewed. They may contain errors, bias or omissions — when in doubt, read the linked original source.