30 Years of Records, Barely a Signal

🔥 What happened London researchers pretrained a 34.6M-parameter transformer on 1.4B coded events from 3.6M primary-care patients, then tested how far back a model must look to predict colorectal cancer. AUROC rose only from 0.551 at 2 years to 0.590 at 18 years — then flatlined. 💡 Why it matters 80% of the gain arrives within 10 years. A plain XGBoost baseline hits the same ceiling (0.589). Worse, RoPE aliases beyond ~20 years at default base frequency — the architecture itself swallows the long-range signal. ⚡ Our take 30-year context sounds like deep learning but delivers a spreadsheet model. Anyone selling long-history EHRs as a moat should check their positional embeddings first.