Ethics & SecurityNovember 02, 2026
-38
🧊

Google Unifies AI Opt-Out in robots.txt

Google to honor a single robots.txt token blocking AI training from 2027.

#robots.txt#Google#IETF#AI training#crawlers
Google bündelt KI-Widerspruch in robots.txt
Share Article
🔥 What happened Google will support a unified policy token in robots.txt starting 2027, letting site operators centrally opt out of AI training. Instead of maintaining separate blocks for each AI vendor, one signal will cover all participating developers. 💡 Why it matters Publishers previously had to block dozens of crawler tokens like GPTBot or ClaudeBot one by one – a patchwork. The new token, based on an IETF draft, is voluntary: only cooperative AI providers will honor it, and it can't technically enforce anything. ⚡ Our take A win for publishers on paper, but a toothless tiger: anyone already ignoring robots.txt will ignore this too. Without enforcement, it's just a gentleman's agreement in the AI era.
The title, summary and analysis of this item were produced automatically by an AI system and have not been editorially reviewed. They may contain errors, bias or omissions — when in doubt, read the linked original source.