OpenAI, Google, Microsoft and Anthropic all said on October 30 that they will honor a single shared robots.txt policy token, __usage_ai-training, so a website can opt out of AI training by every participating company in one entry instead of blocking each crawler separately. The signal comes from an IETF Internet-Draft authored by Kevin Kelley of Anthropic, building on the IETF's AI Preferences working group.

The token covers training only. It does not block crawling, search indexing, or AI answers that quote a site's pages, and it does not remove material already collected into existing training sets.

The timelines differ sharply. Anthropic says it has already started implementing support, OpenAI says it aims to implement the signal in the coming months, Microsoft targets February 1, 2027 — and says it will add separate page-level controls for training and AI grounding — and Google targets 2027, treating an opt-out as equivalent to blocking Google-Extended unless that is set directly. The underlying draft is not yet a formal standard and expires in May 2027 unless updated.