AI Training Opt-Out: OpenAI, Google and Microsoft Back One Rule

Hakkı ALKAN

October 31, 2026

OpenAI, Google, Microsoft and Anthropic said on October 30, 2026 that they will honor a single AI training opt-out rule in robots.txt, letting publishers block their content from being used to train AI models with one entry instead of separate rules for every AI company. Anthropic has already started rolling it out, Microsoft will begin on February 1, 2027, Google aims for 2027 and OpenAI says “in the coming months”.

The signal is a new robots.txt “policy token” called __usage_ai-training. It comes from the IETF’s AI Preferences (AIPREF) working group and is described in an Internet-Draft written by Kevin Kelley, a standards lead at Anthropic. The Verge first reported the joint backing after Search Engine Roundtable spotted it.

How the AI Training Opt-Out Works

Today, a site that wants to stay out of AI training has to block each crawler by name, such as GPTBot, Google-Extended and ClaudeBot, and keep the list current as new bots appear. The shared token replaces that with one group in robots.txt:

User-Agent: __usage_ai-training
Disallow: /

Publishers can also opt out only certain folders, or allow specific paths, using the normal robots.txt path rules. The draft does not add any new directives, so existing robots.txt parsers can read it.

The key limitation: the token covers training only. It is not a crawling directive, it does not affect search indexing, and it does not stop AI products from using pages to ground or cite answers. Content stays available for training by default until a site adds the rule.

What Each Company Said

• Google said in a Search Central blog post that when a site opts out with the shared token and does not mention Google-Extended separately, it will treat that as a Google-Extended block. Company-specific controls keep working, and Google aims to support the new signal in 2027.
• Microsoft will start honoring the preference from February 1, 2027 as Bing recrawls pages. It is also adding a page-level training setting and a separate “AI Grounding” preference that decides whether a page can feed answers in Bing, Copilot and Microsoft 365, with a “DisallowExtended” option that limits use to Microsoft’s own products. Bing’s old NoArchive and NoCache tags stop counting as AI preferences after March 1, 2027.
• OpenAI plans to implement it in the coming months, while Anthropic says it has already begun.
• Meta spokesperson Andy Stone called a common signal “an important step” but did not say whether Meta will honor it.

What It Means for Publishers

The move follows years of complaints and lawsuits from news organizations, including The New York Times’ 2023 case against OpenAI and Microsoft, and regulators are paying closer attention, as the UK’s recent Meta CMA investigation over publisher content shows. Google also notes that the approach fits the EU’s code of practice for general-purpose AI models, which treats robots.txt as a primary way to express machine-readable preferences.

The trade-off is visibility. Both Google and Microsoft warn that opting out of AI uses can reduce how often a site shows up in AI experiences, at a time when Google search traffic is already falling for many sites and Google keeps expanding ads in AI Mode. Alternatives also exist: Cloudflare has blocked AI crawlers by default on new sites since July 2025, and the Really Simple Licensing (RSL) standard lets publishers charge bots for access.

For now, nothing changes on most sites. Publishers who want out can add the token today so it takes effect as each company switches it on, while keeping their existing per-crawler rules in place.