Decide which AI tools can read your content, one bot at a time
Every major AI company now runs at least two kinds of bots. One is for training, which collects content to help build future AI models. The other is for live answers, which fetches pages on demand when a user asks a question right now. The difference matters. Blocking a training bot keeps your content out of future models. Blocking a live-answer bot also stops your site from showing up when someone asks ChatGPT or Perplexity about your topic.
This tool gives you a simple on or off switch for 20 of the most active AI bots, with a one-line note on what each one does. Pick what to allow and what to block, and the tool gives you the matching rules for your setup: a robots.txt file, a Cloudflare rule, or a Next.js snippet.
The four presets, explained simply
- Allow all. The default. AI bots can read your site freely. Best if you want the most AI search visibility and don't mind your content being used in future models.
- Block all AI. The strictest option. No AI tools can train on or quote your content. Also gives up AI search visibility entirely.
- Block training only. The balanced option. Live-answer bots (Perplexity, ChatGPT, Claude) can still read your pages so you can be quoted, but training bots (GPTBot, ClaudeBot, CCBot) are blocked. A solid choice for most publishers.
- Block live answers only. An unusual choice. Your content can be used for training but not for live AI answers. Sometimes used by brands that want to stay in future models without giving up traffic to AI search products today.
An important note: robots.txt is a polite request, not a wall
The robots.txt file is a respected request, not a hard block. The major AI companies (OpenAI, Anthropic, Google, Perplexity) follow it. Some smaller bots and most basic scrapers do not. If you need real enforcement, use the Cloudflare rule. That stops bots at the network edge, before they ever reach your website. The Next.js option is best for sites built on Next.js where you want bot detection inside your application code.
What the list includes
The tool currently includes 20 bots across OpenAI, Anthropic, Google, Perplexity, Apple, Meta, Common Crawl, ByteDance, Amazon, Mistral, Cohere, DuckDuckGo, You.com, and Diffbot. Each one is labeled as a training bot, a live-answer bot, or both, with a short note so you always know what you are toggling. We update the list as new bots show up.
Pairs naturally with the Schema Markup Generator as part of a one-time setup for your website.