https://faqpages.com/robots-txt-and-the-three-chatgpt-bots-a-simple-policy-for-do
GPTBot, OAI-SearchBot and robots.txt Guide
May 26, 2026 - A plain-English policy for GPTBot, OAI-SearchBot, ChatGPT-User, plus copy-paste robots.txt and X-Robots-Tag examples.
gptbotoaisearchbotrobotstxt
https://www.ioriver.io/questions/block-ai-crawlers-cdn-edge
Blocking AI Crawlers at the CDN Edge: GPTBot vs. OAI-SearchBot
Blocking AI crawlers like GPTBot at the CDN edge: when to block training bots, allow search crawlers, and verify identity in logs.
ai crawlersat theblockingcdnedge
https://grow.contact/glossary/claudebot
ClaudeBot / Claude-SearchBot — GEO Glossary
Anthropic's crawlers — ClaudeBot for general fetches, Claude-SearchBot for Claude's live web search.
claudebotsearchbotgeoglossary