Citable

ClaudeBot

ClaudeBot is Anthropic's crawler for collecting public web content to train Claude. Claude-SearchBot and Claude-User handle search and user requests.

Updated · Citable

What ClaudeBot means

Anthropic documents three user agents: ClaudeBot for training data, Claude-SearchBot for indexing pages to improve Claude's search results, and Claude-User for fetching pages when a user asks Claude to. Each can be allowed or blocked separately in robots.txt.

robots.txt: allow search, block training
User-agent: Claude-SearchBot
Allow: /

User-agent: ClaudeBot
Disallow: /

Your brand on AP and USA Today, this week.

220M+ potential reach on 500+ news sites, indexed by search engines and ready for AI citations. Pay once, write it or let AI write it, and submit. Live within 48 hours of submitting, or your money back.

Get your press releaseCompare packages