Robots.txt Generator

Generate a correct robots.txt with crawl rules and sitemap.

Site URL

Sitemap URL

Block these crawlers: GPTBot (OpenAI) ClaudeBot (Anthropic) PerplexityBot CCBot (Common Crawl) Google-Extended (Gemini training only)

Additional Disallow Paths (one per line)