Cloudflare Radar
Monthly shares of web traffic from Cloudflare Radar since May 2022: AI crawlers such as GPTBot and ClaudeBot as a share of all requests, the largest crawlers individually, bot against human traffic, and the share of sampled sites whose robots.txt blocks or allows each crawler. Every value is a percentage, not a count.
Cloudflare decides which visitors count as AI crawlers. The list also holds normal search crawlers such as Googlebot. Radar publishes shares, not request counts.
AI bots
Crawler traffic mix
These three lines do not add up to 100%. Cloudflare puts the rest in a mixed class it counts as both human and bot.
Human
Non-AI bots
Top AI crawlers
Each line is a share of the requests from AI crawlers, not of all web traffic. Together the lines add up to about 100%.
GPTBot (OpenAI)
ClaudeBot (Anthropic)
Bytespider (ByteDance)
Amazonbot
Meta
Googlebot (Google)
Bingbot (Microsoft)
Applebot (Apple)
Bot vs human traffic
⚠ Cloudflare counts these two lines with an older method than the crawler lines. It gives a share about 23 points lower, so compare the trends, not the numbers.
Bots (all traffic)
Humans (all traffic)
robots.txt sample size
robots.txt files sampled
Sites blocking AI crawlers
Cloudflare reads about 4,000 robots.txt files each week. Each line shows the share of those files that block the whole site to one crawler.
GPTBot (OpenAI) derived
ClaudeBot (Anthropic) derived
CCBot (Common Crawl) derived
Google-Extended derived
Bytespider (ByteDance) derived
meta-externalagent (Meta) derived
Amazonbot derived
Applebot-Extended (Apple) derived
PerplexityBot derived
Sites explicitly allowing AI crawlers
Each line shows the share of the same files that give one crawler full access. ⚠ A site can block one part and allow another, so it can appear in both groups.
GPTBot (OpenAI) derived
ClaudeBot (Anthropic) derived
Google-Extended derived
PerplexityBot derived
snapshots · collecting daily
These start accruing from first fetch — a curve builds over the coming days.
Figures
| Series | Latest | Change | Min | Max | Coverage |
|---|---|---|---|---|---|
| AI bots | 5.8% | +55.4% 8 mo | 3.7% | 5.8% | Oct 2025 — Jun 2026 |
| Human | 42.7% | −13.5% 8 mo | 42.7% | 49.4% | Oct 2025 — Jun 2026 |
| Non-AI bots | 48.2% | +14.5% 8 mo | 42.0% | 48.2% | Oct 2025 — Jun 2026 |
| GPTBot (OpenAI) | 9.4% | −28.2% yr | 9.4% | 13.1% | Jul 2025 — Jun 2026 |
| ClaudeBot (Anthropic) | 19.8% | +75.5% yr | 6.9% | 19.8% | Jul 2025 — Jun 2026 |
| Bytespider (ByteDance) | 7.6% | +178.9% yr | 2.1% | 10.2% | Jul 2025 — Jun 2026 |
| Amazonbot | 5.4% | −18.4% yr | 4.1% | 7.4% | Jul 2025 — Jun 2026 |
| Meta | 10.2% | +21.0% yr | 7.9% | 16.5% | Jul 2025 — Jun 2026 |
| Googlebot (Google) | 25.2% | −40.9% yr | 25.2% | 47.8% | Jul 2025 — Jun 2026 |
| Bingbot (Microsoft) | 7.8% | −24.5% yr | 7.8% | 13.0% | Jul 2025 — Jun 2026 |
| Applebot (Apple) | 6.0% | +271.2% yr | 1.6% | 9.2% | Jul 2025 — Jun 2026 |
| Claude-SearchBot (Anthropic) | 3.3% | +46.6% 1 mo | 2.2% | 3.3% | May 2026 — Jun 2026 |
| Bots (all traffic) | 34.8% | +14.7% yr | 27.7% | 34.8% | May 2022 — Jun 2026 |
| Humans (all traffic) | 65.2% | −6.4% yr | 65.2% | 72.3% | May 2022 — Jun 2026 |
| robots.txt files sampled | 4.2k | +9.5% yr | 3.7k | 4.2k | Sep 2024 — Jun 2026 |
| GPTBot (OpenAI) | 10.1% | +50.6% yr | 6.4% | 10.1% | Sep 2024 — Jun 2026 |
| ClaudeBot (Anthropic) | 9.0% | +123.3% yr | 2.6% | 9.0% | Sep 2024 — Jun 2026 |
| CCBot (Common Crawl) | 9.8% | +61.8% yr | 4.6% | 9.8% | Sep 2024 — Jun 2026 |
| Google-Extended | 8.1% | +87.1% yr | 3.6% | 8.1% | Sep 2024 — Jun 2026 |
| Bytespider (ByteDance) | 9.1% | +135.1% yr | 2.6% | 9.1% | Sep 2024 — Jun 2026 |
| meta-externalagent (Meta) | 7.7% | +203.2% yr | 0.6% | 7.7% | Sep 2024 — Jun 2026 |
| Amazonbot | 7.3% | +214.5% yr | 1.5% | 7.3% | Sep 2024 — Jun 2026 |
| Applebot-Extended (Apple) | 6.8% | +105.9% yr | 1.6% | 6.8% | Sep 2024 — Jun 2026 |
| PerplexityBot | 4.1% | +25.4% yr | 2.2% | 4.3% | Sep 2024 — Jun 2026 |
| GPTBot (OpenAI) | 3.5% | +651.0% yr | 0.0% | 3.5% | Sep 2024 — Jun 2026 |
| ClaudeBot (Anthropic) | 2.8% | ×11 yr | 0% | 2.8% | Sep 2024 — Jun 2026 |
| Google-Extended | 3.1% | ×11 yr | 0% | 3.1% | Sep 2024 — Jun 2026 |
| PerplexityBot | 3.9% | +512.7% yr | 0% | 3.9% | Sep 2024 — Jun 2026 |