y-axis

Cloudflare Radar

Monthly shares of web traffic from Cloudflare Radar since May 2022: AI crawlers such as GPTBot and ClaudeBot as a share of all requests, the largest crawlers individually, bot against human traffic, how much of Cloudflare's verified bot traffic comes from AI crawlers rather than search engines, and the share of sampled sites whose robots.txt blocks or allows each crawler. Every value is a percentage, not a count.

Cloudflare decides which visitors count as AI crawlers. The list also holds normal search crawlers — Googlebot is the largest one in it. Radar publishes shares, not request counts.

cadence · monthly 30 series updated 10 Aug 2026 radar.cloudflare.com/bots ↗
release markers

All AI crawlers

5.6% of HTTP requests +51.3% 9 mo as of Jul 2026
01234562023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Crawler traffic mix

These three lines do not add up to 100%. Cloudflare puts the rest in a mixed class it counts as both human and bot.

Human

41.4% of HTTP requests −16.1% 9 mo as of Jul 2026
010203040502023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Non-AI bots

49.6% of HTTP requests +17.8% 9 mo as of Jul 2026
010203040502023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Top AI crawlers

Each line is a share of the requests from AI crawlers, not of all web traffic. Together the lines add up to about 100%.

GPTBot (OpenAI)

9.7% of AI-bot requests −25.6% yr as of Jul 2026
0246810122023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

ClaudeBot (Anthropic)

16.2% of AI-bot requests +44.0% yr as of Jul 2026
051015202023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Bytespider (ByteDance)

5.1% of AI-bot requests +86.2% yr as of Jul 2026
02468102023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Amazonbot

6.0% of AI-bot requests −9.6% yr as of Jul 2026
012345672023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Meta

12.7% of AI-bot requests +50.5% yr as of Jul 2026
02468101214162023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Googlebot (Google)

24.6% of AI-bot requests −42.3% yr as of Jul 2026
010203040502023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Bingbot (Microsoft)

8.9% of AI-bot requests −13.9% yr as of Jul 2026
0246810122023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Applebot (Apple)

6.5% of AI-bot requests +304.3% yr as of Jul 2026
024682023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Claude-SearchBot (Anthropic)

3.5% of AI-bot requests +58.2% 2 mo as of Jul 2026
00.511.522.533.52023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Bot vs human traffic

⚠ Cloudflare counts these two lines with an older method than the crawler lines. It gives a share about 23 points lower, so compare the trends, not the numbers.

Bots (all traffic)

34.8% of HTTP requests +13.2% yr as of Jul 2026
051015202530352023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Humans (all traffic)

65.2% of HTTP requests −5.9% yr as of Jul 2026
0102030405060702023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

robots.txt sample size

robots.txt files sampled

4.2k +11.5% yr as of Jul 2026
01.0k2.0k3.0k4.0k2023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss
Every robots.txt share on this page is a percentage of this number. It grew from 3,700 to 4,200 files since 2024, so we show shares, not site counts.

Sites blocking AI crawlers

Cloudflare reads about 4,000 robots.txt files each week. Each line shows the share of those files that block the whole site to one crawler. ⚠ Since July 2025 Cloudflare adds this block to new sites itself, so a block is not always the site owner's choice.

GPTBot (OpenAI) derived

11.0% of sites +72.6% yr as of Jul 2026
0246810122023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

ClaudeBot (Anthropic) derived

10.1% of sites +161.7% yr as of Jul 2026
02468102023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

CCBot (Common Crawl) derived

11.1% of sites +90.9% yr as of Jul 2026
0246810122023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Google-Extended derived

9.2% of sites +100.8% yr as of Jul 2026
02468102023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Bytespider (ByteDance) derived

10.6% of sites +178.5% yr as of Jul 2026
02468102023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

meta-externalagent (Meta) derived

8.7% of sites +268.9% yr as of Jul 2026
024682023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Amazonbot derived

8.4% of sites +232.4% yr as of Jul 2026
024682023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Applebot-Extended (Apple) derived

8.0% of sites +179.2% yr as of Jul 2026
024682023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

PerplexityBot derived

4.5% of sites +42.9% yr as of Jul 2026
012342023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Sites explicitly allowing AI crawlers

Each line shows the share of the same files that give one crawler full access. ⚠ A site can block one part and allow another, so it can appear in both groups.

GPTBot (OpenAI) derived

4.2% of sites +684.8% yr as of Jul 2026
012342023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

ClaudeBot (Anthropic) derived

3.5% of sites ×11 yr as of Jul 2026
012342023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Google-Extended derived

3.8% of sites ×12 yr as of Jul 2026
012342023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

PerplexityBot derived

4.4% of sites +492.6% yr as of Jul 2026
012342023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Bot traffic composition

Cloudflare verifies each crawler before it counts it, and it adds new ones over time. We use only the crawlers that were on its list in every month, so a new arrival cannot change these charts.

AI crawlers and assistants derived

21.4% of verified bot requests +26.0% yr as of Jul 2026
051015202023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss
⚠ Cloudflare does not verify ClaudeBot, Bytespider or PerplexityBot, so this chart leaves them out.

Search engine crawlers derived

35.5% of verified bot requests −0.5% yr as of Jul 2026
010203040502023202420252026Copilot GAChatGPTClaude 3o1DeepSeek-R1GPT-5 / gpt-oss

Figures

SeriesLatestChangeMinMaxCoverage
All AI crawlers5.6%+51.3% 9 mo3.7%5.8%Oct 2025 — Jul 2026
Human41.4%−16.1% 9 mo41.4%49.4%Oct 2025 — Jul 2026
Non-AI bots49.6%+17.8% 9 mo42.0%49.6%Oct 2025 — Jul 2026
GPTBot (OpenAI)9.7%−25.6% yr9.4%13.1%Jul 2025 — Jul 2026
ClaudeBot (Anthropic)16.2%+44.0% yr6.9%19.8%Jul 2025 — Jul 2026
Bytespider (ByteDance)5.1%+86.2% yr2.1%10.2%Jul 2025 — Jul 2026
Amazonbot6.0%−9.6% yr4.1%7.4%Jul 2025 — Jul 2026
Meta12.7%+50.5% yr7.9%16.5%Jul 2025 — Jul 2026
Googlebot (Google)24.6%−42.3% yr24.6%47.8%Jul 2025 — Jul 2026
Bingbot (Microsoft)8.9%−13.9% yr7.8%13.0%Jul 2025 — Jul 2026
Applebot (Apple)6.5%+304.3% yr1.6%9.2%Jul 2025 — Jul 2026
Claude-SearchBot (Anthropic)3.5%+58.2% 2 mo2.2%3.5%May 2026 — Jul 2026
Bots (all traffic)34.8%+13.2% yr27.7%34.8%May 2022 — Jul 2026
Humans (all traffic)65.2%−5.9% yr65.2%72.3%May 2022 — Jul 2026
robots.txt files sampled4.2k+11.5% yr3.7k4.2kSep 2024 — Jul 2026
GPTBot (OpenAI)11.0%+72.6% yr6.4%11.0%Sep 2024 — Jul 2026
ClaudeBot (Anthropic)10.1%+161.7% yr2.6%10.1%Sep 2024 — Jul 2026
CCBot (Common Crawl)11.1%+90.9% yr4.6%11.1%Sep 2024 — Jul 2026
Google-Extended9.2%+100.8% yr3.6%9.2%Sep 2024 — Jul 2026
Bytespider (ByteDance)10.6%+178.5% yr2.6%10.6%Sep 2024 — Jul 2026
meta-externalagent (Meta)8.7%+268.9% yr0.6%8.7%Sep 2024 — Jul 2026
Amazonbot8.4%+232.4% yr1.5%8.4%Sep 2024 — Jul 2026
Applebot-Extended (Apple)8.0%+179.2% yr1.6%8.0%Sep 2024 — Jul 2026
PerplexityBot4.5%+42.9% yr2.2%4.5%Sep 2024 — Jul 2026
GPTBot (OpenAI)4.2%+684.8% yr0.0%4.2%Sep 2024 — Jul 2026
ClaudeBot (Anthropic)3.5%×11 yr0%3.5%Sep 2024 — Jul 2026
Google-Extended3.8%×12 yr0%3.8%Sep 2024 — Jul 2026
PerplexityBot4.4%+492.6% yr0%4.4%Sep 2024 — Jul 2026
AI crawlers and assistants21.4%+26.0% yr12.2%21.4%Sep 2024 — Jul 2026
Search engine crawlers35.5%−0.5% yr33.3%48.8%Sep 2024 — Jul 2026
← back to all sources