Agent Web Census

How many websites block GPTBot? (Nov 2026)

15.3%of sites with a readable robots.txt block it by name, site-wide
19.0%effectively blocked (incl. * rule)
3.8%partially restricted
2.5%explicitly allowed
#3/28most-blocked rank

GPTBot is operated by OpenAI; purpose: training. Scan of the top 10,000 websites (Tranco list V38GN) on 2026-11-18; repeated weekly.

Key facts

Cite this: Agent Web Census, “How many websites block GPTBot?”, https://agents.datacensus.org/ai-crawlers/gptbot/, scan of the top 10,000 websites (Tranco list V38GN) on 2026-11-18.
Definitions

By popularity rank

Bucketn (readable robots.txt)Fully blockedPartially
top1005221.2%1.9%
top1k51416.5%4.9%
top10k5,80615.3%3.8%

By TLD

TLDn (readable robots.txt)Fully blockedNote
.com2,98514.8%
.net25915.4%
.org31016.1%
.ru1527.2%
.io1037.8%
.jp11414.0%
.de13524.4%
.edu1035.8%
.gov801.2%small sample (n=80), treat with caution
.in3912.8%small sample (n=39), treat with caution
.cn320.0%small sample (n=32), treat with caution
.br6311.1%small sample (n=63), treat with caution
.fr7533.3%small sample (n=75), treat with caution
.uk6023.3%small sample (n=60), treat with caution
.it5831.0%small sample (n=58), treat with caution
.tv476.4%small sample (n=47), treat with caution
.co329.4%small sample (n=32), treat with caution
.es4325.6%small sample (n=43), treat with caution
.pl4226.2%small sample (n=42), treat with caution
.ai3810.5%small sample (n=38), treat with caution

n = domains of that TLD with a readable robots.txt; percentages are of n. TLDs with n < 30 are omitted.

How to block or allow GPTBot

# block
User-agent: GPTBot
Disallow: /

# allow
User-agent: GPTBot
Allow: /

robots.txt is a voluntary standard (RFC 9309); compliance depends on the crawler operator.

← all AI crawlers

Last updated: (UTC). Numbers are recomputed automatically from primary sources; see methodology.