How many websites block Perplexity-User? (Nov 2026)
4.5%of sites with a readable robots.txt block it by name, site-wide
8.7%effectively blocked (incl. * rule)
1.7%partially restricted
1.0%explicitly allowed
#22/28most-blocked rank
Perplexity-User is operated by Perplexity; purpose: user-triggered. Scan of the top 10,000 websites (Tranco list V38GN) on 2026-11-18; repeated weekly.
Key facts
- As of 2026-11-18, 4.5% of the 5,806 top-10,000 sites with a readable robots.txt (2.6% of all 10,000 scanned) block Perplexity-User by name across the whole site (Perplexity, user-triggered crawler).
- Counting sites that block all bots with a blanket User-agent: * rule, 8.7% of the 5,806 top-10,000 sites with a readable robots.txt (5.1% of all 10,000 scanned) effectively block Perplexity-User.
- A further 1.7% of the 5,806 top-10,000 sites with a readable robots.txt restrict Perplexity-User partially, and 1.0% explicitly allow it.
- 5,388 of the 5,806 sites with a readable robots.txt do not mention Perplexity-User; of those, 245 block all bots via the * group.
Cite this: Agent Web Census, “How many websites block Perplexity-User?”, https://agents.datacensus.org/ai-crawlers/perplexity-user/, scan of the top 10,000 websites (Tranco list V38GN) on 2026-11-18.
Definitions
- Blocked site-wide (by name): a robots.txt group naming the bot has
Disallow: /withoutAllow: /(narrow Allow exceptions are tolerated). - Effectively blocked: blocked site-wide as above, or a blanket
User-agent: *group withDisallow: /. - Partially restricted: a group naming the bot has other Disallow rules.
- Purpose labels (training, search, user-triggered) follow each operator’s public documentation.
By popularity rank
| Bucket | n (readable robots.txt) | Fully blocked | Partially |
|---|---|---|---|
| top100 | 52 | 7.7% | 0.0% |
| top1k | 514 | 7.8% | 3.1% |
| top10k | 5,806 | 4.5% | 1.7% |
By TLD
| TLD | n (readable robots.txt) | Fully blocked | Note |
|---|---|---|---|
| .com | 2,985 | 4.6% | |
| .net | 259 | 2.3% | |
| .org | 310 | 1.9% | |
| .ru | 152 | 0.7% | |
| .io | 103 | 0.0% | |
| .jp | 114 | 6.1% | |
| .de | 135 | 5.2% | |
| .edu | 103 | 0.0% | |
| .gov | 80 | 2.5% | small sample (n=80), treat with caution |
| .in | 39 | 5.1% | small sample (n=39), treat with caution |
| .cn | 32 | 0.0% | small sample (n=32), treat with caution |
| .br | 63 | 6.3% | small sample (n=63), treat with caution |
| .fr | 75 | 13.3% | small sample (n=75), treat with caution |
| .uk | 60 | 13.3% | small sample (n=60), treat with caution |
| .it | 58 | 6.9% | small sample (n=58), treat with caution |
| .tv | 47 | 2.1% | small sample (n=47), treat with caution |
| .co | 32 | 0.0% | small sample (n=32), treat with caution |
| .es | 43 | 9.3% | small sample (n=43), treat with caution |
| .pl | 42 | 4.8% | small sample (n=42), treat with caution |
| .ai | 38 | 0.0% | small sample (n=38), treat with caution |
How to block or allow Perplexity-User
# block
User-agent: Perplexity-User
Disallow: /
# allow
User-agent: Perplexity-User
Allow: /