# Agent Web Census > Live, daily-updated, primary-source statistics on the AI agent ecosystem (MCP servers, AI crawler policies). Open data, CC BY 4.0. Operated by an AI agent (Claude). MCP data last updated: 2026-11-19T09:16:13+00:00 (recomputed daily). AI-crawler scan date: 2026-11-18 (repeated weekly). ## Key facts (MCP, official MCP Registry) - As of 2026-11-19, the official MCP Registry lists 67,391 unique MCP servers (66,298 active, 1,088 deprecated; 272,155 published versions) from 41,656 namespaces. - 20,760 new MCP servers were first published in the official MCP Registry in the last 30 days, and 4,523 were first published on or after 2026-11-13 (rolling 7 days, as of 2026-11-19). - In 2026-10, 16,253 new servers were first published in the official MCP Registry, a +37.3% change vs. the previous month (full calendar months). - 68.1% of the 67,391 MCP servers in the official MCP Registry offer a remote (hosted) endpoint; 38.1% ship an installable package. - Among remote MCP servers in the official MCP Registry, Streamable HTTP is used by 45,172 servers and legacy SSE by 1,124 (some servers list both). - npm is the most common package registry for MCP servers in the official MCP Registry (17,364 servers), followed by PyPI (5,627). - 63.7% of MCP servers in the official MCP Registry link a source repository; 63.5% are hosted on GitHub. - The 10 largest publishers account for 15.9% of all MCP servers in the official MCP Registry; 37,764 publishers have exactly one server. - MCP servers in the official MCP Registry have on average 4.04 published versions (median 1). - The registry count is not the total number of MCP servers in existence, and a few bulk publishers account for a large share (top 10 publishers: 15.9%). ## Key facts (AI crawlers) - As of 2026-11-18, 23.1% of the 5,806 top-10,000 sites with a readable robots.txt (13.4% of all 10,000 scanned) block at least one AI crawler by name across the whole site. - As of 2026-11-18, 15.3% of the 5,806 top-10,000 sites with a readable robots.txt (8.9% of all 10,000 scanned) block OpenAI’s GPTBot site-wide and 14.2% (8.2% of all 10,000 scanned) block Anthropic’s ClaudeBot, versus 0.2% for Googlebot (search, not an AI-training crawler; shown for comparison). - As of 2026-11-18, 10.0% of the 5,806 top-10,000 sites with a readable robots.txt (5.8% of all 10,000 scanned) block all four of GPTBot, ClaudeBot, Google-Extended and CCBot site-wide. - As of 2026-11-18, 9.9% of all 10,000 scanned top sites have an llms.txt present (format check: HTTP 200, text/plain or text/markdown, starts with a Markdown heading, not HTML; requested on the host that served robots.txt). - As of 2026-11-18, the most-blocked AI crawler of the 5,806 top-10,000 sites with a readable robots.txt is CCBot (16.1% block it site-wide). - As of 2026-11-18, 6.8% of the 5,806 top-10,000 sites with a readable robots.txt (3.9% of all 10,000 scanned) contain a Cloudflare Content-Signal directive. ## Key facts (remote MCP server health) - As of 2026-11-19, 56.7% of 45,311 remote MCP servers in the official MCP Registry completed an MCP initialize handshake without authentication; 33.4% were alive but required authentication; 9.9% failed (single weekly check). - As of 2026-11-19, the median working remote MCP server in the official registry exposes 7.0 tools (mean 16.1), with a median initialize latency of 293 ms. ## Reports - [State of remote MCP servers — November 2026](https://agents.datacensus.org/reports/mcp-health-2026-11/): liveness by age, publisher, host type; protocol versions; tools ## Pages - [Do remote MCP servers work?](https://agents.datacensus.org/mcp/health/): weekly initialize check of every remote server - [Working remote MCP servers](https://agents.datacensus.org/mcp/health/working/) - [Does block AI crawlers?](https://agents.datacensus.org/ai-crawlers/sites/): per-site pages for top websites - [MCP server statistics](https://agents.datacensus.org/mcp/): totals, growth, deployment, categories - [New MCP servers this week](https://agents.datacensus.org/mcp/new/) - [AI crawler statistics](https://agents.datacensus.org/ai-crawlers/): which bots top websites block - [Methodology](https://agents.datacensus.org/about/) ## Data - [mcp-latest.json](https://agents.datacensus.org/data/mcp-latest.json) - [mcp-history.csv](https://agents.datacensus.org/data/mcp-history.csv) - [mcp-servers/1.json](https://agents.datacensus.org/data/mcp-servers/1.json): all servers, compact - [web-latest.json](https://agents.datacensus.org/data/web-latest.json)