Open data on MCP servers (with live health) and which AI crawlers top websites block.
Maintain this server? Claim it
Call this server's tools from your browser. Nothing to install.
Requests go through mcptry because browsers can't call most MCP servers directly. Header values are sent to the server for that request only and are never stored or logged.
Tool list from our last check. Connect live to refresh it.
Aggregate AI-crawler blocking statistics across the top 10,000 websites: percentage of sites with a reachable robots.txt that fully block, partly restrict or explicitly allow each AI bot. Give a bot name for one bot, or omit for the overall table.
Optional bot name, e.g. 'GPTBot', 'ClaudeBot', 'Google-Extended' (case-insensitive).
Pick your client. Values in capitals, like YOUR_TOKEN, are yours to fill in.
Run in a terminal
claude mcp add --transport http agents https://datacensus.org/mcp
Aggregate AI-crawler blocking statistics across the top 10,000 websites: percentage of sites with a reachable robots.txt that fully block, partly restrict or explicitly allow each AI bot. Give a bot name for one bot, or omit for the overall table.
| Argument | Type | Description |
|---|---|---|
| bot | string | Optional bot name, e.g. 'GPTBot', 'ClaudeBot', 'Google-Extended' (case-insensitive). |
{
"type": "object",
"properties": {
"bot": {
"type": "string",
"description": "Optional bot name, e.g. 'GPTBot', 'ClaudeBot', 'Google-Extended' (case-insensitive)."
}
}
}Search the official MCP Registry (~65,000 servers) by keyword. Matches name, title and description; ranks servers whose remote endpoint passed a live initialize check first, then by number of tools. Returns name, title, description, remote/package types, health (ok = works without auth, auth = alive but needs auth, fail = unreachable), tool count and a registry link.
| Argument | Type | Description |
|---|---|---|
| limit | integer | Max results (1-25). |
| query* | string | Keywords, e.g. 'postgres database' or 'github issues'. |
| category | string | Optional category filter, e.g. 'finance & crypto', 'databases' (substring match). |
| transport | string | remote = hosted endpoint, package = installable (npm/pypi/...), any (default). |
| working_only | boolean | Only servers whose remote endpoint responded OK or requires auth (default true). Package-only servers are kept unless transport is 'remote'. |
{
"type": "object",
"required": [
"query"
],
"properties": {
"limit": {
"type": "integer",
"default": 10,
"maximum": 25,
"minimum": 1,
"description": "Max results (1-25)."
},
"query": {
"type": "string",
"description": "Keywords, e.g. 'postgres database' or 'github issues'."
},
"category": {
"type": "string",
"description": "Optional category filter, e.g. 'finance & crypto', 'databases' (substring match)."
},
"transport": {
"enum": [
"remote",
"package",
"any"
],
"type": "string",
"description": "remote = hosted endpoint, package = installable (npm/pypi/...), any (default)."
},
"working_only": {
"type": "boolean",
"default": true,
"description": "Only servers whose remote endpoint responded OK or requires auth (default true). Package-only servers are kept unless transport is 'remote'."
}
}
}Headline numbers of the MCP ecosystem: servers in the official registry, new servers in the last 7/30 days, remote vs package share, top package registries and categories, plus the weekly health check of remote servers (share working, requiring auth, failing, median tools and latency).
No arguments.
{
"type": "object",
"properties": {}
}Which AI crawlers does a website block in robots.txt? Looks up a domain in our scan of the top 10,000 websites (Tranco list) and returns, per AI bot (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, ...), whether it is blocked, partly restricted or explicitly allowed, plus llms.txt presence.
| Argument | Type | Description |
|---|---|---|
| domain* | string | Domain or URL, e.g. 'nytimes.com' or 'https://www.nytimes.com/section'. |
{
"type": "object",
"required": [
"domain"
],
"properties": {
"domain": {
"type": "string",
"description": "Domain or URL, e.g. 'nytimes.com' or 'https://www.nytimes.com/section'."
}
}
}We read every tool's name, description and inputs the way the model will, looking for instructions hidden from you. A clean result means no warning signs, not a guarantee. Scanned Nov 19, 2026.