Check which AI bots a site's robots.txt allows; read-only facts about Parallel World
Maintain this server? Claim it
Call this server's tools from your browser. Nothing to install.
Requests go through mcptry because browsers can't call most MCP servers directly. Header values are sent to the server for that request only and are never stored or logged.
Tool list from our last check. Connect live to refresh it.
Reads the public robots.txt of a website (one request to https://<host>/robots.txt, nothing else on the site) and says, for each AI crawler and AI agent of the migurei bot registry (training crawlers such as GPTBot and ClaudeBot, search bots, fetchers that act for a user, and others), whether that bot is allowed or disallowed for the path "/", which robots.txt rule decided it, and whether the file exists. Input is a domain or URL; only public domain names are accepted (no IP addresses, no ports, no private hosts). Read-only; the domain is not stored. robots.txt is a request to bots, not access control, and a User-Agent is only a claim.
A public domain name or a URL, for example example.com or https://example.com/page. Only its host is used.
Reads the public robots.txt of a website (one request to https://<host>/robots.txt, nothing else on the site) and says, for each AI crawler and AI agent of the migurei bot registry (training crawlers such as GPTBot and ClaudeBot, search bots, fetchers that act for a user, and others), whether that bot is allowed or disallowed for the path "/", which robots.txt rule decided it, and whether the file exists. Input is a domain or URL; only public domain names are accepted (no IP addresses, no ports, no private hosts). Read-only; the domain is not stored. robots.txt is a request to bots, not access control, and a User-Agent is only a claim.
| Argument | Type | Description |
|---|---|---|
| domain* | string | A public domain name or a URL, for example example.com or https://example.com/page. Only its host is used. |
{
"type": "object",
"required": [
"domain"
],
"properties": {
"domain": {
"type": "string",
"maxLength": 300,
"description": "A public domain name or a URL, for example example.com or https://example.com/page. Only its host is used."
}
},
"additionalProperties": false
}Returns the facts published on this website, in Japanese: service overview, plans, prices and payment terms, how the scanner behaves, how to contact us, and the operating company. Each fact comes with the URL of the page that shows it. Read-only; it returns nothing that is not on the pages.
| Argument | Type | Description |
|---|---|---|
| topic* | string | Which facts to return |
{
"type": "object",
"required": [
"topic"
],
"properties": {
"topic": {
"enum": [
"overview",
"pricing",
"scanner",
"contact",
"company",
"all"
],
"type": "string",
"description": "Which facts to return"
}
},
"additionalProperties": false
}