com.atalaya-ai/ai-crawler-checkv1.1.0 Domain verified
Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.
Maintained byatalaya-ai.com(verified)Also a maintainer?
Try it
Call this server's tools from your browser. Nothing to install.
https://atalaya-ai.com/mcp
Requests go through mcptry because browsers can't call most MCP servers directly. Header values are sent to the server for that request only and are never stored or logged.
Tool list from our last check. Connect live to refresh it.
check_ai_crawler_access Read-only
Checks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and more) robots.txt allows and the exact rule for each, how much text the homepage HTML carries for crawlers that do not run JavaScript (flags client-rendered app shells, common in apps built with Lovable, Bolt, Base44 or Replit), title/meta description/H1, llms.txt, sitemap, and the platform or host setting to change (Cloudflare, Vercel, Netlify). Use it when someone asks why their site or app does not show up in ChatGPT, Claude, Perplexity or other AI answers.
Public domain or URL, for example example.com
Language of the verdict text (default en)
Install
Pick your client. Values in capitals, like YOUR_TOKEN, are yours to fill in.
Run in a terminal
claude mcp add --transport http ai-crawler-check https://atalaya-ai.com/mcp
Tools (2)
check_ai_crawler_accessChecks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and more) robots.txt allows and the exact rule for each, how much text the homepage HTML carries for crawlers that do not run JavaScript (flags client-rendered app shells, common in apps built with Lovable, Bolt, Base44 or Replit), title/meta description/H1, llms.txt, sitemap, and the platform or host setting to change (Cloudflare, Vercel, Netlify). Use it when someone asks why their site or app does not show up in ChatGPT, Claude, Perplexity or other AI answers.
read-only
check_ai_crawler_access
Checks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and more) robots.txt allows and the exact rule for each, how much text the homepage HTML carries for crawlers that do not run JavaScript (flags client-rendered app shells, common in apps built with Lovable, Bolt, Base44 or Replit), title/meta description/H1, llms.txt, sitemap, and the platform or host setting to change (Cloudflare, Vercel, Netlify). Use it when someone asks why their site or app does not show up in ChatGPT, Claude, Perplexity or other AI answers.
| Argument | Type | Description |
|---|---|---|
| lang | string | Language of the verdict text (default en) |
| domain* | string | Public domain or URL, for example example.com |
Input schema (JSON)
{
"type": "object",
"required": [
"domain"
],
"properties": {
"lang": {
"enum": [
"en",
"es"
],
"type": "string",
"description": "Language of the verdict text (default en)"
},
"domain": {
"type": "string",
"description": "Public domain or URL, for example example.com"
}
},
"additionalProperties": false
}list_ai_crawlersLists the AI crawler user-agent tokens Atalaya checks, by vendor and purpose (training, search index, user-triggered fetch), so you can write a robots.txt that keeps AI search while opting out of training.
read-only
list_ai_crawlers
Lists the AI crawler user-agent tokens Atalaya checks, by vendor and purpose (training, search index, user-triggered fetch), so you can write a robots.txt that keeps AI search while opting out of training.
No arguments.
Input schema (JSON)
{
"type": "object",
"properties": {},
"additionalProperties": false
}Security checkA100/100
We read every tool's name, description and inputs the way the model will, looking for instructions hidden from you. A clean result means no warning signs, not a guarantee. Scanned Nov 3, 2026.
- Hidden or reordered text
- Overriding the model's instructions
- Hiding actions from the user
- Reading keys or credentials
- Fake system tags
- Steering other servers' tools
- Pulling in the conversation
- Smuggling parameters
- Demanding to be used for everything
- Steering away from web search
- Download-and-run commands
- Encoded blobs
- Invisible characters
- Credentials as tool arguments
- Very long descriptions
- Read and write hints