CrawlWitness

Working Security A

com.crawlwitness/oraclev1.0.52 Domain verified

AI consent & licensing check for any domain, with signed on-chain receipts. Paid via x402.

Legal & Government

Watch Website Registry entry

Maintain this server? Claim it

Try it

Call this server's tools from your browser. Nothing to install.

https://api.crawlwitness.com/mcp

Requests go through mcptry because browsers can't call most MCP servers directly. Header values are sent to the server for that request only and are never stored or logged.

Tool list from our last check. Connect live to refresh it.

ai_consent_score Read-only

Free summary for a domain: 0-100 signal score and AI-training stance (reservado = reserved, permitido = allowed, sin_senal = no signal), from cache when available.

domainstring, required

Domain to check, without scheme or path (e.g. nytimes.com)

Tools (6)

ai_consent_score

Free summary for a domain: 0-100 signal score and AI-training stance (reservado = reserved, permitido = allowed, sin_senal = no signal), from cache when available.

read-only

Argument	Type	Description
domain*	string	Domain to check, without scheme or path (e.g. nytimes.com)

Input schema (JSON)

{
  "type": "object",
  "required": [
    "domain"
  ],
  "properties": {
    "domain": {
      "type": "string",
      "maxLength": 253,
      "minLength": 3,
      "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
    }
  },
  "additionalProperties": false
}

can_ai_use

Fast yes/no verdict for one domain, one AI bot and one use (train, rag or search), from robots.txt, IETF AIPREF, W3C TDMRep, noai directives and RSL licenses, with the reasons and a signed receipt. Use it in a crawler or RAG pipeline before fetching each new domain. Costs 0.001 USD on Base (0.002 on Solana) via x402. Invalid domain or unknown bot is never charged.

read-only

Argument	Type	Description
bot*	string	AI user-agent token (e.g. GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot)
use	string
domain*	string	Domain to check, without scheme or path (e.g. nytimes.com)

Input schema (JSON)

{
  "type": "object",
  "required": [
    "domain",
    "bot"
  ],
  "properties": {
    "bot": {
      "enum": [
        "GPTBot",
        "ClaudeBot",
        "anthropic-ai",
        "Google-Extended",
        "GoogleOther",
        "CloudVertexBot",
        "Applebot-Extended",
        "meta-externalagent",
        "Amazonbot",
        "amazon-kendra-",
        "Bytespider",
        "CCBot",
        "cohere-training-data-crawler",
        "AI2Bot",
        "Diffbot",
        "Timpibot",
        "ImagesiftBot",
        "omgili",
        "PetalBot",
        "KimiBot",
        "ICC-Crawler",
        "Cotoyogi",
        "Reflectionbot",
        "CloudflareBrowserRenderingCrawler",
        "LinerBot",
        "FishBot",
        "Novellum",
        "bigsur.ai",
        "NavuBot",
        "QualifiedBot",
        "atlassian-bot",
        "make.com",
        "magpie-crawler",
        "Awario",
        "w4mwnpbXf3MFAbxOkJRw",
        "SemrushBot-OCOB",
        "SemrushBot-SWA",
        "BorderxBot",
        "SelectikaScraper",
        "CitibotSiteCrawler",
        "AdpResearchBot",
        "WARDBot",
        "ygs-scraper-bot",
        "OAI-SearchBot",
        "Claude-SearchBot",
        "PerplexityBot",
        "Applebot",
        "Amzn-SearchBot",
        "Bravebot",
        "Cloudflare-AI-Search",
        "Cloudflare-AI-Search-External",
        "DuckAssistBot",
        "YouBot",
        "KernelSearchBot",
        "ShapBot",
        "alphalens-bot",
        "Anomura",
        "Element451Bot",
        "ChatGPT-User",
        "Claude-User",
        "Perplexity-User",
        "meta-externalfetcher",
        "meta-externaltest",
        "muse-",
        "MistralAI-User",
        "cohere-ai",
        "Google-Agent",
        "GrokAgent",
        "Devin",
        "ApifyWebsiteContentCrawler",
        "Instapaper",
        "Retool",
        "QATechBot",
        "TwinAgent",
        "CledaraBot",
        "HarkBot",
        "HIFIBot",
        "ChathiveCrawler",
        "EasyScan",
        "Nava",
        "ZipchatBot"
      ],
      "type": "string",
      "description": "AI user-agent token (e.g. GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot)"
    },
    "use": {
      "enum": [
        "train",
        "rag",
        "search"
      ],
      "type": "string",
      "default": "train"
    },
    "domain": {
      "type": "string",
      "maxLength": 253,
      "minLength": 3,
      "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
    }
  },
  "additionalProperties": false
}

check_ai_consent

Full AI-consent report for a domain: robots.txt rules for each AI bot (training, search and user-agent crawlers), RSL license, IETF AIPREF Content-Usage, W3C TDMRep, noai meta tags, llms.txt and C2PA, combined into a 0-100 score and a training stance, plus a verifiable receipt (EAS attestation anchored on Base). Use it before crawling, training on or ingesting a site. Costs 0.005 USD in USDC via x402 (Base or Solana); without a payment it returns the x402 payment options. Unreachable sites are not charged.

read-only

Argument	Type	Description
pay	boolean	Pay directly instead of using the small daily free quota
fresh	boolean	Force a fresh analysis instead of the cached one (<24 h)
domain*	string	Domain to check, without scheme or path (e.g. nytimes.com)

Input schema (JSON)

{
  "type": "object",
  "required": [
    "domain"
  ],
  "properties": {
    "pay": {
      "type": "boolean",
      "default": false,
      "description": "Pay directly instead of using the small daily free quota"
    },
    "fresh": {
      "type": "boolean",
      "default": false,
      "description": "Force a fresh analysis instead of the cached one (<24 h)"
    },
    "domain": {
      "type": "string",
      "maxLength": 253,
      "minLength": 3,
      "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
    }
  },
  "additionalProperties": false
}

check_wallet

Sanctions screening of a crypto address against the US OFAC SDN list (all its digital-currency addresses: EVM, BTC, TRX, SOL, USDT, USDC…). Use it before an agent pays, receives from or interacts with a wallet. Returns sanctioned true/false, the listed entity and programs, the exact list version used and a signed receipt. Costs 0.005 USD via x402. Screening aid, not legal advice.

read-only

Argument	Type	Description
address*	string	Crypto address (EVM 0x…, BTC, TRX, SOL…)

Input schema (JSON)

{
  "type": "object",
  "required": [
    "address"
  ],
  "properties": {
    "address": {
      "type": "string",
      "maxLength": 110,
      "minLength": 20,
      "description": "Crypto address (EVM 0x…, BTC, TRX, SOL…)"
    }
  },
  "additionalProperties": false
}

signal_history

What a domain's robots.txt (or llms.txt, tdmrep.json, RSL license, homepage headers) said at a given instant, with the SHA-256 of each version and its EAS attestation anchored on Base. Without `source` it returns the index of versions; with `source` (and optional `at`, RFC 3339) the document in force at that instant. Covers only domains already analysed.

read-only

Argument	Type	Description
at	string	Instant (RFC 3339); default now
domain*	string	Domain to check, without scheme or path (e.g. nytimes.com)
source	string	portada = homepage headers

Input schema (JSON)

{
  "type": "object",
  "required": [
    "domain"
  ],
  "properties": {
    "at": {
      "type": "string",
      "format": "date-time",
      "description": "Instant (RFC 3339); default now"
    },
    "domain": {
      "type": "string",
      "maxLength": 253,
      "minLength": 3,
      "description": "Domain to check, without scheme or path (e.g. nytimes.com)"
    },
    "source": {
      "enum": [
        "robots",
        "llms",
        "tdmrep",
        "rsl",
        "portada"
      ],
      "type": "string",
      "description": "portada = homepage headers"
    }
  },
  "additionalProperties": false
}

verify_receipt

Returns the signed EAS attestation of a paid query and, once anchored, its Merkle proof (OpenZeppelin format) and the on-chain attestation on Base that contains it.

read-only

Argument	Type	Description
uid*	string	Attestation UID (0x + 64 hex)

Input schema (JSON)

{
  "type": "object",
  "required": [
    "uid"
  ],
  "properties": {
    "uid": {
      "type": "string",
      "pattern": "^0x[0-9a-fA-F]{64}$",
      "description": "Attestation UID (0x + 64 hex)"
    }
  },
  "additionalProperties": false
}

CrawlWitness

Try it

ai_consent_score Read-only

CrawlWitness

Try it

ai_consent_score Read-only

Install

Tools (6)

Security checkA100/100