Ai Crawler Policy
Workingio.github.lazymac2x/ai-crawler-policy
Cloudflare Workers MCP server: ai-crawler-policy
463 servers in Web Search
Cloudflare Workers MCP server: ai-crawler-policy
Google SERP as JSON: organic, AI Overview, People Also Ask, AI Mode, news, shopping. No Cloud setup.
Brave Search MCP — independent web index (no Google/Bing dependency)
DuckDuckGo search as JSON. Ranked results, ads in their own array, 37 regions.
Bing search results with the ad block and Copilot answer, geo and market targeting, as JSON.
Hyperbrowser MCP — wraps the Hyperbrowser AI-agent browsing API (hyperbrowser.ai). Scrape…
x402 MCP with exact and batched payments for recurring wallet, trading, market and web workflows.
Common Crawl's public web archive — find every time a URL was crawled since 2008 and read the…
Arquivo.pt MCP — full-text search over the Portuguese web archive.
Google Images results with source pages, thumbnails and full-resolution URLs, as JSON.
Recurring webpage change, price, inventory and metadata monitoring for autonomous agents.
Archive MCP — wraps the Internet Archive APIs (free, no auth)
Cloudflare Solver: Scraping API designed to bypass Cloudflare protection.
Apify MCP — run web-scraping Actors and fetch their dataset results.
Bypass Cloudflare API: A powerful scraping API designed to bypass Cloudflare protection.
web-fetch — fetch ONE public URL live from its origin, verbatim.
Image Search: Fast and Simple Image Search API. You can get 100+ search results in one query!.
Search job postings across Indeed, LinkedIn, and more from one request - titles, companies
Exa MCP — neural/semantic web search + content retrieval (exa.ai)
SERP data scraper: Scrape Search Results and get all the data you need. On the JoJ API marketplace.
Structured Google search, news, and local results at scale - organic results, news, and local packs
Diffbot MCP — Knowledge Graph company enrichment + web content extraction (diffbot.com)
ScrapeUnblocker: ScrapeUnblocker allows to bypass anti-bot services and scrape the full page source.
Turn any webpage into structured JSON with CSS-selector schemas - strings select text, attributes
Decodo MCP — wraps the Decodo Web Scraping API (decodo.com, formerly Smartproxy). Rotating…
google search: google web search api, web, images, videos, news, music, favicon, proxy, audio.
Crawlbase MCP — wraps the Crawlbase Crawling API (crawlbase.com, formerly ProxyCrawl). One…
Google Web Search: Google Web Search API. Search the world’s information, including webpages.
ScrapingBee MCP — wraps the ScrapingBee headless-scraping API (scrapingbee.com)
ScrapingAnt MCP — wraps the ScrapingAnt web scraping API (scrapingant.com)