doc-scraper
Local installio.github.Sriram-PR/doc-scraper
Crawl documentation sites into local corpora agents can search, read, and diff fully offline.
372 servers in Web Search
Crawl documentation sites into local corpora agents can search, read, and diff fully offline.
Evidence-backed web research for AI agents with citations and confidence scores.
Google SERP + AI Overviews, AI Mode, SEA news mentions, Maps, Jobs, Trends and 13 more data tools.
A polite crawler for agents: obeys robots.txt, fetches each page once, finds and verifies quotes.
Keyless web access for AI agents: read bot-walled pages as Markdown, search, grab assets
MCP server for the Serper.dev search API with per-call credit accounting
Build job feeds from careers pages; query enriched jobs and hiring stats. Needs a YubHub API key
Local Go MCP server: DuckDuckGo/Bing web + image search, readability scraping to Markdown.
Token-optimized HTTP client for LLMs: fetches any URL as clean markdown. CLI and MCP server.
Convert a web page URL to clean Markdown for LLM context. Local, no external API calls.
Free proxies that actually work: checked HTTP/SOCKS proxies, and pages fetched through them
Unofficial read-only metadata search of Soft98, P30Download, Downloadly and VGDL. Links only.
Search LinkedIn, Indeed, ZipRecruiter, Glassdoor and regional job boards
Provider-neutral web_search for AI agents. Grounding + agent search APIs, keyless defaults.
Sync public web sources into cited context packs for AI agents, RAG, and MCP clients.
Turn any URL into clean Markdown or text for LLMs, RAG and agents. Strips nav, ads and footers.
Free search: up to 25/day per IP, shared capacity. Paid extract $0.005/call (x402); Review Board.
Web pages, PDFs and documents as small, clean, cached Markdown for AI agents
Scrape, crawl, and map websites to Markdown or JSON via local CLI.
Open-source web scraper for AI agents with scrape, crawl, and map tools
Web intelligence for AI agents: fetch, render, extract, research. x402 micropayments, no API keys.
In-memory Bloom filter for URL and document hash deduplication preventing redundant web crawler inge
Live Google search results, domain positions and open markets via the Serper.live SERP API.
Live jobs from company Greenhouse, Lever and Ashby boards with age, long-open and repost signals.
Turn any web page into clean Markdown, text or token-bounded RAG chunks (via RapidAPI).
Read any web page and search the web: 28 tools, cheapest first, every page checked.
Web access for AI agents: URL to Markdown, link metadata, screenshots, links and images of any URL.
Find a named page type on a company's own website from its domain, in 11 languages.
Pay-per-call x402 agent tools (USDC): web reader, token/wallet/email/site checks, screenshots, PDFs.
Web scraping MCP server — scrape, extract structured data, screenshot any site with anti-bot bypass.