Quarry is a self-hosted scraper and crawler built for AI agent context. Scrape to clean markdown, crawl entire sites, monitor for changes, and extract structured data, all on infrastructure you own, not ours.
curl -fsSL https://raw.githubusercontent.com/samuelorobosa/quarry/main/install.sh | bash> curl -X POST http://localhost:3000/scrape \
-H "Content-Type: application/json" \
-d '{ "url": "https://example.com" }'
{
"url": "https://example.com",
"title": "Example Domain",
"markdown": "# Example Domain...",
"scraped-at": "2026-06-20T10:00:00Z"
}No vendor API keys. No usage caps. No one else's servers touching your pages.
Single page in, clean markdown out. No boilerplate, no ads, no nav chrome.
Multi-page crawl as an async job. Depth limits, include/exclude patterns, webhook on completion.
Recurring crawl on a schedule. Webhook fires only when content actually changes.
Page to structured JSON via your configured LLM. Missing fields return null, never hallucinated.
Jobs, monitors, queue depth, and config, all in one ops view.
Prometheus metrics for everything the worker touches.
Nothing leaves your infrastructure. No third-party scraping API in the loop.
OpenAI, Anthropic, Ollama, or any OpenAI-compatible endpoint for extraction.
AGPL-3.0, no open-core bait. Fork it, run it, ship it. The source stays open.
Clone it, self-host it, point it at a URL.
curl -fsSL https://raw.githubusercontent.com/samuelorobosa/quarry/main/install.sh | bash