Crawl website sitemap

VerifiedGET /web/scrape/sitemapPublished by Context.dev
curl -X POST https://api.orthogonal.com/v1/run \
-H 'Authorization: Bearer orth_live_••••' \
-H 'Content-Type: application/json' \
-d '{
"api": "context-dev",
"path": "/web/scrape/sitemap",
"method": "GET",
"query": {
"domain": ""
}
}'
{
"success": true,
"data": {
"meta": { … },
"urls": [ 1 item ],
"domain": "example",
"success": true,
"key_metadata": { … }
}
}
Example values, built from the documented fields.

$0.0300per call

API-wide: 98.4% success · 18,774 ms median · 30 days

About this endpoint

Crawls the sitemap of the given domain and returns all discovered page URLs. Supports sitemap index files (recursive), parallel fetching with concurrency control, deduplication, and filters out non-page resources (images, PDFs, etc.).

Query parameters

Response types

More from Context.dev

  1. Retrieve brand data by email addressGET/brand/retrieve-by-email$0.0300 per call
  2. Query website data using AIPOST/brand/ai/query$0.0300 per call
  3. Scrape images from a URLGET/web/scrape/images$0.0300 per call
  4. Scrape ScreenshotGET/web/screenshot$0.0300 per call
  5. Retrieve simplified brand data by domainGET/brand/retrieve-simplified$0.0300 per call
  6. Retrieve brand data by stock tickerGET/brand/retrieve-by-ticker$0.0300 per call
  7. Prefetch brand data for a domainPOST/brand/prefetch$0.0300 per call
  8. Crawl website and extract MarkdownPOST/web/crawl$0.0300 per call
  9. Retrieve brand data by domainGET/brand/retrieve$0.0300 per call
  10. Scrape raw HTML from a URLGET/web/scrape/html$0.0300 per call
  11. Classify NAICS IndustriesGET/web/naics$0.0300 per call
  12. Retrieve brand data by ISINGET/brand/retrieve-by-isin$0.0300 per call
All 26 Context.dev endpoints