Website crawl

POST/web/crawl

Create a crawl job to extract content from all pages on a website.

Listed by OrthogonalPublished by SupadataPublisher docs
curl -X POST https://api.orthogonal.com/v1/run \
-H 'Authorization: Bearer orth_live_••••' \
-H 'Content-Type: application/json' \
-d '{
"api": "supadata",
"path": "/web/crawl",
"method": "POST",
"body": {
"url": "",
"limit":
}
}'
{
"success": true,
"data": {
"jobId": "00416487-3bb5-48be-8a7c-0e6ba6292140"
}
}
A real response, recorded when Orthogonal checked this endpoint.

Changes with the parameters.

$0.0109per page

1 page
$0.0109
100 pages (default)
$1.09

Body parameters

Response types

More from Supadata

  1. Video transcriptGET/transcript$0.0109 base price
  2. YouTube metadata batchPOST/youtube/video/batch$0.0109 per call
  3. YouTube channel metadataGET/youtube/channel$0.0109 per call
  4. YouTube batch resultsGET/youtube/batch/{jobId}Free
  5. YouTube searchGET/youtube/search$0.0109 per call
  6. Social media metadataGET/metadata$0.0109 per call
  7. Website URL mapGET/web/map$0.0109 per call
  8. YouTube channel videosGET/youtube/channel/videos$0.0109 per call
  9. Video data extractionPOST/extract$9.81 per call
  10. Website crawl resultsGET/web/crawl/{jobId}Free
  11. YouTube playlist metadataGET/youtube/playlist$0.0109 per call
  12. YouTube transcript translationGET/youtube/transcript/translate$0.3270 base price
All 19 Supadata endpoints