AI agents need data from the web. Whether it's researching competitors, monitoring prices, enriching leads, or gathering training data—web access is fundamental to what agents do.
Today we're excited to highlight Olostep, a powerful web data API now available on Orthogonal.
What is Olostep?
Olostep is a comprehensive web data platform built specifically for AI applications. It goes beyond basic scraping to offer a complete toolkit: real-time scraping, multi-page crawling, batch processing at scale, AI-powered search answers, and site mapping.
Trusted by leading AI startups like Gumloop, Openmart, Athena, and dozens more, Olostep has become the default web infrastructure layer for companies building intelligent applications.
Key Features
Scrape - Clean Data from Any URL
Get LLM-friendly Markdown, HTML, or structured JSON from any webpage. Full JavaScript rendering and residential IPs are included on every request—no extra configuration needed.
Crawl - Explore Entire Websites
Recursively crawl through a website's subpages and gather all content. Works without a sitemap, making it perfect for documentation sites and knowledge bases.
Answers - AI-Powered Web Search
Search the web and get direct LLM-powered answers to questions, complete with sources. Ideal for research agents that need to find and synthesize information.
Batches - Process at Scale
Submit up to 100,000 URLs and get results back in 5-7 minutes. Essential for deep research, monitoring, and large-scale data aggregation.
Maps - Discover All URLs
Get a complete list of all URLs on any website. Perfect for understanding site structure before crawling or building comprehensive sitemaps.
Using Olostep with Orthogonal
Scrape - Get Markdown from Any URL
import Orthogonal from "@orth/sdk";
const orthogonal = new Orthogonal({
apiKey: process.env.ORTHOGONAL_API_KEY,
});
// Scrape a webpage and get clean markdown
const result = await orthogonal.run({
api: "olostep",
path: "/v1/scrapes",
body: {
url_to_scrape: "https://docs.stripe.com/api"
}
});
console.log(result.data.markdown_content);Answers - AI-Powered Web Search
// Search the web and get AI-powered answers
const result = await orthogonal.run({
api: "olostep",
path: "/v1/answers",
body: {
task: "What are the top 3 competitors to Notion?",
json: {
competitors: [],
summary: ""
}
}
});
console.log(result.data);Crawl - Get All Pages from a Site
// Crawl an entire documentation site
const result = await orthogonal.run({
api: "olostep",
path: "/v1/crawls",
body: {
start_url: "https://docs.example.com",
include_urls: ["/**"],
max_pages: 50
}
});
console.log(result.data);Using x402 Protocol
Olostep on Orthogonal also supports x402—an open protocol that enables native payments in HTTP. With x402, your agents can pay for API calls directly using USDC stablecoins, with no API keys required.
// Install: npm install x402-fetch viem
import { wrapFetchWithPayment } from "x402-fetch";
import { privateKeyToAccount } from "viem/accounts";
const privateKey = process.env.PRIVATE_KEY;
if (!privateKey) {
throw new Error("PRIVATE_KEY environment variable is required");
}
const account = privateKeyToAccount(
privateKey.startsWith("0x") ? privateKey : `0x${privateKey}`,
);
const fetchWithPayment = wrapFetchWithPayment(fetch, account);
// Make a paid request - payment is automatic
const url = "https://x402.orth.sh/olostep/v1/scrapes";
const response = await fetchWithPayment(url, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url_to_scrape: "https://docs.stripe.com/api" })
});
const data = await response.json();
console.log(data);x402 enables a new paradigm where AI agents can autonomously pay for services using stablecoins—no subscriptions, no API key management, just seamless agent-to-agent payments.
Why Agents Choose Olostep
- LLM-optimized output: Get clean Markdown that feeds directly into your AI models
- Scale without limits: Process up to 100k URLs with batch endpoints
- AI-powered answers: Get direct answers to questions with source citations
- Site discovery: Maps endpoint reveals all URLs on any website
- Reliability: 99.5% uptime with premium proxies on every request
Available Endpoints
Olostep offers a comprehensive set of endpoints through Orthogonal:
| Endpoint | Method | Price | Description |
|---|---|---|---|
| /v1/scrapes | POST | $0.01 | Initiate a web page scrape |
| /v1/crawls | POST | Dynamic | Start a new crawl (1-10 mins depending on depth) |
| /v1/answers | POST | $0.05 | AI-powered web search with direct answers |
| /v1/batches | POST | Dynamic | Start a new batch job |
| /v1/maps | POST | $0.01 | Get all URLs on a website |
Try It Today
Sign up for Orthogonal and get $10 free credits to try Olostep and dozens of other APIs. No API keys to manage, no accounts to create—just instant access to powerful tools for your agents.