Documentation Index
Fetch the complete documentation index at: /llms.txt
Use this file to discover all available pages before exploring further.
List visited, pending, failed, or skipped crawler URLs.
scrapfly.listCrawlUrls reads one materialized page from Scrapfly’s streaming URL endpoint.
import { automation, onInvocation } from "automate.ax"
import { scrapfly } from "automate.ax/scrapfly"
export default automation("List failed crawl URLs", () => {
onInvocation()
scrapfly.listCrawlUrls({
crawlerId: "crawl-id",
status: "failed",
perPage: 250,
})
})
| Input | Type | Required | Default | Description |
|---|---|---|---|---|
crawlerId | string | Yes | — | Crawler job ID. |
status | 'visited' | 'pending' | 'failed' | 'skipped' | No | Visited | URL state filter. |
page | number | No | 1 | One-based page. |
perPage | number | No | 100 | Records per page, up to 1,000. |
{ page, perPage, urls }. Failed and skipped entries can include reason. Scrapfly does not return a global total; increment page until urls is empty.