> ## Documentation Index
> Fetch the complete documentation index at: https://docs.automate.ax/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Automate.ax automations are TypeScript programs.
> Use Bun for installation and command examples.
> Use Automate.ax for the product name and automate.ax for the package, CLI, and domain.
> Treat documented public APIs as current; do not invent transitional or deprecated names.

# Scrape URLs with Scrapfly

> Scrape up to 100 independently configured URLs through Scrapfly's streaming batch endpoint.

`scrapfly.scrapeUrls` sends one atomic batch while preserving a correlation ID for every result.

Scrapfly offers the Batch API on paid plans. Free-plan requests fail before any URL is scraped.

```ts theme={null}
import { automation, onInvocation } from "automate.ax"
import { scrapfly } from "automate.ax/scrapfly"

export default automation("Scrape product pages", () => {
  onInvocation()
  scrapfly.scrapeUrls({
    requests: [
      {
        correlationId: "starter",
        url: "https://example.com/starter",
        format: "markdown",
      },
      { correlationId: "pro", url: "https://example.com/pro", renderJs: true },
    ],
  })
})
```

| Input                      | Type                    | Required | Default                     | Description                                                    |
| -------------------------- | ----------------------- | -------- | --------------------------- | -------------------------------------------------------------- |
| `requests`                 | `ScrapflyScrapeInput[]` | Yes      | —                           | One to 100 inputs accepted by `scrapfly.scrapeUrl`.            |
| `requests[].correlationId` | `string`                | No       | `request-1`, `request-2`, … | Stable ID used to match streamed parts. Values must be unique. |

Returns `{ results }`. Each entry contains `correlationId` and either the same `result` shape as `scrapfly.scrapeUrl` or `{ error: { name, message } }`. Scrapfly can complete parts out of order; use `correlationId`, not array position, to join them.