Create web scrape - Automate.ax

closeBot.createWebScrape crawls a website into a CloseBot library file.

automations/create-closebot-web-scrape.automation.ts

import { automation, onDashboardRun } from "automate.ax"
import { closeBot } from "automate.ax/closebot"

export default automation("Create CloseBot web scrape", () => {
  onDashboardRun({ title: "Create web scrape" })
  closeBot.createWebScrape({
    scrapeUrl: "https://example.com/docs",
    maxDepth: 2,
    maxBreadth: 20,
  })
})

Inputs

Provide scrapeUrl, nonnegative maxDepth, and nonnegative maxBreadth crawl limits.

Output

Returns the new file id, a provider message, and totalSize. Use getLibraryFile or listLibraryFiles to monitor processing status.