Developers building crawlers or data pipelines that feed scraped web content into AI and RAG systems.

A Node.js and TypeScript library for web scraping and browser automation that unifies Playwright, Puppeteer, Cheerio, and raw HTTP crawling with built-in proxy rotation and queue management.

Runs as a library inside Node.js scraping jobs, either headless or headful, locally or in the cloud.

Use when a project needs dependable, resumable crawls to collect training data or documents for retrieval.

It handles the retry, queueing, and anti-blocking plumbing that hand-rolled scrapers usually get wrong.

Wraps Playwright, Puppeteer, Cheerio, and JSDOM behind one consistent crawler API.

Open source, Apache 2.0 licensed, maintained by Apify; free to use with optional paid Apify cloud hosting.