Crawlee

Crawlee is a web scraping and browser automation library for Node.js and TypeScript, aimed at building reliable crawlers with minimal extra configuration. It's designed so crawlers behave in a more human-like way by default, helping them avoid triggering common bot-detection systems, while giving you the tools to crawl for links, scrape data, and store results to disk or the cloud.

It integrates with Playwright (already used elsewhere in this catalog for browser automation) as an optional dependency when a crawler needs a real browser rather than raw HTTP requests, and a separate Crawlee for Python package exists for teams working in Python instead of the JavaScript/TypeScript ecosystem this npm package targets.

Crawlee is a fit for JavaScript/TypeScript teams building scrapers or data-extraction pipelines - including for feeding LLM or RAG pipelines - who want a maintained library with built-in anti-detection defaults, complementing Python-first tools like Scrapy in the current catalog rather than duplicating them.