// Read docs & web pages
crawleev3.18.2
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Open source Maintained For agents
IN ONE PARAGRAPH
crawlee is crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Good for: Developers who want their agent (Claude Code, Codex, Cursor…) to read docs & web pages.
01How to start
- 1Read the official README for requirements and permissions.
- 2Install it from the official package registry:
npm install crawlee - 3Tell your agent the command exists (e.g. in AGENTS.md / CLAUDE.md) so it knows when to call it.