Node.js Web Scraping: Puppeteer, Cheerio, or a Scraping API?

The short answer

Start with Cheerio plus fetch. If the data you want is already in the HTML the server sent, Cheerio parses it in milliseconds, adds about one dependency, and costs nothing. Escalate to a real browser only when the page genuinely renders client-side, and when you do, reach for Playwright rather than Puppeteer: for most new Node work in 2026 it is the better default, with cleaner contexts, built-in waiting and multi-browser support. Reach for a scraping API when rendering, retries, proxy rotation, crawl orchestration and typed output would otherwise become a permanent side project for your team. Most Node projects should begin with Cheerio and only move up the ladder when a specific page forces them to.

Is Puppeteer or Cheerio better for web scraping?

Neither is better in general, because they do different jobs. Cheerio is a parser: you hand it an HTML string and it gives you a jQuery-style API over it. Puppeteer is a browser driver: it launches Chromium, executes the page's JavaScript, and lets you interact. If the data is in the server response, Cheerio wins on every axis. If it is not, Cheerio cannot help you at all.

The practical test takes ten seconds. Run curl -s https://example.com/page | grep "the value you want". If it comes back, you need a parser. If it does not, you need a renderer. Teams skip this test constantly and end up launching Chromium to scrape a static WordPress blog, which is roughly a hundred times slower and a hundred times more fragile than the alternative.

How do I scrape a website in Node.js?

Fetch the HTML with the built-in fetch (stable since Node 18), parse it with Cheerio, and select the fields you need. That is the whole loop for a static page. The only parts worth adding on day one are a real User-Agent, a timeout, and a check that the response was actually a 200 before you parse it.

import * as cheerio from "cheerio";

const res = await fetch("https://example.com/products/widget", {
  headers: { "User-Agent": "AcmeResearch/1.0 (+contact@acme.com)" },
  signal: AbortSignal.timeout(15_000),
});
if (!res.ok) throw new Error(`HTTP ${res.status}`);

const $ = cheerio.load(await res.text());

const product = {
  name:  $("h1.product-title").first().text().trim(),
  price: Number($("span[data-price]").attr("data-price")),
  stock: $("[data-in-stock]").length > 0,
};

Two things about that snippet matter more than they look. $("h1.product-title").text() returns an empty string when the selector matches nothing, so a site redesign gives you silently blank records rather than a stack trace. Validate before you write to a database. And Number(undefined) is NaN, which will happily flow through your pipeline into a chart. Cheerio is fast and cheap, but it will never tell you it stopped working.

Can Cheerio scrape JavaScript-rendered pages?

No. Cheerio does not execute JavaScript, does not have a DOM in the browser sense, and does not make network requests at all. It parses the string you give it. On a React or Vue app, that string is usually a near-empty shell with a <div id="root"></div> and a bundle tag, so there is nothing for Cheerio to find.

There is a middle path worth trying before you install a browser. Many single page apps ship their data as JSON inside a <script type="application/json"> block, a Next.js __NEXT_DATA__ payload, or a JSON-LD block. Cheerio can grab that script tag and you can JSON.parse it, which gets you structured data without a browser and is often more stable than any CSS selector. Failing that, check whether the page's own XHR endpoint is public. If neither works, you need rendering, either your own or a service that renders JavaScript pages before extracting. We wrote up the tradeoffs in more depth in when you actually need to render JavaScript while scraping.

Is Playwright better than Puppeteer for scraping?

For most new scraping projects, yes, though the gap is smaller than the internet suggests. Playwright gives you Chromium, Firefox and WebKit from one API, auto-waiting on locators that removes a whole genre of flaky waitForSelector code, cheap isolated browser contexts, and first-class request interception. Puppeteer remains excellent, is Chrome-only in practice, and if you already have a working Puppeteer fleet there is no reason to rewrite it.

The auto-waiting difference is the one you feel at 3am. In Puppeteer you write explicit waits and then discover the selector appears before the data populates. Playwright's locators retry until the element is actionable, so the same scrape gets meaningfully less flaky without you doing anything clever.

import { chromium } from "playwright";

const browser = await chromium.launch();
const context = await browser.newContext({
  userAgent: "AcmeResearch/1.0 (+contact@acme.com)",
});
const page = await context.newPage();

// Skip images and fonts: cuts bandwidth and page time a lot.
await page.route("**/*.{png,jpg,jpeg,webp,woff2}", (r) => r.abort());

try {
  await page.goto("https://example.com/products/widget", {
    waitUntil: "domcontentloaded",
    timeout: 30_000,
  });
  const price = await page.locator("[data-price]").first().textContent();
  console.log(price?.trim());
} finally {
  await context.close();
  await browser.close();
}

Cheerio, Puppeteer, Playwright and an API side by side

The failure mode that actually kills Node scrapers: memory

Selectors break loudly and you fix them in an afternoon. Memory kills a scraper at 4am on a Sunday, and the postmortem is always the same two mistakes.

Leaked pages and contexts. Every newPage() that is not closed keeps a renderer process alive. Miss the close on an exception path and the box climbs from 400 MB to 6 GB over a few hours until the OOM killer takes it, or worse, Chromium survives as a zombie after your Node process dies and nothing reaps it. Close pages and contexts in a finally block, always, and recycle the whole browser every 200 to 500 pages regardless of how healthy it looks. Chromium accumulates state you cannot see, and relaunching costs a second.

Unbounded concurrency. await Promise.all(urls.map(scrape)) over 5,000 URLs does not queue anything. It starts 5,000 requests at once, exhausts file descriptors, times out most of them, and hammers the target site hard enough to earn a block you deserved. Use a pool.

import pLimit from "p-limit";

const limit = pLimit(5);           // 5 in flight, not 5000
const results = await Promise.all(
  urls.map((url) => limit(async () => {
    const page = await context.newPage();
    try {
      return await scrape(page, url);
    } catch (err) {
      return { url, error: String(err) };   // never reject the whole batch
    } finally {
      await page.close();
    }
  })),
);

Two more habits worth stealing: set --disable-dev-shm-usage in Docker, because the default 64 MB /dev/shm makes Chromium crash in ways that look like network errors, and log RSS every few minutes so you can see the leak before the pager does. If you are running this yourself, the boring parts of getting the service deployed and kept running on a server are a real part of the total cost, and they rarely show up in the build-versus-buy spreadsheet.

How do I avoid getting blocked when scraping with Node.js?

Mostly by being slower and more honest than you want to be. The single biggest cause of blocks is request rate, not fingerprinting. Cap concurrency per host, add jitter between requests, cache aggressively so you never refetch an unchanged page, and back off exponentially on 429 and 5xx responses instead of retrying immediately in a tight loop.

Beyond rate: send a real User-Agent that identifies you and provides contact details, keep cookies across requests in a session so you look like one client rather than a thousand strangers, use HTTP/2 where the target supports it, and prefer the site's own JSON endpoints over rendering when they are public. Headless Chromium is detectable by design and you should not build a business on losing an arms race about that. If a site is clearly refusing automated access after you have behaved well, that is a decision, not a puzzle.

Where an API fits

The third option moves the fetching, rendering, retrying and parsing off your infrastructure. You POST a URL and a schema, and get typed fields back, which means your Node code stops containing CSS selectors entirely.

const res = await fetch("https://api.clawengine.ai/v1/extract", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CLAWENGINE_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://example.com/products/widget",
    format: "json",
    render: true,
    schema: { product_name: "string", price: "number", in_stock: "boolean" },
  }),
});

const { data } = await res.json();

What changes is the failure surface. There is no browser in your container, no memory ceiling to tune, no zombie Chrome, and adding the tenth source costs the same as adding the second, because a Node.js web scraping API takes a URL rather than a bespoke parser. Every record comes back with the same keys, which matters more than it sounds when the output lands in Postgres or a vector store.

What does not change: it costs money per request, forever, and Cheerio is free. You inherit a vendor's uptime and pricing, and you give up the fine control that a Playwright script gives you over clicks, scrolls and interception. For five stable static sources, writing thirty lines yourself is the better deal, and our plans start at $39 a month precisely because they are not aimed at that case.

Scrape the pages you are allowed to scrape

Same rules regardless of tool. Public, logged-out pages only. Read robots.txt and respect it, including crawl-delay. Back off on 429 rather than retrying harder. Identify yourself with a User-Agent that a site owner could act on. Do not scrape behind logins, paywalls or anything gated by anti-bot measures, and do not treat a CAPTCHA as a challenge to solve: it is a site telling you no. We do not bypass logins, paywalls, CAPTCHAs or anti-bot systems, and any vendor promising otherwise is selling you a legal problem. Our position is written out on the compliance page.

Which one should you pick

A decision rule beats a feature matrix.

Data is in the raw HTML, few sources, breakage is cheap: Cheerio plus fetch. Nothing else. Adding a browser here is the most common overengineering mistake in Node scraping, and paying an API for it would be worse.

Data needs a browser, you have a handful of sites, and you need real interaction: Playwright, with a concurrency pool and browser recycling from day one. Use Puppeteer instead if your team already runs it. Add Crawlee if you need queueing and dedup and would rather not write your own scheduler.

Ten or more sources, they render client-side, they change, and the output feeds something a customer sees: that is when a hosted extraction layer earns its cost, because the per-site selector work and the browser fleet are both permanent, and neither is your product.

Most teams end up mixing: Cheerio for the five stable sources that have not changed in two years, an API for the twenty that keep moving. Spending money only where breakage actually hurts is a better policy than picking a side. If you work across languages, the same comparison for Python reaches a structurally identical conclusion with different tools, which is a decent sign the reasoning is about the problem rather than the ecosystem.