By capability · Table extraction

Extract tables from website pages: scrape table data to JSON or CSV with one API call

The short answer

To extract tables from a website, send the page URL to an extraction API along with a schema naming the columns you want, and you get typed rows back instead of HTML to parse. ClawEngine renders the page first, so tables that JavaScript builds after load come back complete, then returns every row as JSON or CSV in a single call. Because the same call can follow a result list, a table split across 40 pages of pagination arrives as one dataset. It reads public, no-login pages only and respects robots.txt. Plans start at $39 a month.

Clean markdown & JSON · JavaScript rendered · robots.txt respected

Last updated August 2026

Live Extraction

Endpoint · POST /v1/extract

POST

try:

Hit Extract to turn this page into clean, LLM-ready data.

robots.txt respected · public data only

Markdown · JSON · structured fields, from one API call. Crawling, rendering and extracting ...

Copying a table out of a web page is trivial once. The problem is the second time, and the four hundredth. A pricing table you need every Monday, a state licensing register that runs 60 pages deep, a supplier catalog that quietly adds a column in March: each of those turns a five-minute copy and paste into a script somebody has to own.

The DIY route is well travelled and genuinely works for one-off jobs. Pandas will read simple markup in a line, and BeautifulSoup handles the awkward cases. Both parse the HTML they are handed, which is the catch: if the table is drawn client-side by JavaScript, the markup contains an empty shell and you get nothing back. From there you are running a headless browser, and the job stops being a script.

An extraction API takes the parsing and the browser off your plate. You name the columns you want and their types, and every row arrives in that shape whether the source used a clean table element, nested divs pretending to be a grid, or merged header cells. When the site is redesigned, the schema usually still describes what you asked for, so there are no selectors to chase.

One honest limit worth stating up front: this works on tables in web pages. ClawEngine does not run OCR, so a table locked inside a scanned PDF is out of scope. If the source publishes a CSV or an open API, use that instead. It is cheaper, faster and more stable than any crawler.

CRAWL RENDER JS EXTRACT MARKDOWN JSON

Any URL in LLM-ready data out

robots.txt respected public data only

Why it works

What you get with Table extraction

Columns by name, not by position

You describe the fields you want and their types. A new column, a reworded header or a merged cell does not break the job, because nothing is pinned to the third td in the second tr.

Renders the grid before reading it

Plenty of tables are drawn client-side after load, which is why a plain HTML parse comes back empty. Each page loads in a real browser environment first, so the rows exist by the time extraction runs.

Pagination is part of the same call

A register that runs 60 pages deep is one request, not 60 plus the glue between them. Rows arrive as a single typed dataset with the source URL kept on each one for auditing.

What it handles

Any URL in, clean structured data out

Point ClawEngine at a public page and it crawls, renders the JavaScript and extracts clean markdown or typed JSON in one call. Define a schema for structured fields, and respect robots.txt and Terms of Service by default.

Turns HTML tables into typed JSON or CSV rows
Reads grids built client-side by JavaScript
Follows pagination across a full result set
Survives merged cells and multi-row headers
Keeps the source URL on every extracted row
Reads robots.txt and public, no-login pages only

POST /v1/extract extraction result

200 · JSON

{
  "url": "https://example.com/products/atlas",
  "title": "Atlas Field Notebook",
  "markdown": "# Atlas Field Notebook\n\nDurable...",
  "data": {
    "name": "Atlas Field Notebook",
    "price": 24.00,
    "currency": "USD",
    "rating": 4.7
  },
  "links": [ "/products", "/cart" ],
  "metadata": { "rendered": true }
}

JS rendered · boilerplate stripped ✓ robots.txt respected

Why ClawEngine

One API that crawls, renders and extracts

Not a raw HTML dump, not a headless browser fleet to run, and not a brittle parser to maintain. One call crawls a public page, renders its JavaScript and returns clean markdown or typed JSON, built for RAG pipelines and AI agents.

LLM-ready output

Clean markdown or typed JSON with the boilerplate stripped, so the data drops straight into a vector store, a prompt or an agent without a cleanup step.

JavaScript rendered

Each page loads in a real browser environment before extraction, so single-page apps and client-rendered content come back complete, not as an empty shell.

Compliance-first

ClawEngine works on public, permitted data only. It respects robots.txt and site Terms of Service and honors crawl-delay, so responsible scraping is the default.

Code examples

Pull a table into typed rows

Post the page URL and a schema describing one row. ClawEngine renders the page, finds the grid carrying those columns and returns every row in that shape. Point it at a listing instead of a single page to walk pagination in the same call.

curl Extract one table as typed rows (POST /v1/extract)

curl https://api.clawengine.ai/v1/extract \
  -H "Authorization: Bearer $CLAWENGINE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/suppliers/price-list",
    "format": "json",
    "render": true,
    "schema": {
      "rows": [{
        "sku": "string",
        "description": "string",
        "unit_price": "number",
        "currency": "string",
        "in_stock": "boolean"
      }]
    }
  }'

python Walk a paginated table and load it into a DataFrame

import os, requests, pandas as pd

BASE = "https://api.clawengine.ai/v1"
headers = {"Authorization": f"Bearer {os.environ['CLAWENGINE_API_KEY']}"}

row_schema = {
    "license_number": "string",
    "holder_name": "string",
    "status": "string",
    "issued_date": "string",
    "expires_date": "string",
}

# Crawl the register and extract the same row shape from every result page.
resp = requests.post(
    f"{BASE}/crawl",
    headers=headers,
    json={
        "url": "https://licensing.example.gov/search?status=active",
        "follow": "pagination",   # keep going while a next-page link exists
        "limit": 200,             # cap the crawl so a runaway list cannot bill forever
        "render": True,
        "format": "json",
        "schema": {"rows": [row_schema]},
    },
)
resp.raise_for_status()

# Flatten every page into one table, keeping provenance on each row.
records = []
for page in resp.json()["pages"]:
    for row in page["data"]["rows"]:
        records.append({**row, "source_url": page["url"]})

df = pd.DataFrame(records)
df.to_csv("licenses.csv", index=False)
print(f"{len(df)} rows from {len(resp.json()['pages'])} pages")

Extract tables from a website: the questions buyers ask

How do I extract tables from a website?

Send the page URL to an extraction API with a schema listing the columns and their types, and it returns typed rows. That skips writing selectors against the markup. For a one-off job on a static page, pandas can read simple tables in a single line. Use an API when the table is rendered by JavaScript, spans many pages, or has to keep working next quarter.

How do I scrape a table from a website using Python?

For static markup, pandas reads tables directly from HTML into a DataFrame, and BeautifulSoup gives you row-level control when the structure is messy. Neither runs JavaScript, so a client-side grid returns empty. In that case either drive a headless browser yourself or post the URL to an extraction API and load the JSON it returns straight into a DataFrame.

How do I convert an HTML table to JSON?

Map each header cell to a key and each body row to an object, so a table with four columns becomes an array of objects with four fields each. Doing this by hand breaks on merged cells, multi-row headers and footer totals. An extraction API applies the schema you defined instead, which keeps types consistent even when the header wording changes.

How do I extract table data from multiple web pages?

Treat it as a crawl rather than a fetch. Collect the result URLs from the listing or follow the next-page link, extract each page against the same schema, then concatenate. Keep the source URL on every row so you can re-fetch and audit a value later. ClawEngine does the crawl and the extraction in one call, so pagination is not separate plumbing.

Can I extract a table from a website to Excel?

Yes, by way of CSV. ClawEngine returns rows as JSON or CSV, and CSV opens directly in Excel or Google Sheets. For a single static table, Excel's own Get Data from Web and the Sheets IMPORTHTML function both work without any code. They struggle once the table needs a login, renders through JavaScript, or spans more pages than you want to refresh by hand.

Can you extract tables from a PDF?

Not with ClawEngine. It reads web pages and does not run OCR, so a table inside a scanned or image-based PDF is out of scope. That is a document extraction job rather than a crawling one, and the tools built for it read page layout instead of markup. If your source publishes the same figures as an HTML page, extract those instead.

Why do table scrapers break?

Almost always because they target position rather than meaning. A selector pinned to the third column breaks the moment a column is inserted, and one pinned to a generated class name breaks at the next deploy. Merged cells, multi-row headers and footer totals cause the rest. Describing the fields you want by name and type survives all of those.

Is it legal to scrape tables from a website?

In the United States, collecting publicly available data from pages that need no login is broadly lawful, and courts have repeatedly declined to treat it as unauthorized access under the Computer Fraud and Abuse Act. What still binds you is the site's Terms of Use, its robots.txt, and privacy law once rows contain personal data. Facts in a table are generally not copyrightable, though a curated selection can be. This is general information, not legal advice.

Good questions

Questions about Table extraction

Your schema decides. Because you name the fields you want rather than an index, the extraction targets the table that actually carries those columns, which is more reliable than asking for the second table on the page. If you genuinely need all of them, run one extraction per schema and join the results downstream.

Hash the fields you care about on each row and store the digest beside the record. On the next run, compare digests: new keys are additions, changed digests are edits, and missing keys are removals. That turns a full re-read into a diff, which is cheaper and gives you a change feed you can alert on.

Read every web scraping question

Explore more

More ways to turn the web into data with ClawEngine

See every use case See pricing Back to the web scraping API

Stop wrangling raw HTML. Get LLM-ready data.

Point ClawEngine at a public page and one call crawls, renders the JavaScript and extracts clean markdown or typed JSON, ready for your RAG pipeline or AI agent. Public, permitted data only.

See pricing

Crawl · render JS · extract markdown & JSON · robots.txt respected, public data only