How to Scrape a Table From a Website Into JSON or CSV

Getting one table out of one page is a solved problem, and it takes about four minutes. The reason people go looking for a method is almost always the second requirement hiding behind the first: the table runs 60 pages deep, or it has to refresh every Monday morning, or the page renders an empty shell until JavaScript fills it in. Those change the answer completely.

Here are the four routes that actually work in 2026, what each one costs you in effort, and where each falls over.

How do I scrape a table from a website?

Pick the route by two questions: is the table in the HTML source, and how long does this job need to keep working? Everything else is detail.

Scraping a static table with pandas

If the table markup is in the page source, this is genuinely a one-liner. read_html() finds every table element on the page and hands back a list of DataFrames.

import pandas as pd

tables = pd.read_html("https://example.com/suppliers/price-list")
print(len(tables))      # how many tables were found
df = tables[0]          # usually not the one you want, see below
df.to_csv("prices.csv", index=False)

Two things bite here. First, tables[0] is a positional guess: layout tables, navigation and footers all count, so the table you want may be index 3 today and index 4 after a banner is added. Passing match="SKU" to filter on text the target table contains is far more stable than an index. Second, read_html parses markup and does not execute anything, which leads directly to the most common failure.

Why does pandas return an empty list or no table?

Because the table is not in the HTML that the server sent. Modern grids frequently ship as an empty container plus a JavaScript bundle that fetches rows and injects them after load. read_html, requests and BeautifulSoup all see the page before that happens, so they find nothing to parse.

You can confirm this in about ten seconds. View the page source (not the inspector, which shows the live DOM) and search for a value you can see in the table. If it is missing from the source, the table is client-side and you need a rendered page rather than a raw fetch. From there you either run a headless browser yourself, or send the URL to something that renders before extracting.

Occasionally there is a better answer than either. Open the network tab, reload, and look for the request the page makes to populate the grid. If it returns JSON, use that endpoint directly: it is faster, lighter, and gives you clean typed data with no parsing at all. Never scrape what the site already hands out as structured data.

How do I scrape a table that spans multiple pages?

Treat it as a crawl, not a fetch. Collect every result URL, extract the same row shape from each, then concatenate and keep the source URL on every row so you can re-check a value later without re-running the whole job.

The pattern is simple; the maintenance is not. Pagination links change, some sets use infinite scroll rather than page numbers, and a run that silently stops at page 12 of 60 produces a dataset that looks complete and is not. Cap the crawl, count the pages you actually retrieved, and compare that to the total the site reports. This is the point where doing it by hand stops paying, and where an API that extracts tables from website pages across pagination in one call removes the plumbing entirely.

How do I convert an HTML table to JSON?

Map each header cell to a key and each body row to an object, so a four-column table becomes an array of objects with four fields each. That is the easy version, and it holds until the markup does something normal that the naive loop did not expect.

Declaring the shape you want up front handles all four, because the extraction is told that unit_price is a number rather than being asked to guess from whatever the third cell contained. That is the practical difference between parsing a table and scraping a website to JSON against a schema.

Why do table scrapers break?

Almost always because they target position instead of meaning. A selector pinned to the third column breaks the moment somebody inserts a column. One pinned to a generated class name like css-1x9fk2 breaks at the next frontend deploy, which nobody will announce. Neither failure is loud: the job keeps running and quietly writes wrong values into your dataset.

Two habits help more than any tool choice. Assert on shape after every run, so a scrape returning 4 columns when it returned 5 last week fails loudly instead of writing nulls. And store a hash of each row so re-running becomes a diff, which is cheaper than reprocessing and gives you a change feed worth alerting on. That is the same discipline behind deduplicating pages during a crawl.

Can you scrape tables from a PDF?

Not with a web scraper, and this trips people up because the request sounds identical. A crawler reads markup: rows, cells and headers are explicit in the document it receives. A scanned PDF has none of that. It has glyphs at coordinates, and reconstructing a table means inferring columns from whitespace and alignment, which is a layout problem rather than a parsing one. ClawEngine does not run OCR, so PDF tables are outside what it does.

If your source is a PDF, you want software built to read document layout instead of markup, which handles the coordinate maths and the scan quality. And check first whether the same figures are published as an HTML page somewhere on the site, because agencies and vendors very often post both, and the web version is dramatically easier to work with.

Is it legal to scrape tables from a website?

In the United States, collecting publicly available data from pages that require no login is broadly lawful, and courts have repeatedly declined to treat access to public web data as unauthorized access under the Computer Fraud and Abuse Act. Facts arranged in a table are generally not copyrightable either, although a creative selection or arrangement can be.

What still binds you is the site's Terms of Use, its robots.txt, and privacy law the moment rows contain personal data. Rate limits are a courtesy that also protects you: hammering a small county server is both rude and the easiest thing for anyone to point at afterwards. The legal guide to web scraping and robots.txt covers the boundaries in detail. This is general information, not legal advice.

Which method should you use?

For one static table you need once, use a spreadsheet import or pandas.read_html() with a match filter and move on. Do not build infrastructure for a five-minute job.

For a table you need repeatedly, or one that renders client-side, or one that spans more pages than you want to click through, the maintenance cost is what decides it. Writing the scraper is the small part; keeping it alive through redesigns is the part that lands on somebody's backlog every quarter. A schema-based API removes both the browser and the selector maintenance, and it is worth it precisely when the job is recurring. If you are weighing that trade-off more broadly, we wrote up using a web scraping API versus building your own with the numbers.

Either way, start by checking the page source for the values you want. That one test tells you which half of this article applies, and it takes ten seconds.

Last updated August 2026.