By output · HTML to markdown

URL to markdown API: webpage to markdown and HTML to markdown in one call

The short answer

An HTML to markdown API converts a live web page into clean markdown in one call: it fetches the URL, renders the JavaScript, strips the navigation, ads and scripts, and returns just the content with headings, lists, tables and links intact. Markdown matters here because it carries the document structure a model needs while costing far fewer tokens than raw HTML, which is why it has become the default input format for RAG pipelines and AI agents. ClawEngine renders before converting, so single page apps come back complete rather than empty. It runs on public, permitted pages only. Plans start at $39 a month.

Clean markdown & JSON · JavaScript rendered · robots.txt respected

Last updated August 2026

Live Extraction

Endpoint · POST /v1/extract

POST

try:

Hit Extract to turn this page into clean, LLM-ready data.

robots.txt respected · public data only

Markdown · JSON · structured fields, from one API call. Crawling, rendering and extracting ...

A naive HTML to markdown converter chokes on modern sites, because the content is not in the HTML until JavaScript runs. ClawEngine is an HTML to markdown API that renders the page in a real browser environment first, then converts the resulting DOM into clean markdown with the boilerplate stripped out.

Give it a public URL and you get back tidy markdown that preserves headings, lists, tables and links, ready to embed or index. There is no browser or converter to run on your side. ClawEngine processes public, permitted pages only, respects robots.txt and site Terms of Service, and honors crawl-delay, so the conversion is both clean and compliant.

Worth being straight about where this is and is not the right tool. If you already hold the HTML and the page is plain static markup, a local library like Turndown, html2text or markdownify does the job for free, and you should use one. An API earns its keep when the page needs rendering, when you are converting at volume, or when you want a whole site rather than a single document. That is the line we would draw if you asked us directly.

CRAWL RENDER JS EXTRACT MARKDOWN JSON

Any URL in LLM-ready data out

robots.txt respected public data only

Why it works

What you get with HTML to markdown

Renders, then converts

JavaScript runs before conversion, so content that only exists in the rendered DOM makes it into the markdown, unlike a static HTML parser.

Clean, faithful markdown

Headings, lists, tables and links are preserved while ads, navigation and scripts are dropped, so the markdown mirrors the real content.

No browser to run

The rendering and conversion happen in the API, so you skip running and scaling a headless browser just to get markdown.

What it handles

Any URL in, clean structured data out

Point ClawEngine at a public page and it crawls, renders the JavaScript and extracts clean markdown or typed JSON in one call. Define a schema for structured fields, and respect robots.txt and Terms of Service by default.

Converts any public URL to markdown
Renders JavaScript before converting
Strips ads, navigation and scripts
Preserves headings, lists, tables and links
Returns LLM-ready, embed-ready output
Stays on public, permitted pages only

POST /v1/extract extraction result

200 · JSON

{
  "url": "https://example.com/products/atlas",
  "title": "Atlas Field Notebook",
  "markdown": "# Atlas Field Notebook\n\nDurable...",
  "data": {
    "name": "Atlas Field Notebook",
    "price": 24.00,
    "currency": "USD",
    "rating": 4.7
  },
  "links": [ "/products", "/cart" ],
  "metadata": { "rendered": true }
}

JS rendered · boilerplate stripped ✓ robots.txt respected

Why ClawEngine

One API that crawls, renders and extracts

Not a raw HTML dump, not a headless browser fleet to run, and not a brittle parser to maintain. One call crawls a public page, renders its JavaScript and returns clean markdown or typed JSON, built for RAG pipelines and AI agents.

LLM-ready output

Clean markdown or typed JSON with the boilerplate stripped, so the data drops straight into a vector store, a prompt or an agent without a cleanup step.

JavaScript rendered

Each page loads in a real browser environment before extraction, so single-page apps and client-rendered content come back complete, not as an empty shell.

Compliance-first

ClawEngine works on public, permitted data only. It respects robots.txt and site Terms of Service and honors crawl-delay, so responsible scraping is the default.

Code examples

Convert a URL to markdown in one call

Send a public URL and read the markdown out of the response. The second sample crawls a whole documentation site and writes one markdown file per page, which is the usual way teams build a knowledge base from docs.

bash One page to markdown

curl -X POST https://api.clawengine.ai/v1/extract \
  -H "Authorization: Bearer $CLAWENGINE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/docs/getting-started",
    "format": "markdown",
    "render": true
  }'

python Crawl a whole site to markdown files

import os, pathlib, requests

headers = {"Authorization": f"Bearer {os.environ['CLAWENGINE_API_KEY']}"}

res = requests.post("https://api.clawengine.ai/v1/crawl", headers=headers, json={
    "url": "https://example.com/docs",
    "path_prefix": "/docs",   # stay inside the docs tree
    "limit": 500,
    "format": "markdown",
    "render": True,
})
res.raise_for_status()

out = pathlib.Path("docs_markdown")
out.mkdir(exist_ok=True)

for page in res.json()["pages"]:
    name = page["url"].rstrip("/").split("/")[-1] or "index"
    (out / f"{name}.md").write_text(page["markdown"])

print(f"wrote {len(res.json()['pages'])} markdown files")

javascript Markdown plus typed fields in one request

const res = await fetch("https://api.clawengine.ai/v1/extract", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.CLAWENGINE_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    url: "https://example.com/blog/post",
    format: "markdown",
    render: true,
    schema: {
      title: "string",
      author: "string",
      published_at: "string",
    },
  }),
});

const { markdown, data } = await res.json();
console.log(data.title, data.published_at);
console.log(markdown.slice(0, 500));

HTML to markdown API: the questions buyers ask

What is an HTML to markdown API?

It is a hosted endpoint that takes a URL, loads the page, and gives you back markdown instead of HTML. The service handles the fetch, the JavaScript rendering and the cleanup, so you receive the article content with its heading structure preserved and the navigation, ads and scripts removed. You call it like any REST API and get text you can embed, index or pass to a model directly.

How do I convert a URL to markdown?

Send the URL to a conversion endpoint and read the markdown out of the response. With ClawEngine that is a single POST with the URL and a format of markdown, and the reply contains the cleaned document. Doing it locally instead means fetching the page yourself, running a headless browser if the site is JavaScript-heavy, and then passing the DOM through a converter library.

Why convert HTML to markdown for LLMs?

Raw HTML spends most of its tokens on markup, classes and scripts that carry no meaning, which inflates cost and buries the actual content. Markdown keeps the parts a model needs, headings, lists, tables and emphasis, in a fraction of the tokens. Cleaner input also produces measurably better retrieval, because the embedded chunks contain content rather than boilerplate.

Can I convert HTML to markdown for free?

Yes, if you already have the HTML. Turndown for JavaScript, html2text and markdownify for Python are free, mature libraries that convert markup you hold locally, and for static pages they are the correct choice. What they cannot do is fetch and render the page. On a site that builds its content client-side, the HTML you feed the library is an empty shell, and that is the gap an API fills.

How do I convert a whole website to markdown?

Crawl it rather than converting page by page. Give the crawl endpoint a seed URL, a path prefix to stay inside and a page limit, and it walks the site and returns one markdown document per page in the same shape. That is the usual way teams turn a documentation site into a knowledge base, and it is one job instead of thousands of individual conversions.

Does an HTML to markdown API handle JavaScript-rendered pages?

Only if it renders, and many do not. A converter that reads the raw HTTP response sees whatever the server sent, which on a React or Vue site is often a nearly empty document. ClawEngine loads each page in a real browser environment and waits for the content to build before converting, so client-side content and lazy-loaded sections appear in the markdown just like static markup.

Does markdown conversion preserve tables and links?

Yes. Tables become pipe tables, links keep their href targets, and headings map to their matching markdown levels, so the document hierarchy survives the conversion. That matters more than it sounds: chunking a document for retrieval usually splits on headings, and a converter that flattens them to plain text destroys the structure your pipeline depends on.

Good questions

Questions about HTML to markdown

It is not better in every case, and we would rather say so. A local library converts HTML you already hold, for free, and on static pages that is the right answer. The difference shows up when the content only exists after JavaScript runs, or when you are converting thousands of pages: then you need the fetch, the render and the retries handled for you, which is what the API does.

You give ClawEngine a public URL and it fetches, renders and converts the page for you. It targets public, permitted pages only and respects robots.txt and Terms of Service.

Jina Reader is the best known free option and it is genuinely good for one-off conversions: prefix a URL and you get markdown back. If that covers your use, use it. ClawEngine is aimed at the case where you need a whole-site crawl with scope rules, typed JSON extraction alongside the markdown, and predictable throughput on a paid plan rather than a shared free endpoint.

Yes. Pass a schema alongside the markdown format and the response carries both: the cleaned markdown document and the typed fields you named. That saves a second request when you want the readable text for a model and specific values, a price or a published date, for a database at the same time.

We do not do that. ClawEngine converts public and permitted pages, and it does not authenticate into accounts or work around paywalls and anti-bot systems on your behalf. If the content sits behind a login, the route is a licensed feed or the site owner's own API, not a scraping tool.

Read every web scraping question

Explore more

More ways to turn the web into data with ClawEngine

See every use case See pricing Back to the web scraping API

Stop wrangling raw HTML. Get LLM-ready data.

Point ClawEngine at a public page and one call crawls, renders the JavaScript and extracts clean markdown or typed JSON, ready for your RAG pipeline or AI agent. Public, permitted data only.

See pricing

Crawl · render JS · extract markdown & JSON · robots.txt respected, public data only