HTML to Markdown: choose the step your pipeline needs

An HTML page can contain article text, navigation, scripts, tables, and interactive controls. Converting it to Markdown changes the representation; selecting its main content changes what the result contains. A useful pipeline makes both choices explicitly.

For LLM data preparation, begin with the input you have and the output you need. A converter, an extractor, a generative model, and a hosted crawler take on different parts of the work.

Separate the responsibilities

• Format conversion translates the supplied HTML's supported structures into Markdown.
• Content extraction chooses which page regions to retain before conversion.
• Structured extraction returns fields such as a title, price, or date according to a schema.
• Fetching and crawling obtain pages, potentially render JavaScript, and follow links.

A tool may combine these steps, but the combination is not evidence that every page or structure will be handled correctly. Inspect a representative sample at each boundary.

Check for published Markdown

Some origins publish Markdown alongside HTML, advertise an alternate representation, or respond to Accept: text/markdown. Markdownee's optional markdownDiscovery can use an accepted representation as the source for all output formats. Its default is off.

Discovery coverage varies by site. The Markdown guide explains advertised links, negotiation, sibling probing, validation, and request budgets. When no usable representation exists, continue with HTML extraction.

Convert HTML you already trust to contain the right content

Turndown

Turndown converts supplied HTML in JavaScript and supports custom element rules.¹ Its GFM plugin adds tables, strikethrough, and task lists. For example:

turndownService.addRule('highlight', {
  filter: 'mark',
  replacement: function(content) {
    return '==' + content + '==';
  }
});

Choose the code-block configuration deliberately: the default uses indented code; fenced blocks can carry language hints. A full page passed directly to Turndown can retain navigation and footer text because content selection is outside the converter's job.

markdownify

Python markdownify builds on BeautifulSoup. Its MarkdownConverter class can be extended through convert_* methods, and code_language_callback lets callers supply code-language detection.² Evaluate the options against the actual tables, code blocks, and links in your input.

html-to-markdown in Go

Johannes Kaufmann's library offers HTML-to-Markdown conversion through Go APIs and a CLI, with CommonMark and GFM-related support.³ It is an option for Go applications or command-line conversion. The cited documentation does not establish a universal speed ranking against JavaScript and Python tools.

Select page content before converting

Trafilatura

Trafilatura combines extraction heuristics and fallbacks with several output formats, including Markdown.⁴ Its behavior has been evaluated on article-extraction datasets.⁵ Those measurements describe the tested versions and corpora, not every current page type.

Use output_format='markdown' to request Markdown in Python. Inspect content selection as well as formatting: a well-formed Markdown result can still omit a paragraph or include a sidebar.

Readability followed by a converter

Mozilla Readability returns selected article HTML rather than Markdown. Passing that HTML to Turndown separates selection from conversion. This is useful when you want independent control of each step, and it gives you two outputs to inspect when something is missing.

Neither supplied-HTML step renders JavaScript. Capture a rendered document first when the page's initial response lacks the desired content.

Generate Markdown with ReaderLM-v2

ReaderLM-v2 treats supplied HTML as a model input and can generate Markdown or schema-shaped JSON. Jina's January 2025 launch evaluation reports ROUGE-L 0.84 for v2 and 0.86 for v2-pro, against 0.69 for GPT-4o in that evaluation.⁶ These are checkpoint-specific results, not a general quality ranking for all converters.

The model card describes a roughly 1.54B-parameter model based on Qwen2.5-1.5B-Instruct, with up to 512K combined context tokens and 29 languages. It supports CPU as well as GPU inference. Memory, input/output length, precision, and hardware affect its practical cost.⁷

Distinguish the model from Jina Reader, the hosted r.jina.ai service. Reader has rule-based conversion profiles; ReaderLM-v2 is an optional experimental profile.⁸ Model generation should be checked against the source for omissions, substitutions, and formatting changes.

Delegate fetching to a service

Firecrawl

Firecrawl's API combines fetching with output options such as Markdown and HTML. Its broader interface also offers crawling, search, and schema-shaped extraction.⁹ Hosted and self-hosted operation have different deployment responsibilities.

Credits depend on the endpoint and selected features. Consult current pricing and measure a sample workload instead of treating a plan's headline credit count as a fixed page allowance.¹⁰

ScrapingAnt

ScrapingAnt provides fetching, proxy, and JavaScript-rendering services with a Markdown transformation endpoint.¹¹ Its credit requirements vary with request options, including browser rendering and proxy selection.¹² Compare both the content returned and the request features your sites require.

Combine fetching and filtering with Crawl4AI

Crawl4AI provides Python crawling and Markdown generation. DefaultMarkdownGenerator produces Markdown; pruning and BM25 filters can produce a more selective “fit Markdown” result. Model-assisted filtering is optional.¹³

Browser crawling is one strategy; Crawl4AI also supports an HTTP-only strategy.¹⁴ Select the strategy according to the page rather than assuming each run must launch a browser. Its CLI, service, and MCP interfaces provide additional integration choices.

Compare responsibilities, then measure your data

Responsibilities across HTML-to-Markdown approaches

The table preserves project-specific license labels; consult each project's terms for your chosen version and use.

Measure token counts using the same captured pages, tool versions, settings, and downstream tokenizer before comparing costs. Research such as IndexLM studies a particular extraction method, not these tools on a shared input.¹⁵ API input-token rates also vary by model and change over time.¹⁶

Match the output to the next consumer

For already-selected HTML, start with a converter in your application's language. For article content, add a selection step and inspect what it removes. For JavaScript content, arrange browser capture. For arbitrary fields, evaluate a schema-oriented extractor and validate values separately from JSON shape.

Markdownee combines these collection and conversion steps around Trafilaturacore. Its playground previews a page and generates commands. Choose the output format based on the consuming application, then compare representative results before expanding the crawl.

Updated: September 7, 2026