Library

Markdown uses visible text markers to preserve headings, lists, and links. Markdownee outputs Markdown by default, while plain text, readable HTML, and Minified HTML remain available.

Minified HTML keeps Markdownee's cleaned HTML structure in a compact form, with the same selected layout as readable HTML. It uses the minified-html format token, the minifiedHtml structured field, and a distinct .min.html filename without being confused with original raw HTML.

The easiest way to try Markdownee — one npx command extracts a page straight to your terminal, with no browser install (you can install it later) and no API key — you host it yourself:

npx markdownee crawl-one https://example.com/ --crawler-type cheerio

--crawler-type cheerio fetches over plain HTTP, so no headless browser is downloaded. Need a whole-site crawl or specific formats? Use the playground to build a more advanced command visually, then copy it.

An HTML parser constructs a document tree from elements, text, and attributes. Markdownee can return that content as readable html or compact minified-html, using a fragment with minimal layout or a complete document with standard or enhanced. The separate original format keeps the crawler capture before extraction, which may already reflect rendering and consent handling.

Plain text represents characters without document markup. Encodings range from 7-bit ASCII and legacy code pages to UTF-8; receivers still need to know the encoding. Markdownee's text output retains paragraph and list separation and table delimiters. For embedding pipelines and classification, compare the output with the structure your model needs.

Consent interfaces can add unwanted text or replace the page a crawler needs. Network filtering, cosmetic hiding, DOM removal, and consent interaction have different effects. Markdownee combines configured Ghostery filters with residual-container removal and targeted wall recovery; inspect the captured content when a page remains blocked or incomplete.

Check whether the HTTP response contains the required content before choosing a fetching method. A browser adds JavaScript execution and interaction; an extractor then selects content from the supplied document. Compare completeness and resource use on your own pages instead of relying on a universal throughput ratio.

Plain text, Markdown, readable cleaned HTML, and Minified HTML carry different structural signals and byte costs. Markdownee can render several formats from one extraction pass, with minimal, standard, or enhanced layouts.

HTML-to-Markdown tools combine different responsibilities. Turndown converts supplied HTML; Trafilatura selects page content before conversion. ReaderLM-v2 generates Markdown and supports CPU as well as GPU inference. Hosted APIs can add fetching, JavaScript rendering, and anti-bot handling. Compare the content each approach retains and the resources it requires using representative inputs.

Selectors locate fields through page structure; models can locate them from content and a requested schema. Both need checks for missing or incorrect values. Combining them can reduce model input, provided the earlier extraction step retains the prices, identifiers, and other evidence the final fields require.

Compare Trafilatura's rule-based selection with ReaderLM-v2's generated Markdown and JSON. Their published evaluations use different datasets, metrics, and checkpoints. Examine output coverage and runtime requirements separately; another port's timing is not a Markdownee measurement, and ReaderLM-v2 can run on CPU as well as GPU.

Python Trafilatura offers extraction and several outputs, readability-lxml returns article HTML, and Newspaper4k combines article parsing with optional NLP. Match the API to your required fields and compare a sample of your pages. Markdownee uses the separate Trafilaturacore engine with its own crawling and conversion layers.

Trafilatura is Adrien Barbaresi's Python toolkit for downloading, crawling, and extracting main text and metadata. go-trafilatura and rs-trafilatura port the approach into Go and Rust while changing fallbacks, outputs, and extraction behavior.

Trafilaturacore is a separate pure-TypeScript port for offline Node use, ported directly from Python Trafilatura v2.2.0 with go-trafilatura used only as a DOM translation aid. Markdownee consumes Trafilaturacore, then separately handles crawling and format conversion.