Library

Trafilatura is Adrien Barbaresi's Python toolkit for downloading, crawling, and extracting main text and metadata. go-trafilatura ports the approach into Go with its own runtime and fallback choices.

Trafilatura Core is an open-source extraction engine based on Trafilatura, implemented in TypeScript and Python. It extracts content from supplied HTML. Its TypeScript engine ports Python Trafilatura v2.2.0's fast=True path, with go-trafilatura used only as a DOM translation aid. Its native Python engine translates that port.

Minified HTML keeps Markdownee's cleaned HTML structure in a compact form, with the same selected layout as readable HTML. It uses the minified-html format token, minifiedHtml in TypeScript/API results, minified_html in Python results, and a distinct .min.html filename without being confused with original raw HTML.

Markdown uses visible text markers to preserve headings, lists, and links. Markdownee outputs Markdown by default, while plain text, readable HTML, and Minified HTML remain available.

Plain text represents characters without document markup. Encodings range from 7-bit ASCII and legacy code pages to UTF-8; receivers still need to know the encoding. Markdownee's text output retains paragraph and list separation and table delimiters. For embedding pipelines and classification, compare the output with the structure your model needs.

Consent interfaces can add unwanted text or replace the page a crawler needs. Network filtering, cosmetic hiding, DOM removal, and consent interaction have different effects. Markdownee's TypeScript crawler and Actor combine configured Ghostery filters with residual-container removal and targeted wall recovery; the native Python library leaves consent handling to the caller.

Plain text, Markdown, readable cleaned HTML, and Minified HTML carry different structural signals and byte costs. Markdownee can render several formats from one extraction pass, with minimal, standard, or enhanced layouts.

HTML-to-Markdown tools combine different responsibilities. Turndown converts supplied HTML; Trafilatura selects page content before conversion. ReaderLM-v2 generates Markdown and supports CPU as well as GPU inference. Hosted APIs can add fetching, JavaScript rendering, and anti-bot handling. Compare the content each approach retains and the resources it requires using representative inputs.

Check whether the HTTP response contains the required content before choosing a fetching method. A browser adds JavaScript execution and interaction; an extractor then selects content from the supplied document. Compare completeness and resource use on your own pages instead of relying on a universal throughput ratio.

Try Markdownee with one npx command — no browser install or API key is needed. Use Node.js 22.22.2+ on the 22.x line, 24.15.0+ on 24.x, or 26+:

npx --package=@markdownee/markdownee markdownee fetch \
  https://en.wikipedia.org/wiki/Web_scraping \
  --crawler-type cheerio --save markdown-file -o page.md

Open page.md to inspect the extracted article. --crawler-type cheerio fetches over plain HTTP. For a whole-site crawl or specific formats, use the playground to build a command visually, then copy it.

An HTML parser constructs a document tree from elements, text, and attributes. Markdownee can return that content as readable html or compact minified-html, using a fragment with minimal layout or a complete document with standard or enhanced. The separate original format keeps the crawler capture before extraction, which may already reflect rendering and consent handling.