Markdown Explained — Lightweight Text Formatting

Markdown expresses document structure with text conventions: headings begin with hash signs, links combine labels and destinations, and lists use visible markers. You can read the source before converting it to HTML.

It was created in 2004 by John Gruber, with substantial design input from Aaron Swartz, and released as a Perl script on Gruber's blog Daring Fireball¹.

Gruber described Markdown as "a text-to-HTML conversion tool for web writers" and released it under a BSD-style open source license. He deliberately kept the syntax description informal — more of a guide than a specification. That decision would cause problems later.

Pick a dialect before relying on a feature

The original syntax left parsing cases open to interpretation. CommonMark, developed by John MacFarlane and collaborators, specifies these details through rules and examples. Its 0.31.2 specification covers headings, lists, blockquotes, links, emphasis, code, and raw HTML.² The project's early naming changes are recorded in Jeff Atwood's account.³

GitHub Flavored Markdown adds extensions to CommonMark. The published 0.29-gfm specification includes tables, task lists, strikethrough, extended autolinks, and disallowed raw HTML.⁴ A hosting product can support features beyond that specification: GitHub supports footnotes, but footnotes are not a GFM 0.29 extension.

Confirm the receiving renderer's dialect. Matching the file extension does not establish matching behavior.

Read and write the main constructs

ATX headings use one to six # markers. Setext headings use an underline of = or - for the first two levels. Asterisks or underscores express emphasis; double markers express strong emphasis. Inline links use [label](destination), while reference links separate the destination from the text. Images add ! before the link syntax.²

A GFM table uses a header, delimiter row, and pipe-separated cells:

| Language | Typing     | First appeared |
|----------|------------|----------------|
| Python   | Dynamic    | 1991           |
| Rust     | Static     | 2010           |
| Go       | Static     | 2009           |

Task lists use bracketed checkboxes:

- [x] Parse HTML
- [x] Extract content
- [ ] Convert to Markdown

Use ~~text~~ for GFM strikethrough. CommonMark code can be indented or fenced; a fence can carry an information string that a renderer may use as a language hint:

```python
result = extract(html, output_format="markdown")
```

Prefix blockquote lines with >. Unordered lists accept -, *, or +. Ordered lists accept numeric markers; in CommonMark the first number establishes the list's start. Nesting and indentation follow the chosen parser's rules.²

Understand what conversion can lose

Markdown syntax does not express all HTML structures. GFM pipe tables do not encode merged cells; layouts, interactive widgets, and some element relationships need another representation. Mathematical notation and footnotes require additional support. Raw HTML may be accepted, escaped, filtered, or removed by the receiving system.

For extracted content, inspect tables, code blocks, captions, and links after conversion. Plain text carries less markup structure; readable HTML and Minified HTML retain cleaned elements for consumers that need them. The format comparison helps match a representation to the next processing step.

Markdown in model inputs

Markdown can keep section boundaries and links visible in a text prompt. Its token cost depends on the document and tokenizer; a fixed overhead percentage or per-heading token count cannot characterize it across models. Evaluate the extracted content and downstream task together.

The HTML-to-Markdown guide distinguishes converting tags from selecting main content. Running a converter on a full page can retain navigation and other material you did not intend to include.

Jeremy Howard's September 2024 /llms.txt proposal uses Markdown for a curated site overview, including a title, optional summary, descriptive text, and grouped links.⁵⁶ It also discusses Markdown versions of individual pages. It remains a proposal; the presence of a .md URL or a site overview does not by itself establish extraction quality or model performance.

Markdownee's two Markdown paths

Markdownee normally uses Trafilaturacore to clean and select HTML content. Its conversion package then renders Markdown with Turndown and GFM support. The extraction engine itself returns cleaned HTML; conversion belongs to Markdownee.

Optional markdownDiscovery can instead use a representation published by the origin:

• off, the default, leaves HTML fetching unchanged.
• alternate follows advertised same-origin Markdown links.
• negotiate additionally asks for Markdown through the HTTP Accept header.
• probe additionally tries a .md sibling URL.

The available steps depend on the crawler path. Discovery can try three alternates, a negotiated refetch, and a sibling; robots.txt checks can add an origin-level request. Per-origin budgets bound unsuccessful attempts. Unavailable or rejected representations fall back to HTML extraction.

An accepted representation supplies content for all requested formats. Markdown output can use the served body after source front matter is removed and the selected output layout applied; otherwise it is converted from cleaned HTML. Records carry markdownSource, including whether that body was used verbatim.

Request Markdown output

The Actor defaults to markdown-kvs, storing Markdown in the key-value store, as shown below. Use markdown-dataset instead when the record should contain the text inline:

{
  "startUrls": [{ "url": "https://example.com/" }],
  "save": ["markdown-kvs"]
}

The playground selects Markdown by default and offers .md downloads. CLI crawl defaults to markdown-kvs; crawl-one defaults to markdown-stdout. The npm library's crawlOne defaults to formats: ['markdown'] and returns a format map such as { markdown: '...' }.

Metadata can accompany the body through the selected layout and output records. Choose another format when the next consumer needs plain text or HTML structure.

Updated: September 10, 2026