Markdownee for Python (PyPI)

This Python wrapper for Markdownee is currently an alpha, experimental release — not fully tested or officially supported, though still maintained.

The PyPI package exposes typed Python calls over the bundled Node CLI. Its runtime dependency supplies Node.js; a separate Node installation is unnecessary. For direct Node.js integration, use the npm library.

Install the package and browser

pip install markdownee
python -m markdownee install           # one-time Chromium, for the adaptive and chromium engines
python -m markdownee install firefox   # only if you use the firefox engine

Python 3.12 or later is required. The documented wheel targets are macOS arm64/x86_64, Linux x86_64/aarch64 with glibc 2.28 or later, and Windows x64. musl is unsupported. A compatible published wheel is required; the package does not provide a source distribution.

Chromium serves adaptive and Chromium crawling. Firefox requires its separate browser installation. Cheerio uses HTTP and needs neither. Browser downloads are separate from the wheel and honor PLAYWRIGHT_BROWSERS_PATH.

Write a crawl to files

crawl accepts a URL string or URL list and exports the requested content:

import markdownee

summary = markdownee.crawl(
    ["https://example.com"],
    save=["markdown-kvs"],
    output_dir="./out",
    max_requests_per_crawl=10,
)
print(summary)
# CrawlSummary(total=1, succeeded=1, failed=0, skipped=0,
#                output_dir='/abs/out', manifest_path='/abs/out/manifest.json')

The output directory defaults to ./markdownee-output. Its manifest.json array contains records tagged success, failed, or skipped. The call returns a frozen CrawlSummary:

Use acrawl for the awaitable equivalent with the same options:

import asyncio
import markdownee

async def main():
    summary = await markdownee.acrawl(
        ["https://example.com", "https://example.org"],
        save=["markdown-dataset", "original-kvs"],
        output_dir="./out",
        max_concurrency=5,
    )
    print(summary.succeeded, "of", summary.total)

asyncio.run(main())

Partial page failures remain visible in summary.failed. Invalid configuration, missing runtime prerequisites, and child-process failures can raise errors rather than returning a summary.

Return one page's content

crawl_one follows no links and returns content without an export directory:

import markdownee

url = "https://en.wikipedia.org/wiki/Web_scraping"
contents = markdownee.crawl_one(url)
print(contents.get("markdown"))  # markdown is the default format

One or several requested formats return a CrawlOneResult dictionary. Its keys are txt, markdown, html, minified_html, and original; select compact HTML with the minified-html format token:

contents = markdownee.crawl_one(
    "https://example.com",
    formats=["markdown", "minified-html", "original"],
)
print(contents["markdown"])  # extracted markdown
print(contents["minified_html"])  # compact cleaned HTML
print(contents["original"])  # raw page HTML

The single-page CrawlOneOptions subset includes settings such as proxy, mode, user_agent, cookies, headers, and headless. Crawl-frontier controls such as globs and max_crawl_depth are excluded.

A failed page raises MarkdowneeError. A format yielding no content is omitted, including when it is the only requested format.

Return-only extraction rejects image_handling="save". Use resolved-url for absolute references, storage-backed crawl for KVS images, or npm CLI crawl-one with file output for sibling image assets.

acrawl_one provides the async form:

import asyncio
import markdownee

async def main():
    url = "https://en.wikipedia.org/wiki/Web_scraping"
    contents = await markdownee.acrawl_one(url)
    print(contents.get("markdown"))

asyncio.run(main())

Configure the run

CrawlOptions supplies typed keyword arguments. These technical reference values describe the supported vocabulary:

Readable html and compact minified-html are separate formats. output_layout selects body-only/fragment output (minimal), ordinary metadata and complete HTML (standard), or additional allowlisted metadata and crawl information (enhanced).

markdown_discovery changes the content source for every format. alternate follows advertised same-origin Markdown links; negotiate also requests Markdown with Accept; probe also tries a .md sibling. The default off leaves HTML fetching unchanged. Crawler capabilities and per-origin budgets limit attempts; robots.txt lookup can add an origin-level request. Unavailable or rejected representations fall back to HTML extraction.

Reuse JSON settings

A JSON config file uses camelCase, while Python keywords use snake_case:

{
  "mode": "precision",
  "save": ["markdown-kvs", "minified-html-dataset"],
  "outputLayout": "standard",
  "maxRequestsPerCrawl": 25,
  "maxCrawlDepth": 2
}

markdownee.crawl(
    ["https://example.com"],
    config_file="config.json",
    output_dir="./out",
)

Explicit keyword arguments override config-file values.

Use a proxy

Supported proxy schemes are http, https, socks4, and socks5. Unsupported schemes raise ProxySchemeError before spawning the CLI. The wrapper redacts registered credentials from child diagnostics.

markdownee.crawl(
    ["https://example.com"],
    proxy=["http://user:pass@proxy-host:3128"],
    proxy_rotation="per-request",
    output_dir="./out",
)

proxy_rotation accepts recommended, per-request, or until-failure. The default is recommended; synchronous and async crawl and single-page calls accept the option.

Provision browsers from Python

The same browser setup is available programmatically:

import markdownee

markdownee.install("chromium")  # the adaptive and chromium engines
markdownee.install("firefox")   # only if you use the firefox engine

Handle errors and runtime settings

Inspect partial-result counts as well as exceptions. The wrapper also accepts:

• MARKDOWNEE_NODE_PATH to select a host Node executable.
• storage_dir to reuse a Crawlee directory; otherwise each call uses temporary storage that is cleaned up afterward.
• timeout as a wall-clock limit in seconds for each child process.

Related guides

• Help hub lists the interfaces.
• Playground previews pages and generates code.
• Apify Actor covers hosted execution.
• npm CLI documents the commands invoked by the wrapper.
• npm library covers direct Node.js calls.

Updated: September 10, 2026