Skip to main content

Purge crawl storage

Web content as Markdown — built on Trafilaturacore, free, open source.

Generate purge command

Generate a ready-to-run command or code to clear the Markdownee crawl storage.

Storage

Commands or code only

What to do:

npm package CLI

Clear the Markdownee crawl storage using the markdownee CLI. This permanently deletes the datasets/, key_value_stores/, and request_queues/ storage buckets — it is irreversible, with no confirmation and no undo. Point --storage at the storage directory to clear; with no --storage it resolves the default location (./storage or the shared XDG storage), which may wipe unrelated crawl data. Requires Node 22.22.2+ on the 22 line, 24.15.0+ on the 24 line, or 26+.

Content variants

What is Markdownee?

Collect the main content of web pages for research, dataset preparation, or a retrieval pipeline. Markdownee combines page fetching with configurable extraction and conversion. Preview one URL here, then use the generated commands or code to run your own collection.

  • Boilerplate removal is powered by Trafilaturacore, our open-source pure-TypeScript port of Trafilatura. Its extraction core is a direct port of Python Trafilatura — with go-trafilatura used only as a DOM translation aid — and applies Trafilatura's own heuristics to strip navigation, sidebars, footers, and similar clutter
  • Fetch rendered pages through Crawlee and Playwright, then pass their HTML to Trafilaturacore
  • No Python runtime or GPU is required
  • Choose open-source, self-hosted execution through the npm CLI or npm library, or use the hosted Apify Actor; source code is on GitHub
  • Save image files with the optional image downloading mode

Getting started

The easiest way to try Markdownee — one npx command extracts a page straight to your terminal, with no browser install (you can install it later) and no API key — you host it yourself:

npx markdownee crawl-one https://example.com/ --crawler-type cheerio

--crawler-type cheerio fetches over plain HTTP, so no headless browser is downloaded. Need a whole-site crawl or specific formats? Use the playground to build a more advanced command visually, then copy it.