Website to Markdown API

Yes. It never logs in and needs no account on the site, only a free Apify account to run it. It reads the pages a logged-out visitor can see.

No. It uses plain HTTP without a browser, so pages that need JavaScript to show their content are skipped and not charged.

For sites that serve their text in HTML, yes. This Actor charges a flat $0.65 per 1,000 pages on the free plan and $0.60 on paid plans at 256 MB, and failed, duplicate and empty pages are not charged. apify/website-content-crawler bills compute units at 8 GB default memory, and apify/rag-web-browser costs $2.28 and $1.90. Both of those render JavaScript; this Actor reads server HTML only, so pick them for JavaScript-only sites.

Two ways. For a fixed workflow, add Apify's n8n or Make node, choose **Run Actor**, enter aurelix/website-to-markdown and map the dataset to the next step. For an AI agent (Claude, ChatGPT, Cursor or an n8n AI Agent node), add https://mcp.apify.com/?tools=aurelix/website-to-markdown as an MCP server and ask for pages as Markdown in plain words; setup for each client is in the section above. Runs bill your own Apify account at the same per-result price.

$0.65 per 1,000 pages ($0.60 on paid Apify plans), billed through your Apify account; you pay only for results. Apify's free plan includes $5 of credit a month, enough for about 7,692 pages.

Always, and it cannot be switched off. Disallowed URLs are never requested, Crawl-delay is honoured, redirects into disallowed paths are not followed, and a site whose robots.txt returns a server error is not crawled. Links on pages marked nofollow are not followed.

The page's <article> or <main> element when it has one, otherwise the largest content block. Navigation, headers, footers, tables of contents, cookie banners, site banners, version switchers, Wikipedia language and category boxes, share buttons, forms, scripts and hidden elements are removed. Prose sidebars and footnotes inside the content are kept; link-list sidebars are dropped.

No. It reads the HTML the server sends, which covers documentation sites, blogs, news, knowledge bases and most company sites, including server-rendered React and Next.js. Pages that are an empty JavaScript app shell are detected, listed in the log and not charged. A headless browser would need about 1 GB of memory per run, so it is left out to keep runs cheap.

Use the start URL's folder (scope 'Under the start path'), or include and exclude globs: '**' matches anything, '*' anything except '/'. Example: include https://example.com/docs/** and exclude **/v1/**.

No. A URL is returned once, and a page whose Markdown is identical to one already returned (e.g. / and /index.html) is skipped for free.

Run this Actor once for the full copy, then re-crawl only what changed with our Website Change Tracker on the same start URLs: it returns added, changed and removed pages, so you re-embed only those.

Yes. The inputs are described for agents: give 'startUrls' and optionally 'maxPages'. It works through Apify's MCP server and API.

It reads only public data any logged-out visitor can see: the website you choose (crawl only sites you have the right to crawl). It goes at a polite pace, respects robots.txt, never logs in and never solves CAPTCHAs. It returns each page's main content as Markdown with title, headings and links; the content is returned as the site published it; copyright in it stays with its owner. To have a record removed, open an issue on this Actor's Issues tab; if a site owner formally asks us to stop, this Actor will stop. You are responsible for using the data lawfully, incl. anti-spam and privacy laws; this is not legal advice.

Only the $0.001 start fee. Pages blocked by robots.txt, fetch errors, HTTP errors such as 404, 429 and 500, non-HTML files, duplicates, pages over 2 MB, pages that need JavaScript and pages with no main content are never charged. The run's status message lists how many pages were skipped for each reason.

No. Max pages counts only pages converted and charged, so a crawl with many errors or duplicates still returns up to that many pages if the site has them. The highest value is 10,000.

About 1,000 to 1,300 large documentation pages an hour at the default 256 MB, because the conversion is CPU-bound. A run with the default 1-hour timeout fits roughly 1,000 to 3,000 pages, depending on page size. The run stops itself before its timeout and ends SUCCEEDED with the pages done so far, so raise the timeout or the memory for bigger crawls.

Yes. HTTP 429 and 5xx responses and connection errors are retried up to 3 times in all, waiting for the site's Retry-After header when it sends one (up to 30 seconds). A page still failing after that is skipped and not charged; a 429 is named as throttled in the status message, so you know to try again later.

No. It fetches pages directly, identifies itself as AurelixBot in the user agent, and sends at most two new requests a second per site with up to 3 in flight. Sites that block bots or show a challenge page will not be converted.

No. www. and the bare domain count as one site; other subdomains and other sites are not followed. Add each subdomain as its own start URL if you need it.

crawl_depth is the number of links followed from a start URL to reach the page (0 for the start URL itself). http_status is the final HTTP status of the page after redirects; it is always 200 on returned rows, because other statuses are skipped.

Yes. The Markdown keeps the page's own text in any language, and the language field is read from the page's HTML lang attribute (empty when the page does not set one).

Download the run's results as CSV or Excel and open them in Google Sheets, or add Apify's Google Sheets integration to the run or a schedule so every run's results land in a sheet.