Substack Publication Scraper

Substack Publication Scraper: every post in an archive, one row each

Give it a Substack handle like astralcodexten and get the publication's archive back as rows. Title, link, date, author, word count, comments, reactions, restacks, and whether the post sits behind the paywall.

Read this before you buy: no row carries the article text. A publication's public archive does not publish post bodies, for free posts either, so there is nothing to hand you. What you do get is audience, which is the real free-or-paid flag, and Substack's own short preview on about half the posts.

📰 What Substack Publication Scraper does

It reads a publication's own archive and walks it to the depth you ask for. A handle on its own is enough. If the publication moved onto its own domain, give that domain: a publication on its own domain only answers there, and its old substack.com address will hand back an empty archive rather than an error.

You can point it at 20 publications in one run, and each gets its own post budget, so one prolific newsletter cannot eat the whole run.

A word on what arriving means. Three different wrong answers all come back as HTTP 200 here: a publication that has moved returns an empty list, one of them redirects to a profile page and answers with a web page, and a site that is not Substack at all serves its own page from the same address. Each of those is checked and turned into an uncharged row that names what to do about it. Nothing is billed for a response that was not an archive.

📥 What you give it

{
  "publications": ["astralcodexten", "www.bigtechnology.com"],
  "maxPostsPerPublication": 50,
  "sort": "new"
}

📤 What you get back

A real row from a recent run:

{
  "ok": true,
  "charged": true,
  "recordType": "post",
  "publication": "astralcodexten.substack.com",
  "title": "Your Book Review: This Is Going To Hurt",
  "url": "https://www.astralcodexten.com/p/your-book-review-this-is-going-to",
  "postDate": "2026-09-18T20:53:58.344Z",
  "audience": "everyone",
  "isPaywalled": false,
  "subtitle": "Finalist #9 in the Book Review Contest",
  "description": "Finalist #9 in the Book Review Contest",
  "previewText": null,
  "wordCount": 12639,
  "commentCount": 295,
  "reactionCount": 152,
  "restackCount": 7,
  "authors": "Scott Alexander",
  "postType": "newsletter",
  "slug": "your-book-review-this-is-going-to",
  "postId": "216363681",
  "coverImage": "https://substack-post-media.s3.amazonaws.com/public/images/916daf3c-...png",
  "language": "en",
  "scrapedAt": "2026-09-21T02:09:44.003Z"
}

🧾 Reading the output

Three kinds of row can land in your dataset, and recordType tells them apart.

The charged field is a label on the row, not a receipt. It is stamped when the row is built, which is before billing happens, so treat it as the same signal as recordType: it separates real posts from the sample and diagnostic rows, nothing more.

One bad publication never stops the others, and a diagnostic row never fails the run.

▶️ How to run it

1. Open Substack Publication Scraper and click Try for free. 2. Put one handle per line into Substack publications. 3. Set Posts per publication. Start around 10 while you are looking at the output shape. 4. Pick an Order if reverse date order is not what you want. 5. Click Start, then download the dataset as JSON, CSV or Excel, or read it from the Apify API.

💰 How much does it cost?

$0.26 per 1,000 posts. Flat on every Apify plan, no volume tiers.

You pay per post row delivered. Duplicates, the sample row, diagnostic rows and a publication that returns nothing are all free.

💡 What people use it for

• Tracking which posts a newsletter puts behind the paywall and which it leaves open, using

audience over time.

• Ranking a publication's back catalogue by reactionCount or commentCount to see what actually

landed.

• Pulling 20 newsletters in one field into a single sheet to compare posting cadence.
• Watching a competitor's publishing rhythm on a schedule, deduped on postId.

🚧 What it does not do

• No article text. The archive does not publish post bodies. previewText is Substack's own

snippet and it is null about half the time.

• No subscriber counts, no revenue, no email list size. None of that is public.
• No comments themselves, only how many there are.
• A publication on its own domain only answers on that domain. Its old substack.com address

returns an empty archive, and you will get a row saying so.

• top and community are Substack's own orderings, applied before paging, so asking for 20 of

them gives you the 20 it serves first.

• No Substack Notes, no podcasts beyond the post record, and no cross-publication search.
• Rows are a snapshot. Reaction and comment counts move, and scrapedAt records when they were

read.

• Up to 20 publications per run. Split a longer list across runs.

🧭 Which news scraper do you need?

❓ Questions people ask

Do I need a Substack account? No. No account, no cookies, no browser.

Can I get the post text? No, and nothing else can either from a public archive. Substack does not publish bodies there, for free posts or paid ones.

How do I know if a post is paywalled? isPaywalled, or audience if you want Substack's own wording. Every row has it.

What do I put in for a publication on its own domain? That domain. www.bigtechnology.com, not the old substack.com address, which answers empty.

Can I watch several newsletters at once? Yes, up to 20 per run, each with its own post budget.

Is scraping Substack archives legal? These are public archive pages. Rows carry author names, which is personal data under GDPR, so have a reason for collecting it. Apify's write-up on scraping and the law is a sensible starting point, and we are not lawyers.

🆘 If something breaks

Open the Issues tab on the actor page. Send the run ID and the publication you used. The errorCode on the diagnostic row usually names the problem on its own.