Package Registry Scraper (npm + PyPI)

Package Registry Scraper: npm and PyPI metadata in one row shape

Look up packages by exact name on npm or PyPI, or search npm by keywords, and get one row each: name, latest version, description, author, homepage, repository, licence, keywords and the project page URL. npm rows can also carry last month's download count.

The two registries are not symmetrical, and it matters before you plan a run. npm does keyword search and exact names. PyPI does exact names only, because it publishes no clean search API for anyone to call.

🔍 What Package Registry Scraper does

Two ways in, and you can use both at once on npm.

Exact names. Put a list in packageNames and each one is looked up directly. Scoped npm names like @types/node work. A name that does not exist gets its own NOT_FOUND row, uncharged, and the run carries on with the rest.

npm keyword search. Put words in searchQuery and the registry's own search decides what matches, ordered by its relevance score, up to your maxItems.

There is a real difference between the two that the field list hides: a search row has no licence and usually no repository link. npm's search index does not carry them. If you are doing a licence audit, feed the names in through packageNames and look them up properly.

includeDownloads is on by default and adds last month's download count to npm rows. PyPI has no public endpoint for that, so PyPI rows never carry it.

📥 What you give it

{
  "registry": "npm",
  "packageNames": ["react", "@types/node"],
  "includeDownloads": true
}

packageNames has no ceiling. Paste a 4,000-line requirements.txt and you get up to 4,000 charged rows, because maxItems is not watching that path. Trim the list to what you need.

📤 What you get back

A real row from a recent npm search:

{
  "ok": true,
  "registry": "npm",
  "name": "unstated-next",
  "version": "1.1.0",
  "description": "200 bytes to never think about React state management libraries ever again",
  "author": "thejameskyle",
  "homepage": null,
  "repository": null,
  "license": null,
  "keywords": [],
  "score": 364.87637,
  "url": "https://www.npmjs.com/package/unstated-next",
  "monthlyDownloads": 330524
}

That row shows the search-path gap plainly: license, repository, homepage and keywords are all empty, because the search index does not carry them. Look the same package up by name and they are filled in.

🧾 Reading the output

Packages carry ok: true. Anything with ok: false carries an errorCode and is not charged. One run can contain both, which is normal when a name list has a typo in it.

The Console's Overview table hides license and repository, which are the two columns a licence audit actually needs. Download the dataset as JSON, CSV or Excel, or switch the table to all fields.

▶️ How to run it

1. Open Package Registry Scraper and click Try for free. 2. Pick a Registry. 3. Paste names into Package names, one per line. For an npm search, type into Search query instead. 4. Leave Include monthly downloads on unless you do not want it, then click Start. 5. Download the dataset as JSON, CSV, Excel or XML.

💰 How much does it cost?

$2.00 per 1,000 packages. Flat on every Apify plan, no volume tiers, and the same on both registries.

You pay per package row. The same name twice in one run is charged once. NOT_FOUND names, a search that matched nothing and every diagnostic row are not charged. Turning downloads on does not change the price.

💡 What people use it for

• Auditing the licences behind a package.json or a requirements.txt by feeding the names in and

reading the license column.

• Comparing two libraries side by side on version, downloads and last release before choosing one.
• Watching a dependency on a schedule and firing an alert when version moves.
• Sizing a niche: search npm for the keywords, then rank by monthlyDownloads to see what people

actually install.

🚧 What it does not do

• No PyPI search. Exact names only on that side.
• No PyPI download counts. There is no public endpoint to read them from.
• No licence on npm search rows. Use packageNames when the licence is the point.
• No dependency trees and no version history. One row is the current state of one package.
• No download trend. monthlyDownloads is last month's total, a single number, not a series.
• No GitHub stars or issues. repository gives you the link to go and look.
• Rows are a snapshot. Versions and download counts move daily.

🧭 Which developer data scraper do you need?

❓ Questions people ask

Can I search PyPI by keyword? No. PyPI has no clean public search API, so names are the only route. Find the name elsewhere and look it up here.

Why is license empty? You came in through the npm search. Search results do not carry it. Take the names from that run and look them up in a second run.

Can I mix a search and a name list? On npm, yes. Both run and the results are deduplicated.

Does maxItems protect me from a huge name list? No. It caps the search only. Trim the list yourself.

What happens to a name that does not exist? It comes back as its own NOT_FOUND row, you are not charged for it, and the other names still run.

Is this legal? Both registries publish this through open APIs meant to be read, and it is package metadata rather than personal data. Apify's write-up on scraping and the law is a good starting point, and we are not lawyers.

🆘 If something breaks

Open the Issues tab on the actor page. Send the registry, the names or query you used and the run id. The errorCode on the diagnostic row usually names the problem by itself.