OpenAlex Scholarly Works Scraper

OpenAlex Scraper: search papers and get the abstract as readable text

Search OpenAlex by keyword and get back the paper, the authors, the institutions behind them, the year, the citation count, the DOI, whether it is open access and where to read it. The abstract comes back as ordinary prose.

That last part is the work. OpenAlex does not store abstracts as text. It stores a word-position index, and this actor rebuilds the sentences from it. Papers that have no index have no abstract, and the field is null rather than a guess.

🔍 What OpenAlex Scraper does

Your keywords go against OpenAlex's own search, which reads titles, abstracts and full text where it has them. The run then pages with a cursor, fifty at a time, until it has the number of unique works you asked for or OpenAlex runs out.

You can order by relevance, by citation count or by date, and set a published-on-or-after floor. For anything more specific there is a raw filter field that goes straight through to OpenAlex, so type:article,is_oa:true works exactly as their documentation describes it.

Nothing is written to your dataset until the whole set has been collected, so you get either the full result set or a diagnostic row explaining why not.

📥 What you give it

{
  "query": "protein folding",
  "sort": "citations",
  "fromDate": "2020-01-01",
  "maxItems": 500
}

filter is handed to OpenAlex as you typed it, and a mistake in it does not come back as an error. OpenAlex answers a malformed filter with an empty result set, so the run finishes on NO_RESULTS and you are left thinking your query matched nothing. Check the filter keys against OpenAlex's documentation before blaming the query.

If your filter already contains from_publication_date, the fromDate field is ignored. Set the date in one place, not both.

📤 What you get back

A real row from a recent run, with the author and institution lists cut short here for length:

{
  "ok": true,
  "openalexId": "https://openalex.org/W2101234009",
  "doi": "https://doi.org/10.48550/arxiv.1201.0490",
  "title": "Scikit-learn: Machine Learning in Python",
  "authors": ["Fabián Pedregosa", "Gaël Varoquaux", "Alexandre Gramfort", "..."],
  "institutions": ["Commissariat à l'Énergie Atomique et aux Énergies Alternatives", "..."],
  "year": 2012,
  "publicationDate": "2012-01-02",
  "type": "article",
  "venue": "ORBi (University of Liège)",
  "citations": 63984,
  "concepts": ["Python (programming language)", "Documentation", "Computer science", "..."],
  "isOpenAccess": true,
  "oaUrl": "https://orbi.uliege.be/handle/2268/225787",
  "abstract": "Scikit-learn is a Python module integrating a wide range of ... machine learning algorithms for medium-scale supervised and unsupervised problems.",
  "url": "https://openalex.org/W2101234009"
}

🧾 Reading the output

Two kinds of row land in your dataset, and ok tells them apart.

The overview table in the Apify console shows the work columns only, so a diagnostic row looks blank there. Switch to the JSON or All fields view to read it.

▶️ How to run it

1. Open OpenAlex Scraper and click Try for free. 2. Type your keywords into Search query. 3. Set Max works. Start around 50 to see the row shape. 4. Pick a Sort by, add a From publication date if you want, then click Start. 5. Download the dataset as JSON, CSV or Excel, or read it from the Apify API.

💰 How much does it cost?

$2.00 per 1,000 works. Flat on every Apify plan, no volume tiers.

You pay per work delivered. Works that appear twice across pages are dropped before they are counted, diagnostic rows are not charged, and a search that matches nothing is not charged.

💡 What people use it for

• Pulling a field's most-cited work with the abstracts attached, in one file, for a reading list.
• Feeding titles and abstracts to a model that has to summarise research it was not trained on.
• Finding which institutions keep appearing on a topic, using the institutions field.
• Filtering to open access with is_oa:true so every row has somewhere to read it for free.
• Watching a topic on a schedule with sort on date, so each run surfaces what is new.

🚧 What it does not do

• Keyword search over works only. No author, institution or funder lookup by entity.
• No full text. You get the abstract and a link, not the paper.
• Abstracts are missing where OpenAlex has no index for them, and that is common on older and

paywalled records.

• citations cannot tell you zero from unknown. Both read 0.
• A bad filter looks like an empty search, not like an error.
• No end date field. Use the raw filter if you need an upper bound.
• Institutions and concepts are OpenAlex's matching, and it is not perfect. Treat them as hints,

not as ground truth.

• All or nothing. A failure partway through discards what had been collected, so you get a

diagnostic row rather than a partial set.

🧭 Which research scraper do you need?

❓ Questions people ask

Do I need an API key? No. OpenAlex is open and the actor talks to it directly.

Why is the abstract null on some rows? OpenAlex only keeps a word-position index for a work if the publisher made one available. No index, no abstract to rebuild.

Why is venue a repository instead of a journal? OpenAlex picks one primary location per work, and for a paper with a preprint or a repository copy it sometimes picks that. The DOI still points at the published version.

Can I search by author or institution? Not as a mode. You can approximate it through the raw filter field using OpenAlex's own filter keys.

Why did my filtered search return nothing? Most often a filter key or value that OpenAlex does not recognise. It answers with an empty set rather than an error, so check the filter first.

Is this legal? OpenAlex publishes this data openly under a public domain dedication and invites this kind of use. Apify's write-up on scraping and the law is a good starting point, and we are not lawyers.

🆘 If something breaks

Open the Issues tab on the actor page. Send the run ID, the query and any filter you used. The errorCode on the diagnostic row usually names the problem on its own.