Scientific Literature · jablonkagroup
chempile-paper
A corpus of 13.9 billion tokens from chemistry papers and preprints for training and evaluating language models.
What chempile-paper is
ChemPile-Paper is a curated text dataset comprising approximately 13.9 billion tokens across 11.7 million documents from scientific publications and preprints. Developed by the Jablonka Group as a component of the broader ChemPile initiative, it gathers chemistry-focused literature alongside relevant work in materials science, chemical physics, biology, and health sciences.
Training and evaluating domain-adapted language models requires structured exposure to scientific terminology, experimental findings, and chemical reasoning. ChemPile-Paper meets this need by standardising articles collected from arXiv, bioRxiv, medRxiv, ChemRxiv, and Europe PMC into consistent text splits designed for language modeling and natural language inference.
How it was made
Articles are retrieved from public preprint servers and databases using the PaperScraper package. Full-text content from bioRxiv, medRxiv, and ChemRxiv is parsed using the Nougat optical character recognition (OCR) model. For Europe PMC, which spans a broader medical corpus, chemistry articles are filtered using a BERT-based multilabel classifier trained on CAMEL datasets (20,000 examples per discipline) that inspects the first five 512-token segments of each text with 50-token overlaps, achieving an F1-score of approximately 0.77 against FineWebMath annotations.
Quality control routines remove extraneous non-chemical sections such as author lists, acknowledgements, and page numbers. The resulting data are divided into seven specific repository configurations, structured into training, validation, and testing splits, and stored in Parquet format.
Key facts
- Total tokens
- ~13.9 billion
- Total documents
- ~11.7 million
- Modality
- Text
- Format
- Parquet
- Primary language
- English
- Source repositories
- ArXiv, bioRxiv, medRxiv, ChemRxiv, Europe PMC
What you can use it for
- Language model training
- Pre-training or fine-tuning foundation models on specialised chemical text, terminology, and research prose.
- Scientific knowledge retrieval
- Building domain-specific search engines and indexing systems across peer-reviewed chemistry literature and preprints.
- Natural language inference
- Evaluating reasoning and claim verification capabilities using chemical and biomedical literature.
- Automated research synthesis
- Training systems to produce scientific summaries and generate domain-specific academic text.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from jablonkagroup/chempile-paper at the moment you asked. Nothing is cached or stored — every row above came from that request.
Strengths and limits
Good at
- Spans nearly 13.9 billion tokens across full-text articles and abstracts from five scientific repositories.
- Employs automated classifier-based filtering to select relevant chemistry literature from large biomedical archives.
- Includes dedicated configurations with standard splits for training, validation, and evaluation.
- Removes non-chemical text such as author affiliations and acknowledgements during post-processing.
Watch out for
- Limited to English-language scientific literature.
- Text quality varies across repositories, with potential optical character recognition errors in older documents.
- Inherits topical and demographic representation biases present in academic scientific publishing.
- Subject to temporal cutoffs, omitting research published after the data collection period.
Questions
Can I look inside chempile-paper without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is chempile-paper under?
Its authors declare cc-by-nc-nd-4.0. Read the terms on the original page before reusing the data.
Which repositories are covered in this dataset?
The corpus includes materials science and chemical physics papers from arXiv, chemistry preprints from ChemRxiv, biology preprints from bioRxiv, health sciences preprints from medRxiv, and both abstracts and full articles from Europe PMC.
How was chemistry literature distinguished from medical text in Europe PMC?
A BERT-based multilabel classifier trained on CAMEL datasets evaluated the initial five 512-token chunks of each document to identify chemistry content before inclusion.
What metadata fields accompany the extracted text?
Depending on the subset, entries include article titles, authors, DOIs, PubMed identifiers, PubMed Central identifiers, topic classifications, and classification confidence scores.
How was text extracted from preprint PDFs?
Preprint PDFs from bioRxiv, medRxiv, and ChemRxiv were collected using the PaperScraper package and parsed into text using the Nougat OCR tool.
How to cite it
Adrian Mirza, Nawaf Alampara, Martiño Ríos-García and others, 'ChemPile: A 250GB Diverse and Curated Dataset for Chemical Foundation Models', arXiv preprint arXiv:2505.12534, 2025.
Written from chempile-paper's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.