Chemistry Code Corpus · jablonkagroup
chempile-code
A curated Python code corpus for training models on computational chemistry, biology, and materials workflows.
What chempile-code is
ChemPile-Code is a curated collection of scientific software scripts spanning chemistry, biology, and materials science. Created by jablonkagroup as a constituent dataset of the wider ChemPile project, it is designed to support the training and adaptation of language models on computational science tasks.
General code corpora often contain very little domain-specific scientific code, hindering the ability of models to assist with specialised simulations or cheminformatics pipelines. ChemPile-Code addresses this gap by compiling approximately 17.9 billion tokens across 2.27 million documents, focusing on widely used scientific libraries and execution scripts.
How it was made
Data were collected by filtering existing open code repositories, specifically the GitHub-Code subset of CodeParrot and the StarCoder subset of The Stack. Relevance to chemistry, biology, and materials science was identified using keyword matching targeting domain libraries, including molecular simulation engines like LAMMPS, GROMACS, and OpenMM, informatics toolkits like RDKit and ASE, analysis packages like MDTraj, and visualisation utilities like PyMOL and VMD.
The extracted files underwent deduplication using exact hash matching on code contents to remove identical snippets. The curated records were subsequently organised into standardized train, validation, and test splits alongside metadata such as file paths, matched keywords, and source repository names.
Key facts
- Total tokens
- ~17.9 billion
- Total documents
- ~2.27 million
- Primary language
- Python
- Source datasets
- The Stack (StarCoder) and GitHub-Code (CodeParrot)
- File format
- Parquet
- Subsets
- codeparrot_github-code-chemistry-python-default, starcoder-chemistry-default
What you can use it for
- Scientific Code Generation
- Training or fine-tuning foundation models to generate and autocomplete computational chemistry and physics scripts.
- Domain Adaptation
- Adapting general-purpose programming language models to specialised scientific computing libraries and domain syntax.
- Scientific Computing Assistance
- Powering automated assistants that construct simulation pipelines in tools such as LAMMPS, GROMACS, and OpenMM.
- Code Search and Analysis
- Enabling indexing and semantic search over scientific repositories to locate specific algorithmic implementations.
- Automated Documentation
- Generating documentation, docstrings, and explanatory summaries for complex scientific codebases.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from jablonkagroup/chempile-code at the moment you asked. Nothing is cached or stored — every row above came from that request.
How the columns are distributed
Over 186,878 rows of codeparrot_github-code-chemistry-python-default/train
keyword
list- min
- 1
- median
- 1
- mean
- 1.13
- max
- 23
language
string_label- Python100%
license
string_label- mit24%
- gpl-3.023%
- bsd-3-clause14%
path
string_text- min
- 4
- median
- 31
- mean
- 34.22
- max
- 191
repo_name
string_text- min
- 5
- median
- 18
- mean
- 19.33
- max
- 86
size
int- min
- 12
- median
- 7,511
- mean
- 19335.09
- max
- 1.0M
text
string_text- min
- 12
- median
- 7,510
- mean
- 19294.16
- max
- 1.0M
text_hash
string_text- min
- 64
- median
- 64
- mean
- 64
- max
- 64
Strengths and limits
Good at
- Provides a focused corpus of roughly 17.9 billion tokens dedicated to scientific computing scripts.
- Eliminates duplicate files through exact cryptographic hash filtering.
- Captures code from specialised chemical and physical simulation libraries such as ASE, RDKit, and OpenMM.
- Includes pre-partitioned train, validation, and test splits for structured evaluation.
Watch out for
- Primarily limited to Python, lacking broad coverage of other scientific programming languages.
- Contains code of variable quality from public repositories, including incomplete scripts or potential programming errors.
- Hash-based deduplication removes only exact text matches, leaving possible semantic duplicates in the collection.
Questions
Can I look inside chempile-code without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is chempile-code under?
Its authors declare agpl-3.0. Read the terms on the original page before reusing the data.
Which software libraries are included in the dataset?
The dataset specifically filters for libraries used in molecular dynamics and simulation, such as LAMMPS, GROMACS, and OpenMM, informatics toolkits like RDKit and ASE, trajectory analysis packages like MDTraj, and structural visualisers such as PyMOL and VMD.
What metadata fields accompany each code snippet?
Depending on the subset, entries include the code snippet text, source repository name, repository file path, language, content size in bytes, matched keywords, and an exact content hash.
How to cite it
Adrian Mirza, Nawaf Alampara, Martiño Ríos-García, and others. ChemPile: A 250GB Diverse and Curated Dataset for Chemical Foundation Models. arXiv preprint arXiv:2505.12534 (2025).
Written from chempile-code's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.