MSA / Training Corpus · LiteFold
OpenProteinSet
A massive open-source training corpus of protein sequence alignments and template structures for structure prediction.
What OpenProteinSet is
OpenProteinSet is an open-access biological dataset developed by the OpenFold team to reconstruct and expand the training data pipeline of AlphaFold2. While original AlphaFold2 model training relied on an unreleased corpus of multiple sequence alignments (MSAs) and template structures, this resource provides the necessary data to train protein folding architectures from scratch to performance parity with AlphaFold2.
The dataset contains over 16 million precomputed MSAs, structural template search hits from the Protein Data Bank (PDB), and AlphaFold2 structure predictions. It addresses the computational bottleneck of sequence searching across massive databases, enabling structural biologists and computational researchers to train, benchmark, and fine-tune macromolecular structure models without generating alignments independently.
How it was made
The corpus is organised into two primary components. The first is a PDB-derived training set consisting of MSAs and template hits for every unique PDB chain, constructed using JackHMMER against UniRef90 and MGnify, HHBlits against BFD and Uniclust30, and HHSearch against PDB70. The second component covers the Uniclust30 database with one alignment per cluster representative—yielding roughly 16 million MSAs—alongside a curated, deep, and diverse subset paired with AlphaFold2 self-distillation predictions for noisy student training.
Files include sequence alignments in .a3m format, homology search outputs in .hhr format, and metadata tables in CSV and JSON formats.
Key facts
- Publisher
- LiteFold
- Alignment files (.a3m)
- 393,000 files (535.65 GiB)
- Homology search files (.hhr)
- 131,454 files (64.40 GiB)
- Total file count
- 524,458 files
- Total uncompressed payload
- 652.27 GiB
- Archived tar payload
- 601.38 GiB across 31 shards
- Precomputed MSAs
- Over 16 million
- Primary sequence sources
- UniRef90, MGnify, BFD, Uniclust30, and PDB70
What you can use it for
- Training Protein Structure Prediction Models
- Train models like OpenFold and AlphaFold2-style architectures from scratch using precomputed alignments and PDB template hits.
- Self-Distillation and Noisy Student Training
- Apply precomputed AlphaFold2 structure predictions paired with deep Uniclust30 alignments to execute large-scale distillation protocols.
- Evolutionary Feature Extraction
- Extract pre-built multiple sequence alignments and HHSearch homology matches across PDB chains for downstream biological machine learning tasks.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from LiteFold/OpenProteinSet at the moment you asked. Nothing is cached or stored — every row above came from that request.
Strengths and limits
Good at
- Faithfully reconstructs the previously unreleased training data pipeline used for AlphaFold2.
- Provides over 16 million precomputed multiple sequence alignments, eliminating high-compute search steps for users.
- Includes self-distillation structure predictions across diverse sequence clusters for noisy student learning.
- Offers structured metadata tables to allow streaming or selective extraction of individual files from tar archives.
Watch out for
- Accessing specific individual files requires extracting members from multi-gigabyte tar shards or reassembling split byte files.
Questions
Can I look inside OpenProteinSet without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is OpenProteinSet under?
Its authors declare other. Read the terms on the original page before reusing the data.
What tools were used to build the alignments in OpenProteinSet?
Alignments were built using JackHMMER against UniRef90 and MGnify, HHBlits against BFD and Uniclust30, and HHSearch against PDB70.
What file formats are contained in the dataset?
The archive primarily consists of .a3m alignment files and .hhr homology search files, alongside zipped mmCIF structures, CSV metadata, and JSON manifest files.
How are large files such as pdb_mmcif.zip retrieved?
Large files exceeding typical upload boundaries are stored in split byte parts that must be downloaded and concatenated sequentially using the part paths specified in metadata.csv.
How to cite it
Ahdritz, Gustaf, Nazim Bouatta, Sachin Kadyan, Lukas Jarosch, Daniel Berenberg, Ian Fisk, Andrew M. Watkins, Stephen Ra, Richard Bonneau, and Mohammed AlQuraishi. "OpenProteinSet: Training Data for Structural Biology at Scale." Advances in Neural Information Processing Systems 36: Datasets and Benchmarks Track (2023). arXiv:2308.05326.
Written from OpenProteinSet's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.