Protein Sequences · LiteFold
UniRef90
A shard index of UniProt protein sequences clustered at 90% identity for sequence search and language model training.
What UniRef90 is
UniRef90 provides clustered protein sequences derived from UniProt, grouping entries so that each cluster contains sequences sharing at least 90% sequence identity. This organisation reduces redundancy while retaining comprehensive sequence diversity, providing a foundational resource for computational biology and genomics.
Published by LiteFold, this dataset provides a structured index over partitioned UniRef90 data. Covering 188,848,220 records and 66,359,825,357 amino acid residues, it provides a manifest for accessing and loading large sequence datasets in distributed computational workflows.
How it was made
The dataset is structured as a tabular index of 194 rows tracking partitioned files. It indexes 189 sequence shards stored as Zstandard-compressed FASTA files alongside metadata stored in JSONL format. Each row defines file roles, storage paths, byte sizes, compression formats, and record counts.
Data partitioning into training and evaluation sets is deterministic, computed from the SHA-256 hash of each file identifier modulo 10. Files falling into bucket 0 constitute the test split, while buckets 1 through 9 make up the training split.
Key facts
- Publisher
- LiteFold
- Clustering identity
- 90%
- Total records
- 188,848,220
- Total residues
- 66,359,825,357
- Sequence shards
- 189
- Index rows
- 194
- Sequence format
- FASTA compressed with Zstandard (.fasta.zst)
- Index format
- Parquet
What you can use it for
- Sequence similarity search
- Use 90% identity cluster representatives as search databases to detect sequence homology efficiently.
- Annotation transfer
- Map functional annotations across closely related proteins within the same sequence identity cluster.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from LiteFold/UniRef90 at the moment you asked. Nothing is cached or stored — every row above came from that request.
How the columns are distributed
Over 175 rows of default/train
access_note
string_label- File/shard index for UniRef90; download sequence shards for FASTA records.100%
compression
string_label- zstd97%
2.9% empty
download_pattern
string_label- sequences/sequence_uniref90_uniref90.fasta.gz/shard-*.fasta.zst97%
- _MANIFEST.json1%
- README.md1%
file_id
string_text- min
- 9
- median
- 68
- mean
- 66.89
- max
- 68
is_metadata_records
bool- False99%
- True1%
is_sequence_shard
bool- True97%
- False3%
path
string_text- min
- 9
- median
- 68
- mean
- 66.89
- max
- 68
records_total
int- min
- 189M
- median
- 189M
- mean
- 189M
- max
- 189M
repo_id
string_label- LiteFold/UniRef90100%
residues_total
int- min
- 66.4B
- median
- 66.4B
- mean
- 66.4B
- max
- 66.4B
role
string_label- sequence_shard97%
- git_attributes1%
- metadata_records1%
shard_index
float- min
- 0
- median
- 96.50
- mean
- 95.35
- max
- 188
2.9% empty
size_bytes
int- min
- 464
- median
- 175M
- mean
- 587M
- max
- 66.4B
source_sha
string_label- ee8996a7f408c0f9ab4e8a6bc8739ce2a4c785db100%
source_slug
string_label- sequence_uniref90_uniref90.fasta.gz100%
split_bucket
int- min
- 1
- median
- 5
- mean
- 5.19
- max
- 9
total_shards
int- min
- 189
- median
- 189
- mean
- 189
- max
- 189
Strengths and limits
Good at
- Substantially reduces sequence redundancy while maintaining broad coverage across known protein space.
- Organises massive biological sequence data into 189 manageable, Zstandard-compressed shards.
- Provides deterministic, hash-based train and test splits across all partitioned files.
Watch out for
- Accessing the underlying sequence data requires downloading and decompressing external Zstandard FASTA files.
- Sequences with greater than 90% identity are clustered together, omitting individual duplicate variants.
Questions
Can I look inside UniRef90 without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is UniRef90 under?
Its authors declare cc-by-4.0. Read the terms on the original page before reusing the data.
What is the primary content of the index table?
The table records paths, sizes, shard indexes, record totals, residue totals, and roles for both FASTA sequence shards and JSONL metadata files.
How are the sequence files compressed?
Sequence shards are formatted as FASTA files and compressed using Zstandard (.fasta.zst).
How are train and test splits allocated?
Splits are computed deterministically using the formula sha256(file_id) % 10, where bucket 0 forms the test split and buckets 1 to 9 form the train split.
How to cite it
Suzek, Baris E. and Wang, Yuqi and Huang, Hongzhan and McGarvey, Peter B. and Wu, Cathy H. and UniProt Consortium. UniRef clusters: a comprehensive and scalable alternative for improving sequence similarity searches. Bioinformatics, 31(6), 926–932, 2015. doi:10.1093/bioinformatics/btu739.
Written from UniRef90's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.