Trait Prediction Benchmark · HuggingFaceBio
traitgym
Evaluates genomic foundation models predicting causal non-coding variant effects across diverse human phenotypes.
What traitgym is
TraitGym is a multi-task benchmark designed to evaluate genomic foundation models on their ability to identify causal regulatory variants across diverse human phenotypes. This repackaging provides the original benchmark variants along with pre-extracted 8,192 base pair (bp) sequence windows from the hg38 human reference genome for both the reference sequence and the alternative sequence.
The benchmark assesses zero-shot variant effect prediction in the non-coding genome, which is critical for interpreting disease risk. It covers both monogenic Mendelian conditions curated from OMIM and complex polygenic traits from the UK Biobank, pairing causal variants against matched non-causal controls.
How it was made
The dataset consists of two subsets: mendelian_traits and complex_traits. The Mendelian subset contains 3,380 samples, comprising 338 putative-causal non-coding variants from 113 diseases matched 9:1 against gnomAD common variants based on chromosome, molecular consequence, and distance to the nearest transcription start site (TSS). The complex traits subset contains 11,400 samples, comprising 1,140 putative-causal variants identified via statistical fine-mapping (posterior inclusion probability, PIP > 0.9) from 83 traits matched 9:1 against non-causal controls (PIP < 0.01) based on chromosome, consequence, TSS distance, minor allele frequency, and linkage disequilibrium score.
For each single nucleotide variant, the dataset supplies an 8,192 bp window centred on the variant position (index 4096) on the forward strand from UCSC hg38. The alternative sequence substitutes the alternate allele at index 4096. Evaluation is conducted by computing the difference in sequence log-likelihood between the alternative and reference sequences, scored against binary labels using AUROC, AUPRC, or chromosome-weighted average AUPRC, with optional reverse-complement log-likelihood ratio averaging.
Key facts
- Variant type
- Single nucleotide variants (SNVs)
- Reference genome
- hg38
- Window size
- 8,192 bp
- Matching ratio
- 9:1 matched controls to positive variants
- File format
- Parquet
What you can use it for
- Mendelian regulatory variant prioritisation
- Evaluate how accurately sequence models discriminate causal non-coding disease variants in OMIM conditions from consequence-matched common controls.
- Polygenic trait fine-mapping benchmarking
- Benchmark the ability of genomic models to identify high-confidence fine-mapped non-coding variants from UK Biobank genome-wide association studies.
- Standardised sequence log-likelihood evaluation
- Compute zero-shot variant effect log-likelihood differences using pre-extracted, consistent 8,192 bp sequence contexts without requiring reference genome lookups.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from HuggingFaceBio/traitgym at the moment you asked. Nothing is cached or stored — every row above came from that request.
Strengths and limits
Good at
- Provides pre-extracted 8,192 bp sequence windows for reference and alternative alleles, removing the requirement to extract genomic context from FASTA files.
- Incorporates rigorous 9:1 control matching accounting for genomic location, consequence, distance to transcription start sites, allele frequency, and linkage disequilibrium.
- Covers both monogenic Mendelian disease genetics and polygenic complex traits.
Watch out for
- Restricted to single nucleotide variants, omitting insertions, deletions, and structural variants.
- Confined to a fixed 8,192 bp window, which cannot evaluate regulatory interactions occurring outside 4,096 bp from the variant site.
Questions
Can I look inside traitgym without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is traitgym under?
Its authors declare mit. Read the terms on the original page before reusing the data.
How are effect scores computed from these sequences?
Scores are typically calculated as the delta between the model log-likelihood of the alternative sequence window and the reference sequence window. Users can also average scores across the forward sequence and its reverse-complement.
How are the negative control variants selected?
In the Mendelian configuration, controls are common gnomAD variants matched 9:1 on chromosome, consequence, and TSS distance. In the complex traits configuration, controls are non-causal variants (PIP < 0.01) matched 9:1 on chromosome, consequence, TSS distance, minor allele frequency, and LD score.
What metrics are standard for evaluating this benchmark?
Performance is assessed using AUROC, AUPRC, and chromosome-weighted average AUPRC against the binary causal status labels.
How to cite it
Benegas, Gonzalo, Gokcen Eraslan, and Yun S. Song. 'Benchmarking DNA Sequence Models for Causal Regulatory Variant Prediction in Human Genetics.' bioRxiv (2025). doi:10.1101/2025.02.11.637758.
Written from traitgym's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.