Interpretability · biohub
ESMC-SAE-Features
A feature table annotating 16,384 latent features from a sparse autoencoder trained on the ESMC-6B protein model.
What ESMC-SAE-Features is
ESMC-SAE-Features is an interpretability dataset compiled by Biohub containing annotations and statistics for the 16,384 features of a sparse autoencoder (SAE) trained on layer 60 of the ESMC-6B protein language model. Sparse autoencoders decompose complex neural representations into distinct, human-interpretable directions, helping computational biologists understand the specific biological concepts captured by deep protein models.
The dataset details the concepts that activate each feature, including functional classifications, residue-level activation behaviours, and representative protein families derived from Swiss-Prot activations. It also provides frequency, maximum activation, and inverse document frequency metrics computed across 208 million UniRef90 proteins, enabling downstream analysis and interpretation of ESMC model activations.
How it was made
The table is built from the ESMC-6B-sae-layer60-k64-codebook16384 model, which applies a k-sparse autoencoder with a 16,384-codebook size to layer 60 representations. Concept descriptions were generated using a multi-agent system that evaluated feature activation profiles across Swiss-Prot. This process extracted a biological summary, the residue-level spatial activation pattern, and exemplar protein families for each feature, followed by classification into one of twelve agent-assigned categories.
Quantitative statistics were derived by screening feature activations across UniRef90 and a non-redundant Swiss-Prot subset clustered at 30% sequence identity. The dataset records UniRef90 activation frequencies, inverse document frequency (IDF) values, maximum activation values, lists of the top 100 activating proteins from both databases, reliability thresholds, and the 10 nearest neighbour features within the SAE decoder latent space.
Key facts
- Feature count
- 16,384
- Source model
- ESMC-6B (layer 60)
- SAE architecture
- k-sparse autoencoder (k=64, codebook size 16,384)
- Format
- Parquet tabular data
- Reference databases
- UniRef90 (208 million proteins) and Swiss-Prot (30% identity cluster subset)
- Category count
- 12 functional categories
What you can use it for
- Mechanistic interpretability of protein language models
- Investigate how layer 60 of ESMC-6B internally represents biological motifs, functions, and sequences by mapping latent directions to annotated features.
- Decoding protein language model activations
- Interpret feature activations from downstream biological sequences using precomputed reliability thresholds, summaries, and residue-level activation patterns.
- Feature space neighbourhood analysis
- Explore conceptual and semantic relationships between features using the precomputed 10 nearest neighbours from the SAE decoder latent space.
Explore the data
The schema, real rows, full-text search and a SQL filter — read in your browser, nothing downloaded.
Explore
Read the real rows without downloading anything
Reading rows…
LiveRead from biohub/ESMC-SAE-Features at the moment you asked. Nothing is cached or stored — every row above came from that request.
How the columns are distributed
Over 16,384 rows of default/train
activation_pattern
string_text- min
- 0
- median
- 374
- mean
- 386.62
- max
- 1,233
category
string_label- Structural motif19%
- Ligand-binding site18%
- Domain14%
1.3% empty
decoder_nearest_neighbors
list- min
- 10
- median
- 10
- mean
- 10
- max
- 10
description
string_text- min
- 43
- median
- 1,839
- mean
- 1838.75
- max
- 3,531
exemplar_protein_families
string_text- min
- 0
- median
- 1,032
- mean
- 1035.07
- max
- 2,376
feature_id
int- min
- 0
- median
- 8191.50
- mean
- 8191.50
- max
- 16,383
summary
string_text- min
- 10
- median
- 365
- mean
- 375.22
- max
- 1,198
threshold
float- min
- 2.0e-5
- median
- 0.25
- mean
- 0.30
- max
- 1.22
0.08% empty
top_100_uniref_ids
list- min
- 100
- median
- 100
- mean
- 100
- max
- 100
top_swissprot_activations
list- min
- 20
- median
- 20
- mean
- 20
- max
- 20
uniref90_frequency
int- min
- 1.4M
- median
- 9.4M
- mean
- 26M
- max
- 208M
uniref90_idf
float- min
- 0
- median
- 3.10
- mean
- 2.75
- max
- 5.00
uniref90_max_activation
float- min
- 7.80
- median
- 20.59
- mean
- 20.69
- max
- 50.72
Strengths and limits
Good at
- Combines natural language explanations with residue-level activation patterns and exemplar protein families for each feature.
- Supplies global population statistics and normalisation factors calculated across 208 million UniRef90 proteins.
- Includes empirical activation thresholds indicating where feature descriptions can be reliably trusted.
- Identifies semantic proximity between latent concepts via precomputed decoder nearest neighbours.
Watch out for
- Descriptions generated by the multi-agent system may not be comprehensive and can miss nuances of features with complex behaviour or sparse activations in Swiss-Prot.
- Listed exemplar protein families and functional annotations are derived only from Swiss-Prot activations and are not exhaustive.
- The table is specific to layer 60 of the ESMC-6B architecture and cannot be applied directly to other layers or models.
Questions
Can I look inside ESMC-SAE-Features without downloading it?
Yes. This page reads its schema, real rows and per-column statistics in the browser, with full-text search and a SQL filter. Nothing is downloaded to your machine.
What licence is ESMC-SAE-Features under?
Its authors declare mit. Read the terms on the original page before reusing the data.
Which model was used to generate these features?
The features come from the ESMC-6B-sae-layer60-k64-codebook16384 sparse autoencoder, which was trained on layer 60 of the ESMC-6B protein language model.
How were the feature descriptions generated?
Descriptions were produced by a multi-agent system that analysed feature activation patterns across Swiss-Prot proteins and segmented findings into a summary, an activation pattern, and exemplar protein families.
What categories are assigned to the features?
Features are classified into 1 of 12 categories, which include concepts such as catalytic function, compositional bias, and structural motifs.
What is the threshold column used for?
The threshold column specifies the activation level above which the natural language feature description can be relied upon for accuracy.
How were the top Swiss-Prot activations selected?
The top 100 Swiss-Prot activations were compiled using a non-redundant subset clustered at 30% sequence identity and filtered for high annotation scores.
Written from ESMC-SAE-Features's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.