Multi-Modal Foundation · tahoebio
Tahoe-x1
Perturbation-trained transformer foundation models for single-cell cancer research and drug discovery.
What Tahoe-x1 is
Tahoe-x1 is a family of perturbation-trained single-cell foundation models developed by Tahoe Therapeutics. Built to address computational bottlenecks and biological complexity in oncology and drug discovery, the family spans model sizes from 70 million up to 3 billion parameters. The models are designed to capture transcriptional responses and biological variations across massive transcriptomic datasets.
The models are pretrained on 266 million single-cell transcriptomic profiles gathered from CellxGene, scBaseCamp, and the Tahoe-100M perturbation compendium. By integrating large-scale perturbation biology into pretraining, Tahoe-x1 enables researchers to generate expressive cell and gene representations, infer perturbation outcomes, and assess gene functions in cancer contexts.
In cancer-relevant benchmarks, Tahoe-x1 operates with 3- to 30-fold greater compute efficiency compared to existing cell-state models. The architecture supports standard single-cell analysis pipelines using AnnData structures, facilitating downstream integration for classification, pathway recovery, and clustering tasks.
How it works
Tahoe-x1 is based on a transformer encoder architecture trained using a masked language modelling (MLM) objective over single-cell RNA profiles. It incorporates modern efficiency techniques including FlashAttention, fully sharded data parallel (FSDP) execution, and mixed-precision arithmetic implemented through MosaicML Composer.
During pretraining, the model processes 266 million single-cell profiles pooled across three main repositories: the Tahoe-100M perturbation dataset, CellxGene, and scBaseCamp. Input single-cell count matrices are mapped into tokenised vocabularies to generate cell-level embeddings stored in `obsm` and gene-level embeddings stored in `varm`.
Key facts
- Base architecture
- Transformer encoder pretrained with masked language modelling (MLM)
- Model sizes
- Tx1-70M (~70M parameters), Tx1-1B (~1.3B parameters), Tx1-3B (~3B parameters)
- Training data volume
- 266 million single-cell transcriptomic profiles
- Primary training sources
- Tahoe-100M, CellxGene, scBaseCamp
- Training framework
- MosaicML Composer with FlashAttention and FSDP
- Supported formats
- PyTorch, Safetensors, AnnData (.h5ad)
What you can use it for
- Predicting cancer gene dependencies
- Apply learned representations to predict gene essentiality in cancer cell lines based on DepMap benchmark tasks.
- Transcriptional perturbation prediction
- Model and forecast single-cell transcriptomic shifts and cellular responses following experimental perturbations.
- Biological pathway and hallmark recovery
- Extract high-dimensional gene embeddings to identify pathway memberships and enrichments across MSigDB Hallmarks.
- Cross-tissue cell-type classification
- Derive dense cell embeddings from transcriptomic counts to classify cell identities across diverse tissue environments.
- Single-cell clustering and visualisation
- Generate low-dimensional cell representations directly saved into AnnData objects for UMAP visualisation and unsupervised clustering.
Run it here
We list this model but cannot run it yet: no pipeline_tag — needs a human.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Operates with 3- to 30-fold higher compute efficiency than other cell-state models on cancer-focused tasks.
- Pretrained on a diverse corpus of 266 million single-cell profiles, including the Tahoe-100M perturbation compendium.
- Scales across three distinct sizes (70M, 1.3B, and 3B parameters) to balance computational resources with capacity.
- Integrates directly with Scanpy workflows by extracting embeddings into AnnData obsm and varm matrices.
Watch out for
- Full-scale execution of the largest 3B parameter model and batch processing may require substantial GPU memory and distributed setups.
- Extraction of high-dimensional gene embeddings across very large single-cell datasets requires lower-level batching APIs to manage memory.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · apache-2.0
The apache-2.0 licence carries terms worth reading before you depend on it.
Questions
Can I use Tahoe-x1 commercially?
Check the licence before commercial use. The apache-2.0 licence carries terms worth reading before you depend on it.
What model variants are available in the Tahoe-x1 family?
Tahoe-x1 comes in three parameter configurations: Tx1-70M (~70 million parameters), Tx1-1B (~1.3 billion parameters), and Tx1-3B (~3 billion parameters).
What training data was used for Tahoe-x1?
The models were pretrained on 266 million single-cell transcriptomic profiles compiled from CellxGene, scBaseCamp, and the Tahoe-100M perturbation compendium.
How do I extract cell and gene embeddings into Scanpy?
Using the tahoex library, models can be loaded via `ComposerTX.from_hf()` and evaluated on an AnnData object. Extracted cell embeddings are written to `adata.obsm`, while gene embeddings can be saved to `adata.varm`.
Can Tahoe-x1 be fine-tuned on custom single-cell data?
Yes, models can be fine-tuned from pretrained checkpoints or trained from scratch using MosaicML Composer configuration scripts.
How to cite it
Gandhi, Shreshth and Javadi, Farnoosh and Svensson, Valentine and Khan, Umair and Jones, Matthew G. and Yu, Johnny and Merico, Daniele and Goodarzi, Hani and Alidoust, Nima. Tahoe-x1: Scaling Perturbation-Trained Single-Cell Foundation Models to 3 Billion Parameters. bioRxiv (2025). DOI: 10.1101/2025.10.23.683759
Written from Tahoe-x1's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.