Multi-Modal Foundation · Merck
TEDDY
Single-cell foundation models trained on 116 million cells for cross-donor and cross-disease representations.
What TEDDY is
TEDDY (Transformer for Enabling Drug Discovery) is a family of foundation models developed by Merck for single-cell biology, genomics, and drug discovery applications. Trained on approximately 116 million single-cell RNA sequencing (scRNA-seq) profiles from human and mouse tissues, TEDDY addresses the challenge of capturing disease-relevant signals and biological variation across heterogeneous single-cell datasets.
The models are designed to learn robust representations of cell states and gene expression that generalize across biological contexts. By incorporating both masked language modelling and ontology-based supervision, TEDDY enables researchers to represent cells and genes in ways that extrapolate to unseen donors and unseen diseases.
How it works
TEDDY models are transformer-based architectures trained on preprocessed single-cell transcriptomics data. The pipeline normalizes expression counts (to 10,000 per cell) and applies median normalization. Cells are then converted into tokens using rank-based continuous encoding (TEDDY-G) or binned encoding (TEDDY-X), with a maximum sequence length of 2,048 tokens. Token sequences can optionally incorporate biological annotation tokens such as disease, tissue, cell type, and sex.
During pretraining, models are optimized using a combination of masked language modelling on gene expression profiles and ontology classification across metadata annotations. Across the model family, architectures scale from 10 million to 400 million parameters, with the primary release providing rank-encoded TEDDY-G checkpoints at 70M, 160M, and 400M parameters.
Key facts
- Architecture
- Transformer
- Model variants released
- TEDDY-G 70M, TEDDY-G 160M, TEDDY-G 400M
- Parameter count range
- 10M to 400M parameters (family-wide); 70M to 400M (released TEDDY-G checkpoints)
- Training data volume
- ~116 million single cells
- Organisms
- Human and mouse
- Maximum sequence length
- 2048 tokens
What you can use it for
- Cell and gene embedding generation
- Extract dense vector representations of single cells and genes from tokenized transcriptomic profiles for clustering or downstream analysis.
- Cross-donor cell classification
- Classify cellular phenotypes and disease states in datasets containing unseen donors, leveraging invariant biological features learned during pretraining.
- Cross-disease generalization
- Predict biological characteristics and disease associations for conditions not included in the fine-tuning or training datasets.
Run it here
We list this model but cannot run it yet: no pipeline_tag — needs a human.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Trained on an exceptionally large dataset of approximately 116 million single cells spanning diverse tissues, diseases, and species.
- Incorporates ontology classification alongside masked language modelling to ground representations in biological metadata such as cell type, tissue, and disease.
- Demonstrates strong generalisation on held-out donor and held-out disease benchmarks compared to alternative foundation models.
Watch out for
- Input sequences are truncated to a maximum sequence length of 2,048 tokens, limiting the number of expressed genes that can be processed per cell.
- Only TEDDY-G (rank-based) models at 70M, 160M, and 400M parameter sizes are included in the repository release, omitting TEDDY-X variants and smaller configurations.
- Pretraining and fine-tuning require dedicated GPU infrastructure due to model scale and single-cell dataset size.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · apache-2.0
The apache-2.0 licence carries terms worth reading before you depend on it.
Questions
Can I use TEDDY commercially?
Check the licence before commercial use. The apache-2.0 licence carries terms worth reading before you depend on it.
What is the difference between TEDDY-G and TEDDY-X?
TEDDY-G uses rank-based continuous encoding for gene expression values, whereas TEDDY-X uses binned encoding. Only TEDDY-G checkpoints are provided in the current public release.
What input format does TEDDY accept?
The preprocessing pipeline ingests single-cell data in AnnData (.h5ad) format along with corresponding metadata files.
What annotations can be embedded during tokenization?
The tokenization module supports biological ontology annotations including disease, tissue, cell type, and sex mappings.
How to cite it
Alexis Chevalier, Soumya Ghosh, Urvi Awasthi, James Watkins, Julia Bieniewska, Nichita Mitrea, Olga Kotova, Kirill Shkura, Andrew Noble, Michael Steinbaugh, Julien Delile, Christoph Meier, Leonid Zhukov, Iya Khalil, Srayanta Mukherjee, Judith Mueller. TEDDY: A Family Of Foundation Models For Understanding Single Cell Biology. arXiv:2503.03485, 2025.
Written from TEDDY's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.