Multi-Modal Foundation · recursionpharma
OpenPhenom
Generates phenomic embeddings from multichannel cell microscopy images for drug discovery and perturbation analysis.
What OpenPhenom is
OpenPhenom is a foundation model designed to extract biologically informative representations from high-content fluorescence microscopy images. Developed by Recursion, it featurises cellular morphology from images acquired through profiling assays such as Cell Painting, supporting downstream biological analysis, drug discovery, and genetic or chemical perturbation studies.
High-content microscopy experiments frequently produce large image volumes across multiple fluorescence channels, making automated phenotypic profiling essential. OpenPhenom addresses this by transforming raw cellular images into low-dimensional vector representations that capture morphological variations and cellular phenotypes.
How it works
The model implements a channel-agnostic masked autoencoder (CA-MAE) using a Vision Transformer small backbone with 16-by-16 patch tokens (ViT-S/16). It applies channelwise cross-attention over patch tokens to build contextualised representations independently for each imaging channel, outputting an embedding vector of dimension 384, or 384 multiplied by the channel count when channel-specific embeddings are requested.
OpenPhenom was trained to reconstruct masked microscopy images across three datasets: RxRx3, JUMP-CP overexpression, and JUMP-CP gene-knockouts. Its channel-agnostic architecture can process variable channel numbers and utilise the full autoencoder framework to predict missing fluorescence stains.
Key facts
- Architecture
- Channel-Agnostic Masked Autoencoder (CA-MAE) with ViT-S/16 backbone
- Training data
- RxRx3, JUMP-CP overexpression, and JUMP-CP gene-knockouts
- Supported modalities
- Multichannel fluorescence microscopy (Cell Painting)
- Output dimension
- 384 features (or 384 per channel)
- Training hardware
- Nvidia H100 Hopper nodes (400 hours)
What you can use it for
- Cellular Phenotyping
- Extract biological embeddings from multichannel Cell Painting microscopy images to profile morphological states.
- Perturbation Profiling
- Featurise high-content screening images to assess phenotypic changes induced by chemical compounds, gene overexpressions, or gene knockouts.
- Stain Imputation
- Employ the complete autoencoder encoder and decoder to predict missing channels or stains in incomplete microscopy panels.
- Encoder Fine-Tuning
- Adapt the vision transformer encoder backbone for downstream tasks such as cellular phenotype classification.
Run it here
No installation and no GPU of your own: fill in the inputs and it runs on our servers. Results stay in your browser unless you sign in.
Runs on On-demand GPU
Runs on our servers with on-demand compute. A first run needs time to load the model; active capacity can be reused and scales down when idle.Run this model
OpenPhenom
Strengths and limits
Good at
- Accommodates arbitrary numbers of imaging channels through a channel-agnostic cross-attention mechanism.
- Extracts both pooled whole-image embeddings and channel-specific contextualised representations.
- Reconstructs or predicts missing fluorescence channels using its integrated decoder architecture.
Watch out for
- Requires batch correction post-processing, such as Typical Variation Normalisation or PCA-CenterScale, for embeddings to capture biological signals effectively.
- Underperforms on small-scale experiments, such as a single assay plate, where supervised bespoke models may be preferable.
- Is unsuited to brightfield microscopy or out-of-domain medical images such as haematoxylin and eosin (H&E) histology.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · See model card
The See model card licence carries terms worth reading before you depend on it.
Questions
Can I use OpenPhenom commercially?
Check the licence before commercial use. The See model card licence carries terms worth reading before you depend on it.
How do I run OpenPhenom without installing anything?
Use the form under “Run it here” on this page: fill in the inputs and press Run model. It runs on our servers on CPU only, typically in 12s, and a run costs about 28 credits. Visitors without an account get 1,000 credits a month.
What input dimensions does OpenPhenom expect?
The model accepts multichannel image tensors, typically evaluated on 256 by 256 pixel images across variable channel counts such as 1, 4, 6, or 11 channels.
How should embeddings be post-processed after inference?
Extracted embeddings should undergo batch correction, ideally using Typical Variation Normalisation or by fitting a PCA kernel on control images and standardising per experimental batch.
Can the model predict missing fluorescence channels?
Yes, the full masked autoencoder architecture, combining the encoder and decoder, can be used to reconstruct and predict absent stains.
Can OpenPhenom be fine-tuned?
The encoder backbone can be fine-tuned for downstream machine learning tasks, such as supervised image classification.
How to cite it
Kraus, O., Kenyon-Dean, K., Saberian, S., Fallah, M., McLean, P., Leung, J., Sharma, V., Khan, A., Balakrishnan, J., Celik, S., et al. (2024). Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11757-11768.
Written from OpenPhenom's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.