Multi-Modal Foundation · ratschlab
DeepSpot-M
Multimodal deep-learning model that predicts spatially resolved gene expression directly from H&E whole-slide images — a LoRA adapter over the Midnight pathology foundation model, covering 19k+ genes. Used to generate the TCGA and HEST virtual spatial transcriptomics atlases for computational pathology and oncology research.
What DeepSpot-M is
DeepSpot-M is a multimodal deep-learning model developed by ratschlab that infers spatially resolved gene expression directly from standard haematoxylin and eosin (H&E) whole-slide histology images. It addresses the practical challenge of obtaining dense spatial molecular information across histological sections without requiring physical spatial sequencing on every sample.
It has been applied in oncology and computational pathology to generate large-scale virtual spatial transcriptomics resources, such as datasets based on TCGA and HEST collections.
How it works
DeepSpot-M is structured as a Low-Rank Adaptation (LoRA) adapter applied to the Midnight pathology foundation model (kaiko-ai/midnight).
Associated training and atlas generation resources include the TCGA virtual spatial transcriptomics atlas and the HEST Xenium virtual spatial transcriptomics dataset.
Key facts
- Base model
- kaiko-ai/midnight
- Architecture
- LoRA adapter on the Midnight pathology foundation model
- Input modality
- H&E whole-slide images
- Library
- deepspotm
What you can use it for
- Virtual spatial transcriptomics
- Predicting spatial gene expression profiles directly from H&E-stained whole-slide tissue sections across thousands of gene targets.
- Oncology tissue characterisation
- Assessing spatial molecular expression in cancer histology images to investigate tumour heterogeneity without performing sequencing.
- Histology atlas construction
- Generating large-scale virtual spatial transcriptomic atlases from historical or archival whole-slide collections such as TCGA.
Run it here
We list this model but cannot run it yet: gated on the Hub — never run past a gate the user did not accept.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Uses parameter-efficient LoRA adaptation on top of the Midnight pathology foundation model.
- Derives spatial transcriptomic information directly from standard H&E whole-slide images.
Watch out for
- Requires the Midnight (kaiko-ai/midnight) base model to run the adapter weights.
- Generates predicted or virtual spatial expression rather than ground-truth molecular sequencing measurements.
Inputs and outputs
You provide
Image
PNG, JPEG or TIFF. Sent to the model as-is.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · cc-by-nc-sa-4.0
This model is gated by its authors — its licence has to be accepted on the source repository before it will run.
Questions
Can I use DeepSpot-M commercially?
Check the licence before commercial use. This model is gated by its authors — its licence has to be accepted on the source repository before it will run.
What inputs does DeepSpot-M require?
DeepSpot-M takes haematoxylin and eosin (H&E) whole-slide histology images as input.
Which base model does DeepSpot-M adapt?
It is built as a LoRA adapter over the Midnight pathology foundation model (kaiko-ai/midnight).
What reference atlases have been created with DeepSpot-M?
The model has been used to produce the TCGA virtual spatial transcriptomics atlas and the HEST Xenium virtual spatial transcriptomics dataset.
Written from DeepSpot-M's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.