Multi-Modal Foundation · MahmoodLab
HEST Tissue Segmentation
Segments tissue from background in H&E and IHC histology images for spatial transcriptomics and slide processing.
What HEST Tissue Segmentation is
HEST Tissue Segmentation is a semantic segmentation model developed by the Mahmood Lab at Harvard Medical School. Created alongside the HEST-1k spatial transcriptomics resource, it automates the delineation of tissue areas from slide background in whole-slide pathology images.
Isolating tissue from glass slide background is a fundamental preparatory step in computational histology. By identifying valid tissue regions across haematoxylin and eosin (H&E) and immunohistochemistry (IHC) preparations, the model enables automated downstream workflows such as patch extraction for spatial transcriptomic and histological analysis.
How it works
The model is based on the DeepLabV3 architecture and has been fine-tuned on the HEST-1k and Acrobat datasets to identify and segment tissue regions across both H&E and IHC stained histology images.
Key facts
- Architecture
- DeepLabV3
- Supported slide preparations
- H&E and IHC
- Fine-tuning datasets
- HEST-1k and Acrobat
- Accompanying publication
- arXiv:2406.16192
What you can use it for
- Tissue Background Separation
- Distinguishing tissue specimens from empty slide background in digital whole-slide images.
- Histology Patch Extraction
- Guiding automated patch extraction tools to sample tiles exclusively from valid tissue areas.
- Spatial Transcriptomics Pre-processing
- Filtering tissue boundaries on H&E images aligned with spatial transcriptomics data in the HEST-1k pipeline.
- Immunohistochemistry Slide Processing
- Segmenting tissue sections on immunohistochemistry (IHC) slides prior to downstream feature analysis.
Run it here
We list this model but cannot run it yet: no pipeline_tag — needs a human.
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Supports both haematoxylin and eosin (H&E) and immunohistochemistry (IHC) stained slides.
- Fine-tuned directly on diverse pathology data from the HEST-1k and Acrobat datasets.
- Employs the established DeepLabV3 semantic segmentation architecture.
Watch out for
- The model identifies tissue versus background and does not classify sub-tissue structures or disease phenotypes.
- Workflow execution and integration rely on the external HEST library repository.
Inputs and outputs
You provide
Sequence or text
A protein or nucleotide sequence, or free text — whichever this model was trained on.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · cc-by-nc-sa-4.0
The cc-by-nc-sa-4.0 licence carries terms worth reading before you depend on it.
Questions
Can I use HEST Tissue Segmentation commercially?
Check the licence before commercial use. The cc-by-nc-sa-4.0 licence carries terms worth reading before you depend on it.
What stains can this model segment?
The model is fine-tuned to segment tissue on both haematoxylin and eosin (H&E) and immunohistochemistry (IHC) images.
What is the primary objective of this model?
It identifies and separates tissue from background in whole-slide histology images to enable downstream patch extraction.
What architecture is used?
The model uses the DeepLabV3 architecture.
Which datasets were used to train the model?
It was fine-tuned on the HEST-1k and Acrobat datasets.
How to cite it
Jaume, Guillaume, Paul Doucet, Andrew H. Song, Ming Y. Lu, Cristina Almagro-Perez, Sophia J. Wagner, Anurag J. Vaidya, Richard J. Chen, Drew F. K. Williamson, Ahrong Kim, and Faisal Mahmood. 'HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis.' arXiv preprint arXiv:2406.16192 (2024).
Written from HEST Tissue Segmentation's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.