Multi-Modal Foundation · MahmoodLab
CONCH
A pathology vision-language model for zero-shot classification, retrieval, and analysis on H&E whole-slide images.
What CONCH is
CONCH is a contrastive vision-language model designed for computational pathology, developed by MahmoodLab. It connects microscopic morphological features with descriptive natural language.
The model is built to handle haematoxylin and eosin (H&E) stained whole-slide histology images, providing a foundation for automated and assisted histological evaluation across medicine and biology.
How it works
CONCH is trained using a contrastive learning objective on 1.17 million histology image–caption pairs. This allows visual features from tissue sections to be aligned directly with pathology text.
Built for image feature extraction, the model interfaces with PyTorch via the timm library to generate representations suitable for multimodal evaluation and retrieval.
Key facts
- Model type
- Contrastive vision-language model
- Training data
- 1.17 million histology image–caption pairs
- Primary imaging modality
- H&E whole-slide images
- Supported library
- timm (PyTorch)
- Primary pipeline task
- Image feature extraction
What you can use it for
- Zero-Shot Histology Classification
- Categorise tissue samples into clinical or morphological classes using text prompts without task-specific fine-tuning.
- Histopathological Image and Text Retrieval
- Search and match relevant microscopic regions and corresponding descriptive text captions across histology collections.
- Report-Grounded Slide Analysis
- Analyse H&E whole-slide images in context with pathology reports and diagnostic descriptions.
- Histology Feature Extraction
- Derive dense visual embeddings from whole-slide images using the timm pipeline for downstream analytical tasks.
Run it here
We list this model but cannot run it yet: excluded (gated): Gated and CC-BY-NC-ND-4.0..
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Trained on 1.17 million domain-specific histology image–caption pairs.
- Enables zero-shot classification without requiring retraining on specific diagnostic categories.
- Natively supports cross-modal retrieval and report-grounded evaluation on H&E stained slides.
Watch out for
- Training and functionality are focused on H&E whole-slide images, limiting out-of-the-box utility on other staining modalities.
- Downstream tasks depend on the alignment established from histology image–caption pairs.
Inputs and outputs
You provide
Image
PNG, JPEG or TIFF. Sent to the model as-is.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · cc-by-nc-nd-4.0
This model is gated by its authors — its licence has to be accepted on the source repository before it will run.
Questions
Can I use CONCH commercially?
Check the licence before commercial use. This model is gated by its authors — its licence has to be accepted on the source repository before it will run.
What training data was used to develop CONCH?
CONCH was trained on a dataset of 1.17 million histology image–caption pairs.
What types of slides does CONCH analyse?
The model is specifically designed for haematoxylin and eosin (H&E) whole-slide histology images.
Can CONCH be used as a standard feature extractor?
Yes, it is implemented with the timm library in PyTorch and configured for image feature extraction.
Written from CONCH's own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.