Compare · 19 models
Pathology and medical imaging models you can run online
Pathology and medical imaging models analyse histology slides and clinical scans for cancer subtyping, tissue segmentation, multimodal clinical question answering, and spatial molecular profiling.
What pathology and medical imaging models do
Pathological and medical image analysis encompasses diverse computational workflows ranging from tissue-level feature extraction to multi-modality clinical reasoning. In digital pathology, whole-slide histology images must typically be divided into smaller image patches, which are then processed by vision encoders into dense vector representations. Multiple-instance learning architectures subsequently aggregate these distributed patch embeddings to perform slide-level cancer subtyping or biomarker identification without requiring manual pixel-level annotations.
Beyond histological classification, biomedical imaging foundation models address varied analytical tasks across pathology, radiology, dermatology, and ophthalmology. Dual-encoder vision-language models align visual features with text descriptions to enable zero-shot classification and cross-modal retrieval, while multimodal generative language models answer clinical questions using both 2D imagery and volumetric 3D scans. Complementary models perform semantic or prompt-guided image segmentation to separate tissue from slide background or isolate anatomical targets, and specialised adapters infer spatially resolved gene expression directly from histological morphology.
Compared
- HuatuoGPT-Vision 7BFreedomIntelligenceA multimodal medical model that answers clinical questions across radiology, pathology, and dermatology imagery.Check termsOn demand
- ABMIL (CONCH v1.5 features, pc108-24k)MahmoodLabAttention-based MIL classifier for slide-level cancer subtyping using CONCH v1.5 features.See its pageNot yet
- ABMIL (UNI features, pc108-24k)MahmoodLabSlide-level attention-based classifier aggregating UNI patch features across pan-cancer histology images.See its pageNot yet
- ABMIL (UNI2 features, pc108-24k)MahmoodLabAttention-based multiple-instance learning classifier for whole-slide histology inference using UNI2 patch features.See its pageNot yet
- CONCHMahmoodLabA pathology vision-language model for zero-shot classification, retrieval, and analysis on H&E whole-slide images.See its pageNot yet
- CONCH v1.5MahmoodLabA pathology vision-language model for image feature extraction and zero-shot or few-shot tasks.See its pageNot yet
- DeepSpot-MratschlabMultimodal deep-learning model that predicts spatially resolved gene expression directly from H&E whole-slide images — a LoRA adapter over the Midnight pathology foundation model, covering 19k+ genes. Used to generate the TCGA and HEST virtual spatial transcriptomics atlases for computational pathology and oncology research.See its pageNot yet
- HEST Tissue SegmentationMahmoodLabSegments tissue from background in H&E and IHC histology images for spatial transcriptomics and slide processing.See its pageNot yet
- MadeleineMahmoodLabA companion pathology model developed by Mahmood Lab for open histology evaluation pipelines.See its pageNot yet
- MedGemma 1.5 4BgoogleA 4-billion-parameter multimodal model interpreting medical imaging and clinical text for biomedical researchers.See its pageNot yet
- MedGemma 27BgoogleAn instruction-tuned 27B medical AI for clinical question answering, health records, and medical image interpretation.See its pageNot yet
- MedSAMwanglabA promptable medical image segmentation model fine-tuned on CT, MRI, X-ray, ultrasound, and histology data.See its pageNot yet
- MedSigLIPgoogleA dual-encoder vision-language model mapping medical images and text into a shared embedding space for healthcare AI.See its pageNot yet
- Path FoundationgoogleVision transformer developed by Google to extract feature embeddings from histopathology image patches.See its pageNot yet
- RudolfV 2AignosticsA 1.1-billion-parameter pathology foundation model for image feature extraction in digital histopathology.See its pageNot yet
- SEALMahmoodLabA pathology foundation model for whole-slide image representation, cancer classification, and biomarker tasks.See its pageNot yet
- TITANMahmoodLabSlide-level pathology foundation model generating whole-slide embeddings for oncology and biomarker prediction.See its pageNot yet
- UNIMahmoodLabA self-supervised Vision Transformer patch encoder for computational pathology and histology workflows.See its pageNot yet
- UNI2-hMahmoodLabVision transformer generating 1,536-dimensional embeddings from histology patches for pathology analysis.See its pageNot yet
How to choose
For patch-level feature extraction in computational pathology, models differ in parameter scale, pre-training corpus, and stain compatibility. UNI2-h scales up to a 681-million-parameter ViT-H/14 architecture pre-trained on over 200 million tiles across both H&E and immunohistochemistry preparations. RudolfV 2 offers an even larger 1.1-billion-parameter ViT-g architecture that incorporates a dedicated post-training stage, along with a distilled 86-million-parameter ViT-B variant for lower-resource environments. Path Foundation provides Vision Transformer patch embeddings trained specifically on 60 million patches from The Cancer Genome Atlas.
To perform slide-level classification from patch features, choice depends on the upstream encoder used. The ABMIL classifiers are trained on the pc108-24k pan-cancer dataset: ABMIL (UNI features, pc108-24k) requires embeddings from UNI, ABMIL (UNI2 features, pc108-24k) pairs with representations from UNI2-h, and ABMIL (CONCH v1.5 features, pc108-24k) takes patch vectors from CONCH v1.5. Instead of relying on patch extraction and multiple-instance learning pooling stages, TITAN directly generates slide-level representations from whole tissue sections via a self-supervised Vision Transformer, while SEAL also focuses on whole-slide representation for cancer classification and biomarker discovery.
For vision-language alignment without text generation, CONCH aligns H&E histology patches with descriptions using 1.17 million image-caption pairs for zero-shot classification and retrieval, and CONCH v1.5 provides an updated image encoder trained on an expanded corpus. MedSigLIP extends dual-encoder representations across radiology, pathology, dermatology, and ophthalmology with a 64-token text limit. When conversational interaction or clinical text generation is required, HuatuoGPT-Vision 7B provides English and Chinese question answering across medical disciplines using a 7B vision-language backbone trained on PubMedVision, requiring 40 GB of GPU memory. MedGemma 1.5 4B and MedGemma 27B interpret medical text alongside 2D clinical imagery and 3D CT or MRI scans, with MedGemma 27B also handling structured electronic health record data.
For delineation tasks, HEST Tissue Segmentation applies DeepLabV3 to segment tissue from background on H&E and IHC slides. In contrast, MedSAM accepts spatial points and bounding boxes to isolate user-specified structures across CT, MRI, X-ray, ultrasound, and histology scans.
Questions
Can I run these models without a GPU or installing anything?
1 of the 19 models here run on our servers from their page: open one, fill in the inputs and press Run. Those marked "Not yet" are listed with their licence and documentation but have no loader here yet.
What is the difference between the three ABMIL models?
Each ABMIL model uses an attention-based multiple-instance learning pooling architecture trained on the pc108-24k pan-cancer cohort, but they accept different input embeddings. ABMIL (UNI features, pc108-24k) requires pre-computed UNI embeddings, ABMIL (UNI2 features, pc108-24k) requires representations from UNI2-h, and ABMIL (CONCH v1.5 features, pc108-24k) ingests features from CONCH v1.5. None of them take raw whole-slide image pixels directly.
Which models generate natural language answers rather than numerical feature embeddings?
HuatuoGPT-Vision 7B, MedGemma 1.5 4B, and MedGemma 27B are generative multimodal models that produce natural language text in response to clinical and radiological queries. Models such as UNI, UNI2-h, Path Foundation, RudolfV 2, CONCH, and MedSigLIP generate feature embeddings or similarity scores rather than open-ended text completions.
How do MedSAM and HEST Tissue Segmentation differ in scope?
HEST Tissue Segmentation uses the DeepLabV3 semantic segmentation architecture specifically to separate tissue from background across H&E and IHC histology images for downstream patch extraction. MedSAM is a promptable segmentation model built on the Segment Anything Model and ViT-Base, allowing users to guide segmentation targets with coordinates or bounding boxes across CT, MRI, X-ray, ultrasound, and histology images.
What prerequisite model is needed to run DeepSpot-M?
DeepSpot-M is implemented as a Low-Rank Adaptation (LoRA) adapter and requires the Midnight pathology foundation model (kaiko-ai/midnight) as its base model to infer spatially resolved gene expression from H&E whole-slide images.
Written from each model's own documentation, as summarised on its page here. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.