Multi-Modal Foundation · MahmoodLab
ABMIL (UNI2 features, pc108-24k)
Attention-based multiple-instance learning classifier for whole-slide histology inference using UNI2 patch features.
What ABMIL (UNI2 features, pc108-24k) is
ABMIL (UNI2 features, pc108-24k) is a slide-level multiple-instance learning classifier developed by MahmoodLab for digital pathology and biomedical research. The model addresses the computational challenge of classifying entire histology slides without requiring manual patch-level annotations across gigapixel images.
The classifier operates downstream of patch extraction, specifically designed to process feature representations extracted by the UNI2-h foundation model. By learning attention weights across individual tissue patches, it aggregates local morphological representations into unified slide-level predictions across pan-cancer datasets.
How it works
The system uses an attention-based multiple-instance learning (ABMIL) architecture. In this paradigm, a whole-slide image is treated as a collection (bag) of patch embeddings (instances). An attention mechanism assigns learned weights to each patch embedding according to its diagnostic relevance before pooling them into a final slide-level representation.
The model is trained on the pc108-24k pan-cancer dataset, specifically taking feature vectors generated by the UNI2-h patch encoder as its input.
Key facts
- Architecture
- Attention-based multiple-instance learning (ABMIL)
- Supported inputs
- UNI2 / UNI2-h patch embeddings
- Training data
- pc108-24k pan-cancer dataset
- Format
- PyTorch (safetensors)
What you can use it for
- Pan-cancer slide classification
- Predicting diagnostic slide-level outcomes across diverse histological tissue types using pre-computed UNI2 embeddings.
- Attention-guided pathology triage
- Identifying salient, morphologically significant tissue regions within whole-slide images via learned attention scores.
- Downstream digital pathology pipelines
- Integrating with the UNI2-h patch encoder to provide end-to-end slide-level inference workflows in biomedical studies.
Run it here
We list this model but cannot run it yet: excluded (gated): Gated and CC-BY-NC-ND-4.0..
Everything on this page still applies. Running it needs a loader for this repository, not a bigger machine.
Strengths and limits
Good at
- Trained on the broad pc108-24k pan-cancer dataset for multi-tissue histology analysis.
- Directly compatible with representations from the UNI2-h patch encoder.
- Applies attention pooling to weight diagnostic patches without requiring pixel-level annotations.
Watch out for
- Cannot process raw histology image pixels directly, requiring prior patch extraction and feature generation with UNI2-h.
- Performance and applicability remain constrained to domains compatible with the pc108-24k training distribution.
Inputs and outputs
You provide
Image
PNG, JPEG or TIFF. Sent to the model as-is.
You get back
Mean-pooled representation, one vector per input.
Licence
Check the licence before commercial use · cc-by-nc-nd-4.0
This model is gated by its authors — its licence has to be accepted on the source repository before it will run.
Questions
Can I use ABMIL (UNI2 features, pc108-24k) commercially?
Check the licence before commercial use. This model is gated by its authors — its licence has to be accepted on the source repository before it will run.
Which patch encoder is required for this model?
The classifier is designed to pair with patch-level feature representations extracted by the UNI2-h encoder.
What training data was used for this checkpoint?
This model was trained on the pc108-24k pan-cancer dataset.
What type of classification architecture does the model employ?
It uses an attention-based multiple-instance learning (ABMIL) framework that pools patch-level embeddings into a slide-level prediction.
How to cite it
arXiv:2506.09022
Written from ABMIL (UNI2 features, pc108-24k)'s own documentation (model card). Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.