Pathology Benchmark · MahmoodLab
Patho-Bench
A benchmark dataset of standardised splits and labels for evaluating whole-slide image foundation models.
What Patho-Bench is
Patho-Bench provides standardised benchmark splits and labels designed to evaluate patch and slide encoder foundation models on whole-slide images across an array of histopathology tasks. Developed by the Mahmood Lab at Harvard Medical School and Brigham and Women's Hospital, the benchmark addresses the need for systematic evaluation protocols across clinical, morphological, and molecular pathology targets.
The benchmark compiles 95 public evaluation tasks sourced across 33 distinct histology datasets. These tasks span seven core analytical categories: morphological subtyping, tissue microenvironment characterisation, tumour grading, molecular subtyping, mutation prediction, treatment response assessment, and patient survival prediction.
By formalising data partitions and task metadata across diverse cancer types and data repositories, Patho-Bench establishes reproducible testing grounds for computational pathology researchers comparing vision models.
How it was made
Patho-Bench structures evaluation using predefined data splits and metadata files rather than serving raw image files directly. For every supported task, the benchmark provides a YAML file detailing task metadata alongside a TSV file listing sample identifiers, slide identifiers, and associated target labels.
Researchers retrieve the required raw whole-slide images from their respective external repositories and apply the benchmark's YAML and TSV split definitions, either through the dedicated Patho-Bench software library or within custom computational pipelines.
Key facts
- Funding
- NIH NIGMS R35GM138216
- Total tasks
- 95 public tasks
- Total datasets
- 33 datasets
- Task families
- 7
- Data format
- YAML (metadata) and TSV (splits and labels)
- Download size
- 4.2 MB (splits and labels)
What you can use it for
- Genomic Mutation Prediction
- Evaluating foundation models across 34 specific mutation prediction tasks, such as predicting KRAS alterations directly from histology images.
- Tissue Microenvironment Characterisation
- Testing encoders on 16 tasks that predict microenvironmental morphological features, including tumour necrosis, vascular invasion, and immune response.
- Tumour Grading and Subtyping
- Benchmarking classification systems on cellular differentiation, growth patterns, and distinct morphological or molecular disease subtypes.
- Patient Survival and Treatment Assessment
- Measuring model performance when forecasting clinical survival outcomes, risk stratification, and patient therapeutic response.
Explore the data
No queryable preview
This dataset is not available in a queryable form — usually an unusual file format or a repository too large to convert. The original source may still be readable.Strengths and limits
Good at
- Provides uniform, pre-configured partitions across 95 clinical and molecular evaluation tasks.
- Spans 33 independent histopathology datasets covering diverse cancer types and imaging sources.
- Very compact download footprint of approximately 4.2 MB by providing splits and metadata rather than raw slide files.
Watch out for
- Does not supply raw image files, requiring researchers to separately acquire whole-slide images from external repositories.
- Evaluations are constrained to the predefined tasks and splits across the compiled datasets.
Questions
Can I look inside Patho-Bench without downloading it?
Not here: it is not available in a queryable form. Its description and statistics are still shown.
What licence is Patho-Bench under?
Its authors declare cc-by-nc-nd-4.0. Read the terms on the original page before reusing the data.
Does Patho-Bench include the whole-slide images?
No. Patho-Bench only distributes the data splits, slide identifiers, and task labels. Raw image files must be obtained directly from each respective source repository.
What models is Patho-Bench intended to evaluate?
It is designed to evaluate both patch-level and slide-level encoder foundation models trained for whole-slide histology analysis.
What file formats contain the benchmark definitions?
Each task is defined using a YAML metadata file paired with a TSV file specifying sample IDs, slide IDs, and ground truth labels.
How can the benchmark splits be loaded?
The splits can be downloaded per task, per dataset, or in full using the Hugging Face datasets Python library, or accessed via the official Patho-Bench codebase.
How to cite it
Zhang, Andrew, Guillaume Jaume, Anurag Vaidya, Tong Ding, and Faisal Mahmood. Accelerating Data Processing and Benchmarking of AI Models for Pathology. arXiv preprint arXiv:2502.06750 (2025).
Written from Patho-Bench's own dataset card. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.