nf-core pipeline · cell-lines, cross-validation, deep-learning
nf-core/drugresponseeval
Evaluate and benchmark computational drug response prediction models using standardised cross-validation protocols.
What nf-core/drugresponseeval is
nf-core/drugresponseeval, also known as DrEval, is a bioinformatics pipeline for testing and benchmarking drug response prediction models. Created by researchers at the Technical University of Munich and Freie Universität Berlin, it establishes standardised evaluation workflows and automated preprocessing routines to ensure that performance comparisons are reproducible, biologically meaningful, and statistically rigorous.
Assessing machine learning models trained on pharmacological and molecular data often suffers from inconsistent cross-validation strategies, differing data splits, and biased parameter selection. DrEval resolves these challenges by handling hyperparameter tuning and model assessment under unified, fair protocols across diverse architectures ranging from traditional statistical models to deep neural networks.
What the pipeline does
The pipeline begins by loading drug response data and initiating a cross-validation framework across specified models and baselines. For every cross-validation split, it carries out a grid search to discover the optimal hyperparameters for each model. The model is then retrained on the complete training split (combining training and validation data) using the selected parameters to predict responses on the unseen test set.
If enabled, the pipeline executes additional checks to evaluate biological signal and prediction stability. In randomization tests, models trained on the training partition predict against randomized test data. In robustness tests, models are retrained multiple times on the training partition with their chosen hyperparameters. Baseline models bypass these checks. Finally, DrEval compiles the performance metrics, plots summaries, and visualises model rank comparisons via critical difference diagrams evaluated with the Nemenyi test.
Key facts
- Pipeline name
- nf-core/drugresponseeval (DrEval)
- Latest release
- 1.2.2 (2026-06-23)
- Original authors
- Judith Bernett (TUM) and Pascal Iversen (FU Berlin)
- Python package
- drevalpy (available via PyPI)
- Model support
- Statistical baselines to complex neural networks
- Hyperparameter tuning
- Grid search per cross-validation split
- Statistical testing
- Nemenyi test with critical difference diagrams
What you can use it for
- Benchmarking novel prediction models
- Compare newly developed neural networks or machine learning models against established baselines using identical cross-validation splits and preprocessing workflows.
- Sanity checking predictions via randomization
- Determine whether predictive accuracy reflects genuine biological signals rather than dataset artefacts by testing trained models against randomised test data.
- Testing model robustness
- Quantify variance and stability by repeatedly training models on the full training partition with fixed optimal hyperparameters.
- Automated hyperparameter optimization
- Conduct unbiased grid searches across cross-validation folds to systematically identify top-performing parameter sets for each model.
Run it here
The research agent runs nf-core/drugresponseeval with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
Strengths and limits
Good at
- Automates standardised data preprocessing and fair cross-validation splits across multiple models.
- Accommodates diverse modeling approaches, from baseline statistical algorithms to complex deep learning architectures.
- Includes dedicated randomization and repeated-training robustness tests to assess model validity.
- Visualises statistically significant differences between models using Nemenyi tests and critical difference diagrams.
Watch out for
- Baseline models cannot undergo randomization or robustness testing within the pipeline.
- Automated hyperparameter optimization is restricted to grid search strategies.
Questions
Can I run nf-core/drugresponseeval without my own cluster?
The research agent runs nf-core/drugresponseeval with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
What licence is nf-core/drugresponseeval under?
MIT, like every nf-core pipeline.
What types of models can be evaluated?
DrEval offers a flexible model interface that supports any model type, spanning simple statistical algorithms and baselines to complex deep neural networks.
How are model hyperparameters tuned?
For each cross-validation split, the pipeline runs a grid search across predefined hyperparameter configurations to select the best setup prior to final training on that split.
What is drevalpy?
drevalpy is the underlying standalone Python package on PyPI that implements the DrEval evaluation framework used by the Nextflow pipeline.
How does DrEval establish whether one model outperforms another?
It computes average model ranks across datasets and evaluation metrics, assessing statistically significant rank differences using the Nemenyi test and visualising them on critical difference diagrams.
How to cite it
Bernett, J., Iversen, P., Picciani, M., Wilhelm, M., Baum, K., & List, M. Critical Evaluation of Drug Response Prediction Models with DrEval. Nature Communications 17, 4238 (2026). DOI: 10.1038/s41467-026-72903-w
Written from nf-core/drugresponseeval's own README. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.