nf-core pipeline · artificial-intelligence, dataprocessing, deeplearning
nf-core/deepmodeloptim
Augments biological data towards optimal task-specific training sets for PyTorch deep learning models.
What nf-core/deepmodeloptim is
nf-core/deepmodeloptim, implementing the STIMULUS (Stochastic Testing and Input Manipulation for Unbiased Learning Systems) approach, is a bioinformatics pipeline designed to augment biological datasets into task-specific training sets for machine learning. The workflow addresses the challenge that model architectures often yield comparable results, making the structure and quality of the training data the primary driver of deep learning performance.
Originally written by Mathys Grapotte with contributions from the nf-core community, the pipeline allows researchers to prototype, test, and optimise model training on complex biological data types such as genomics.
What the pipeline does
The pipeline takes tabular biological data, a JSON experiment configuration file passed via --exp_conf, and a user-provided PyTorch model script (.py). Tabular inputs are supplied as CSV files where column headers use a name:type:class naming schema. Columns designated as 'input' are passed directly to the model, 'meta' columns are retained without transformation, and 'label' columns serve as ground-truth targets.
User models must adhere to specific structural requirements: exactly one class name must start with 'Model', the forward method must use parameter names matching the input names defined in the CSV, and the class must provide a batch function that processes input and target dictionaries to handle loss calculation and optimization steps.
Key facts
- Original developer
- Mathys Grapotte
- Model framework
- PyTorch
- Data input format
- CSV with name:type:class column formatting
- Experiment configuration format
- JSON
- Supported data classes
- dna, float (and other implemented encoding classes)
What you can use it for
- Genomic sequence predictive modelling
- Training PyTorch neural networks on genomic DNA sequence inputs mapped to continuous functional targets, such as RNA-seq expression values.
- Data-centric model optimization
- Augmenting and manipulating input biological training sets to find optimal data configurations for specific learning tasks.
- Deep learning model prototyping
- Testing custom PyTorch model architectures against standardized biological input pipelines using structured batch functions.
Run it here
The research agent runs nf-core/deepmodeloptim with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
Strengths and limits
Good at
- Focuses computational effort on biological data augmentation to maximise performance in accordance with neural scaling principles.
- Decouples data handling from model execution through a flexible name:type:class column specification.
- Enables custom PyTorch neural network designs via a standardised batch execution interface.
Watch out for
- Requires custom PyTorch models to adhere strictly to internal interface rules, including specific class naming and argument bindings.
- Restricted to data types and features for which dedicated encoding classes have been implemented.
- Demands an external experiment configuration file supplied in JSON format.
Questions
Can I run nf-core/deepmodeloptim without my own cluster?
The research agent runs nf-core/deepmodeloptim with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
What licence is nf-core/deepmodeloptim under?
MIT, like every nf-core pipeline.
What format does the input data table require?
Data must be provided as a CSV file where headers follow the name:type:class convention. The type must be set as input, meta, or label, and class denotes a supported encoding type such as dna or float.
What requirements are imposed on the PyTorch model file?
The Python file must contain exactly one class whose name starts with 'Model'. The forward pass variables must match the column names defined in the CSV, and it must include a batch function to execute training and evaluation steps.
How is the experiment configured?
Experiment details describing how the data is handled before tuning are supplied in a JSON file provided using the --exp_conf parameter.
How to cite it
Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.
Written from nf-core/deepmodeloptim's own README. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.