nf-core pipeline · corruption, data-cleaning, data-recovery
nf-core/fastqrepair
A Nextflow pipeline to recover corrupted FASTQ.gz files, fix uncompliant reads, and restore read pairing.
What nf-core/fastqrepair is
nf-core/fastqrepair is a bioinformatics pipeline designed and written by Tommaso Mazza to rescue and clean sequencing datasets. It tackles common technical faults found in FASTQ data, including corrupted gzipped files, uncompliant or malformed read structures, disordered entries, and orphaned reads that have lost their pairs.
Corrupted or improperly formatted sequencing files frequently break standard downstream analysis workflows. By systematically recovering readable data from damaged archives, standardising read syntax, and matching paired reads, the pipeline helps researchers salvage usable sequencing reads from otherwise unusable files.
What the pipeline does
The workflow begins by extracting readable sequences from damaged FASTQ.gz files using gzrt. Recovered sequences are then processed with wipertools, which drops or rectifies non-compliant reads to ensure the output conforms strictly to FASTQ formatting rules.
For paired-end experiments, the pipeline runs bbmap/repair.sh to re-synchronise disordered read pairs and discard unpaired singletons. Finally, the recovered and repaired reads are evaluated using FastQC, with overall quality metrics and cleaning action summaries collated into a MultiQC report.
Key facts
- Pipeline engine
- Nextflow
- Lead developer
- Tommaso Mazza
- Pipeline version
- 1.1.1
- Input formats
- FASTQ, FASTQ.gz (single-end or paired-end)
- Output formats
- FASTQ, text reports, HTML
- Key tools
- gzrt, wipertools, bbmap, FastQC, MultiQC
What you can use it for
- Corrupted archive recovery
- Salvaging valid read records from damaged or prematurely truncated FASTQ.gz files using gzrt.
- FASTQ format validation
- Correcting or removing invalid read lines with wipertools so that files adhere to standard FASTQ syntax.
- Paired-end re-synchronisation
- Re-aligning disordered paired reads and stripping out orphaned reads using bbmap repair.sh.
Run it here
The research agent runs nf-core/fastqrepair with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
Strengths and limits
Good at
- Recovers readable sequence information from damaged gzip compressed files.
- Accommodates both single-end and paired-end sequencing datasets.
- Generates both textual summaries of cleaning actions and visual MultiQC quality reports.
Watch out for
- Samplesheet rows containing differing file extensions for paired-end files (such as pairing a compressed and uncompressed file) are not permitted.
Questions
Can I run nf-core/fastqrepair without my own cluster?
The research agent runs nf-core/fastqrepair with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
What licence is nf-core/fastqrepair under?
MIT, like every nf-core pipeline.
What inputs does the pipeline accept?
It accepts single-end and paired-end sequencing reads in FASTQ or FASTQ.gz format, supplied via a comma-separated samplesheet.
Can uncompressed FASTQ files be paired with compressed files?
No, rows in the samplesheet that contain differing file extensions for the paired reads are not allowed.
Which tools perform the recovery and repair steps?
The pipeline applies gzrt for corrupted archive recovery, wipertools to ensure well-formed reads, and bbmap/repair.sh to re-pair disordered reads.
How to cite it
10.5281/zenodo.14796348
Written from nf-core/fastqrepair's own README. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.