nf-core pipeline · 10x-genomics, atera, image-processing
nf-core/spatialaxe
Pipeline for segmentation, transcript assignment and quality control of 10x Genomics Xenium and Atera spatial data.
What nf-core/spatialaxe is
nf-core/spatialaxe is a bioinformatics pipeline designed for the analysis, processing, and quality assessment of high-resolution in situ spatial transcriptomics data. Developed by Sameesh Kher, Dongze He, and Florian Heyl, it primarily processes datasets from the 10x Genomics Xenium platform, with support being extended to the 10x Genomics Atera system.
Spatial transcriptomics platforms detect hundreds to thousands of distinct RNA species directly within tissue sections at subcellular resolution. Turning these optical and coordinate detections into biologically meaningful single-cell data requires multiple non-trivial steps: identifying cellular boundaries through segmentation, assigning transcripts to segmented compartments, and tracking technical artefacts such as non-specific hybridisation.
The pipeline unites established tools for both image-based and coordinate-based cell segmentation, segmentation-free spatial analysis, and quality assurance into reproducible execution workflows.
What the pipeline does
The pipeline ingests sample metadata, morphology images, and platform output bundles to run through configurable analysis modes. For image-guided boundary detection, it couples deep learning cell segmentation methods such as Cellpose or StarDist with Baysor and Xenium Ranger import, before formatting into SpatialData. For coordinate-based approaches, spatial transcript positions are processed using Proseg or Segger to resolve cell units directly.
Alternative operational modes permit segmentation-free spatial profiling with Ficture or Baysor, rapid segmentation previewing, or isolated quality control execution. Technical quality metrics are evaluated using the MultiQC Xenium Extra Plugin alongside the Off-target Probe Tracker (OPT). Resource utilization varies by tool: Segger requires a graphics processing unit (GPU) across all stages, Cellpose offers significant memory and runtime reductions on GPUs compared to CPUs, and StarDist executes efficiently on CPUs.
Key facts
- Pipeline engine
- Nextflow (nf-core)
- Supported platforms
- 10x Genomics Xenium (extension for 10x Genomics Atera in progress)
- Input formats
- CSV samplesheet with paths to Xenium bundles and OME-TIFF morphology images
- Segmentation methods
- Baysor, Cellpose, StarDist, Xenium Ranger
- Transcript assignment methods
- Segger, Proseg
- Segmentation-free methods
- Ficture, Baysor
- Quality control tools
- MultiQC Xenium Extra Plugin, OPT
- Lead developers
- Sameesh Kher, Dongze He, Florian Heyl
- Latest release
- 1.0.1 (2026-08-06)
What you can use it for
- Image-based cell segmentation
- Identifies nuclear and cellular boundaries from morphology OME-TIFF images using Cellpose or StarDist, integrates transcript coordinates via Baysor, and imports results back through Xenium Ranger.
- Coordinate-based cell assignment
- Applies Proseg or Segger to define cell boundaries and assign transcripts directly from transcript coordinate clouds without solely relying on morphology stains.
- Segmentation-free transcript analysis
- Runs Baysor Segfree or Ficture to model spatial expression trends and resolve tissue organisation independently of nuclear or cell boundary detection.
- Quality control and off-target probe assessment
- Assesses run performance and evaluates potential off-target probe hybridisation using the MultiQC Xenium Extra Plugin and OPT without re-running full segmentation.
Run it here
The research agent runs nf-core/spatialaxe with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
Strengths and limits
Good at
- Supports image-based, coordinate-based, and segmentation-free transcriptomics analysis workflows in a single framework.
- Accommodates major specialized spatial algorithms including Baysor, Cellpose, StarDist, Proseg, Segger, and Ficture.
- Produces standard SpatialData data objects for downstream analytical compatibility.
- Validated with automated continuous integration tests using full-sized datasets on AWS cloud infrastructure.
Watch out for
- Conda is not currently supported for software dependency management.
- Segger requires GPU hardware across its dataset creation, training, and prediction stages.
- Pipeline parameters cannot be supplied through custom configuration files passed with the Nextflow -c flag.
- The software is under active development, meaning features and implementation details may change frequently.
Questions
Can I run nf-core/spatialaxe without my own cluster?
The research agent runs nf-core/spatialaxe with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
What licence is nf-core/spatialaxe under?
MIT, like every nf-core pipeline.
What inputs are needed to run the pipeline?
A comma-separated samplesheet specifying the sample identifier, the directory path to the Xenium bundle, and the file path to the corresponding morphology OME-TIFF image.
Is a GPU required to run the pipeline?
A GPU is only strictly required if using Segger, which relies on GPU acceleration for all three steps. Tools like StarDist run efficiently on CPUs, while Cellpose runs on CPU but achieves substantial speedups and lower memory usage on a GPU.
What execution modes are available?
Modes include image-based segmentation, coordinate-based segmentation, segmentation-free analysis, preview mode, and quality-control-only mode.
How should custom parameters be passed to the pipeline?
Parameters must be provided via the command-line interface or a Nextflow parameters file (-params-file); custom configuration files specified via -c cannot be used to declare pipeline parameters.
How to cite it
10.5281/zenodo.20733817
Written from nf-core/spatialaxe's own README. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.