nf-core pipeline ·
nf-core/meerpipe
Processes MeerKAT pulsar data to generate calibration products, pulse arrival times and timing residuals.
What nf-core/meerpipe is
nf-core/meerpipe is an astronomy pipeline developed as part of the MeerTime project to process pulsar observations captured by the MeerKAT radio telescope. It automates the preparation of calibrated data products, pulse timing measurements, and observational diagnostic figures. The pipeline was originally written by Nick Swainston.
Radio pulsar timing analysis depends on obtaining precise pulse times of arrival, interstellar dispersion measures, and rotation measures from raw observations that are subject to radio-frequency interference and instrumentation effects. The pipeline cleans, calibrates, and decimates raw telescope archives, computing timing solutions and uploading data products to the MeerTime data portal for scientific inspection.
What the pipeline does
The workflow begins by querying the MeerTime database for observations matching user-defined observation time and pulsar filters, then downloads associated metadata, ephemerides, and profile templates. Raw observation archives are combined with initial and final subintegrations removed, cleaned of radio-frequency interference using MeerGuard, calibrated for polarisation using South African Radio Astronomy Observatory Jones matrices in pac, corrected for delays via dlyfix, and adjusted for rotation measure and flux density. For observations with a signal-to-noise ratio exceeding 20, times of arrival are extracted to derive dispersion measures using tempo2 and rotation measures using rmfit; otherwise, dispersion measures are estimated using pdmp.
The calibrated archives then have edge channels excised and are decimated across varied combinations of frequency channels, polarisation states, and time subintegrations. Pulse times of arrival are calculated with pat for decimation settings under 32 channels. Once arrival times are uploaded, tempo2 downloads all current and historical arrivals to generate timing residuals, while psrplot and Python tools such as scintools generate diagnostic plots and dynamic spectra.
Key facts
- Primary developer
- Nick Swainston
- Associated telescope
- MeerKAT
- Project affiliation
- MeerTime project
- Input parameters
- Pulsar name, start UTC, end UTC
- RFI mitigation tool
- MeerGuard
- Calibration tools
- pac, pam, dlyfix, fluxcal_meerkat
- Timing tools
- tempo2, pat, pdmp, rmfit
- Portal integration
- Meertime data portal (pulsars.org.au)
What you can use it for
- Pulsar timing analysis
- Generates pulse times of arrival and multi-epoch timing residuals using tempo2 to track pulsar spin and orbital dynamics.
- Radio-frequency interference excision and calibration
- Removes radio interference with MeerGuard, corrects polarisation using Jones matrices, and applies bootstrap flux calibration to raw MeerKAT archives.
- Interstellar medium measurement
- Measures dispersion measures and rotation measures across observation sessions to evaluate interstellar medium properties along the line of sight.
- Observation quality screening
- Produces dynamic spectra with scintools and diagnostic images for review on the MeerTime data portal.
Run it here
The research agent runs nf-core/meerpipe with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
Strengths and limits
Good at
- Automates the entire processing sequence from raw telescope archives to timing residual generation.
- Adapts dispersion and rotation measure calculations based on whether the observation signal-to-noise ratio meets a threshold of 20.
- Integrates with the MeerTime data portal to publish residuals, times of arrival, and images directly for collaborative review.
Watch out for
- The rotation measure calculation uses rmfit, which has documented fitting inaccuracies.
- Observations with a signal-to-noise ratio below 20 do not generate rotation measures and rely on alternative dispersion calculation methods.
- Requires matching ephemerides and template files from the MeerTime repository to compute dispersion measures and times of arrival.
Questions
Can I run nf-core/meerpipe without my own cluster?
The research agent runs nf-core/meerpipe with Nextflow on its own machine — up to 32 cores, 256 GiB of memory and 24 hours per run, which covers the test profile and real single-sample runs. It builds the samplesheet, checks it against the pipeline's schema and reads the results back. When a run is too big for one machine, it gives you the exact command for your own cluster instead.
What licence is nf-core/meerpipe under?
MIT, like every nf-core pipeline.
What inputs are required to run the workflow?
The pipeline requires an output directory specification along with pulsar identification and target UTC start and end timestamps to filter and download raw observations from the MeerTime database.
How are low signal-to-noise observations handled during dispersion calculation?
When the signal-to-noise ratio is below 20, the pipeline calculates the dispersion measure using pdmp instead of tempo2, and it omits rotation measure calculation entirely.
How is telescope flux calibrated in this pipeline?
Flux calibration uses fluxcal_meerkat via a bootstrap approach, deriving an initial sky temperature from an existing sky map and refining it using high-latitude pulsar observations collected during the session.
Where can the final timing files and diagnostic plots be accessed?
All diagnostic images, summary results files, and pulse times of arrival are uploaded to the MeerTime data portal, where they can be queried using psrdb.
How to cite it
Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen. The nf-core framework for community-curated bioinformatics pipelines. Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.
Written from nf-core/meerpipe's own README. Statements citing a figure that does not appear there are removed before publication. Licence and running details come from our registry.