Instruments and measurements · What you can do here

Build a PLS calibration from spectra

Build partial least squares calibrations from spectral matrices, assess predictive performance with robust cross-validation schemes, inspect VIP profiles, and resolve mixture spectra using MCR-ALS.

Start this in the research agent

What this involves

A spectral dataset pairs an instrument matrix of wavelengths or wavenumbers with response values from a reference method, such as concentration, sugar content, or moisture. Building a calibration requires looking at raw profiles, removing physical interference such as baseline drifts, scattering, and offsets, fitting a latent-variable model, and rigorously demonstrating that it predicts unseen samples.

Doing this properly requires strict control over preprocessing and validation. Preprocessing must match the physical interferents, whether through extended multiplicative scatter correction (EMSC), Whittaker smoothing and derivatives, or asymmetric baseline fitting. Cross-validation must be guarded against data leakage and small-sample variance, and the resulting latent-variable choice must prioritise model parsimony over marginal gains in calibration metrics.

When physical decomposition of spectral mixtures into pure components is required alongside or instead of regression, multivariate curve resolution alternating least squares (MCR-ALS) can resolve pure component spectra and sample contributions using explicit physical constraints.

How it goes

1Visualisation and Axis VerificationData is loaded into NumPy and pandas arrays, and series are plotted using chemometrics.plot_colored_series coloured by the reference response to identify spectral outliers, baseline offsets, and chemical trends. The wavelength or wavenumber axis is verified directly from instrument headers or user specifications rather than assumed.
2Spectral PreprocessingNon-chemical variations are eliminated using chemometrics.Whittaker for baseline-slope removal and derivatives, chemometrics.Emsc for multiplicative scatter correction and background projection, or chemometrics.AsymWhittaker for asymmetric baseline penalty fitting.
3Latent Variable Calibration and Model SelectionA PLS pipeline combining StandardScaler and chemometrics.PLSRegression is evaluated across latent variables with RepeatedKFold or GroupKFold cross-validation via chemometrics.fit_pls. Selection favours the smallest latent variable count within noise of the optimal median Q².
4Diagnostic Evaluation and VIP MappingThe agent inspects leverage, residuals, observed versus predicted values, and variable importance in projection (vip_) scores from the fitted PLSRegression to correlate influential wavelengths with known vibrational overtones or bands.
5Mixture Resolution (Optional)When resolving component profiles, singular value decomposition via chemometrics.plot_svd establishes rank, and chemometrics.mcr.McrAR estimates pure component spectra and fractional concentrations subject to non-negativity and normalisation constraints.

Start it here

Ask in plain language. The agent works on a Linux machine of its own — it installs and runs what the job needs, keeps every file, and reads the results back with the numbers explained. Nothing to set up, and a long run continues after you close the page.

“Build a PLS calibration from spectra. NIR, Raman, FTIR and UV-Vis: baseline, scatter and derivative preprocessing, then PLS with honest cross-validation.”

Open the research agent

What to watch for

The default cross-validation in chemometrics.fit_pls is unshuffled KFold(5), which produces biased metrics if the data file is ordered by response, batch, or date.
Applying Whittaker smoothing or derivatives reduces the total column count (for example, deriv=2 drops two columns), requiring alignment of the wavelength axis before plotting.
Replicate scans or samples belonging to the same preparation batch must be split using GroupKFold; otherwise, cross-validation evaluates sample re-identification rather than generalisable prediction.
Whittaker cannot be cloned inside a scikit-learn cross-validation pipeline due to attribute handling, so it must be applied to the spectral matrix beforehand with a fixed penalty when airtight folds are required.
MCR-ALS solutions suffer from rotational ambiguity and arbitrary component ordering, so resolved profiles cannot be attributed to specific chemical compounds without correlation against authentic pure-component reference spectra.
A wide divergence between calibration R² and cross-validated Q² indicates overfitting, meaning higher latent variables must not be selected purely on calibration fit.

Questions

Do I need to install anything to build a pls calibration from spectra?

No. The research agent does this work on a Linux machine of its own — it installs what the job needs, keeps the files, and reads the results back to you. Long runs continue if you close the page.

Why should I use RepeatedKFold instead of standard 5-fold cross-validation?

On small spectral datasets of a few dozen samples, the apparent optimum for latent variable counts fluctuates significantly between random fold splits. Repeated cross-validation (such as 5 splits across 10 repeats) stabilises median Q² estimates and helps avoid choosing an overfitted latent variable count based on partition noise.

How are variable importance in projection (VIP) scores interpreted?

Variable importance in projection values are extracted directly from the chemometrics.PLSRegression model step, where values greater than 1 typically indicate influential wavelengths. These spectral regions should be checked against known chemical transitions, such as water or C–H overtones, rather than assumed to be predictive without verification.

Can I project out known solvent spectra during scatter correction?

Yes. The chemometrics.Emsc class supports passing interferent spectra, such as pure water or solvent baselines, directly into its background parameter to remove their contributions alongside polynomial baseline terms and scatter correction.

How should predictive calibration performance be reported?

Performance should report RMSECV in the response variable's original measurement units, accompanied by median Q², the specific latent variable count, sample numbers, and preprocessing parameters. Calibration R² alone does not prove the model can predict new samples.

Written from the method the research agent follows for this work and the documentation of the tools listed above. Statements citing a figure that does not appear there are removed before publication. Corollary is a non-commercial research demonstration; results are not for clinical use.