Local Adaptive Fusion Regression (LAFR) for Local Linear Multivariate Calibration: Application to Large Datasets

Abstract

Impeding linear calibration models from accurately predicting target sample analyte amounts are the target sample-wise deviations in measurement profiles (e.g., spectra) relative to calibration samples. Target sample measurement shifts are due to uncontrollable factors, compositely termed matrix effects, such as temperature, instrument drift, and sample composition divergences relative to analyte and other species amounts altering inter and intramolecular interactions. One approach to circumvent the matrix effect matching problem is to use local modeling where a library with thousands of samples and respective reference analyte values is mined for unique calibration sets matched to each target sample, including analyte amounts between calibration and target samples. Current local modeling methods suffer because it is wrongly assumed similar measurements between calibration and target samples translate to a complete locally matched calibration set. Measurements can be similar, but the underlying matrix effects (and analyte amount) can be drastically different. The presented procedure named local adaptive fusion regression (LAFR) solves this matrix effect matching problem with crucial local modeling paradigm shifts. Expertise with LAFR is unnecessary because input hyperparameters are self-optimized. The capabilities of LAFR to form highly dense localized linear calibration sets matched to target samples spectrally and analyte amounts are verified using a well-studied nonlinear benchmark near-infrared (NIR) meat dataset, a NIR sugarcane dataset covering four major process steps with multiple subgroups within, and a NIR soil database of 98 910 samples spanning the contiguous USA. While LAFR is tested on NIR datasets, it is applicable to other measurement systems affected by matrix effects in a broad sense.

Graphical abstract

This is a visual representation of the abstract.