A novel methodology for determining effectiveness of preprocessing methods in reducing undesired spectral variability in near infrared spectra

Abstract

This study uses a novel analysis methodology based on the Hierarchical Clustering Analysis (HCA) to determine the effectiveness of different preprocessing methods in minimizing undesired spectral variability in near infrared spectroscopy due to both the consecutive and repetitive acquisition of the spectrum and the sample temperature. Nine preprocessing methods and different combinations of them were evaluated in four case studies: reproducibility, repeatability, sample temperature, and combination of the before mentioned cases. Eighty-four spectra acquired on seven different hydrocarbon samples from catalytic conversion processes have been selected as the real case study to illustrate the potential of the mentioned methodology. The approach proposed allows a more detailed discriminatory analysis compared to the classical methods for comparing the between-class and the within-class variances, such as the Wilks’ lambda criterion, and hence constitutes a powerful tool to determine adequate spectral preprocessing strategies. This study also proves the potential of the discrimination analysis methodology as a general scheme to identify atypical behaviors either in the spectrum acquisition or in the measured samples.