Perspective on the Capacity of the Rashomon Effect in Multivariate Data Analysis

Abstract

Presented is a perspective proposing to expand some fragmented spectroscopic modeling and data analysis practices by incorporating multivariate ideologies. For example, through recognizing the theory of analytic chemistry (TAC) by Booksh and Kowalski, it is common to use the multivariate processes (higher orders) of multiple wavelengths for regression and prediction or classification, fusing multiple instruments, or applying multi-way methods such as parallel factor analysis (PARAFAC). Each wavelength, instrument, PARAFAC order deliver different views of underlying sample-wise full matrix effects adding more information per dimension for improved data characterizations. Reasoned here is that model selection, figures of merit, and sample similarity assessments for model prediction reliability, outlier detection, or classification purposes can meaningfully progress by recognizing the multivariate principles of the TAC and additionally, the importance of the Rashomon effect. Applying the Rashomon effect with the TAC removes conventional fragmented data analysis approaches bringing a more wholeness to data analysis. Included in this discussion is that due to the Rashomon effect, interpretation of spectral models is not reasonable. For an uncommon view of these concepts, the perspective ends with drawing parallels between sample-wise matrix effects and the concepts explicate and implicate orders from physicist David Bohm’s depiction of our physical and conscious world and universe. It is hoped that this perspective tempts reflection in your particular area of spectroscopy.

Graphical abstract

This is a visual representation of the abstract.