From MIsgivings to MIse-en-scène: The role of invariance in personality science

Abstract

There are increasing vocal concerns about the application of measurement invariance testing arguing that it is overly strict and arbitrary. We argue that invariance is not just a procedural hurdle but a substantive tool that enhances the understanding of psychological constructs across diverse populations and has important implications for both theory testing and theory development. First, we outline the importance of how invariance, in a broad sense, plays a role at all the major steps within a research cycle, involving both theoretical and methodological concerns. Second, we suggest a list of points linked to these invariance concerns that can benefit research reports to improve reliability, validity, and fairness. We see invariance as a crucial part of scientific inquiry and an informative tool for empirical research. We agree with Funder and Gardiner’s point that “Data are data,” but would like to add that invariance inquiries and their implications help making sense of the data and the underlying world.

Plain Language Summary

A number of recent articles have criticized the practice of statistical invariance testing in personality, social, and psychological science. We reflect on the broader questions that underlie these invariance tests and argue they are substantive tools that can help us better understand personality processes across diverse populations. We discuss how the broader questions about the comparability of theory and data are central concerns throughout the full research cycle. These points highlight that an exclusive focus on criterion validity, that is, the associations with third variables, can be misleading. We suggest a list of invariance-related questions for researchers that can improve replicability, robustness, and fairness of psychological, especially cross-cultural, research. We agree that “data are data” and that paying attention to data patterns in a systematic, transparent, and open-minded way (including those arising from invariance tests) can help us make better sense of personality and the world that humans inhabit.