When dimensions collide: The risks of unjustified simplification of complex leadership concepts

Abstract

Although prior research has shown that behavioral and nonbehavioral items are not equivalent, researchers have failed to recognize that these items elicit reliance on different memory processes and predict different outcomes. The typical use of multidimensional scales compounds this problem even further when researchers ignore the multidimensional nature of the construct and smash disparate dimensions based on semantic and episodic memory sources into a single composite score, a practice we refer to as DIMSmash. We conduct a literature review of four popular leadership measures to examine the prevalence of DIMSmash, demonstrate how a memory-based approach to measurement suggests that DIMSmash is ill-advised, and conduct a computer simulation to show that the consequences of DIMSmash are not trivial. We found that the practice of DIMSmash is pervasive; 89% of the reviewed articles collapsed multidimensional leadership measures into a single score, even though the consequences of DIMSmash matter. Proceeding with DIMSmash despite the warning signs yields misleading results that obscure relationships with outcomes, which leads to incorrect conclusions. We suggest that the use of a bifactor model can overcome the limitations of DIMSmash.