Drawing Generalizable Conclusions From Multilevel Models: Commentary on Van de Calseyde and Efendić (2022)

Abstract

In a recently published article, Van de Calseyde and Efendić (2022) argue that inner-crowd wisdom (i.e., the reduction in error afforded by aggregating two estimates from a given person relative to a single initial estimate from that person) is enhanced when people are instructed to adopt the perspective of someone with whom they disagree prior to making a second estimate. Here, I present a reanalysis of Van de Calseyde and Efendić’s data and argue that evidence supporting their primary claim spuriously arises from anticonservative multilevel models. Specifically, Van de Calseyde and Efendić assess their data via random-intercept models and fail to account for item-level effects of experimental condition. Such an approach generally allows analysts to reap the enhanced statistical power of multilevel models without implementing appropriate checks on that power; in this case, underestimation of item-level variance appears to have driven an illusory benefit of perspective taking.