In defense of the RMSEA-P for latent variable model evaluation: A failed reproducibility study of Lance et al. (2016)

Abstract

Management researchers now have the option of using the path model fit index RMSEA-P as they evaluate their latent variable models to overcome limitations of global fit indices (CFI, RMSEA, SRMR). Two simulation studies of the effectiveness of the RMSEA-P reached different conclusions about its merit, one favorable and the other very unfavorable ( Lance et al., 2016). The present research is a constructive reproduction of Lance et al. with analytical improvements. We failed to reproduce their key negative findings that correctly specified models were retained in less than 50% of cases, and incorrectly specified models were retained in over 50% of cases. Instead, using our improved analysis techniques, we found that correct models were identified in 92% of cases and incorrect models were retained in less than 10% of cases. Our results also show the RMSEA-P works best when used with a Traditional Model Selection Strategy.