CALMs : A Shiny Application for Comprehensive Analysis of Latent Means

Abstract

This article presents a Shiny application CALMs for comprehensively comparing groups via latent means, which includes the examination of group equivalency, propensity score analysis, measurement invariance analysis, and assessment of latent mean differences of equivalent groups with invariant data. Despite the importance of these techniques, their application can be complex and time-consuming, particularly for researchers not experienced in advanced statistical methods. The Shiny application CALMs makes this cumbersome process more accessible to a broader range of users. In addition, it allows researchers to focus more on the interpretation aspect of the research rather than the labor required for testing. The practical utility of the CALMs application is demonstrated using real-world data, highlighting the potential of the application to enhance the validity and reliability of group comparison studies in psychological research.

Introduction

One of the most basic analyses in psychology is the ability to compare groups. However, this seemingly straightforward task is often complicated by two significant challenges: ensuring group equivalence and assessing measurement invariance. If not adequately addressed, these challenges can lead to misleading conclusions and undermine the validity of research findings.

To comprehensively compare groups, analysts should confirm that groups are equivalent on all measures other than what is being compared. If groups are deemed non-equivalent, propensity score analysis can identify a subset of the data that yields equivalent groups. Once equivalent groups have been created, measurement invariance tests should be conducted before comparing groups. If data from the measurement instrument does not demonstrate invariance, any observed differences could reflect differences in interpretation rather than true differences in the variables of interest. If measurement issues yield non-invariance, additional analyses, including the factor ratio test and list-and-delete procedures, can identify subsets of items that can be validly used to compare groups.

We note that propensity score analysis, tests of measurement and structural invariance, and tests of partial invariance are not new to the literature with several articles that speak to these techniques individually but not cohesively. Due to the disparate availability of literature and software, their application can be complex and time-consuming as applied to the analysis of latent means. This can particularly be the case for researchers who are not well-versed in these methods.

Program Description

The CALMs package is a Shiny application designed to facilitate comprehensive analyses of latent means in group comparison studies. Through a user-friendly graphical interface, researchers can upload their own datasets, along with accompanying metadata files, and perform a variety of analyses, including group equivalency checks, propensity score analysis, measurement invariance analysis, and latent means comparisons across invariant, equivalent groups. Additional features include an embedded user manual, a guided video demonstration, interactive dataset exploration, and downloadable files.

Group equivalency and propensity score analyses are implemented using the MatchIt package (Ho et al., 2011), ensuring robust control for covariates. Structural equation modeling and invariance testing rely heavily on the lavaan package (Rosseel, 2012), providing a flexible and powerful modeling framework. The application also integrates modified functions from Hammack-Brown et al. (2022) to support factor ratio testing and the list-and-delete procedure.

Availability, Documentation, and Distribution

The CALMs package was written in the R language (R Core Team, 2025) and is compatible with Windows, Mac, and Linux systems. The source code and documentation are freely available from cran.r-project.org/package=CALMs.

The application can also be accessed at evaluent.shinyapps.io/CALMs/.