Abstract
Whether it stems from participant attrition, nonresponse, unwillingness to disclose information, technical errors, or flawed collection methods, incomplete data pose significant challenges to researchers in psychology. Although a rich methodological literature exists, applied researchers often lack clear guidance for aligning missing-data methods with study design, assumptions, and analytic goals. In this article, I provide a practical, assumption-aware framework for reasoning about missing data in psychology, emphasizing how missingness operates as a selection process and how method choice depends on the underlying data-generating structure. I review commonly used approaches, including likelihood-based estimation, multiple imputation, Bayesian data augmentation, and pattern-mixture models, highlighting their assumptions, strengths, and limitations. To support implementation and pedagogy, I introduce DataPatch, an interactive tool that allows users to simulate missing-data mechanisms, apply alternative handling strategies, and examine their consequences for estimation and interpretation (davidmoreau.shinyapps.io/DataPatch/). Together, the conceptual framework and accompanying tool aim to promote more transparent, principled, and informed handling of missing data in psychological research.
Keywords
Missing data is a ubiquitous and pervasive issue in psychological research. This phenomenon is not merely a technical inconvenience; it poses a profound challenge, affecting the validity, reliability, and interpretability of research findings (McKnight, 2007; Pampaka et al., 2016). Over the past few decades, the handling of missing data has been a subject of significant concern (Graham et al., 2003) because improper approaches can lead to biased results, reduced statistical power, and ultimately, conclusions that may be misleading or erroneous (Donders et al., 2006; K. J. Lee et al., 2021; R. J. A. Little & Rubin, 2019; Schafer & Graham, 2002).
Incomplete data sets can arise from a multitude of sources: Participants may choose not to respond to certain questions (Mirzaei et al., 2022), drop out of longitudinal studies (Nooraee et al., 2018), or be unavailable for follow-up (Twisk & de Vente, 2002). Missing data can also stem from the juxtaposition of information collected at different times or in different contexts, for example, in the case of demographics or population-level data assembled by governments and organizations (Von Hippel, 2007). Furthermore, technical issues, such as data-entry errors or loss of data, can also contribute to this problem (R. J. A. Little & Rubin, 2019; Troyanskaya et al., 2001). The occurrence of missing data is arguably not a question of if but when, making it a ubiquitous challenge in psychological research (Golino & Gomes, 2016; Gomila & Clark, 2022; Newman, 2014).
The implications of such missingness are wide-ranging. At a fundamental level, missing data can compromise the integrity of statistical conclusions—when portions of data are missing, researchers may inadvertently draw inferences based on a nonrepresentative sample of the population, thereby skewing results and potentially leading to invalid conclusions (Schafer, 1999; Schafer & Graham, 2002). In the context of psychological research, in which studies often seek to understand complex human behaviors, emotions, and cognitive processes, the impact of missing data can be particularly pronounced (Baraldi & Enders, 2010). Psychological constructs are typically nuanced and multifaceted (Carver, 1989; F. F. Chen et al., 2012), making accurate and complete data collection paramount. Furthermore, the ubiquity of self-reported measures (Baumeister et al., 2007; Paulhus & Vazire, 2007; Schwarz, 1999), which are especially susceptible to nonresponse (Chan, 2010; Fox-Wasylyshyn & El-Masri, 2005), highlights the critical nature of effective handling of missing data. In psychology, this issue is further exacerbated given the prevalence of longitudinal designs (Nicholson et al., 2017; Troxel et al., 1998), which naturally introduce multiple opportunities for data attrition because of factors such as participant dropout, reduced engagement, or personal circumstances that interrupt study participation (Enders, 2011a; Jelicˇić et al., 2009; T. D. Little et al., 2014). The challenge, therefore, is twofold: Researchers must not only grapple with the technical aspects of managing incomplete data but also consider the implications of missingness on the interpretation of their findings.
Despite the prevalence of missing data, however, the literature lacks recent, practical guidance for researchers in psychology on how to best handle such situations (but for a worked example in developmental psychology, see Woods et al., 2023). Existing literature tends to be fragmented, often buried in specialized journals that may not be readily accessible or easily interpretable. In the present article, I seek to bridge this gap by distilling and synthesizing the wealth of information available on missing-data techniques, including recent advances, making it accessible and applicable to the broad psychological-research community.
More specifically, the aim here is threefold. First, I seek to inform psychology researchers about the implications of missing data on the validity of their research. Second, I provide an overview of current methods for handling missing data, illustrated with causal diagrams to clarify how missingness arises, when bias is introduced, and how method choice depends on the underlying structure of the study. In addition to providing a theoretical and practical overview of methods for handling missing data, I also highlight common pitfalls and misconceptions in this area, for example, about the nature and impact of different types of missing data and the capabilities and limitations of various methods for dealing with missing data (for an overview of relevant terms, see Table 1). Finally, to offer pragmatic support and actionable guidance, the article is accompanied by a Shiny app that allows implementing these methods with one’s own data and simulating their impact for robustness or pedagogical purposes.
Glossary of Terms
Understanding Missingness
Before selecting a strategy for handling missing data, researchers must consider how and why data are missing (Peeters et al., 2015). The most widely used framework for characterizing missingness distinguishes among missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR; Rubin, 1976). These categories remain a useful starting point because they clarify the assumptions under which common analytic methods yield valid inference (Enders, 2010; R. J. A. Little & Rubin, 2019; for an illustration, see Fig. 1). However, they are best understood as idealized reference cases rather than exhaustive or mutually exclusive descriptions of real-world missingness processes (Rubin, 1996).
Example representations of data that are missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). In the MCAR scenario, missingness is entirely random, so missing values are scattered uniformly across the data. In the case of MAR, missingness in the outcome depends on the observed values of the predictor; here, higher values for the predictor are more prone to missingness. For MNAR, missingness depends on the unobserved values of the outcome; here, higher values for the outcome are missing more frequently. Note that these are only illustrative depictions of MCAR, MAR, and MNAR and that the scenarios are not exhaustive of all possibilities for these data types.
In practice, missing data in psychological research rarely arise from a single, clean mechanism. Instead, missingness often reflects a combination of design features, participant behavior, and unobserved processes (Graham, 2009). As a result, diagnosing missingness solely by assigning a data set to one of the three canonical categories can be misleading. Recent methodological work has emphasized this point by examining finer distinctions within MAR (e.g., linear vs. nonlinear forms; Enders, 2022; Little & Rubin, 2019), subtypes of MNAR (Gomer & Yuan, 2021), and strategies for handling mixtures of mechanisms (Gomer & Yuan, 2023).
From this perspective, MCAR represents a rare and fragile condition in which missingness is unrelated to both observed and unobserved data. Although MCAR implies unbiased estimation under many standard analyses, it is seldom plausible in applied psychological studies and should not be assumed by default. MAR, which allows missingness to depend on observed variables, underlies many modern methods, such as likelihood-based estimation, multiple imputation (MI), and Bayesian data augmentation. However, MAR is fundamentally an assumption about the adequacy of the observed data: If important predictors of missingness are unmeasured or omitted from the model, bias may persist even when MAR is assumed (R. J. A. Little, 1998; Molenberghs et al., 2014).
MNAR encompasses situations in which missingness depends on unobserved values, latent traits, or other unmeasured processes. Addressing MNAR typically requires explicit modeling of the missingness process (e.g., selection or pattern-mixture models [PMMs]; R. J. A. Little, 2008) and careful sensitivity analyses because conclusions necessarily depend on untestable assumptions (Daniels & Hogan, 2008; Robins et al., 2000; Steck, 2010). Importantly, recent work suggests that MNAR should not be treated as a single condition but as a family of mechanisms with distinct implications for identifiability and inference.
Rather than treating MCAR, MAR, and MNAR as labels to be diagnosed conclusively, a more productive approach is to view them as anchors for structured reasoning about missingness. This involves combining substantive theory, study-design considerations, and diagnostic exploration to assess which assumptions are most plausible and which estimands remain recoverable under those assumptions. This concern is especially salient in psychological research, in which construct validity depends on accurately capturing complex patterns of behavior, cognition, and experience, and in which missingness may itself be systematically related to the constructs under study (Moreau & Wiebels, 2022; Smith, 2005; Strauss & Smith, 2009). As developed in subsequent sections, causal representations and interactive tools can support this process by making missingness assumptions explicit, clarifying selection effects, and highlighting cases in which standard methods may succeed or fail (Wiebels & Moreau, 2023).
A Causal Perspective on Missing Data
Discussions of missing data often focus on statistical mechanisms (e.g., MCAR, MAR, and MNAR) without explicit reference to the underlying causal structure of the study. However, whether missing data lead to biased estimates depends not only on the missingness mechanism but also on the estimand of interest and the causal relationships among variables. As several authors have argued, missing data are fundamentally a problem of causal inference rather than purely statistical adjustment (Hernán & Robins, 2020; Mohan & Pearl, 2021).
Causal diagrams, typically represented as directed acyclic graphs (DAGs), provide a useful framework for making these relationships explicit (for a primer, see Rohrer, 2018). DAGs encode assumptions about how variables influence one another, including the processes that generate missingness, and thereby clarify when particular analytic strategies are likely to yield valid inferences. Crucially, they highlight that the same pattern of missing data may be largely inconsequential in one study design but highly problematic in another.
Missingness as a selection process
From a causal perspective, missing data can be understood as a form of selection. Let R denote an indicator of whether a variable is observed. Bias arises when conditioning on R = 1 (i.e., restricting analyses to observed cases) opens or closes causal paths in a way that distorts the relationship between variables of interest. DAGs make these selection effects transparent by explicitly representing the variables that influence missingness and the consequences of conditioning on observation.
Figure 2 summarizes several common causal structures through which missing data arise. In Figure 2a, missingness in the outcome depends on an observed covariate Z but not on the outcome itself. Under this structure, conditioning on Z (typically through likelihood-based estimation or MI) can block spurious paths between missingness and the outcome, rendering the effect of X on Y recoverable under a MAR assumption. In contrast, Figure 2b illustrates outcome-dependent missingness, in which the probability that the outcome is observed depends directly on its (unobserved) value. Conditioning on observed cases induces selection bias because the analyzed sample systematically underrepresents certain outcome values. In this setting, standard MAR-based approaches may fail regardless of the richness of the imputation or analysis model.
Causal structures underlying common missing-data scenarios. (a) Missing outcome depending on an observed covariate (missing at random [MAR]). Outcome missingness (Ry) depends on an observed variable (Z) but not on the outcome itself. Under this structure, conditioning on Z (e.g., via likelihood-based estimation or multiple imputation) can yield unbiased estimates of the effect of X on Y, assuming correct model specification. (b) Outcome-dependent missingness (missing not at random). The probability that the outcome is observed depends directly on the outcome. Conditioning on observed cases induces selection bias such that MAR-based methods may fail even when imputation models are otherwise well specified and even after conditioning on observed predictors such as X. (c) Missingness as a collider. An unobserved variable (U ) affects both the outcome and its probability of being observed. Conditioning on Ry = 1 (a descendant of U ) opens a noncausal path between X and Y, inducing bias even though missingness does not depend directly on the predictor. (d) Missing confounder in an observational study. A latent or partially observed confounder (Z) influences both exposure (X ) and outcome (Y ). When missingness in Z depends on Z itself, conditioning on observed values (Rz = 1) restricts analysis to a selected subset, leaving the confounding path unblocked and potentially biasing causal estimates. Dashed arrows denote causal influences on the missingness process, shaded nodes indicate latent or unobserved variables, and boxed nodes represent selection indicators on which analyses condition when using observed data. Conditioning on a selection indicator (R = 1) corresponds to analyses restricted to observed cases (e.g., complete-case analysis).
Figure 2c highlights a subtler but equally important source of bias. Here, an unobserved variable influences both the outcome and the probability of observation. Conditioning on Ry = 1 induces collider bias, opening a noncausal path between the predictor and the outcome even when missingness does not depend directly on either. This structure helps explain why complete-case analysis can be biased in practice even when missingness appears unrelated to key study variables.
Finally, Figure 2d illustrates the case of a missing confounder in an observational study. A latent or partially observed variable influences both the exposure and the outcome, creating confounding. When this confounder is missing for some units (and missingness depends on the confounder itself), conditioning on observed values restricts the analysis to a selected subset of the population. Even when missingness does not depend on the exposure or outcome, standard missing-data methods may fail to recover the causal effect of interest because the confounding path remains unblocked.
Clarifying bias, estimands, and study design
A central advantage of a causal approach is that it forces researchers to articulate the estimand—the quantity they wish to estimate—before selecting a method for handling missing data. Estimating an average treatment effect in an experimental design raises different concerns than estimating a regression coefficient in an observational study subject to confounding even when the observed missingness patterns appear similar.
Without an explicit causal model, claims that a method is “biased” or “unbiased” are difficult to interpret. Bias is always defined relative to an estimand and a set of assumptions. DAGs provide a formal language for stating these assumptions and for assessing whether a given strategy, such as listwise deletion, MI, or full-information maximum likelihood (FIML), targets the intended estimand under plausible conditions (Pepinsky, 2018; Schlomer et al., 2010).
This perspective clarifies why missing-data methods cannot be meaningfully evaluated independently of study design. In randomized experiments, missing outcomes may primarily threaten precision, whereas in observational studies, they may exacerbate confounding or selection bias. Likewise, missing covariates pose fundamentally different challenges from missing outcomes even when the proportion of missing data is identical.
Missing data affect not only bias in parameter estimates but also model evaluation and model selection because common fit and comparison indices depend on quantities that change when data are incomplete and when different missing-data-handling strategies are used. In the context of structural equation models (SEMs), Davey et al. (2005) showed that missing data can substantially reduce the power to detect model misspecification and that the behavior of widely used fit indices depends on both the missing-data mechanism (e.g., MCAR vs. MAR) and the location of misfit (e.g., measurement vs. structural components).
Recent simulation work by Heo et al. (2024) extended this concern to Bayesian longitudinal models, demonstrating that increasing amounts of missing data and model misspecification degrade the performance of Bayesian model-fit and -selection procedures, including posterior predictive checks and information criteria, even when missingness is handled via Bayesian data augmentation under ignorable assumptions. Together, these results underscore that missing-data decisions have consequences not only for bias reduction but also for hypothesis testing, model-fit conclusions, and model comparison, reinforcing the need to align missing-data methods with study design, estimands, and inferential goals.
Implications for method choice
Viewing missing data through a causal lens has direct implications for method selection. It underscores the importance of including appropriate covariates, particularly confounders and predictors of missingness, in imputation or likelihood models, as shown in Figure 2a. At the same time, it highlights situations in which standard MAR-based approaches are insufficient, such as outcome-dependent missingness (Fig. 2b), collider-induced selection (Fig. 2c), or missing confounders (Fig. 2d). In these cases, sensitivity analyses or explicit MNAR models are not optional refinements but necessary components of principled inference (Yang et al., 2019).
Finally, causal diagrams can also inform the design phase of a study. Because they help anticipate which variables are likely to influence both missingness and outcomes, causal diagrams can help researchers collect auxiliary information, mitigate selection effects, or implement planned missing-data designs that preserve identifiability by construction. In this sense, DAGs serve not only as analytic tools but also as guides for prevention and transparent reporting.
Strategies for Prevention
Although much of the missing-data literature focuses on statistical remedies after data loss has occurred, an equally important and often underemphasized approach involves preventing missingness through thoughtful study design and data-collection practices. From a causal perspective, prevention strategies are not merely pragmatic conveniences; they shape the missingness mechanism itself and can determine whether valid inference is possible (Kang, 2013). This is particularly relevant for longitudinal and large-scale psychological studies, in which participant burden, attrition, and logistical constraints frequently contribute to systematic data loss (Jelicˇić et al., 2009; T. D. Little et al., 2014).
One of the most effective preventive approaches is the planned-missing-data design, sometimes referred to as “matrix sampling” or a “three-form design.” Rather than treating missingness as an unanticipated flaw to be corrected post hoc, planned missingness deliberately introduces structured patterns of missing data to reduce participant burden, optimize resource allocation, and maintain statistical efficiency (Graham et al., 2006; T. D. Little & Rhemtulla, 2013). Critically, when implemented correctly, these designs enforce a missingness mechanism that is completely at random by design, corresponding to the causal structure illustrated in Figure 2a.
Planned-missing-data designs operate by randomly assigning participants to subsets of items or measures such that no individual completes the full battery but each item is observed for a sufficient proportion of the sample. Because assignment to missingness is randomized, the probability that a variable is observed is independent of its value and of other study variables, satisfying the assumptions required for valid inference under likelihood-based estimation or MI (Enders, 2011b, 2011; Rhemtulla & Little, 2012). From a causal standpoint, these designs avoid the selection mechanisms illustrated in Figures 2b and 2c and reduce the risk of missing confounders as depicted in Figure 2d.
Planned missingness is particularly advantageous in longitudinal studies, in which repeated measurement increases participant fatigue and dropout risk over time. By reducing the number of items administered at each wave, researchers can mitigate burden while preserving the integrity of the measurement model and the causal estimand of interest (Wu et al., 2016). When combined with auxiliary variables and appropriate analytic strategies, planned-missing-data designs can yield unbiased parameter estimates and in some cases, greater statistical efficiency than traditional complete-case designs (Enders, 2010; Graham et al., 2006).
Implementing a planned-missing-data design requires careful upfront planning. Missingness must be introduced randomly, each variable must be observed frequently enough to support the intended analyses, and the resulting missingness patterns must align with the assumptions of the chosen estimation method. Poorly implemented designs—for example, those that systematically omit key covariates or confounders—can undermine identifiability despite appearing statistically efficient. When feasible, preregistering the planned-missingness structure and analytic approach can enhance transparency and guard against post hoc rationalization.
Together, these preventive strategies illustrate that missing data is not solely an analytic problem to be solved after data collection but a design choice with direct implications for causal validity (Dziura et al., 2013; Enders, 2013). Via active shaping of the missingness mechanism itself, researchers can reduce reliance on strong, untestable assumptions and improve both the quality and interpretability of their findings.
Choosing a Method Based on Design and Goal
When preventive strategies are insufficient or missingness remains despite best efforts, researchers rely on statistical methods to address incomplete data and minimize bias. Broadly, approaches to missing data can be grouped into deletion methods, imputation-based methods, and likelihood- or model-based approaches (Carpenter & Kenward, 2013; Kenward & Carpenter, 2007; Kenward & Molenberghs, 1998; Perkins et al., 2018; Silva & Zárate, 2014). Although often discussed as interchangeable technical options, these methods differ fundamentally in the assumptions they require and in the estimands they target. Consequently, method choice should be guided by study design, the causal structure generating missingness, and the substantive research goal rather than by convenience or convention. 1
Method choice as a function of causal structure
From a causal perspective, missing-data methods are not neutral tools but strategies whose validity depends on how missingness arises and how variables relate to one another. DAGs provide a principled way to formalize these assumptions and to assess whether a given method identifies the intended estimand under plausible conditions. When missingness depends only on observed variables (Fig. 2a), conditioning on those variables (via regression adjustment, likelihood-based estimation, or MI) can yield unbiased estimates under MAR assumptions. In such settings, methods that retain incomplete cases, such as FIML or MI, are typically preferable to deletion approaches because they preserve efficiency while respecting the assumed causal structure (Patrician, 2002; Pedersen et al., 2017; Pigott, 2001; Raaijmakers, 1999).
In contrast, when missingness depends directly on unobserved outcomes or latent traits (Fig. 2b), standard MAR-based approaches may fail regardless of statistical sophistication. Here, the problem is one of identifiability rather than power: Causal effects cannot be recovered without additional, untestable assumptions. Sensitivity analyses, selection models, or PMMs are therefore required to evaluate how inferences depend on assumptions about the missingness mechanism. Importantly, deletion methods implicitly condition on the observation indicator, which can induce collider bias when missingness is influenced by unobserved variables related to the outcome (Fig. 2c). This explains why complete-case analyses can be biased even when missingness appears unrelated to the predictor of interest.
Finally, the role of the missing variable itself matters. Missing outcomes, missing predictors, and missing confounders pose qualitatively different inferential challenges. Missing confounders are particularly problematic in observational studies because they can invalidate causal interpretations even when outcomes are fully observed. Accordingly, method choice should be guided not only by the proportion or pattern of missingness but also by a careful assessment of the causal structure underlying the research question.
Deletion methods
Deletion methods, including listwise (complete-case) and pairwise deletion, exclude cases with missing values and remain the default in many statistical-software packages (Enders, 2010; King et al., 1998). These approaches are simple to implement but are valid only under MCAR and can substantially reduce power and introduce bias when missingness depends on observed or unobserved variables (R. J. A. Little & Rubin, 2019; Schafer & Graham, 2002). Pairwise deletion may retain more data than listwise deletion (Newman & Cottrell, 2014) but can yield inconsistent sample bases across analyses, complicating interpretation and inference (Allison, 2002; Shi et al., 2020). From a causal standpoint, both approaches condition on the observation indicator and are therefore vulnerable to selection and collider bias (Fig. 2c). As a result, deletion methods are generally discouraged for confirmatory analyses.
Single-imputation methods
Simple single-imputation methods, such as mean substitution and regression imputation, are still encountered in applied research but are rarely appropriate for confirmatory inference (Dodeen, 2003). Mean substitution preserves sample size but distorts variances and attenuates associations, leading to biased standard errors and inflated Type I error rates (Enders, 2010; Schafer & Graham, 2002). Regression imputation improves on mean substitution by conditioning on observed predictors and auxiliary variables (Gold & Bentler, 2000), but it imputes fixed values and fails to propagate uncertainty, resulting in overly precise estimates and confidence intervals (Cheung, 2007; Musil et al., 2002). Both approaches rely on MAR assumptions and are sensitive to model misspecification, limiting their utility beyond exploratory analyses or as components in MI frameworks (R. J. A. Little & Rubin, 2019; Rubin, 1987).
MI
MI addresses missing data by generating multiple plausible values for each missing observation, analyzing each completed data set, and pooling results to reflect both within- and between-imputations uncertainty (Cummings, 2013; Enders, 2010; Rubin, 1987; Sinharay et al., 2001). Under MAR assumptions, MI yields unbiased parameter estimates while preserving multivariate relationships and incorporating auxiliary variables to improve the plausibility of the imputation model (Bernaards et al., 2007; Gondara & Wang, 2017; Schafer & Graham, 2002; van Buuren, 2018). MI is compatible with a wide range of statistical models and software environments (Carpenter et al., 2023; de Goeij et al., 2013), making it a practical default in many psychological applications (e.g., Chen & Sun, 2010).
A widely used implementation is MI by chained equations (MICE), which imputes variables sequentially using conditional models tailored to variable type (Azur et al., 2011; Resche-Rigon & White, 2018; van Buuren & Groothuis-Oudshoorn, 2011; White et al., 2011). MICE offers substantial flexibility for data sets with mixed variable types but requires careful model specification and attention to congeniality between imputation and analysis models (Bartlett et al., 2015; Royston & White, 2011; Rubin, 1987). Poorly specified imputation models can introduce bias or fail to recover the underlying data structure (Akande et al., 2017; Audigier et al., 2018). From a causal perspective, MI can be understood as a form of data augmentation in which missing values are treated as latent variables; regardless of whether estimation proceeds via Rubin’s rules or Bayesian inference, the validity of results depends on the causal assumptions encoded in the model (Sterne et al., 2009; Wayman, 2003; Wulff & Jeppesen, 2017; Zhang, 2016a, 2016b).
A critical limitation of MI that is often underemphasized in applied practice concerns uncertainty in the imputation model itself rather than uncertainty in its estimated parameters. Standard implementations of MI, even when described as “Bayesianly proper,” typically condition on a single, user-specified imputation model and therefore do not account for uncertainty about which predictors, functional forms, or dependencies should be included in that model. When important predictors of the missing data or the missingness process are omitted or when model structure is misspecified, MI can yield biased imputations and misleading inferences (Enders et al., 2016).
Kaplan and Yavuz (2020) addressed this problem by incorporating Bayesian model averaging into the imputation process, allowing uncertainty about the imputation model to be explicitly represented alongside parameter uncertainty. Their results demonstrate that ignoring imputation-model uncertainty can materially affect performance, even under MCAR and MAR conditions, and that model-averaged imputation can yield improved predictive accuracy and distributional recovery relative to conventional normal-theory MI. Importantly, this work highlights that the validity of MI depends not only on assumptions about the missingness mechanism but also on the adequacy and robustness of the imputation model itself. For applied researchers, these considerations underscore the need for sensitivity analyses, model checking, and cautious interpretation when key predictors of missingness are unmeasured or uncertain.
Likelihood-based approaches
Maximum-likelihood (ML) methods estimate model parameters directly from the observed data by integrating over the distribution of missing values rather than explicitly imputing them (Hayes & Enders, 2023; R. J. A. Little & Rubin, 2019; Shin et al., 2017). Under MAR assumptions and correct model specification, ML yields unbiased and efficient estimates while making full use of partially observed cases. ML approaches are particularly well suited to complex models (Chung et al., 2013; Dempster et al., 1977), including SEMs and multilevel models (Bryk & Raudenbush, 1992), in which missingness may occur at multiple levels (Enders, 2010; T. Lee & Shi, 2021).
FIML is especially prominent in SEMs and longitudinal research because it estimates parameters directly from all available data without discarding incomplete cases (Bodner, 2008; Enders & Bandalos, 2001; Newman, 2003). FIML has been shown to outperform deletion and simple imputation methods in terms of bias and efficiency under MAR (Larsen, 2011; Lang & Little, 2018). As with MI, however, ML methods rely on correct model specification and are vulnerable to bias when data are MNAR or when key predictors of missingness are omitted (R. J. A. Little & Rubin, 2019). Related estimation strategies treat missing data as latent variables within a Bayesian framework, as described next.
Bayesian data augmentation
An alternative model-based approach to handling missing data is Bayesian estimation via data augmentation, in which missing values are treated as latent variables and jointly estimated with model parameters. Rather than imputing missing values in a separate preprocessing step, data augmentation integrates missingness directly into the estimation procedure by iteratively sampling from the posterior distribution of both parameters and missing data (Tanner & Wong, 1987).
This framework is widely applicable to statistical models commonly used in psychology, including regression models, multilevel models, SEMs, and latent-variable models (Enders, 2022; S. Y. Lee, 2007). Under assumptions analogous to MAR, Bayesian data augmentation yields valid inference by propagating uncertainty about missing values through the posterior distribution, thereby avoiding the need for ad hoc deletion or single imputation.
From a conceptual perspective, Bayesian data augmentation is closely related to likelihood-based estimation and MI. Like FIML, it relies on an explicit model of the data-generating process; like MI, it represents uncertainty about missing values rather than treating them as fixed quantities. However, as with other MAR-based approaches, its validity depends on the plausibility of the assumed missingness mechanism and the adequacy of the specified model. When missingness depends on unobserved values or latent traits, Bayesian estimation does not circumvent identifiability problems and must be complemented by sensitivity analyses or explicit modeling of the missingness process.
PMMs
When missingness is plausibly MNAR, PMMs provide a flexible framework for incorporating assumptions about the relationship between missingness and unobserved values (R. J. A. Little, 1993, 1994). PMMs are an extension of mixture models (McLachlan, 1999; McLachlan et al., 2019; McLachlan & Rathnayake, 2014; Rasmussen, 1999; Reynolds, 2009), which have become popular for a range of uses in psychology, from accounting for latent distributions of effect sizes in meta-analysis (Moreau, 2021; Moreau & Corballis, 2019) to modeling p-value distributions (Gronau et al., 2017). Unlike selection models, which focus on the probability of data being missing, PMMs stratify the data by missingness pattern and model the observed data within each pattern, combining results using explicit assumptions about unobserved values. This structure facilitates sensitivity analyses by making assumptions transparent and allowing researchers to evaluate how conclusions depend on alternative specifications (Hedeker & Gibbons, 1997; Ratitch et al., 2013). Although powerful, PMMs require careful justification of assumptions and can be computationally demanding, particularly in complex or high-dimensional settings.
Additional considerations
Consider a longitudinal study examining the effect of academic stress (X ) on depressive symptoms (Y ) in university students, assessed via repeated self-report questionnaires. Suppose students with more severe depressive symptoms are less likely to complete follow-up assessments, leading to outcome-dependent dropout. Although stress (X ) is fully observed and can be included in the analysis or imputation model, missingness depends directly on the unobserved values of Y. Under this causal structure (Fig. 2b), conditioning on observed cases or applying standard MAR-based methods such as MI or FIML does not recover the target estimand because selection bias is induced by restricting analysis to students who remain observed. Importantly, increasing model complexity or adding auxiliary variables does not resolve this problem unless assumptions about the missingness mechanism are explicitly modified or sensitivity analyses are conducted. From a causal perspective, the issue is not one of statistical power or model flexibility but of identifiability under the assumed missingness process.
Beyond method choice, features of the missingness pattern itself can constrain identifiability and guide analytic decisions. These include whether missingness is univariate or multivariate (Abayomi et al., 2008; Schafer, 1997) or monotone or nonmonotone (R. J. A. Little & Rubin, 2002) and whether missingness patterns are connected or disconnected (Robins & Gill, 1997). Certain variable types (e.g., derived variables or key confounders) also warrant special attention to preserve substantive interpretation. These considerations should be evaluated alongside causal assumptions and study-design features rather than treated as purely technical details. For a summary of the primary advantages, limitations, and assumptions of the methods discussed in this section, see Table 2.
Advantages and Limitations of Common Methods for Handling Missing Data
Note: MCAR = missing completely at random; MAR = missing at random; MNAR = missing not at random; SEMs = structural equation models; MCMC = Markov chain Monte Carlo; ML = maximum likelihood; FIML = full-information maximum likelihood.
Using DataPatch to Diagnose and Handle Missing Data
To support principled decision-making in applied settings, I developed DataPatch, an interactive Shiny application designed to make missing-data mechanisms, assumptions, and consequences explicit (available at davidmoreau.shinyapps.io/DataPatch/). Rather than recommending a single “best” method, DataPatch encourages users to explore how different handling strategies behave under varying patterns of missingness, consistent with the causal perspective outlined above.
Figure 3 provides an overview of the application interface. Users may upload their own data set or load a sample data set and then select between two workflows: “Simulate Missing Data” or “Impute Missing Data.” This structure allows researchers to either diagnose existing missingness or conduct sensitivity analyses by imposing controlled missing-data mechanisms.
Overview of the DataPatch interface and workflow selection. The main DataPatch interface illustrating data-set upload, explicit workflow selection (simulation vs. imputation), and available missing-data-handling options. The sidebar emphasizes the upfront specification of analytic strategy before inspection or modification of the data. From this interface, users can simulate missingness under different assumptions (missing completely at random, missing at random, missing not at random), apply selected handling methods, inspect diagnostics and visualizations, and export processed data sets for downstream analysis.
Simulating missingness
In the simulation workflow (Fig. 4), the app allows users to generate missing data under explicitly specified assumptions about both the extent and mechanism of missingness. Users first define the target proportion of missing values and then select a missingness mechanism (MCAR, MAR, or MNAR), which determines how missingness is causally linked to observed and/or unobserved variables. Once missingness has been generated, users select a handling strategy from the set of available methods (Fig. 5), allowing direct comparison of how different approaches perform under the same simulated mechanism.
Simulating data mechanisms interface. Simulation workflow in DataPatch, showing user-defined control over the proportion of missing data, the assumed missingness mechanism (missing completely at random, missing at random, missing not at random), and the handling method selected for downstream analysis. The main panel displays the data set after simulated missingness has been introduced, with missing values highlighted. This interface allows users to operationalize theoretical missingness assumptions and examine their implications before applying methods to empirical data.
Missing-data-handling methods available in DataPatch. Method-selection panel in DataPatch. Available options include deletion-based methods, single-imputation approaches, multiple-imputation techniques, machine-learning-based methods, likelihood-based estimation, and pattern-mixture model. Methods are presented to facilitate comparison of assumptions, behavior, and performance under different missingness mechanisms rather than as equally recommended defaults for confirmatory inference.
Under MCAR, missing values are introduced independently of the data, serving as a baseline for comparison. Under MAR and MNAR, missingness is generated as a function of observed or latent values, respectively, enabling users to examine how violations of ignorability affect parameter estimates, uncertainty, and model performance. By making these assumptions explicit and manipulable, the simulation workflow operationalizes the causal distinctions discussed earlier in the article.
This functionality is particularly useful for pedagogical purposes and for stress-testing analytic strategies under plausible departures from MAR. For example, a researcher examining longitudinal associations between perceived stress and academic performance may simulate MNAR dropout driven by low-performing students, reflecting realistic attrition processes in educational psychology. Comparing results across simulated mechanisms allows researchers to assess the sensitivity of their conclusions to untestable missingness assumptions before applying methods to empirical data (Collins et al., 2001).
Diagnosing missing-data patterns
Before any imputation or estimation is performed, DataPatch provides a set of diagnostic summaries that describe the extent and structure of missingness in the data set (Fig. 6). These summaries include variable-level missingness rates, overall proportions of incomplete cases, and tabular overviews that allow users to quickly identify where data loss is most pronounced.
Diagnostic summary of missing-data patterns. Visualization of variable-level missingness percentages and a textual summary of overall missing-data characteristics. These diagnostics are presented before imputation to encourage explicit assessment of missing-data patterns and their potential implications for bias and identifiability.
These diagnostics help users assess whether missingness disproportionately affects key outcomes, focal predictors, or potential confounders and whether the observed patterns are compatible with the assumptions underlying their planned analytic strategy. In particular, examining which variables exhibit higher rates of missingness can inform the selection of auxiliary variables and help anticipate whether standard MAR-based approaches are likely to be adequate.
Because it encourages diagnosis before adjustment, this step operationalizes long-standing recommendations in the missing-data literature that emphasize understanding how and where data are missing before choosing a handling method. Rather than defaulting to a preferred technique, users are prompted to consider whether the observed missingness patterns suggest potential selection effects, unmeasured confounding, or the need for sensitivity analyses.
Imputing missing data
Once a handling strategy is selected, DataPatch applies the chosen method and provides visual and numerical summaries of its effects on the data. The application implements a range of commonly used approaches, including deletion-based methods (listwise and pairwise deletion), single-imputation approaches (mean substitution, regression imputation), MI techniques (MICE, Bayesian MI via Amelia, Bayesian data augmentation), machine-learning-based methods (k-nearest neighbors and random-forest imputation), likelihood-based estimation, and PPM.
For imputation-based methods, the app displays the relationship between observed and imputed values using scatterplots and summary metrics, allowing users to assess whether imputations preserve the scale, variability, and associations present in the observed data. These diagnostics help identify potential distortions introduced by particular methods, especially under departures from MAR.
As one illustrative example, DataPatch supports Bayesian data augmentation implemented via Gibbs sampling, in which missing values are treated as unknown parameters and iteratively sampled from their posterior predictive distributions conditional on observed data and current parameter estimates (Fig. 7). This approach provides a coherent Bayesian framework for propagating uncertainty through the imputation process. Users may control the number of iterations and burn-in period, reinforcing the distinction between fully Bayesian estimation and single-imputation approximations.
Visualization of imputation impact on observed and missing values. Scatterplot comparing observed values with values obtained after missing-data handling, illustrating agreement, dispersion, and proximity following imputation. The example shown uses Bayesian data augmentation, allowing users to assess the plausibility and stability of imputed values relative to the observed data distribution.
Across all methods, the resulting data sets, whether imputed or otherwise processed, can be exported as comma-separated (.csv) files for downstream statistical analysis, facilitating integration with standard analytic workflows.
Transparent reporting
Finally, DataPatch supports transparent reporting by automatically generating a structured methods report that documents how missing data were handled (Fig. 8). The report summarizes the extent and pattern of missingness, the assumed missingness mechanism, the selected handling strategy, and key model or algorithmic parameters (e.g., number of imputations, iterations, or tuning settings). Because it externalizes these details in a standardized format, the report reduces ambiguity in methodological descriptions and minimizes the risk that critical decisions remain implicit or underreported.
Automatically generated methods report for missing-data handling. Excerpt from the DataPatch methods report summarizing the selected workflow, missingness assumptions, imputation method, and key tuning parameters (e.g., number of Gibbs iterations and burn-in). This feature supports transparent and reproducible reporting of missing-data procedures in applied research.
This feature is intended to support reproducible research practices by making missing-data assumptions and analytic choices explicit and inspectable. Rather than relying on ad hoc narrative descriptions, researchers can use the generated report as a template for method sections or as supplementary material, ensuring that readers, reviewers, and replicators have access to the information required to evaluate the plausibility of the analysis.
In doing so, DataPatch aligns with broader efforts in psychological science to improve transparency and methodological accountability (Wagenmakers et al., 2021). The app helps lower the practical burden associated with documenting missing-data procedures, thus encouraging more complete reporting, and facilitates critical appraisal of how missingness may have influenced substantive conclusions.
Practical Recommendations
Given the variety of methods available, researchers are often confronted with important decisions about how to handle missingness appropriately. Below, I outline evidence-based, actionable recommendations to guide researchers in selecting and implementing imputation techniques.
Recommendation 1: articulate the estimand and causal structure before addressing missing data
Decisions about missing-data handling should be grounded in an explicit statement of the estimand and the assumed causal structure of the study. Whether the goal is to estimate a causal effect, a predictive association, or a descriptive parameter fundamentally shapes which missing-data assumptions are relevant and which methods are appropriate. Causal diagrams are particularly useful at this stage because they make explicit whether missingness acts as a selection process, a collider, or a confounder and clarify which paths must be blocked to recover the target estimand (see Figs. 2a–2d). Without this step, claims about bias or robustness are difficult to interpret and may be misleading.
Recommendation 2: treat missingness as a selection process, not a nuisance
Missing data should be conceptualized as a form of selection rather than random noise. Conditioning on observed cases (e.g., analyzing only cases with complete outcomes) implicitly conditions on a selection variable, which may induce bias depending on the causal structure. Researchers should explicitly consider whether conditioning on missingness opens noncausal paths between variables of interest (Fig. 2b and 2c), particularly in longitudinal and observational designs. This perspective helps explain why identical proportions of missing data can have very different implications across studies.
Recommendation 3: align the handling method with the missingness assumptions and state them explicitly
No missing-data method is assumption-free. Methods such as MI, FIML, and Bayesian data augmentation yield valid inference only under specific assumptions, typically variants of MAR and correct model specification. When missingness plausibly depends on unobserved values or latent traits (MNAR), standard MAR-based approaches are insufficient, and sensitivity analyses or explicit MNAR models become necessary rather than optional. Authors should clearly state which assumptions they rely on, why these assumptions are plausible given the study design and theory, and what would invalidate them.
Recommendation 4: prefer model-based approaches that propagate uncertainty
Methods that properly propagate uncertainty because of missing data, such as MI, likelihood-based estimation, and Bayesian data augmentation, should generally be preferred over ad hoc approaches (e.g., mean substitution or single imputation). Bayesian data augmentation is particularly flexible because it integrates imputation directly into the estimation process via iterative sampling and can be applied to a wide range of models common in psychological research. Regardless of the framework, researchers should ensure that auxiliary variables predictive of missingness and the incomplete variables are incorporated to improve recoverability.
Recommendation 5: use diagnostics and sensitivity analyses as substantive evidence, not formalities
Diagnostics should be used to evaluate whether the chosen method is behaving plausibly under the assumed causal structure. This includes examining distributions of observed versus imputed values, assessing convergence (for iterative methods; Brooks & Gelman, 1998), and checking whether key relationships are preserved after handling missing data. Sensitivity analyses, such as varying imputation models, priors, or assumptions about the missingness mechanism, are especially important when MNAR cannot be ruled out. Rather than treating these analyses as robustness add-ons, researchers should use them as evidence for or against the credibility of their conclusions.
Recommendation 6: report missing-data decisions transparently and interpret results accordingly
Transparent reporting of missing data is essential for cumulative science. Authors should report the extent and patterns of missingness, the assumed missingness mechanism, the causal rationale for the chosen method, and the implications of these choices for interpretation and generalizability. Importantly, conclusions should be framed on the basis of the missing-data assumptions required for identification. When results depend strongly on untestable assumptions, this dependence should be made explicit rather than minimized.
Future Directions and Conclusion
Missing data remains a pervasive and consequential challenge in psychological research, with direct implications for bias, identifiability, and the interpretation of empirical findings. Although the methodological literature has produced a wide range of principled statistical solutions, these advances have not always translated into consistent or transparent practice in applied research. A central theme of this article is that this gap is not merely technical but conceptual: Missing-data problems cannot be fully understood or resolved without explicit consideration of causal structure and the estimands researchers seek to recover.
Future work would benefit from deeper integration between missing-data methodology and modern causal-inference frameworks. DAGs in particular provide a unifying language for representing missingness as a selection process, clarifying when standard methods are sufficient and when bias is unavoidable without stronger assumptions or sensitivity analyses. Extending these ideas to more complex designs (e.g., multilevel, longitudinal, and intensive longitudinal data) remains an important direction for both methodological and applied research (e.g., Audigier et al., 2016).
At the same time, continued development of flexible estimation approaches, including Bayesian data augmentation and machine-learning-based methods (Kokla et al., 2019), offers promising avenues for handling high-dimensional and nonlinear data structures common in contemporary psychology (Ipsen et al., 2020; Jadhav et al., 2019). However, increased methodological sophistication must be accompanied by clear guidance regarding assumptions, diagnostics, and interpretation. Without such guidance, advanced methods risk being applied as black-box solutions, undermining their potential benefits.
Ultimately, improving practice around missing data will require more than new methods. Tools such as DataPatch can play a complementary role alongside methodological advances by lowering barriers to engagement with best practices. In particular, they can support training, facilitate sensitivity analyses, and encourage clearer reporting by making the implications of missing-data decisions more visible. Continued development in this space should provide improved guidance, interpretability, and integration with substantive research workflows in an effort to support more principled decision-making, clearer reporting, and more credible inferences in psychological research.
Footnotes
Acknowledgements
Transparency
Action Editor: Yasemin Kisbu-Sakarya
Editor: David A. Sbarra
Author Contributions