Estimating the complier average causal effect via a latent class approach using gsem

Abstract

In randomized controlled trials, intention-to-treat analysis is customarily used to estimate the effect of the trial. However, in the presence of noncompliance, this can often lead to biased estimates because intention-to-treat analysis completely ignores varying levels of actual treatment received. This is a known issue that can be overcome by adopting the complier average causal effect approach, which estimates the effect the trial had on the individuals who complied with the protocol. When compliance is unobserved in the control group, the complier average causal effect estimate can be obtained via a latent class specification using the gsem command.

1 Introduction

In a standard randomized controlled trial (rct), we compare the outcome of interest in two groups: 1) the treatment group, which is composed of those participants who are randomly selected to a new treatment; and 2) the control group, which is composed of those who are randomly selected to continue with the usual practice (standard treatment or no treatment at all). When all participants who are randomized to treatment receive the treatment in full or comply with all the requirements of the rct, then the trial arm assignment is enough to estimate the difference between both groups. This is the standard intention-to-treat (itt) approach.

However, in real-world scenarios, participants assigned to treatment may not receive the treatment in full or engage in all the activities required by the rct. This is a common occurrence that can bias the itt estimate, which is why compliance needs to be accounted for.

In this article, we show how to fit a complier average causal effect (cace) model using the Stata gsem command. This type of model can also be fit in Stata by using the community-contributed command gllamm (Rabe-Hesketh, Skrondal, and Pickles 2004). Even though cace models can be fit with gllamm, most of gllamm’s features have been incorporated in gsem from Stata 14 onward.

To the best of our knowledge, there is no worked example available of how to fit a cace model in Stata using gsem. In this article, we explain the main features of this model and how to fit it, providing a reproducible example and using the main options available in a user-friendly way. We aim to provide a practical introduction to cace modeling in Stata for researchers who are already familiar with rct designs, statistical analysis, and Stata’s main capabilities at an intermediate level but who may not be acquainted with the latent class approach to cace and gsem’s capabilities.

The next section discusses the cace approach to addressing the issue of noncompliance in rcts.

2 Addressing noncompliance in RCTs

The cace estimate is defined as the difference between the outcome in those participants who complied with the intervention and those participants who would have complied if assigned to treatment. The assumptions of cace are discussed in detail by Imbens and Rubin (1997) and Little and Yau (1998). Peugh et al. (2017) also provide a thorough introduction to cace modeling and recent extensions.

The overarching principle of cace is that the causal effect does not happen because of the mere offer of the treatment itself (itt) but rather because of the actual treatment received. This is the reasoning behind the “exclusion restriction” assumption in cace modeling (Little and Yau 1998), which states that the treatment has no effect on the outcome in those participants who did not comply with the intervention.

The main difficulty of estimating the causal effect of compliance is that, while it is straightforward to determine compliance with the intervention in the treatment group, it remains unknown or unobservable in the control group. It is necessary, therefore, to deploy a method that allows distinguishing, within the control group, those who would have complied had they been assigned to treatment from those who would not.

Skrondal and Rabe-Hesketh’s (2004) thorough description of the methods to obtain the cace estimate takes the “latent class with training data” approach proposed by Muthén (2002). This is a probabilistic approach that seeks to estimate the “true” compliance status in the control group while treating observed compliance in the treatment arm as “known”.

Consider the example of a school-based intervention where some classrooms are randomized to perform an activity for a prescribed length of time while others are randomized to continue with “business as usual”. During the intervention period, researchers record the times at which the activity is conducted in the intervention group and find wide variability. Given that some classrooms performed the activity for longer than others, we might expect the effect of the intervention to be “diluted” by those classrooms that conducted the activity for shorter periods. Dosage would then be key to understand the effect of the trial on the outcome of interest, and this measure can be used to determine compliance status in the intervention group.

Naturally, classrooms under the “business as usual” regime have no records for said activity, but within this group some would have conducted the activity had they had the chance to do so; hence, “true” compliance in the control group is unknown. Additionally, given that there is heterogeneity in the intervention group (that is, there are compliers and noncompliers), assuming homogeneity in the control group would not necessarily be enabled. By virtue of the randomization itself, the characteristics that make children in the intervention group more likely to comply would also make the children in the control group more likely to comply; that is, there is equivalence of groups prior to intervention (Skrondal and Rabe-Hesketh 2004). This is the rationale behind fitting a cace model with a latent class approach.

Humphrey et al. (2022) used this approach when analyzing the effect of the “good behavior game” on health- and education-related outcomes in children attending primary schools in England. This rct aimed to improve classroom behavior through a team game (that is, the treatment) played during school time, and it was expected that this would positively impact on mental health and school attendance mainly.

The following section illustrates how the latent class approach can be implemented in Stata using gsem.

2.1 Specifying a CACE model with gsem

Below, we can see the basic specification of a cace model using gsem. This makes use of [sem] gsem path notation extensions. This is a latent class regression model (also known as a mixture model) with specific constraints that are necessary for cace estimation.

The most basic specification of a cace model would have the following variables:

• depvar is the outcome of interest, which is by default assumed to have Gaussian distribution.

• treatment is a binary variable to indicate whether the observation was assigned to treatment (treatment = 1) or control (treatment = 0).

• comp is a binary variable to indicate compliance in the treatment group; it is missing in the control group.

The following are optional:

• indepvars are the predictors of the outcome of interest.

• varlist is the predictor or set of predictors for compliance.

The first line of the syntax is simply the call to gsem. The second and third lines compose the regression model for the outcome depvar:

(1: depvar <- i.treatment@0 [indepvars] [, family (familyname) ] )

(2: depvar <- i.treatment [indepvars] [, family (familyname) ] )

(1:) is the regression path for noncompliers, which has the treatment effect fixed to 0; this is the exclusion restriction assumption. (2:) is the regression path for compliers, where the treatment effect is estimated freely; this is the cace estimate. Even though not strictly necessary for the estimation of cace, it is advisable to fit the model with predictors for the outcome of interest (indepvars), for example, baseline measures (Twisk et al. 2018).

The fourth line, (C), is a regression model for the latent class on a set of covariates:

(C <- varlist )

This is not strictly essential in the estimation, but it is preferable to have a set of covariates that are reasonably good predictors of compliance (Jo 2002). If no variables are specified or the whole line is omitted, an intercept-only compliance model is fit.

The fifth and sixth lines compose the latent class model for compliance in the treatment group with special constraints that treat compliance in the treatment arm as known:

(1: comp <- _cons@-15, logit)

(2: comp <- _cons@15, logit)

_cons@-15 fixes the probability of membership to the compliers class of those who did not comply with the treatment to essentially 0. Meanwhile, _cons@15 fixes the probability of membership to the compliers class of those who complied with the treatment to essentially 1.

Finally, the last line states the name of the latent class (C) and the number of classes, which needs to be set to 2 (compliers and noncompliers):

lclass(C 2)

Other available options are discussed in section 3.

2.2 An example CACE application

This example uses data from the jobs II intervention (Vinokur, Price, and Schul 1995). This rct aimed to prevent depression as a result of job loss by providing training to jobseekers. Our example replicates the cace estimate as reported by Little and Yau (1998), which is also replicated in Skrondal and Rabe-Hesketh (2004) using gllamm (Rabe-Hesketh, Skrondal, and Pickles 2004). The aim here is to estimate the causal effect of actually receiving the treatment, that is, attending job training seminars, on the outcome of interest, that is, depression. A brief description of the variables used in this example is presented in table 1.

Table 1.

Variables in the jobs II dataset (Vinokur, Price, and Schul 1995)

Variable	Description
depress	depression score; outcome variable
depbase	baseline depression score
risk	baseline risk; an index of depression, financial strain, and assertiveness
r	dummy variable for being randomized to treatment
c	dummy variable for compliance (valid only for treatment group)
age	age in years
motivate	motivation to attend the job training seminars
educ	school grade completed
assert	assertiveness
single	dummy variable for being single
econ	economic hardship
nonwhite	dummy for not being white versus being white

First, we read the data from the gllamm website, as such:

The variable c is a dummy to indicate compliance in the treatment group. From this variable, we need to derive the variable comp, which is missing (unobserved) in the control group, as such:

. generate comp=c if r==1

(167 missing values generated)

Then, we specify and run the cace model. We constrained the effects of covariates in the regression equations to be equal across classes, that is, @c1 and @c2. These additional constraints were specified to replicate the results in Little and Yau (1998) but are not essential for other applications.

The cace estimate is given on the table for the regression in class 2. The effect of treatment (r) in the compliers class is −0.3098673, which indicates that those who complied in the intervention arm are expected to score 0.31 less in the depression scale than those who would have complied if assigned to treatment.

With gsem, many postestimation functions are available. For example, we can request a summary of model fit information (useful for comparing competing models) by typing

Next we illustrate how we can use margins and marginsplot to obtain predicted values per latent class. First, we run the margins command to obtain some predicted values across classes at the quartiles of baseline depression (depbase) in the noncompliers class (class 1):

Second, we run the marginsplot command to visualize the predicted values in the noncompliers class (class 1):

We omitted the plot because this is an intermediate step for the final plot presented in figure 1. We also requested the predicted values for the noncompliers class overall. There is no need to do this by treatment group—its predicted values are constrained to equality because we specified the exclusion restriction assumption.

Third, we run the margins command again but this time for the predicted values in the compliers class (class 2) across treatment groups:

Afterward, we call the marginsplot command to visualize the predicted values in the compliers class (class 2) by trial arm (plot omitted).

Finally, we call graph combine to plot the predicted values by complier status and trial arm (figure 1):

. graph combine class1 class2, cols(2) scheme(sj)

Figure 1.

Predicted scores by complier status and trial arm

In figure 1, we can see that compliers (right-hand side) in the control group have higher predicted scores for depression at all values of baseline depression. On the other hand, noncompliers (regardless of trial arm) have predicted depression scores lower than compliers in the control group but not as low as compliers in the treatment group. The same steps can be repeated for “risk” scores at baseline to visualize the effect of compliance at different levels of risk.

Some additional options are discussed in the next section.

3 Conclusions

In this article, we presented the main features of a cace model using gsem in Stata. This was aimed at researchers already familiar with rct data analysis and Stata itself, using a reproducible and well-known example. Interested readers are encouraged to explore further options available with gsem, and we mention some potentially useful ones next.

gsem comes with many postestimation functions that can be used in cace modeling. In section 2.2, we provided an example to obtain the information criteria, which can be used for model comparison and selection, as well as an example of margins and visualization. For more details of the available functions, see [sem] gsem postestimation.

In addition, equality constraints can be used potentially to test other hypotheses of interest, such as common or separate coefficients across compliers and noncompliers. For more details, see [sem] sem and gsem option constraints( ) and [sem] sem and gsem path notation.

With gsem, one can also fit cace models for a variety of distributions, hence the option of specifying family() in the above general form of the syntax. For more details of the allowed distribution families and link functions, see [sem] gsem family-and-link options.

Finally, clustering around groups can be accounted for with the vce() option. This is especially relevant for cluster rcts, where whole groups go through the process of randomization before the intervention. Humphrey et al. (2022) recently applied this approach using gsem to fit a cace model in a school-based rct in England. This study fit cace models for generalized outcomes using clustered standard errors, providing a further example of gsem’s functions and extended capabilities. These cace estimates (obtained using gsem) are presented in Troncoso (2021).

To sum up, gsem is a flexible command that can be used to tackle a variety of statistical problems pertaining to the analysis of trial efficacy, including under noncompliance as shown here. The versatility of gsem and Stata’s overall usability makes it a powerful tool for researchers analyzing rct data.

Supplemental Material

Supplemental Material, sj-zip-1-stj-10.1177_1536867X221106416 - Estimating the complier average causal effect via a latent class approach using gsem

Supplemental Material, sj-zip-1-stj-10.1177_1536867X221106416 for Estimating the complier average causal effect via a latent class approach using gsem by Patricio Troncoso and Ana Morales-Gómez in The Stata Journal