Uncovering patterns among latent variables: human rights and de facto judicial independence

Abstract

In this paper, we reexamine the relationship between judicial independence and state respect for human rights by taking advantage of new latent measures of both constructs. In our analysis, we demonstrate a simple method for incorporating the uncertainty of these latent variables. Our results provide strong support for theoretical and empirical claims that independent courts constrain human rights abuses. Although we show that independent courts influence state behavior, the strength of the estimated relationship depends upon whether and to what degree empirical models account for uncertainty in the measurement of the latent variables.

Keywords

human rights judicial independence latent variable model

Introduction

Do independent courts constrain states from abusing human rights? Prior studies have analyzed the extent to which formal provisions for judicial independence, such as constitutional protections against executive interference, are associated with respect for human rights. While the empirical evidence presented in these studies is mixed, recent research suggests a strong positive correlation between judicial independence and state respect for human rights (see Keith (2012) for a comprehensive review of this literature).

By focusing on de jure rather than de facto judicial independence, these studies might incorrectly estimate the true relationship between independent courts and respect for human rights. Most constitutions contain provisions that empower the judiciary to “check state power” (Keith, 2012; 2), but these provisions are not necessarily a strong predictor of the extent to which courts can act independently of the executive (Herron and Randazzo, 2003; Linzer and Staton, 2015). Even if the law guarantees courts specific powers, a regime could still constrain judicial authority by ignoring legal restrictions or by creating other laws, institutions, and norms that erode judicial independence (Silverstein, 2008). Alternatively, even if a state constitution does not empower the judiciary, courts might still find a way to exert influence and limit state terror (Ginsburg and Moustafa, 2008: 17).

To address this important distinction, Keith (2012) examines the effect of de facto judicial independence on political repression and finds compelling evidence that independent courts constrain human rights abuses. The measures that she uses to capture these constructs do not account, though, for the fact that both de facto judicial independence and state respect for human rights are not perfectly observable. That is, they are latent constructs that can only be estimated using observable indicators, which might sometimes be biased relative to the theoretical concepts of interest. Left unacknowledged, this measurement issue may obscure the true relationship between de facto judicial independence and state respect for human rights.

In this research note, we reexamine the relationship between de facto judicial independence and state respect for human rights, taking advantage of new latent variables for both of these important concepts (Fariss, 2014; Linzer and Staton, 2015). Latent variable models focus on the theoretical relationship between data and model parameters and offer scholars a principled way to bring together different pieces of information even if that information is in someway biased relative to the theoretical concept of interest (Fariss, 2015). Thus, the latent human rights and de facto judicial independence variables provide more valid measurements of these important theoretical concepts by bringing together multiple related indicators and linking them together using principled and transparent measurement models. In our analysis, we also demonstrate how to account for the uncertainty in the relative values of the country–year latent variable estimates. The results provide strong support for the theoretical and empirical claims of Keith (2012). The existence of independent courts is associated with greater respect for human rights.

Model specification and results

To examine the relationship between de facto judicial independence and state respect for human rights, we use a common model specification in the human rights literature (Keith, 2002, 2012; Keith et al., 2009; Poe and Tate, 1994). The model regresses a measure of state respect for human rights on a lagged outcome measure and a series of variables that capture differences in “domestic and external threats (civil and/or international war), regime type (democracy, military, and leftist), and socioeconomic conditions (economic development, population size, and colonial legacy)” (Keith, 2012: 79). This model allows us to more easily build upon past empirical findings (Keith, 2012: 68).¹ We make three important changes to the specification of this model.

The first change we make is to replace the usual outcome measures with a latent measure of state respect for human rights. Prior studies of the relationship between judicial independence and state respect for human rights typically use measures provided by or adapted from State Department and Amnesty International country year reports (Cross, 1999; Keith et al., 2009; Powell and Staton, 2009). However, the reporting standards of these organizations have changed, obscuring the true patterns of human rights practices over time (Fariss, 2014). The latent measure accounts for systematic changes to the human rights country reports published annually by the State Department and Amnesty International (Fariss, 2014).

The second change we make is to include a latent measure of de facto judicial independence developed by Linzer and Staton (2015). This variable improves on the measure developed by Keith (2012) in several important ways. The latent variable treats de facto judicial independence as an unobservable construct that can only be measured with uncertainty. This is important because, much like measures of human rights, observers cannot be certain of the precise level of de facto judicial independence for one country–year relative to another.² Uncertainty is important substantive information necessary for comparing the relationship between complex theoretical concepts across political contexts and over time (Fariss, 2015; Schnakenberg and Fariss, 2014). The latent variable model provides a principled method for estimating the uncertainty of the country–year units. If we did not incorporate this information into the regression model, we would need to interpret our results under the strong assumption that we had perfectly operationalized and measured this theoretical concept.

In their measurement model, Linzer and Staton (2015) combine data from 12 different observable indicators (manifest variables) that are theoretically related to de facto judicial independence, which ensures that the latent variable estimates are not overly reliant on any single indicator. The incorporation of many observable indicators is a useful feature of latent variable models both in general and in this particular case. This is because Keith (2012) constructs her measure of de facto judicial independence from a single source, the Department of State human rights reports. These reports are potentially biased in favor of American “allies for security and political reasons” (Keith, 2012: 74) and US trade partners (Poe et al., 2001: 677). These potential biases might obscure the empirical relationship between de facto judicial independence and human rights. As Jackman (2008) points out, a researcher with only one indicator of a latent construct is unable to determine how much variation in the indicator is due to measurement error as opposed to other forms of variation in the latent construct. By using the latent variable of de facto judicial independence in our regression model, we reduce the risk that any possible State Department bias is driving the results.

While the Linzer and Staton (2015) latent variable model incorporates the de facto judicial independence variable developed by Keith (2012), any bias from this particular variable is reduced with respect to the estimate of the latent variable if the other indicators in the measurement model do not share the same biases. Scholars still concerned that one or more manifest variables are biasing the Linzer and Staton (2015) measure can use the publicly available data and code to exclude one or more of these variables from the latent variable model. In light of concerns that the Linzer and Staton (2015) measurement model includes such a measure, the proportion of money that is held in banking institutions or the Contract Intensive Measure score (CIM), we reestimate our regression model with a modified de facto judicial independence variable that excludes it.³

The third change we make to the regression model is to replace the measure of democracy with the Democracy–Dictatorship (DD) measure (Cheibub et al., 2010). Most earlier human rights studies use Polity or Freedom House measures of democracy. This is problematic because the Polity and Freedom House indicators classify regimes, in part, based on their respect for human rights (Hill, 2014; Hill and Jones, 2014). The concern is that using these measures causes us to partially control for the variable we are interested in examining and prevents us from assessing the independent effect of regime type on state respect for human rights. Following Poe and Tate (1994), who state that democracy “must be defined in terms that allow independent operationalization of the concept” (856), we use a measure of democracy that does not include state human rights practices. The DD measure is ideal for this as it measures democracy by whether free and contested elections have occurred (Cheibub et al., 2010: 69). While the results we present include this measure, they are robust to using other alternative indicators of regime type (i.e. the Polity measure used in Keith (2012), the Freedom House and Polity measures used in Keith et al. (2009), and the GWF Autocratic Regimes measure (Geddes et al., 2014).⁴

Since the latent human rights variable is continuous, we test the theoretical expectation that increases in de facto judicial independence are related to increases in state respect for human rights using ordinary least squares (OLS) regression with robust standard errors.⁵ Both of these choices are consistent with models presented in earlier work (Keith et al., 2009; Poe et al., 1999). For our primary models, we use data from (Keith et al. (2009) and Keith (2012) but include different indicators of (1) state respect for human rights (Fariss, 2014), (2) de facto judicial independence (Linzer and Staton, 2015), and (3) democracy (Cheibub et al., 2010).⁶

Figure 1 plots the point estimates from the OLS model along with 90% and 95% confidence intervals. In contrast with previous research (Cross, 1999; Keith, 2012; Keith et al., 2009; Powell and Staton, 2009), this model specification provides no evidence that increased judicial independence decreases state respect for human rights. While the point estimate for de facto judicial independence is positive, the standard error is larger than the point estimate.

Figure 1.

Effect of de facto judicial independence on state respect for human rights.

The results presented in Figure 1, however, do not account for uncertainty in the point estimates of the outcome variable. As discussed above, researchers should take into account uncertainty when they cannot be sure about the precise value of the operationalized construct for one unit relative to another unit.⁷ So far, we have used the point estimates of the latent variable (mean of the posterior distribution) to estimate our model but we have ignored the standard deviation of the posterior distribution. To incorporate the information from these country–year distributions, we follow recommendations from Schnakenberg and Fariss (2014) by duplicating our dataset 1,000 times and assigning a random draw from the posterior distribution of the latent variable to each country–year observation. We use this new value as the outcome measure. We also perform the same procedure for the lagged outcome measure. We then estimate a set of 1,000 OLS models, combining the results across the multiple sets of data to create one set of coefficient and standard error estimates. This procedure is substantively important because it allows us to relax the assumption that theoretically important variables are measured perfectly and without error (Mislevy, 1991; Schnakenberg and Fariss, 2014). The equation used to combine the estimates from each of the 1,000 OLS models was developed by Rubin (1987) to combine estimates from multiply imputed datasets. Mislevy (1991) and Schnakenberg and Fariss (2014) discuss this approach in the context of latent variable models.

Figure 2 presents the results. The point estimate for de facto judicial independence is larger than in the base OLS model and the standard errors are now much smaller. These differences occur because we have relaxed both (a) the assumption that we have perfectly measured the latent variable on the right-hand side of the regression model and also (b) the relationship between the latent human rights variable and its value in the previous year. As a result, de facto judicial independence is now statistically significant and substantively quite large. A change in de facto judicial independence from the 25% to 75% percentile is associated with a 0.11 increase in state respect for human rights. In comparison, the occurrence of civil war, long considered the most important predictor of increased human rights abuse (Keith, 2012), is associated with a 0.15 decrease in state respect for human rights. This evidence suggests that independent courts play a meaningful role in checking human rights abuses.

Figure 2.

Effect of de facto judicial independence on state respect for human rights (accounting for uncertainty in the outcome measure and the lagged outcome measure).

Since our key explanatory variable, de facto judicial independence, is also a latent construct, we need to account for uncertainty in its measurement as well. We do so using the process described above. In Figure 3, we present the results of re-estimating our model with new values for the judicial independence measure. Table 1 presents this model and the earlier models, allowing for easy comparison. As in the previous model, we find that an increase in de facto judicial independence variable is associated with a substantial increase in respect for human rights. While the magnitude of this increase is marginally smaller than in the previous model, the effect of this change is still greater than the individual effect of other variables in the model, with the exceptions of the lagged dependent variable and the civil war measure. This provides additional evidence for a strong positive correlation between judicial independence and state respect for human rights.⁸

Figure 3.

Effect of de facto judicial independent on state respect for human rights (accounting for uncertainty in the outcome measure, the lagged outcome measure, and the independent variable).

Table 1.

State respect for human rights across countries (1980–2004).

	Model 1	Model 2	Model 3
Lagged Outcome Measure	0.967^***	0.859^***	0.860^***
	(0.004)	(0.015)	(0.015)
De Facto Judicial Independence (Latent Measure)	0.015	0.226^***	0.203^***
	(0.021)	(0.065)	(0.061)
Civil War	−0.031^**	−0.150^***	−0.149^***
	(0.013)	(0.036)	(0.036)
International War	0.008	−0.025	−0.024
	(0.025)	(0.082)	(0.082)
Democracy	0.024^**	0.014	0.021
	(0.010)	(0.027)	(0.027)
Military Control	−0.013	−0.023	−0.025
	(0.009)	(0.024)	(0.024)
State-Socialist Regime	0.015	0.023	0.023
	(0.012)	(0.035)	(0.035)
British Col. Exper.	−0.022^***	−0.039^*	−0.037
	(0.007)	(0.023)	(0.023)
Economic Development	0.000^***	0.000^***	0.000^***
	(0.000)	(0.000)	(0.000)
Economic Growth	0.002^***	0.002	0.002
	(0.001)	(0.002)	(0.002)
Logged Population	−0.013^***	−0.042^***	−0.042^***
	(0.002)	(0.007)	(0.007)
Population Growth	0.000	−0.006	−0.006
	(0.003)	(0.008)	(0.008)
Constant	0.214^***	0.600^***	0.602^***
	(0.038)	(0.113)	(0.113)
N	3015	3015	3015

p < 0.10 ; **p < 0.05 ; ***p < 0.01 (two-tailed).

Note: Robust standard errors are shown in parentheses. Data come from 3015 country–year observations from 1980 to 2004. The outcome measure is state respect for human rights. See Keith (2012) for more information about the model and data.

To demonstrate the importance of incorporating uncertainty in the measurement of latent constructs, we compare the estimates of de facto judicial independence. Figure 4 plots the point estimates for de facto judicial independence from each model with 90% and 95% confidence intervals. The figure illustrates that if we do not account for uncertainty in the measurement of the outcome and lagged outcome measures, we might underestimate the possible effect of de facto judicial independence on state respect for human rights. Indeed, we would infer that there was not a statistically significant association. The figure also shows that if we do not take into account uncertainty in the measurement of de facto judicial independence, we would slightly overestimate its effect. Only by accounting for uncertainty in both latent variables can we estimate the relationship between them based on both the uncertainty of the relationship between outcome measure and independent variables (the uncertainty that OLS regression captures) and the uncertainty in the measurement of the independent variables themselves (the uncertainty that a latent variable model captures).

Figure 4.

Comparing the effect of de facto judicial independence across models.

Finally, we consider whether the findings are limited to the cases we include in the analysis. To guard against overfitting and “type III error”, we use k-fold cross-validation (Efron and Gong, 1983; Hill and Jones, 2014; Ward et al., 2010). We run 1,000 simulations, randomly partitioning our data into one training set and nine test sets ( $k = 0$ ) and estimate a series of models. Table 2 presents the OLS results from these models. The baseline model (Model 1) includes only the lagged outcome measure; other models contain combinations of explanatory variables. Figure 5 plots the cross-validation results, specifically the average percent reduction in mean square error of each model compared to the baseline model. This reveals the additional predictive power of individual variables and combinations of variables. As shown in the figure, we reduce more error by including the latent judicial independence measure in the model than we do by including other theoretically important variables. De facto judicial independence is a substantively important predictor of increased state respect for human rights.

Table 2.

State respect for human rights across countries (1980–2004): models used for cross-validation.

	Model 1	Model 2	Model 3	Model 4	Model 5	Model 6
Lagged Outcome Measure	0.918^***	0.940^***	0.935^***	0.900^***	0.874^***	0.860^***
	(0.011)	(0.011)	(0.011)	(0.013)	(0.014)	(0.015)
De Facto Judicial Independence (Latent Measure)	0.211^***	–	–	–	–	0.203^***
De Facto Judicial Independence (Latent Measure)	(0.040)	–	–	–	–	(0.061)
Civil War	–	−0.112^***	–	–	−0.146^***	−0.149^***
	–	(0.036)	–	–	(0.036)	(0.036)
International War	–	−0.017	–	–	−0.016	−0.024
	–	(0.084)	–	–	(0.082)	(0.081)
Democracy	–	–	0.059^***	–	0.079^***	0.021
	–	–	(0.021)	–	(0.022)	(0.027)
Military Control	–	–	−0.047	–	−0.038^*	−0.025
	–	–	(0.023)	–	(0.023)	(0.024)
State-Socialist Regime	–	–	−0.010	–	0.023	0.023
	–	–	(0.035)	–	(0.035)	(0.035)
British Col. Exper.	–	–	−0.004	–	−0.013	−0.037
	–	–	(0.022)	–	(0.022)	(0.023)
Economic Development	–	–	–	0.000^**	0.000^***	0.000^***
	–	–	–	(0.000)	(0.000)	(0.000)
Economic Growth	–	–	–	0.002	0.002	0.002
	–	–	–	(0.002)	(0.002)	(0.002)
Logged Population	–	–	–	−0.032^***	−0.039^***	−0.042^***
	–	–	–	(0.007)	(0.007)	(0.007)
Population Growth	–	–	–	−0.020	−0.012	−0.007
	–	–	–	(0.007)	(0.008)	(0.008)
Constant	−0.066***	0.039***	0.015	(0.533)***	0.619^***	0.602^***
	(0.020)	(0.010)	(0.018)	(0.112)	(0.114)	(0.113)
N	3015	3015	3015	3015	3015	3015

p < 0.10 ; **p < 0.05 ; ***p < 0.01 (two-tailed).

Note: Robust standard errors are shown in parentheses. All models account for uncertainty in the outcome measure, the lagged outcome measure, and the independent variable. Data come from 3015 country–year observations from 1980 to 2004. The outcome measure is state respect for human rights. See Keith (2012) for more information about the model and data.

Figure 5.

Cross-validation results.

Conclusion

We have reexamined the finding from Keith (2012) that de facto judicial independence is positively associated with state respect for human rights, taking advantage of new latent measures of both constructs. We have also demonstrated how to incorporate the uncertainty in the latent variables used in our analysis. Although the relationship depends upon whether and to what degree our empirical models account for uncertainty in the measurement of latent constructs, increased de facto judicial independence appears to be associated with a substantial decrease in human rights abuses. Overall, the results provide strong support for theoretical and empirical claims that the existence of independent courts is associated with greater respect for human rights (Cross, 1999; Keith, 2012; Keith et al., 2009; Lupu, 2013; Powell and Staton, 2009).

Footnotes

Acknowledgements

We would like to thank Luke Keele, Linda Camp Keith, Mark Major, Keith Schnakenberg, and Jeffrey Staton for many helpful comments and suggestions.

Conflict of interest statement

The authors have no conflicts of interest to declare.

Funding

This research was supported in part by The McCourtney Institute for Democracy Innovation Grant, and the College of Liberal Arts, both at Pennsylvania State University.

Supplementary material

The online appendix is available at:

The replication files are available at:

Notes

References

Cheibub

Gandhi

Vreeland

(2010) Democracy and dictatorship revisited. Public Choice 143: 67–101.

Cross

(1999) The relevance of law in human rights protection. International Review of Law and Economics 19: 87–98.

Efron

Gong

(1983) A leisurely look at the bootstrap, the jackknife, and cross-validation. The American Statistician 37: 36–48.

Fariss

(2014) Respect for human rights has improved over time: Modeling the changing standard of accountability. American Political Science Review 108: 297–318.

Fariss

(2015) Human rights treaty compliance and the changing standard of accountability. British Journal of Political Science, in press.

Geddes

Wright

Frantz

(2014) Autocratic breakdown and regime transitions: A new data set. Perspectives on Politics 12: 313–331.

Ginsburg

Moustafa

(2008) Rule by law: the politics of courts in authoritarian regimes. Cambridge: Cambridge University Press.

Herron

Randazzo

(2003) The relationship between independence and judicial review in post-communist courts. Journal of Politics 65(2): 422–438.

Hill

Jr (2014) Democracy and the concept of personal integrity rights in empirical research. Available at: http://myweb.fsu.edu/dwh06c/pages/documents/democ_pi_17Sept14.pdf

10.

Hill

Jr Jones

(2014) An empirical evaluation of explanations for state repression. American Political Science Review 108: 661–687.

11.

Jackman

(2008) Measurement. In: Box-Steffensmeier

Brady

Collier

(eds.) The Oxford Handbook of Political Methodology. Oxford: Oxford University Press.

12.

Keith

(2002) Constitutional provisions for individual human rights (1977-1996): Are they more than mere “window dressing?”. Political Research Quarterly 55: 111–143.

13.

Keith

(2012) Political Repression Courts and the Law. University of Pennsylvania Press.

14.

Keith

Tate

Poe

(2009) Is the law a mere parchment barrier to human rights abuse? The Journal of Politics 71: 644–660.

15.

Linzer

Staton

(2015) Global measure of judicial independence, 1948–2012. Journal of Law and Courts 3: 223–256.

16.

Lupu

(2013) Best evidence: The role of information in domestic judicial enforcement of international human rights agreements. International Organization 67: 469–503.

17.

Mislevy

(1991) Randomization-based inference about latent variables from complex samples. Psychometrika 56: 177–196.

18.

Poe

Carey

Vazquez

(2001) How are these pictures different? a quantitative comparison of the us state department and amnesty international human rights reports, 1976–1995. Human Rights Quarterly 23: 650–677.

19.

Poe

Tate

(1994) Repression of human rights to personal integrity in the 1980s: A global analysis. American Political Science Review 88: 853–872.

20.

Poe

Tate

Keith

(1999) Repression of the human right to personal integrity revisited: A global cross-national study covering the years 1976-1993. International Studies Quarterly 43: 291–313.

21.

Powell

Staton

(2009) Domestic judicial institutions and human rights treaty violation. International Studies Quarterly 53: 149–174.

22.

Rubin

(1987) Multiple Imputation for Nonresponse in Surveys. New York, NY: John Wiley and Sons.

23.

Schnakenberg

Fariss

(2014) Dynamic patterns of human rights practices. Political Science Research and Methods 2: 1–31.

24.

Silverstein

(2008) Singapore: the exception that proves rules matter. In: Rule by law: the politics of courts in authoritarian regimes. Cambridge: Cambridge University Press, pp. 73–101.

25.

Ward

Greenhill

Bakke

(2010) The perils of policy by p-value: Predicting civil conflicts. Journal of Peace Research 47(4): 1–13.