A comprehensive and integrated framework for the analysis of data is offered and used to assess data sets on democracy. The framework first distinguishes among three challenges that are sequentially addressed: conceptualization, measurement, and aggregation. In turn, it specifies distinct tasks associated with these challenges and the standards of assessment that pertain to each task. This framework is applied to the data sets on democracy most frequently used in current statistical research, generating a systematic evaluation of these data sets. The authors’ conclusion is that constructors of democracy indices tend to be quite self-conscious about methodological issues but that even the best indices suffer from important weaknesses. More constructively, the article’s assessment of existing data sets on democracy identifies distinct areas in which attempts to improve the quality of data on democracy might fruitfully be focused.
Get full access to this article
View all access options for this article.
References
1.
Adcock, Robert & Collier, David. (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Review, 95(3), 529-546.
2.
Alvarez, Michael , Cheibub, José Antonio, Limongi, Fernando, & Przeworski, Adam. (1996). Classifying political regimes. Studies in Comparative International Development, 31(2), 1-37.
3.
Arat, Zehra F. (1991). Democracy and human rights in developing countries. Boulder, CO: Lynne Rienner.
4.
Bollen, Kenneth A. (1980). Issues in the comparative measurement of political democracy. American Sociological Review, 45(2), 370-390.
5.
Bollen, Kenneth A. (1986). Political rights and political liberties in nations: An evaluation of human rights measures, 1950 to 1984. Human Rights Quarterly, 8(4), 567-591.
6.
Bollen, Kenneth A. (1989). Structural equations with latent variables. New York: John Wiley.
7.
Bollen, Kenneth A. (1991). Political democracy: conceptual and measurement traps. In Alex Inkeles (Ed.), On measuring democracy: Its consequences and concomitants (p. 3-20). New Brunswick, NJ: Transaction.
8.
Bollen, Kenneth A. (1993). Liberal democracy: Validity and method factors in cross-national measures. American Journal of Political Science, 37(4), 1207-1230.
9.
1998. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2001. Retrieved from http://www.icpsr.umich.edu:8080/ABSTRACTS/02532.xml?format=ICPSR
10.
Bollen, Kenneth A. , & Paxton, Pamela. (2000). Subjective measures of liberal democracy. Comparative Political Studies, 33(1), 58-86.
11.
Carmines, Edward G. , & Zeller, Richard A. (1979). Reliability and validity assessment. Beverly Hills, CA: Sage.
12.
Collier, David , & Adcock, Robert. (1999). Democracy and dichotomies: a pragmatic approach to choices about concepts. Annual Review of Political Science, 2, 537-565.
13.
Coppedge, Michael . (1997). Modernization and thresholds of democracy: evidence for a common path and process. In Manus I. Midlarsky (Ed.), Inequality, democracy, and economic development (p. 177-201). New York: Cambridge University Press.
14.
Coppedge, Michael . (1999). Thickening thin concepts and theories: combining large N and small in comparative politics. Comparative Politics, 31(4), 465-476.
15.
Coppedge, Michael , & Reinicke, Wolfgang H. (1991). Measuring polyarchy. In Alex Inkeles (Ed.), On measuring democracy: Its consequences and concomitants (p. 47-68). New Brunswick, NJ: Transaction.
16.
Dahl, Robert . (1971). Polyarchy. New Haven, CT: Yale University Press.
17.
Elkins, Zachary . (2000). Gradations of democracy? Empirical tests of alternative conceptualizations. American Journal of Political Science, 44(2), 287-294.
18.
Elklit, Jørgen . (1994). Is the degree of electoral democracy measurable? Experiences from Bulgaria, Kenya, Latvia, Mongolia and Nepal. In David Beetham (Ed.), Defining and measuring democracy (p. 89-111). Thousand Oaks, CA: Sage.
19.
Foweraker, Joe , & Krznaric, Roman. (2000). Measuring liberal democratic performance: An empirical and conceptual critique. Political Studies, 48(4), 759-787.
20.
Freedom House . (2000). Annual survey of freedom country scores, 1972-73 to 1999-00. Retrieved from http://freedomhouse.org/ratings/index.htm
21.
Gasiorowski, Mark J. (1996). An overview of the political regime change dataset. Comparative Political Studies, 29(4), 469-483.
22.
Gastil, Raymond D. (Ed.). (1978). Freedom in the world: Political rights and civil liberties, 1978. Boston: G. K. Hall.
23.
Gastil, Raymond D. (1991). The comparative survey of freedom: Experiences and suggestions. In Alex Inkeles (Ed.), On measuring democracy: Its consequences and concomitants (pp. 21-46). New Brunswick, NJ: Transaction.
24.
Gehrlich, Peter . (1973). The institutionalization of European parliaments. In Allan Kornberg (Ed.), European parliaments in comparative perspective (pp. 94-113). New York: D. McKay.
25.
Gifi, Albert . (1990). Nonlinear multidimensional analysis. New York: John Wiley.
26.
Gleditsch, Kristian S. , & Ward, Michael D. (1997). Double take: A reexamination of democracy and autocracy in modern polities. Journal of Conflict Resolution, 41(3), 361-383.
27.
Gurr, Ted Robert , Jaggers, Keith, & Moore, Will H. (1991). The transformation of the western state: The growth of democracy, autocracy, and state power since 1800. In Alex Inkeles (Ed.), On measuring democracy: Its consequences and concomitants (pp. 69-104). New Brunswick, NJ: Transaction.
28.
Guttman, Louis . (1977). What is not what in statistics. Statistician, 26(2), 81-107.
29.
Guttman, Louis . (1994). Louis Guttman on theory and methodology: Selected writings. Brookfield, VT: Dartmouth Publishing.
30.
Hadenius, Axel . (1992). Democracy and development. Cambridge, UK: Cambridge University Press.
31.
Jacoby, William G. (1991). Data theory and dimensional analysis. Newbury Park, CA: Sage.
32.
Jacoby, William G. (1999). Levels of measurement and political research: An optimistic view. American Journal of Political Science, 43(1), 271-301.
33.
Jaggers, Keith , & Gurr, Ted Robert. (1995). Tracking democracy’s third wave with the Polity III data. Journal of Peace Research, 32(4), 469-482.
34.
Kaplan, Abraham . (1964). The conduct of inquiry: Methodology for behavioral science. Scranton, PA: Chandler.
35.
Marshall, Monty G. , & Jaggers, Keith. (2001a). Polity IV project: Political regime characteristics and transitions, 1800-1999. Dataset users manual. Retrieved from http://www.bsos.umd.edu/cidcm/polity/
36.
Marshall, Monty G. , & Jaggers, Keith. (2001b). Polity IV project: Political regime characteristics and transitions, 1800-1999. The Polity IV dataset. Retrieved from http://www.bsos.umd.edu/cidcm/polity/
37.
Mokken, Robert J. (1971). A theory and procedure of scale analysis with applications in political research. Berlin, Germany: Walter de Gruyter.
38.
Przeworski, Adam , & Teune, Henry. (1970). The logic of comparative social inquiry. New York: John Wiley.
39.
Ryan, Joseph E. (1994). Survey methodology. Freedom Review, 25(1), 9-13.
40.
Vanhanen, Tatu . (1993). Construction and use of an index of democracy. In David G. Westendorff & Dharam Ghai (Eds.), Monitoring social progress in the 1990s: Data constraints, concerns and priorities (pp. 301-321). Aldershot, UK: UNRISD/Avebury.
41.
Vanhanen, Tatu . (1997). Prospects of democracy: A study of 172 countries. New York: Routledge.
42.
Vanhanen, Tatu . (2000a). A new dataset for measuring democracy, 1810-1998. Journal of Peace Research, 37(2), 251-265.
43.
Vanhanen, Tatu . (2000b). The polyarchy dataset: Vanhanen’s index of democracy. Retrieved from http://www.svt.ntnu.no/iss/data/vanhanen
