Ambiguous phrases are the bane of researchers' attempts to ensure acceptably high interrater agreement in the encoding of texts. When modal usages and their associated rationales are encoded as part of a text analysis, ambiguities arise in characteristic (and thus identifiable) ways. This article illustrates the typical sources of disagreement among coders involved in encoding data during a modality analysis and provides concrete strategies for improving interrater agreement.
Berelson, B.1952. Content analysis in communication research. New York: Free Press.
2.
Geertz, C.1975. Common sense as a cultural system. Antioch Review33 (1): 5-26.
3.
Holsti, O.R.1969. Content analysis for the social sciences and humanities . London: Addison Wesley.
4.
Horn, L.R.1989. A natural history of negation. Chicago : University of Chicago Press.
5.
Hruschka, D.J., D. Schwartz, D. Cobb St. John, E. Picone-Decaro, R.A. Jenkins, and J.W. Carey.2004. Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods16 (16): 307-31.
6.
Krippendorff, K.2004. Content analysis: An introduction to its methodology . Thousand Oaks, CA: Sage.
7.
Lombard, M., J. Snyder-Duch, and C.C. Bracken.2002. Content analysis in mass communication: Assessment and reporting of intercoder reliability. Human Communication Research28 (4): 587-604.
8.
Markoff, J., G. Shapiro, and S. Weitman.1974. Toward the integration of content analysis and general methodology. In Sociological methodology, 1975, ed. D. R. Heise, 1-58. San Francisco : Jossey-Bass.
9.
Popping, R.1984. AGREE, A package for computing nominal scale agreement . Computational Statistics and Data Analysis2 (2): 182-85.
10.
---. 2000. Computer-assisted text analysis. London : Sage.
11.
Roberts, C.W.1989. Other than counting words: A linguistic approach to content analysis. Social Forces68 (1): 147-77.
12.
---. 1997. Semantic text analysis: On the structure of linguistic ambiguity in ordinary discourse. In Text analysis for the social sciences: Methods for drawing statistical inferences from texts and transcripts , ed. C. W. Roberts, 55-77. Mahwah, NJ: Lawrence Erlbaum.
13.
---. 2008. "The" fifth modality: On languages that shape our motivations and cultures. Leiden, Netherlands: Brill.
14.
Roberts, C.W., R. Popping, and Y. Pan.Forthcoming. Modalities of democratic transformation: Forms of public discourse within Hungary's largest newspaper, 1990-1997. International Sociology.
15.
Roberts, C.W., C. Zuell, J. Landmann, and Y. Wang.2008. Modality analysis: A semantic grammar for imputations of intentionality in texts. Quality and Quantity . (Prepublished September 12, 2008; DOI: 10.1007/s11135-008-9194-7.)
16.
Scott, W.A.1955. Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly19 (3): 321-25.
17.
Shapiro, G., and J. Markoff.1998. Revolutionary demands: A content analysis of the cahiers de doléances of 1789. Stanford, CA: Stanford University Press.
18.
Swidler, A.1986. Culture in action: Symbols and strategies. American Sociological Review51 (2): 273-86.
19.
Van der Auwera, J.1996. Modality: The three-layered scalar square. Journal of Semantics13 (3): 181-95.
