Many social science studies are based on coded in-depth semistructured interview transcripts. But researchers rarely report or discuss coding reliability in this work. Nor is there much literature on the subject for this type of data. This article presents a procedure for developing coding schemes for such data. It involves standardizing the units of text on which coders work and then improving the coding scheme’s discriminant capability (i.e., reducing coding errors) to an acceptable point as indicated by measures of either intercoder reliability or intercoder agreement. This approach is especially useful for situations where a single knowledgeable coder will code all the transcripts once the coding scheme has been established. This approach can also be used with other types of qualitative data and in other circumstances.
ArmstrongDavidGoslingAnnWeinmanJoshMartaeuTheresa. 1997. “The Place of Inter-rater Reliability in Qualitative Research: An Empirical Study.”Sociology31:597–607.
2.
BeckerHoward. 1970. “Problems of Inference in Participant Observation.” Pp. 189–201 in Qualitative Methodology, edited by FilsteadWilliam. Chicago, IL: Rand McNally.
3.
BernardH. Russell. 2000. Social Research Methods: Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage.
4.
BurlaLailaKnierimBirteBarthJürgenLiewaldKatharinaDuetzMargreetAbelThomas. 2008. “From Text to Codings: Inter-coder Reliability Assessment in Qualitative Content Analysis.”Nursing Research57:113–17.
5.
CalarcoJessica. 2011. “‘I Need Help!’ Social Class and Children’s Help-seeking in Elementary School.”American Sociological Review76:862–82.
6.
CampbellJohn L.PedersenOve K.. 2014. The National Origins of Policy Ideas: Knowledge Regimes in the United States, France, Germany and Denmark. Princeton, NJ: Princeton University Press.
7.
DeSouceyMichaela. 2010. “Gastronationalism: Food Traditions and Authenticity Politics in the European Union.”American Sociological Review75:432–55.
8.
DeutscherIrwin. 1970. “Looking Backward: Case Studies on the Progress of Methodology in Sociological Research.” Pp. 202–16 in Qualitative Methodology, edited by FilsteadWilliam. Chicago, IL: Rand McNally.
9.
DunnGraham. 1989. Design and Analysis of Reliability Studies: The Statistical Evaluation of Measurement Errors. New York: Oxford University Press.
10.
EdinKathrynKefalasMaria. 2005. Promises I Can Keep: Why Poor Women Put Motherhood Before Marriage. Berkeley, CA: University of California Press.
11.
EdinKathrynLeinLaura. 1997. Making Ends Meet: How Single Mothers Survive Welfare and Low-wage Work. New York: Russell Sage Foundation.
12.
EngelMimi. 2007. “Mixing Methods: Reliability and Validity across Quantitative and Qualitative Measures of Relationship Quality.” Pp. 255–76 in Unmarried Couples with Children, edited by EnglandPaulaEdinKathryn. New York: Russell Sage Foundation.
13.
EnglandPaulaEdinKathryn. 2007. “Unmarried Couples with Children: Hoping for Love and the White Picket Fence.” Pp. 3–21 in Unmarried Couples with Children, edited by EnglandPaulaEdinKathryn. New York: Russell Sage Foundation.
HruschkaDanielSchwartzDeborahJohnDaphne Cobb St.Picone-DecaroErinJenkinsRichardCareyJames. 2004. “Reliability in Coding Open-ended Data: Lessons Learned from HIV Behavioral Research.”Field Methods16:307–31.
23.
KrippendorffKlaus. 1995. “On the Reliability of Unitizing Continuous Data.”Sociological Methodology25:47–76.
24.
KrippendorffKlaus. 2004. Content Analysis: An Introduction to Its Methodology. 2nd ed. Thousand Oaks, CA: Sage.
25.
KurasakiKaren S.2000. “Intercoder Reliability from Validating Conclusions Drawn from Open-ended Interview Data.”Field Methods12:179–94.
26.
LewinsAnnSilverChristina. 2007. Using Software in Qualitative Research. Los Angeles, CA: Sage.
27.
LombardMatthewSnyder-DuchJenniferBrackenCheryl. 2002. “Content Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability.”Human Communication Research28:587–604.
McCabeJanice. 2011. “Doing Multiculturalism: An Interactionist Analysis of the Practices of a Multicultural Sorority.”Journal of Contemporary Ethnography40:521–59.
30.
MilesMatthew B.Michael HubermanA.. 1984. Qualitative Data Analysis: A Sourcebook of New Methods. Beverly Hills, CA: Sage.
31.
MorrisseyElizabeth R.1974. “Sources of Error in the Coding of Questionnaire Data.”Sociological Methods and Research3:209–32.
32.
PoppingRoel. 1992. “In Search of One Set of Categories.”Quality and Quantity26:147–55.
33.
PoppingRoel. 2010. “Some Views on Agreement To Be Used in Content Analysis.”Quality and Quantity44:1067–78.
34.
RiffeDanielFreitagAlan. 1997. “A Content Analysis of Content Analysis: Twenty-five Years of Journalism Quarterly.”Journalism and Mass Communications Quarterly74:873–82.
35.
RosenthalRobert. 1987. Judgment Studies: Design, Analysis, and Meta-Analysis. New York: Cambridge University Press.
36.
RubinHerbert J.RubinIrene S.. 2005. Qualitative Interviewing: The Art of Hearing Data. 2nd ed. Thousand Oaks, CA: Sage.
37.
StrangDavid. 2010. Learning by Example: Imitation and Innovation at a Global Bank. Princeton, NJ: Princeton University Press.
UseemMichael. 1984. The Inner Circle: Large Corporations and the Rise of Business Political Activity in the U.S. and U.K. New York: Oxford University Press.
WilsonWilliam Julius. 1996. When Work Disappears: The World of the New Urban Poor. New York: Alfred A. Knopf.
42.
ZelditchMorris. 1970. “Some Methodological Problems of Field Studies.” Pp. 217–34 in Qualitative Methodology, edited by FilsteadWilliam. Chicago, IL: Rand McNally.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
