This article presents a computer content analysis approach to the problem of assessing the validity of Likert scales used in survey research. The wording of questions that make up a scale communicates meaning to the respondent in order to acquire relevant responses that, when combined, serve to measure some concept. Traditionally, beyond informal judgments of face validity, investigators examine responses of a sample of subjects to scale questions in order to determine if expected relationships and structure are found. The content analysis procedure proposed here directly examines the wording of scale questions to see if expected meanings, relationships, and structure are found in order to assess the validity of a scale. Possibilities of this approach are illustrated using two previously analyzed sets of scales: alienation and perception of police and crime.
Blalock, H.M. (1982). Conceptualization and measurement in the social sciencesBeverly Hills, CA: Sage .
2.
Bradburn, N.M., & Caplovitz, D. (1965). Reports on happinessChicago : Aldine.
3.
Cleveland, C.E., McTavish, D., & Pirro, E.B. (1974, September 5-13). Contextual content analysis Proceedings of the ISSC/CISS Workshop on Content Analysis m the Social Sciences, a conference sponsored by the Standing Committee on Social Science Data of the International Social Science Council, UNESCO, Centro Nazionale Universitano del Colcolo Electronico (CUNCE), Pisa, Italy.
4.
Dillman, D.A. (1978). Mail and telephone surveys: The total design methodNew York: John Wiley.
5.
Furnham, A., & Henderson, M. (1982). A content analysis of four personality inventoriesJournal of Clinical Psychology, 38(4), 818-825.
6.
Kerlinger, F.N. (1973). Foundations of behavioral research (2nd ed.). New York: Holt, Rinehart and Winston.
7.
Krippendorff, K. (1980). Content analysis: An introduction to its methodologyBeverly Hills, CA: Sage.
8.
Kruskal, J.B., & Wish, M. (1978). Multidimensional scaling (Sage University Paper no. 11). Beverly Hills, CA: Sage .
9.
Labaw, P. (1980). Advanced questionnaire designCambridge, MA: Abt Books.
10.
Litkowski, K.C. (1997a). Category development based on semantic principlesSocial Science Computer Review, 15(4), 394-409. (electronic manuscript available at ken@clres . com)
11.
Litkowski, K.C. (1997b). Desiderata for tagging with WordNet synsets or MCCA categories (4th Meeting of the ACL Special Interest Group on the Lexicon). Washington, DC: Association for Computational Linguistics. (electronic manuscript available at ken@clres. com)
12.
Markoff, J., Shapiro, G., & Weitman, S.R. (1975). Toward the integration of content analysis and general methodology In D. R. Heise (Ed.), Sociological methodology.San Francisco, CA: Jossey-Bass .
13.
Marsh, R.M. (1967). Comparative sociology: A codification of cross-societal analysisNew York: Harcourt, Brace and World.
14.
McTavish, D.G., Litkowski, K.C., & Schrader, S. (1997). A computer content analysis approach to measuring social distance in residential organizations for older peopleSocial Science Computer Review, 15(2), 170-180.
Neal, A.G., & Groat, H.T. (1974). Social class correlates of stability and change in levels of alienation: A longitudinal studySociological Quarterly, 15, 548-558.
17.
Osgood, C.E. (1967). Cross cultural comparability in attitude measurement via multilingual semantic differentials In M. Fishbein (Ed.), Readings in attitude theory and measurement.New York : John Wiley.
18.
Pierce, J.L., McTavish, D.G., & Knudsen, K.R. (1986). The measurement of job characteristics: A content and contextual analytic look at scale validityJournal of Occupational Behavior, 7, 299-313.
19.
Rajecki, D.W., and McTavish, D.G. (1995). Content analysis of attitude scale items from the theory of reasoned action and the tripartite (ABC) model Unpublished manuscript.
20.
Sartori, G. (Ed.). (1984). Social science concepts. Systematic analysis.Beverly Hills, CA: Sage .
21.
Weber, R.P. (1983). Measurement models for content analysisQuality and Quantity, 17 , 127-149.
22.
Weitzman, E.A., & Miles, M.B. (1995). A software sourcebook: Computer programs for qualitative data analysisThousand Oaks, CA: Sage.
23.
Woodworth, D.L. (1982). A content analytic study of religious meaning and life satisfaction Unpublished doctoral dissertation, University of Minnesota.
24.
Zeller, R.A., & Carmines, E.G. (1980). Measurement in the social sciences: The link between theory and dataCambridge, UK: Cambridge University Press.
25.
Zeller, R.A., Neal, A.G., & Groat, H.T. (1980). On the reliability and stability of alienation measures: Alongitudinal analysisSocial Forces , 58(4), 1195-1204.
