Construct Invalidity Comes from Method Effects

Abstract

Method effects are the sources of the construct invalidity in social-behavioral measuring procedures. Some method effects come from inadequacies in the protocol-the sequence of behaviors that is selected and identified solely by the set of conditions under which it occurs. The protocol may be inappropriate to the construct as specified or it may be insufficient because its conditions restrict the range of possible manifestations of construct-relevant behavior. Other method effects are technique effects-errors of commission occurring when a measuring operation is applied to a protocol. These errors may come from the actor's reactions to being measured, from the data-producer, or from the instrument used by the data-producer. Although detecting the presence of method effects is relatively easy, identifying specific effects is more difficult. Such identification usually requires the rescoring or recoding of the data on attributes suspected of contributing systematic but unwanted effects. Method effects also can arise from interactions among such conceptually irrelevant variables or between one of them and the construct of interest. When investigators can understand and improve their methods, they can more adequately understand and interpret their empirical findings.

Get full access to this article

View all access options for this article.

References

Buss, D. M. and Craik, K. H. (1980). The frequency concept of disposition: Dominance and prototypical dominant acts. Journal of Personality, 48, 379-392.

Butt, D. S. and Fiske, D. W. (1968). Comparison of strategies in developing scales for dominance. Psychological Bulletin, 70, 505-519.

Byrne, D. and Holcomb, J. (1962). The reliability of a response measure: Differential recognition-threshold scores. Psychological Bulletin, 59, 70-73.

Cook, T. D. and Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.

Coombs, C. H. (1964). A theory of data. New York: Wiley.

Cooper, H. (1981). Ubiquitous halo. Psychological Bulletin, 90, 218-244.

Cronbach, L. J. and Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.

Duncan, S., Jr. and Fiske, D. W. (1977). Face-to-face interaction: Research, methods, and theory. Hillsdale, NJ: Erlbaum (J. Wiley).

Ericsson, K. A. and Simon, H. W. (1984). Protocol analysis: Verbal reports as data. Cambridge: MIT Press.

10.

Fiske, D. W. (1971). Measuring the concepts of personality. Chicago: Aldine.

11.

Fiske, D. W. (1978). Strategies for personality research: The observation versus interpretation of behavior. San Francisco: Jossey-Bass.

12.

Fiske, D. W. (Ed.) (1981). New directions for methodology of social and behavioral science: Problems with language imprecision (No. 9). San Francisco: Jossey-Bass.

13.

Fiske, D. W. (1982). Convergent and discriminant validation in measurement and research strategies. In D. Brinberg and L. Kidder (Eds.), New directions in methodology of social and behavioral science: Forms of validity in research (No. 12). San Francisco: Jossey-Bass.

14.

Fiske, D. W. (1983, April). Measuring to understand and under-standing measuring. Paper presented at Symposium in honor of S. B. Sells, Fort Worth, TX. (Conference volume to be published)

15.

Fiske, D. W. and Butler, J. M. (1963). The experimental conditions for measuring individual differences. Educational and Psychological Measurement, 23, 249-266.

16.

Fiske, D. W. and Shweder, R. A. (Eds.) (1986). Metatheory in social science: Pluralisms and subjectivities. Chicago: University of Chicago Press.

17.

Hess, E. H. (1975). The tell-tale eye: How your eyes reveal hidden thoughts and emotions. New York: Van Nostrand Reinhold.

18.

Hogarth, R. (Ed.) (1982). New directions for methodology of social and behavioral research: Question framing and response consistency (No. 11). San Francisco: Jossey-Bass.

19.

Jones, M. B. and Fennell, R. S., III. (1965). Runway performance in two strains of rats. Quarterly Journal of the Florida Academy of Science, 28(3), 289-296.

20.

Kelly, E. L. and Fiske, D. W. (1951). The prediction of performance in clinical psychology. Ann Arbor: University of Michigan Press. (Reprinted by Greenwood Press, New York, 1969)

21.

Kuncel, R. B. (1973). Response processes and relative location of subject and item. Educational and Psychological Measurement, 33, 545-563.

22.

Kuncel, R. B. and Fiske, D. W. (1974). Stability of response process and response. Educational and Psychological Measurement, 34, 743-755.

23.

McClintock, M. K. (1981). Simplicity from complexity: A naturalistic approach to behavior and endocrine function. In I. Silverman (Ed.), New directions for methodology of social and behavioral science: Generalizing from laboratory to life (No. 8). San Francisco: Jossey-Bass.

24.

Messick, S. (1984). The psychology of educational measurement. Princeton, NJ: Educational Testing Service Research Report RR-84-6.

25.

Nisbett, R. E. and Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231-259.

26.

Paul, G. L. (in press). Rational operations in residential treatment settings through ongoing assessment of client and staff functioning. In D. R. Peterson and D. B. Fishman (Eds.), Assessment for decision: Psychology in action (Vol. 1). Rutgers Symposia on Applications of Psychology.

27.

Sternberg, R. G. (1979). The nature of mental abilities. American Psychologist, 34, 214-230.

28.

Sudman, S. and Bradburn, N. M. (1982). Asking questions: A practical guide to questionnaire design. San Francisco: Jossey-Bass.

29.

Tversky, A. and Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453-458.

30.

Webb, E. J. , Campbell, D. T. , Schwartz, R. D. , and Sechrest, L. (1966). Unobtrusive measures: Nonreactive research in the social sciences. Chicago: Rand McNally.