Recent developments in the history, philosophy, psychology, and sociology of science
raise serious challenges to our traditional notions about the decisive power of experiments
in the development of scientific knowledge. These developments suggest that the power
of an experiment is only as strong as the clarity af the basic assumptions which underlie it.
Such assumptions not only underlie laboratory experimentation but social evaluation
research as well. A dialectical methodology is proposedfor assessing the influence of key
assumptions in both settings. Among other conclusions, analysis of the role and influence
of key assumptions suggests an additional source of experimental error, termed the error
of the third kind, or E III. E III is defined and discussed as the probability af conducting
the "wrong" experiment when one should have conducted the "right" experiment.
Get full access to this article
View all access options for this article.
References
1.
Argyris, C. (1970) Intervention Theory and Method: A Behavioral Science View. Reading, MA: Addison-Wesley .
2.
Barber, T.X. and M.J. Silver (1968) "Pitfalls in data analysis and interpretation: a reply to Rosenthal." Psych. Bull. Monograph Supplements Part2 (December): 48-62.
3.
Bonoma, T.V. (1977) "Overcoming resistances to changes recommended in operating programs." Professional Psychology8 (November): 451-463.
4.
_______ (1976) "Social evaluation and social psychology." Representative Research in Social Psychology7: 147-159.
5.
_______ (1975) "A new methodology for the study of choice." J. for the Theory of Social Behavior5: 49-62.
6.
_______ and B.R. Schlenker (1978) "The SEU calculus: effects of response mode, sex and sex role on uncertain decision." Decision Sciences (April).
7.
Brush, S. (1974) "Must the history of science be rated X?" Sci.183, 4130.
8.
Campbell, D.T. (1957) "Factors relevant to the validity of experiments in social settings." Psych. Bull.54: 297-312.
9.
_______ and J.C. Stanley (1963) Experimental and Quasi-Experimental Designs for Research. Chicago: Rand-McNally.
10.
Charlesworth, W.R. (1973) "Ethology's contribution to a framework for relevant research." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal , August 27-31.
11.
Chomsky, N. (1966) Cartesian Linguistics. New York : Harper & Row.
12.
_______ (1957) Syntactic Structures. The Hague : Mouton.
13.
Churchman, C.W. (1971) The Design of Inquiring Systems. New York: Basic Books.
14.
Cook, D.L. (1966) "Program evaluation and review technique (PERT ." Office of Education Cooperative Monograph No. 17.
15.
David, E.E., Jr. (1971) "Making objectively credible and acceptable ." Presented at the Seventy-Ninth Annual Meetings of the American Psychological Association, Washington, DC, September.
16.
David, M. (1971) "That's interesting: towards a phenomenology of sociology and a sociology of phenomenology." Philosophy of the Social Services1: 309-344.
17.
DuBois, P. H. and G. D. Mayo [eds.] (1970) "Research strategies for evaluating training." AERA Monograph Series on Curriculum Evaluation, Vol. 4. Chicago: Rand McNally.
18.
Duhem, P. (1914) The Aim and Structure of Physical Theory. New York: Atheneum.
19.
Evans, J.W. (1972) "Evaluating social action programs," pp. 629-639 in G. Zaltman, P. Kotler, and I. Kaufman (eds.) Creating Social Change. New York: Holt, Rinehart & Winston.
20.
Feyerabend, P. (1975) Against Method, Outline of an Anarchistic Theory of Knowledge. London: Humanities Press.
21.
Gergen, K. (1973) "Social psychology as history." J. of Personality and Social Psychology26: 309-320.
22.
Grinker, R. R., Sr. [ed.] (1967) Toward a Unified Theory of Human Behavior. New York: Basic Books.
23.
Grobman, H. (1970) "Evaluation activities of curriculum projects." AERA Monograph Series on Curriculum Evaluation, Vol. 2. Chicago: Rand-McNally .
24.
Guttentag, M. (1973) "Subjectivity and its use in evaluation research " Presented at the annual meeting of the Society for Psychotherapy Research, Philadelphia, June.
25.
Hanson, N.R. (1965) Patterns of Discovery. Cambridge, EnglandCambridge Univ. Press.
26.
Helmreich, R. (1975) "Social psychology: the unfulfilled promise " Personality and Social Bull.4: 548-560.
27.
Holton, G. (1973) Thematic Origins of Scientific Thought. Cambridge: Harvard Univ. Press.
28.
Hull, C. (1935) "The conflicting psycholgies of learning-a way out." Psych. Rev.42: 491-516.
29.
Kaplan, A. (1964) The Conduct of Inquiry. San Francisco: Chandler.
30.
Koch, S. (1971) "Reflections on the state of psychology." Social Research38: 661-709.
31.
Koen, F.M. (1973) "The evaluation of training programs for college teachers." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal , August 27-31.
32.
Krause, M.S. and K.T. Howard (forthcoming) "Program evaluation in the public interest: a new research methodology." Community Mental Health.
33.
Kuhn, T. (1967) The Structure of Scientific Revolutions. Chicago: Univ. of Chicago Press.
34.
Laudan, L. (1965) "On the impossibility of crucial falsifying experiments: Grunbaum on 'The Duhemian Argument'." Philosophy of Sci.32: 295-299.
35.
Levine, M. (1974) "Scientific method and the adversary model, some preliminary thoughts." Amer. Psychologist9: 661-667.
36.
Lewin, K. (1951) Field Theory in Social Science. New York: Harper & Row.
37.
Marx, M.H. and W.A. Hillix (1963) Systems and theories in psychology. New York: McGraw-Hill.
38.
Mason, R.O. (1969) "A dialectical approach to strategic planning." Management Sci. 15: B403-B414.
Mitroff, I.I. (1974a) "Norms and counter-norms in a select group of the Apollo moon scientists: a case study of the ambivalence of scientists ." Amer. Soc. Rev.39: 579-595.
41.
_______ (1974b) The Subjective Side of Science, a Philosophical Inquiry into the Psychology of the Apollo Moon Scientists. Amsterdam: Elsever.
42.
_______ (1973) "Systems, inquiry, and the meanings of falsification ." Philosophy of Sci.40: 255-276.
43.
_______ (1971) "A communication model of dialectical inquirying systems." Management Sci.17: 634-648
44.
_______ and T.R. Featheringham (1974) "On systematic problem solving and the error of the third kind." Behavioral Sci.19: 383-393.
45.
Mitroff, I.I. and I. Fitzgerald (1977) "On the psychology of the Apollo moon scientists: a chapter in the psychology of science." Human Relations30: 651-674.
46.
Mitroff, I.I., J. Nelson, and K. Mason (1974) "On management myth-information systems." Management Sci.21: 371-382.
47.
Mitroff, I.I. and M. Turoff (1974) "On measuring the conceptual errors in large-scale social experiments." J. of Technological Forecasting and Social Change6389-402.
48.
Moores, D.F. (1973) "Moving research across the relevancy continuum ." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal, August 27-31.
49.
Newell, A. (1973) "You can't play 20 questions with nature and win," in W. G. Chase (ed.) Visual Information Processing. New York: Academic Press.
50.
Popper, K.R. (1965) The Logic of Scientific Discovery. New York: Harper & Row.
51.
_______ (1962) Conjectures and Refutations, the Growth of Scientific Knowledge. New York: Basic Books .
52.
Proshansky, H.M. (1974) "The environmental crisis in human dignity." J. of Social Issues29: 1-20.
53.
Quinn, P. (1969) "The status of the D-thesis." Philosophy of Sci.36: 381-399.
54.
Raiffa, H. (1968) Decision Analysis. Reading, MA: Addison-Wesley.
55.
Rossi, P.H. (1977) "Testing for success and failure in social action," in P. H. Rossi and W. Williams (eds.) Evaluating Social Programs . New York: Seminar Press.
56.
Roston, R.A. (1972) "Moral certitude and 'technical validities ' " Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal, August 27-31.
57.
Sagasti, F. and I.I. Mitroff (1973) "Operations research from the viewpoint of general systems theory." Omega1: 695-710.
58.
Samuels, S.J. (1973) "How psychologists can be relevant or you can have your cake and eat it too." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal, August 27-31.
59.
Schlenker, B.R. (1974) "social psychology and science." J. of Personality and Social Psychology29: 1-15.
60.
Scriven, M. (1967) "The methodology of evaluation," in R. Ryler, R. Gagne, and M. Scriven (eds.) Perspectives of Curriculum Evaluation. AREA Monograph Series on Curriculum Evaluation, Vol. 1. Chicago: Rand McNally.
61.
Sechrest, L. (1973) "Training in evaluation research. the development of a program." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal , August 27-31.
62.
Streuning, E. L. and M. Guttentag [eds.] (1975) Handbook of Social Evaluation . Beverly Hills, CA: Sage.
63.
Taylor, J.B. (1973) "Is evaluation sufficient: a possible future." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal, August 27-31.
64.
Webb, E.J., D.T. Campbell, R.D. Schwartz, and L. Sechrest (1966) Unobtrusive Measures: Nonreactive Research in the Social Sciences. Chicago: Rand-McNally .
65.
Wedeking, G. (1969) "Duhem, Qumn, and Grunbaum on falsification." Philosophy of Sci.36: 375-380.
66.
Weiss, C.H. (1973) "Evaluation research in the political context." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal, August 27-31.
Westfall, R.S. (1973) "Newton and the fudge factor." Sci.179: 751-756.
69.
Wolfe, A. (1971) "The myth of the free scholar," pp. 64-67 in R. B. Buckout et al. (eds.) Toward Social Change. New York: Harper & Row.
70.
Wortman, P. (1975) "Evaluation research: a psychological perspective ." Amer Psychologist30: 562-575.
71.
Wozniak, R.H. (1973) "In-context research on children's learning as a basic science prophylatic: or true purity doesn't need to wash." Presented at the Eighty-First Annual Meetings of the American Psychological Association, Montreal, August 27-31.
