Advisingfunders of applied social science on how to optimize the validity ofsuch research
requires an implicit or explicit "sociology of scientific validity. " While such a sociology is
not yet formally available, this essay attempts to make recommendations upon such a
base. The specific target is preventive intervention research in mental health. Twelve of the
specific recommendations are directed toward creating a mutually-monitoring (reinforcing
and criticizing) applied scientific community. One section addresses the undesirable side
effects that monitoring by funders can produce. A third section of four parts addresses
conceptualizations of validity appropriate to applied social science.
Get full access to this article
View all access options for this article.
References
1.
Barnes, B. (1974) Scientific Knowledge and Sociological Theory. London: Routledge & Kegan Paul.
2.
Barnes, B. and D. Bloor (1982) "Relativism, rationality, and the sociology of knowledge," in M. Hollis and L. Lukes (eds.) Rationality and Relativism . Oxford: Blackwell.
3.
Barnow, B.S. (1973) "The effects of Head Start and socioeconomic status on cognitive development of disadvantaged children." Ph.D. dissertation, University of Wisconsin, Department of Economics. (unpublished)
4.
Becker, H.S. (1979) "Do photographers tell the truth?" pp. 99-117 in T. D. Cook and C. S. Reichardt (eds.) Qualitative and Quantitative Methods in Evaluation Research , Beverly Hills, CA: Sage.
5.
Bentler, P.M. and J.A. Woodward (1978) "A Head Start reevaluation: positive effects are not yet demonstrable." Evaluation Q.2: 493-510.
6.
Blau, P. (1955) The Dynamics of Bureaucracy. Chicago: Univ. of Chicago Press.
7.
Bloor, D. (1975) Knowledge and Social Imagery. London: Routledge & Kegan Paul.
8.
Boruch, R.F. (1978) Secondary Analysis. (New directions for program evaluation, No. 4. San Francisco: Jossey-Bass.
9.
Boruch, R.F. and J.S. Cecil (1979) Assuring Privacy and Confidentiality in Social Research. Philadelphia: Univ. of Pennsylvania Press.
10.
Boruch, R. F. and J. S. Cecil [eds.] (1982) Solutions to Ethical and Legal Problems in Social Research. New York: Academic.
11.
Boruch, R.F., D.M. Wortman, D.S. CORDRAY and Associates (1981) Reanalyzing Program Evaluations. San Francisco: Jossey-Bass.
12.
Campbell, D.T. (1957) "Factors relevant to the validity of experiments m social settings." Psych Bulletin54, 4: 297-312.
13.
Campbell, D.T. (1969a) "Reforms as experiments." Amer. Psychologist24: 409-429.
14.
Campbell, D.T. (1969b) "Perspective: artifact and control," pp. 351-382 in R. Rosenthal and Rosnow (eds.) Artifact in Behavior Research. New York: Academic.
15.
Campbell, D.T. (1969c) "Ethnocentrism of disciplines and the fish-scale model of omniscience," in M. Sherif and C. W. Sherif (eds.) Interdisciplinary Relationships in the Social Sciences. Hawthorne, NY: Aldine.
16.
Campbell, D.T. (1971) "Comments on the comments by Shaver and Staines ." Urban Affairs Q.7, 2: 187-192.
17.
Campbell, D.T. (1978) "Qualitative knowing in action research," pp. 184-209 in M. Brenner, P. Marsh, and M. Brenner (eds.) The Social Contexts of Method. London: Croom Helm.
18.
Campbell, D.T. (1979a) "Assessing the impact of planned social change ." Evaluation and Program Planning2: 67-90.
19.
Campbell, D.T. (1979b) "A tribal model of the social system vehicle carrying scientific knowledge." Knowledge: Creation, Diffusion, Utilization2: 181-201.
20.
Campbell, D.T. (1981) "ERISS Conference." 4S Newsletter (Society for the Social Studies of Science)6: 24-25.
21.
Campbell, D.T. (1984) "Can we be scientific in applied social science? " pp. 26-48 in R. F. Conner, D. G. Altman, and C. Jackson (eds.) Evaluation Studies Review Annual, Vol. 9. Beverly Hills, CA: Sage.
22.
Campbell, D.T. (1986a) "Science's social system of validity-enhancing collective belief change and the problems of the social sciences," in D. W. Fiske and R. A. Shweder (eds.) Metatheory in Social Science: Pluralisms and Subjectivities. Chicago: Univ. of Chicago Press.
23.
Campbell, D.T. (1986b) "Science policy from a naturalistic sociological epistemology," in PSA 1984, Vol. 2. Philosophy of Science Association.
24.
Campbell, D.T. (1986c) "Relabeling internal and external validity for applied social scientists," pp. 67-77 in W.M.K. Trochim (ed.) Advances in Quasi-Expenmental Design and Analysis . San Francisco: Jossey-Bass.
25.
Campbell, D.T., R.F. Boruch, R.D. Schwartz, and J. Steinberg (1977) "Confidentiality-preserving modes of access to files and to interfile exchange for useful statistical analysis." Evaluation Q.1, 2: 269-299.
26.
Campbell, D.T. and J.S. Cecil (1982) "A proposed system of regulation for the protection of participants in low-risk areas of applied social research," pp. 96-121 in J. E. Sieber (ed.) The Ethics of Social Research: Fieldwork, Regulation and Publication. New York: Springer-Verlag.
27.
Campbell, D.T. and D.W. Fiske (1959) "Convergent and discriminant validation by the multitrait-multimethod matrix." Psych. Bulletin65: 81-105.
28.
Campbell, D.T. and A.J. Kimmel (1985) "Guiding preventive intervention research centers for research validity." Final report (contract no. SSN 552-12-4531, order no. 83MO54244901D) issued by DHHS, PHS, DMM, Room 5-101, 5600 Fishers Lane, Rockville, MD 20857.
29.
Campbell, D.T. and R.A. LeVine (1970) "Field-manual anthropology," in R. Naroll and R. Cohen (eds.) A Handbook of Method in Cultural Anthropology. New York: Natural History Press/Doubleday.
30.
Campbell, D.T. and J. Stanley (1966) Experimental and Quasi-expenmental Designs for Research. Boston: Houghton Mifflin .
31.
Coleman, J.S. et al. (1966) Equality of Educational Opportunity. Washington, DC: Office of Education, U.S. Department of Health, Education and Welfare.
32.
Collins, H. (1981) "Stages in the empirical programme of relativism ." Social Studies of Sci.11: 3-10.
33.
Collins, H. (1985) Changing Order: Replication and Induction in Scientific Practice. London: Sage.
34.
Cook, T.D., H. Appleton, R. Conner, A. Shaffer, G. Tamkin, and S.J. Weber (1975) Sesame Street Revisited: A Case Study in Evaluation Research. New York: Russell Sage Foundation.
35.
Cook, T.D. and D.T. Campbell (1979) Quasi-Experimentation: Design and Analysis for Field Settings. Boston: Houghton Mifflin.
36.
Cook, T.D. and L.C. Leviton (1980) "Reviewing the literature: a comparison of traditional methods with meta-analysis." J. of Personality48: 449-472.
37.
Cook, T.D. and C.S. Reichardt (1979) Qualitative and Quantitative Methods in Evaluation Research. Beverly Hills, CA: Sage .
38.
Devine, E.C. and T.D. Cook (1983) "Effects of psycho-educational interventions on length of hospital stay: a meta-analytic review of 34 studies," in R. J. Light (ed.) Evaluation Studies Review Annual, Vol. 8. Beverly Hills, CA: Sage.
39.
Dirsmith, M.W. and S.F. Jablonsky (1978) "The pattern of PPB rejection: Something about organizations, something about PPB." Accounting, Organizations and Society3, 3-4.
40.
Fleck, L. (1979) Genesis and Development of a Scientific Fact. Chicago: Univ. of Chicago Press.
41.
Franklin, A.D. (1981) "Millikan's published and unpublished data on oil drops." Historical Studies in the Physical Sciences, 11, 2: 185-201.
42.
Ginsberg, P.E. (1984) "The dysfunctional side effects of quantitative indicator production: illustrations from mental health care." Evaluation and Program Planning7.
43.
Glass, G.V., B. McGaw, and M.L. Smith (1981) Meta-Analysis of Social Research. Beverly Hills, CA: Sage.
44.
Goodman, N. (1972) "Likeness," pp. 421-447 in Problems & Projects. Indianapolis, IN: Bobbs-Merrill .
45.
Habermas, J. (1970a) "On systematically distorted communication." Inquiry13: 205-218.
46.
Habermas, J. (1970b) "Toward a theory of communicative competence." Inquiry13: 360-375.
47.
Holton, G. (1978) "Subelectrons, presuppositions, and the Millikan-Ehrenhaft dispute," pp. 25-83 in The Scientific Imagination: Case Studies. Cambridge: Cambridge Univ. Press.
48.
Kershaw, D. and J. Fair (1976) The New Jersey Income-Maintenance Experiment. Operations, Surveys, and Administration, Vol. 1. New York: Academic.
49.
Knorr-Cetina, K.D. (1971) The Manufacture of Knowledge: An Essay on the Constructivist and Contextual Nature of Science. Oxford : Pergamon.
50.
Lamberth, J. & Kimmel, A.J. (1981) "Ethical issues and responsibilities in applying scientific behavioral knowledge," in A. J. Kimmel (ed.) New Directions for Methodology of Social and Behavioral Science: Ethics of Human Subjects Research. San Francisco: Jossey-Bass.
51.
Latour, B. & S. Woolgar (1979) Laboratory Life: The Social Construction of Scientific Facts. Beverly Hills, CA: Sage.
52.
Light, R.J. and D.B. Pillemer (1984) Summing Up. The Science of Reviewing Research . Cambridge, MA: Harvard Univ. Press .
53.
Magidson, J. (1977) "Toward a causal model approach for adjusting for preexisting differences in the nonequivalent control group situation: A general alternative to ANCOVA." Evaluation Q.1: 399-420.
54.
Merton, R.K. (1973) The Sociology of Science. ( Norman W. Storer, ed.) Chicago: Univ. of Chicago Press.
55.
Orne, M.T. (1969) "Demand characteristics and the concept of quasi-controls," in R. Rosenthal and R. L. Rosnow (eds.) Artifact in Behavioral Research. New York: Academic.
56.
Peltz, D.C. and F.M. Andrews (1966) Scientists in Organizations. New York: Wiley.
57.
Pickering, A. (1984) Constructing Quarks: A Sociological History of Particle Physics. Edinburgh: Edinburgh Univ. Press.
58.
Polanyi, M. (1958) Personal Knowledge: Toward a Post-Critical Philosophy . London: Routledge & Kegan Paul.
59.
Polanyi, M. (1966) "A society of explorers," chap. 3 in The Tacit Dimension. Garden City, NY: Doubleday.
60.
Polanyi, M. (1969) Knowing and Being, Chaps. 4, 5, 6, 13. London: Routledge & Kegan Paul.
61.
Popper, K.R. (1952) The Open Society and Its Enemies, Vol. 2. London: Routledge & Kegan Paul.
62.
Prakhash, P. and A. Rappaport (1977) "Information inductance and its significance for accounting." Accounting, Organizations and Society2: 29-38.
63.
Quine, W.V. (1969) Ontological Relativity and Other Essays. New York: Columbia Univ. Press.
64.
Ravetz, J.R. (1971) Scientific Knowledge and Its Social Problems. Oxford: Clarendon.
65.
Ridgeway, V. (1956) "The dysfunctional consequences of performance measurements." Admin. Sci. Q.1: 240-247.
66.
Shaver, P. & Staines, G. (1971) "Problems facing Campbell's experimenting society ." Urban Affairs Q.7, 2: 173-186.
67.
Smith, M.S. and J.S. Bissell (1970) "Report analysis: The impact of Head Start." Harvard Educational Rev.40: 51-104.
68.
Trochim, W. (1982) "Methodologically based discrepancies in compensatory education evaluations." Evaluation Rev.6: 443-480.
69.
Watts, H. W. and A. Rees (eds.) (1977) The New Jersey Income-Maintenance Experiment, Vols. II and III. New York: Academic Press.
70.
Yeaton, W. H. and P. M. Wortman (eds.) (1984) "Issues in data synthesis." New Directions for Program Evaluation, Vol. 24. San Francisco: Jossey-Bass.
