Donald Campbell and his colleagues have elaborated an approach to research based on
the elimination of threats to four validity types: internal, external, construct, and
statistical conclusion. The central thesis of this article is that the process of eliminating
validity threats depends fundamentally on no-difference findings, a fact that, unfor
tunately, has not been made explicit by researchers. The implications of this neglect are
explored using examples from a number of different substantive areas such as psychology,
health, and medicine. Finally, the intrinsic role of no-difference findings is described in the
context of all four validity types, and suggestions for improving the process are offered.
Get full access to this article
View all access options for this article.
References
1.
Bancroft, T.A. and C.P. Han (1983) "A note on pooling variances." J. of the Amer. Statistical Assn.78: 981-983.
2.
Beard, T.C., H.M. Cooke, W.R. Gray, and R. Barge (1982) "Randomized controlled trial of a no-added-sodium diet for mild hypertension." Lancetii: 455-458.
3.
Beck, J.G., F. Andrasik, and J.G. Arena (1984) "Group comparison designs," in A. S. Bellack and M. Hersen (eds.) Research Methods in Clinical Psychology. New York: Pergamon.
4.
Blackwelder, W.C. (1981) "'Proving the null hypothesis' in clinical trials ." Controlled Clinical Trials3: 345-353.
5.
Byington, R.P., J.D. Curb, and M.E. Mattson (1985) "Assessment of double-blindness at the conclusion of the beta-blocker heart attack trial." J. of the Amer. Medical Assn. 253: 1733-1736.
6.
Campbell, D.T. and J.C. Stanley (1963) Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally .
7.
Carpenter, W.T., J.H. Sadler, P.D. Light, T.E. Hanlon, A.A. Kurland, M.W. Penna, W.P. Reed, E.H. Wilkinson, and J.J. Bartko (1983) "The therapeutic efficacy of hemodialysis in schizophrenia . " New England J. of Medicine308: 669-675.
8.
CASS Principal Investigators and their associates (1983) "Coronary Artery Surgery Study: a randomized trial of coronary artery bypass surgery. Quality of life in patients randomly assigned to treatment groups." Circulation68: 951-960.
9.
Chalmers, T.C., P. Celano, H.S. Sacks, and H. Smith (1983) "Bias in treatment assignment in controlled clinical trials." New England J. of Medicine309: 1358-1361.
10.
Charlson, M.E. and R.I. Horowitz (1984) "Applying results of randomized trials to clinical practice: impact of losses before randomization." British Medical J.289: 1281-1284.
11.
Cohen, J. (1977) Statistical Power Analysis for the Behavioral Sciences . New York: Academic Press.
12.
Cook, T.D. and D.T. Campbell (1979) Quasi-Experimentation: Design and Analysis Issues for Field Settings. Chicago: Rand McNally.
13.
Cronbach, L.J. and G.C. Gleser (1953) "Asessing similarity between profiles." Psych. Bull.50: 456-473.
14.
Dikmen, S., R.M. Reitan, and N.R. Temkin (1983) "Neuropsychological recovery in head injury." Archives of Neurology40: 333-338.
15.
Doherty, M. and P.A. Dieppe (1981) "Effect of intra-articular yttrium-90 on chronic pyrophosphate arthropathy of the knee." Lancetii: 1243-1246.
16.
E.P.S.I.M. Research Group (1982) " A controlled comparison of aspirin and oral anticoagulants in prevention of death after myocardial infarction." New England J. of Medicine307: 701-708.
17.
European Coronary Artery Surgery Study Group ( 1982) "Long-term results of prospective randomized study of coronary artery bypass surgery in stable angina pectoris." Lancetii: 1173-1180.
18.
Federation CECOS, D. Schwartz, and M.J. Mayaux (1982) " Female fecundity as a function of age. Results of artificial insemination in 2193 nulliparous women with azoospermic husbands." New England J. of Medicine306: 404-406.
19.
Fisher, B., M. Bauer, R. Margolese, R. Poisson, Y. Pilch, C. Redmond, E. Fisher, N. Wolmark, M. Deutsch, E. Montague, E. Saffer, L. Wickerham, H. Lerner, A. Glass, H. Shibata, P. Deckers, A. Ketcham, R. Dishi, and I. Russell (1985) "Five-year results of a randomized clinical trial comparing total mastectomy and segmental mastectomy with or without radiation in the treatment of breast cancer." New England J. of Medicine312: 665-673.
20.
Fisher, B., C. Redmond, E.R. Fisher, M. Bauer, N. Wolmark, D.L. Wickerham, M. Deutsch, E. Montague, R. Margolese, and R. Foster (1985) "Ten-year results of a randomized clinical trial comparing radical mastectomy and total mastectomy with or without radiation ." New England J. of Medicine312: 674-681.
21.
Gaw, A.C., W.C. Lennig, L.W. Chang, and L.C. Shaw (1975) "Efficacy of acupuncture on osteoarthritic pain. A controlled, double-blind study." New England J. of Medicine293: 375-378.
22.
Gerstenblith, G. et al. (1982) "Nifedipine in unstable angina. A double-blind, randomized study." New England J. of Medicine306: 885-889.
23.
Heath, C.W., M.R. Nadel, M.M. Zach, A.T.L. Chen, M.A. Bender, and J. Preston (1984) "Cytogenetic findings in persons living near the Love Canal." J. of the Amer. Medical Assn.251: 1437-1440.
24.
Hypertension Detection and Follow-up Program Cooperative Group (1979) "Five-year findings of the Hypertension Detection and Follow-up Program. I. Reduction in mortality of persons with high blood pressure, including mild hypertension." J. of the Amer. Medical Assn.242: 2562-2571.
25.
Lipid Research Clinics Program (1984) "The Lipid Research Clinics coronary primary prevention trial results. Reduction in incidence of coronary heart disease." J. of the Amer. Medical Assn.251: 351-364.
26.
Lipsey, M.W. (1983) "A scheme for assessing measurement sensitivity in program evaluation and other applied research." Psych. Bull.94: 152-165.
27.
Maccoby, E.E. and C.N. Jacklin (1974) The Psychology of Sex Differences. Stanford, CA: Stanford Univ. Press.
28.
Murphy, M.L., H.N. Hultgren, K. Detre, J. Thomsen, and T. Takaro (1977) "Special correspondence: a debate on coronary bypass ." New England J. of Medicine297: 1470.
29.
Parker, E.B. (1963) "The effects of television on public library circulation ." Public Opinion Q.27: 578-589.
30.
Ross, H.L., D.T. Campbell, and G.V. Glass (1970) "Determining the social effects of a legal reform: the British 'breathalyzer' crackdown of 1967." Amer. Behavioral Scientist13: 493-509.
31.
Sacks, H., T.C. Chalmers, and H. Smith (1982) "Randomized versus historical controls for clinical trials." Amer. J. of Medicine72: 233-240.
32.
Scheffé, H. (1959) The Analysis of Variance. New York: John Wiley.
33.
Sechrest, L. and W.H. Yeaton (1982) "Magnitudes of experimental effects in social science research." Evaluation Rev.6: 579-600.
34.
———(forthcoming) "Role of no-difference findings in medical research."
35.
Staines, G.L. (1974) "The strategic combination argument," pp. 417-430 in W. Leinfellner and E. Kohler (eds.) Developments in the Methods of Social Sciences. Dordrecht, Holland: D. Reidel.
36.
Waldman, R.J., W.N. Hall, H. McGee, and G. Van Amburg (1982) "Aspirin as a risk factor in Reye's syndrome." J. of the Amer. Medical Assn.247: 3089-3094.
37.
Wortman, P.M. and W.H. Yeaton (1983) "Synthesis of results in controlled trials of coronary artery bypass graft surgery," pp. 536-551 in R. J. Light (ed.) Evaluation Studies Review Annual, Vol. 8. Beverly Hills, CA: Sage.
38.
Yeaton, W.H. and L. Sechrest (1981) "Critical dimensions in the choice and maintenance of successful treatments: strength, integrity and effectiveness." J. of Consulting and Clinical Psychology49: 156-167.
