Sage Journals: Discover world-class research

Abstract

In this randomized study 160 members of the Evaluation Research Society acted as judges to assess the attributes of research that produce credibility. The study focused on acceptability of no-difference findings, a long ignored but important domain of research. In the context of a hypothetical study, four factors were tested to determine their influence on acceptability of both no-difference and difference findings: randomization/nonran domization, one/three outcomes, power = .80/.60, and equivalence on baseline measures of all eight/all but two of eight. Experts were asked to judge degree of acceptability and to reject or accept findings in both a no-difference and a difference study. Randomization consistently enhanced the believability of outcomes whereas other factors exerted a less consistent influence. Limitations of the study were discussed.

Get full access to this article

View all access options for this article.

References

Bakan, D. (1966) "The test of significance in psychological research ." Psych. Bull. 66: 423-437.

Campbell, D.T. (1982) "Experiments as arguments." Knowledge 3: 327-1337.

Chalmers, T.C. , P. Celano , H.S. Sacks , and H. Smith (1983) "Bias in treatment assignment in controlled clinical trials." New England J. of Medicine 309: 1358-1361.

Cohen, J. (1977) Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press .

Cohen, L.H. (1979) "Clinical psychologists' judgments of the scientific merit and clinical relevance of psychotherapy outcome research." J. of Consulting and Clinical Psychology 47: 421-423.

Cook, T.D. and D.T. Campbell (1979) Quasi-Experimentation. Design and Analysis Issues for Field Settings. Chicago: Rand McNally.

Fisher et al. (1985) "Five-year results of a randomized clinical trial comparing total mastectomy and segmental mastectomy with or without radiation in the treatment of breast cancer." New England J. of Medicine 312: 665-673.

Freiman, J.A. , T.C. Chalmers , H. Smith , and R.R. Keubler (1978) "The importance of beta, the type II error, and sample size in the design and interpretation of the randomized control trial ." New England J. Medicine 299: 690-694.

Havender, W.R. (1983) "The science and politics of cyclamate." Public Interest 71: 17-32.

10.

Heberlein, T.A. and R. Baumgartner (1978) "Factors affecting response rates to mailed questionnaires: a quantitative analysis of the published literature." Amer. Soc. Rev. 43: 447-462.

11.

Riecken, H.W. and R.F. Boruch (1974) Social Experimentation. A Method for Planning and Evaluating Social Intervention New York: Academic Press.

12.

Sears, C. (1985) "Back from Caesareans? New data show normal delivery often safer." Amer. Health (June): 11.

13.

Sechrest, L. (1985) "Experiments and demonstrations in health services research." Medical Care 23: 677-695.

14.

Sechrest, L. and W.H. Yeaton (forthcoming) "Role of no-difference findings in medical research."

15.

Ware, J.E. (1985) "Commentary: monitoring and evaluating health services ." Medical Care 23: 705-709.

16.

Yeaton, W.H. and L. Sechrest (1985) "Testing a strategy for increasing questionnaire return rate: providing space for responses on the prepayment check." (manuscript)

17.

Yeaton, W.H. and L. Sechrest (1986) "Assessing tactics in planning no-difference research ." (manuscript)

Assessing Factors Influencing Acceptance of No-Difference Research

Abstract

Get full access to this article

References