Forty-five complementary and alternative medicine (CAM) efficacy randomized controlled trials (RCTs) from high-impact medical journals (NEJM, JAMA, Annals of Internal Medicine, and Archives of Internal Medicine) were reviewed based on their meeting three validity criteria (the existence of a placebo control, moderate attrition rates, and 50 or more participants per group). Of the 26 efficacy trials meeting all three criteria, only 2 (7.7%) were judged to be positive (i.e., the alternative therapy was significantly superior to its placebo control), while over half (55.5%) of the 19 trials that failed to meet one or more of these criteria reported positive results (p < .001). Of the two positive high-validity trials, one was funded and authored by the herbal company marketing the product tested and one used a placebo-control group of questionable credibility. This analysis is consistent with the hypothesis that CAM therapies are no more effective than placebos when adequate experimental control is present.
Barnes, P.M., Powell-Griner, E., McFann, K., & Nahin, R.L. ( 2004, May). Advance data from vital and health statistics. Centers for Disease Control and Prevention. Hyattsville, Maryland.
2.
Bausell, R.B. ( 2007). Snake oil science: The truth about complementary and alternative medicine. New York: Oxford University Press.
3.
Bausell, R.B. ( 2008). Placebo effect: Why we need less research on alternative medicine, not more. Chronicle of Higher Education, 54, B12-B12.
4.
Bausell, R.B., & Li, Y.F. ( 2002). Power analysis for experimental research: A practical guide for the biological, medical, and social sciences. Cambridge, UK: Cambridge University Press.
5.
Chalmers, T.C., Matta, R.J., Smith, H. Jr., & Kunzler, A.M. ( 1977). Evidence favoring the use of anticoagulants in the hospital phase of acute myocardial infarction. New England Journal of Medicine, 297, 1091-1096.
6.
Egger, M., Ebrahim, S., & Smith, G.D. ( 2002). Where now for meta-analysis? International Journal of Epidemiology, 31, 1-5.
7.
Ernst, E. ( 2002). A systematic review of systematic reviews of homeopathy . British Journal of Clinical Pharmacology, 54, 577-582.
8.
Gould, S.J. ( 1981). The mismeasure of man. New York: W. W. Norton.
Jadad, A.R., Moore, R.A., Carroll, D., et al. (1996). Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled Clinical Trials, 17, 1-12.
11.
Juni, P., Witschi, A., Bloch, R., & Egger, M. ( 1999). The hazards of scoring the quality of clinical trials for meta-analysis. Journal of the American Medical Association , 282, 1054-1060.
12.
Kaptchuk, T.J. ( 2003). Effect of interpretive bias on research evidence. British Medical Journal, 326, 1453-1455.
13.
Kjaergard, L.L., Villumsen, J., & Gluud, C. ( 2001). Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Annals of Internal Medicine, 135, 982-989.
14.
Kohn, A. ( 1988). False prophets. New York: Basil Blackwell.
15.
Laine, C., Goodman, S.N., Griswold, M.E., & Sox, H.C. ( 2007). Reproducible research: Moving toward research the public can trust. Annals of Internal Medicine, 146, 450-453.
16.
Linde, K., Clausius, N., Ramirez, G., Melchart, D., Eitel, F., Hedges, L.W., et al. (1997). Are the clinical effects of homeopathy placebo effects: A meta-analysis of placebo-controlled trials. The Lancet, 350, 834-843.
17.
Linde, K., Scholz, M., Ramirez, G., Clausius, N., Melchart, D., & Jonas, W.B. ( 1999). Impact of study quality on outcome in placebo controlled trials of homeopathic remedies. Journal of Clinical Epidemiology , 52, 631-636.
18.
Moher, D., Jadad, A.R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. ( 1995). Assessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists. Controlled Clinical Trials, 16, 62-73.
19.
Moher, D., Schulz, K.F., & Altman, D.G. ( 2001). The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. Annals of Internal Medicine, 134, 657-662.
20.
O’Connell, N.E., Wand, B.M., & Goldacre, B. ( 2009). Interpretive Bias in Acupuncture Research? A Case Study . Evaluation & the Health Professions, 32(4), 393-409.
21.
Park, R. ( 2000). Voodoo science: The road from foolishness to fraud. New York: Oxford.
22.
Price, D.D., Finniss, D.G., & Benedetti, F. ( 2008). A comprehensive review of the placebo effect: Recent advances and current thought. Annual Review of Psychology , 59, 565-590.
23.
Reed, D.A., Cook, D.A., Beckman, T.J., Levine, R.B., & Kerns, D.E. & Wright, S.M. ( 2007). Association between funding and quality of published medical education research. Journal of the American Medical Association, 298, 1002-1009.
24.
Ridker, P.M., & Torres, J. ( 2006). Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005. Journal of the American Medical Association, 295, 2270-2274.
25.
Schulz, K.F., Chalmers, I., Hayes, R.J., & Altman, D.G. ( 1995). Empirical evidence of bias: Dimensions of methodological quality associated with estimates of treatment effects in controlled trials . Journal of the American Medical Association, 273, 408-412.
26.
Singh, S., & Ernst, E. ( 2008). Trick or treatment? Alternative medicine on trial. London: Bantam Press.
27.
Smith, L.A., Oldman, A.D., McQuay, H.J., & Moore, R.A. ( 2000). Teasing apart quality and validity in systematic reviews: An example from acupuncture trials in chronic neck and back pain. Pain, 86, 119-132.
28.
Solomon, P.R., & Michalczuk, D.E. (2009). Toward establishing guidelines for evaluating cognitive enhancement with complementary and alternative medicine . Evaluation & the Health Professions, 32, 4, 370-392.
29.
Sterne, J.A.C., Gavagham, D., & Egger, M. ( 2000). Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53, 1119-1129.
30.
Tugwell, P., Shea, B., Boers, M., Brooks, P., Simon, L.S., Strand, V., et al. (2004). Evidence based rheumatology. London: BMJ Books.
31.
Turner, E.H., Matthews, A.M., Linardatos, E., Tell, R.A., & Rosenthal, R. ( 2008). Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine , 358, 252-260.
32.
Vickers, A., Goyal, N., Harland, R., & Rees, R. ( 1998). Do certain countries produce only positive results? A systematic review of controlled trials. Controlled Clinical Trials, 19, 159-166.
33.
Wade, N. ( 1976). IQ and heredity: Suspicion of fraud beclouds classic experiment. Science, 194, 916-919.
