The present paper focuses on drug-evaluation research, and describes a method of meta-analysis for evaluating the methodological quality of a body of research focusing on the study of an acute treatment medication. An illustrative application of this procedure to research evaluating lithium treatment for acute mania revealed general problems with diagnostic reliability and the reliability of the dependent variable, and a tendency to ignore the statistical power aspects of determining sample size. The implications of this are discussed in terms of a need to further clarify lithium's role in treating this disorder.
References
1.
GardnerM.J., MachinD., CampbellM.J.Use of check lists in assessing the statistical content of medical studies.Br Med J1986; 292: 810–812.
2.
BartkoJ.J., GurelL., NemiahJ.C.The Journal's new program of statistical review (editorial).Am J Psychiatry1988; 145: 987–988.
3.
BartkoJ.J.Rationale for reporting standard deviations rather than standard errors of mean (editorial).Am J Psychiatry1985; 142: 1060.
4.
NewcombeR.G.Evaluation of treatment effectiveness in psychiatric research.Br J Psychiatry1988; 152: 696–697.
5.
KraemerH.C., PruynJ.P., GibbonsR.D.Methodology in psychiatric research: report on the 1986 MacArthur Foundation Network I Methodology Institute.Arch Gen Psychiatry1987; 44: 1100–1106.
6.
PocockS.J.Current issues in the design and interpretation of clinical trials. Br Med.J1985; 290: 39–42.
7.
NewcombeR.G.Towards a reduction in publication bias.Br Med J1987; 295: 656–659.
8.
HokansonJ.A., BryantS.G., GardnerR.Spectrum and frequency of use of statistical techniques in psychiatric journals.Am J Psychiatry1986; 143: 1118–1125.
9.
Department of Clinical Epidemiology and Biostatistics, McMaster University Health Sciences Centre, Hamilton, Canada.How to read journals: V. To distinguish useful from useless or even harmful therapy.Can Med Assoc J1981; 124: 1156–1162.
10.
ShroutP.E., SpitzerR.L., FleissJ.L.Quantification of agreement in psychiatric diagnosis revisited.Arch Gen Psychiatry1987; 44: 172–177.
11.
KingP.R.Methodological guidelines for reading drug-evaluation research.Can J Psychiatry1984; 29(7): 575–582.
12.
SchouM., Juel-NielsenN., StromgrenE.The treatment of manic psychosis by the administration of lithium salts.J Neurol Neurosurg Psychiatry1954; 17: 250–260.
13.
MaggsR.Treatment of manic illness with lithium carbonate.Br J Psychiatry1963; 109: 56–65.
14.
BunneyW.E., GoodwinF.K., DavisM.J.A behavioural-biochemical study of lithium treatment.Am J Psychiatry1968; 125: 499–512.
15.
JohnsonG., GershonS., HekimianL.J.Controlled evaluation of lithium and chlorpromazine in the treatment of manic states: an interim report.Compr Psychiatry1968; 9: 563–573.
16.
GoodwinF.K., MurphyD.L., BunneyW.E.Lithium carbonate treatment in depression and mania.Arch Gen Psychiatry1969; 21: 486–496.
17.
PlatmanS.R.A comparison of lithium carbonate and chlorpromazine in mania.Am J Psychiatry1970; 127; 351–353.
18.
SpringG., SchweidD., GrayG.A double-blind comparison of lithium and chlorpromazine in the treatment of manic states.Am J Psychiatry1970; 126: 1306–1310.
19.
JohnsonG., GershonS., BurdockE.I.Comparative effects of lithium and chlorpromazine in the treatment of acute manic states.Br J Psychiatry1971; 119: 267–276.
20.
PrienR.F., CaffeyE.M., KlettC.J.Comparison of lithium carbonate and chlorpromazine in the treatment of mania: Report of the Veterans Administration and National Institute of Mental Health Collaborative Study Group.Arch Gen Psychiatry1972; 26: 146–153.
21.
MurphyD.L., BeigelA.Depression, elation and lithium carbonate responses in manic patient subgroups.Arch Gen Psychiatry1974; 31: 643–648.
22.
ShopsinB., GershonS., ThomsonH.Psychoactive drugs in mania: a controlled comparison of lithium carbonate, chlorpromazine and haloperidol.Arch Gen Psychiatry1975; 132: 34–42.
23.
TakahashiR., SakumaA., ItohK.Comparison of efficacy of lithium carbonate and chlorpromazine in mania.Arch Gen Psychiatry1975; 32: 1310–1318.
24.
GarfinkelP.E., StancerH.C., PersadE.A comparison of haloperidol, lithium carbonate and their combination in the treatment of mania.J Affective Disord1980; 2: 279–288.
25.
SchlagenhaufG., TupinJ., WhiteR.B.The use of lithium carbonate in the treatment of manic psychosis.Am J Psychiatry1966; 123: 201–206.
26.
SikesJ.C., SikesS.C.Lithium carbonate treatment in psychiatry.Dis Nerv Syst1970; 31: 52–55.
27.
WhartonR.N., FieveR.R.The use of lithium in affective psychosis.Am J Psychiatry1966; 123: 706–712.
28.
FreimanJ.A., ChalmersT.C., SmithH.The importance of beta, the Type II error and sample size in the design and interpretation of the randomized control trial.N Engl J Med1978; 299: 690–694.
29.
PopeH.G., LipinskiJ.F.Diagnosis in schizophrenia and manic-depressive illness: a reassessment of the specificity of “schizophrenic” symptoms in the light of current research.Arch Gen Psychiatry1978; 35: 811–828.
WittenbornJ.R.Guidelines for clinical trials in psychopharmacology: In: LiptonM.A., DiMascioA., KillamK.F., eds. Psychopharmacology: a generation of progress.New York: Raven Press,1978.
32.
KlermanG.L.The spectrum of mania.Compr Psychiatry1981; 22: 11–30.
33.
ShopsinB.Part 2: Differential diagnostic issues with schizoaffective illness: a critical assessment and implications for research and treatment. In: ShopsinB., ed. Manic illness.New York: Raven Press,1979.
34.
SchouM.Lithium treatment: a refresher course.Br J Psychiatry1986; 149: 541–547.
35.
SmallJ.G., KlapperM.H., KellamsJ.J.Electroconvulsive treatment compared with lithium in the management of manic states.Arch Gen Psychiatry1988; 45: 727–732.
