Increasing the Power of Clinical Trials Through Judgment Analysis

Abstract

A method for increasing the statistical power of clinical trials to detect clinically important differences is described. Inconsistency in physicians' overall judgments of treatment effec tiveness adds "noise" to a trial that may mask either the superiority or the inferiority of particular treatments. The method described here uses "judgment analysis" to reduce errors in an individual's overall judgments of treatment effectiveness. The method can also be used to reduce error variance due to differences in judgment between physicians and may thus be particularly useful in multicenter trials. The method is illustrated with results from a recent trial. Key words: statistical power; experimental design; multi-investigator trials; judgment analysis; clinical judgment. (Med Decis Making 8:33-38, 1988)

Get full access to this article

View all access options for this article.

References

Beal D. , Gillis JS , Stewart TR : The lens model: computational procedures and applications. Perceptual and Motor Skills Monograph Supplement 46:3-28, 1978

Bech P. , Haaber A. , Joyce Crb : Experiments on clinical observation and judgment in the assessment of depression: profiled videotapes and judgment analysis. Psychol Med 16:873-883, 1986

Cohen J. : Statistical Power Analysis for the Behavioral Sciences. Revised edition. New York, Academic Press, 1977

Edlund MJ , Overall JE , Rhoades HM : Beta, or type II error in psychiatric controlled clinical trials. J Psychiat Res 19:563-567, 1985

Fisch H-U. , Hammond KR , Joyce Crb , et al: An experimental study of the clinical judgment of general physicians in evaluating and prescribing for depression. Br J Psychiat 138:100-109, 1981

Hammond KR , Stewart TR , Brehmer B. , et al: Social judgment theory. In: Kaplan MF , Schwartz S (eds): Human Judgment and Decision Processes: Formal and Mathematical Approaches . New York, Academic Press, 1975

Joyce Crb , Hammond KR : Judgment analysis in clinical trials: how to save the baby from the bathwater . Controlled Clin Trials 5:307-308, 1984

Kirwan JR , Chaput De Saintonge DM : How can we discover which treatment responses are clinically important? Br J Clin Pharmacol 23:530-531, 1985

Kirwan JR , Chaput De Saintonge DM , Joyce Crb , et al: Clinical judgment in rheumatoid arthritis. I. Rheumatologists' opinions and the development of 'paper patients.' Ann Rheum Dis 42:644-647, 1983

10.

Overall JE , Klett CJ : Applied Multivariate Analysis. New York, McGraw-Hill, 1972

11.

Pedhazur EJ : Multiple Regression in Behavioral Research. Second edition. New York, Holt, Rinehart and Winston, 1982

12.

Seidl G. : Decision making in drug research. In: Gross F (ed): Cost of Drug Research. New York , Raven Press, 1983

13.

Slovic P. , Lichtenstein S. : Comparison of Bayesian and regression approaches to the study of information processing in judgment. In: Rappoport L , Summers DA (eds): Human Judgment and Social Interaction. New York, Holt, Rinehart and Winston, 1973

14.

Stewart TR , Joyce Crb , Lindell MK : New analyses: application of judgment theory to physicians' judgments of drug effects. In: Hammond KR , Joyce CRB (eds): Psychoactive Drugs and Social Judgment: Theory and Research. New York, John Wiley and Sons, 1975

15.

Ward MJ : Clinical trials in acute severe asthma: are type II errors important ? Thorax 41:824-829, 1986