Reconsidering the Harvard Medical Practice Study Conclusions about the Validity of Medical Malpractice Claims

Weiler P. C. Hiatt H. H. Newhouse J. P. Johnson W. G. Brennan T. A. Leape L. L. , A Measure of Malpractice (Cambridge, MA: Harvard, 1993): At 55.

Id at 75.

Id at 140.

Brennan T. A. Sox C. M. Burstin H. R. , “Relation Between Negligent Adverse Events and the Outcomes of Medical-Malpractice Litigation,” New England Journal of Medicine 335 (1996): 1963–67.

Saks M. , “Medical Malpractice: Facing Real Problems and Finding Real Solutions,” William & Mary Law Journal 35 (1994): 693–726 (a review of Weiler, supra note 1); Bovbjerg R. R. , “Medical Malpractice Research and Reform,” Virginia Law Review 79 (1993): 2155–2208; White M. J. , “The Value of Liability in Medical Malpractice,” Health Affairs 13 (1994): 75–87; Mehlman M. J. , “Saying ‘No’ to No-Fault: What the Harvard Malpractice Study Means for Medical Malpractice Reform,” (New York State Bar Association Special Committee on Medical Malpractice, 1991).

Joint Commission on Accreditation of Healthcare Organizations, Health Care at the Crossroads: Strategies for Improving the Medical Liability System and Preventing Patient Injury (Chicago, Illinois: White Paper, 2005).

Joint Economic Committee, “The Perverse Nature of the Medical Liability System,” Research Report #109–2 (March 2005).

American Medical Association, Medical Liability Reform – NOW! A compendium of facts supporting medical liability reform and debunking arguments against reform (Chicago: AMA, 2004).

Kessler D. P. McClellan M. B. , “How Liability Law Affects Medical Productivity,” Journal of Health Economics 21 (2002): 931–55.

Weiler , supra note 1, at 70.

Danzon P. , “Medical Malpractice: Theory, Evidence, and Public Policy,” (MA: Harvard University Press, 1985) (using data from Mills D. H. Boyden J. S. Rubjam D. S. , California Medical Association & California Hospital Association, Report on the Medical Insurance Feasibility Study (San Francisco: Sutter Publications, 1977, sponsored jointly by the California Medical Association and California Hospital Association), summarized in Mills D. H. , “Medical Insurance Feasibility Study: A Technical Summary,” Western Journal of Medicine 128 (1978): 360–365; Wilson R. M. Runciman W. B. Gibberd R. W. , “The Quality in Australian Health Care Study,” Medical Journal of Australia 163 (1995): 458–71; Thomas E. J. Studdert D. M. Burstin H. R. Orav E. J. Seena T. Williams E. J. Howard K. M. Weiler P. C. Brennan T. A. , “Incidence and Types of Adverse Events and Negligent Care in Utah and Colorado,” Medical Care 38 (2000): 261–71.

Andrews L. B. Stocking C. Krizek T. Gottlieb L. Krizek C. Vargish T. Siegler M. , “An Alternative Strategy for Studying Adverse Events in Medical Care,” Lancet 349 (1997): 309–313.

Weiler , supra note 1, at 55.

Weiler , supra note 1, at 76.

Id. They later found four additional claims but did not include those claims in the analysis. As explained in note 59, including those four additional claims would not have materially changed the results.

Localio A. R. Lawthers A. G. Brennan T. A. Laird N. M. Hebert L. E. Peterson L. M. Newhouse J. P. Weiler P. C. Hiatt H. H. , “Relation Between Malpractice Claims and Adverse Events Due to Negligence, Results of the Harvard Medical Practice Study III,” NEJM 325 (1991): 245–251, at 248, table 3.

Harvard Medical Practice Study (HMPS), “Patients, Doctors, and Lawyers: Medical Injury, Malpractice Litigation and Patient Compensation in New York: The Report of the Harvard Medical Practice Study to the State of New York,” (Cambridge: Harvard University, 1990). Recent references to this HMPS finding and advocating tort reform include Brody W. R. , “Dispelling Medical Malpractice Myths,” Washington Post , November 14, 2004, and Greater New York Hospital Association, Greater New York Hospital Association, Medical Malpractice Insurance Costs and Coverage (2005).

The description of the review process is derived from HMPS, supra note 17, chapter 5.

HMPS, supra note 17, at 6–7.

1 = “little or no evidence for management causation.” 2 = “slight to modest evidence for management causation.” 3 = “management causation not quite likely; less than 50–50 but close call.” 4 = “management causation more likely than not; more than 50–50 but close call.” 5 = “strong evidence for management causation.” 6 = virtually certain evidence for management causation.” From form appended to chapter 5 of HMPS, supra note 17.

“After having considered the factors in 8.1–8.7 you might have reassessed whether negligence occurred. If you feel that there is NO negligence, CHECK THE SPACE ON THE RIGHT AND GO TO Q. 11.” Id.

1 = “little or no evidence for negligence.” 2 = “slight to modest evidence for negligence.” 3 = “negligence not quite likely; less than 50–50 but close call.” 4 = “negligence more likely than not; more than 50–50 but close call.” 5 = “strong evidence for negligence.” 6 = virtually certain evidence for negligence.” From form appended to chapter 5 of HMPS, supra note 17.

HMPS, supra note 17, at 6–10.

For example, the first stage review was verified only if the reviewer sent the record on to the second stage; the physician reviewers were asked to reconsider their opinion only if they concluded that there was evidence of negligence; and the physician reviewers were asked to provide a confidence rating only for an opinion that there was an iatrogenic injury and that the injury was caused by negligence, but not for an opinion that there was not.

E.g., HMPS researchers later estimated that making comparatively modest changes to the review process would increase the rate of medical management injuries by 70%, Runciman W. B. Webb R. K. Helps S. C. Thomas E. J. Sexton E. J. Studdert D. M. Brennan T. A. , “A Comparison of Iatrogenic Injury Studies in Australia and the USA II: Reviewer Behavior and Quality of Care,” International Journal for Quality in Health Care 12 (2000): 379–88.

Danzon , supra note 11; Thomas Studdert Burstin , supra note 11. See also Andrews , supra note 12 (ethnographic approach).

HMPS, supra note 17, at 5–28, noting that there was a “lack of reliability regarding judgments of negligence.” See also Brennan T. A. Localio R. J. Laird N. , “Reliability and Validity of Judgments Concerning Adverse Events Suffered by Hospitalized Patients,” Medical Care 27 (1989): 1148–1158; Brennan T. A. Leape L. L. Laird N. M. Hebert L. Localio A. R. Lawthers A. G. Newhouse J. P. Weiler P. C. Hiatt H. H. , “Incidence of Adverse Events and Negligence in Hospitalized Patients,” New England Journal of Medicine 324 (1991): 370–6.

Brennan , “Reliability and Validity,” supra note 27.

Id. at 1156–57.

Brennan , “Reliability and Validity,” supra note 27, at 375; Localio , “Relation between Malpractice Claims,” supra note 16, at 249. There is an extensive literature examining the problem of reviewer reliability in studies of medical care. E.g., Goldman R. L. , “The Reliability of Peer Assessments of Quality of Care,” JAMA 267 (1992): 958–60; Hayward R. A. McMahon L. F. Bernard A. M. , “Evaluating the Care of General Medicine Inpatients: How Good is Implicit Review?” Annals of Internal Medicine 118 (1998): 550–56; Hofer T. P. Bernstein S. J. DeMonner S. Hayward R. A. , “Discussion Between Reviewers Does Not Improve Reliability of Peer Review of Hospital Quality,” Medical Care 38 (2000): 152–61; Hayward R. A. Hofer T. P. , “Estimating Hospital Deaths Due to Medical Errors: Preventability is in the Eyes of the Beholder,” JAMA 286 (2001): 415–20; Posner K. L. Caplan R. A. Cheney F. , “Variation in Expert Opinion in Medical Malpractice Review,” Anesthesiology 85 (1996):1049–54; Localio A. R. Weaver S. L. Landis J. R. Lawthers A. G. Brennan T. A. Hebert L. Sharp T. , “Identifying Adverse Events Caused by Medical Care: Degree of Physician Agreement in a Retrospective Chart Review,” Annals of Internal Medicine 125 (1996): 457–64; Thomas E. J. Studdert D. M. Brennan T. A. , “The Reliability of Medical Record Review for Estimating Adverse Event Rates,” Annals of Internal Medicine 136 (2002): 812–816; Wilson D. S. McElligott J. Fielding L. P. , “Identification of Preventable Trauma Deaths: Confounded Inquiries?” Journal of Trauma 32 (1992): 45–51. Cf. O'Neil A. C. Petersen L. A. Cook E. F. Bates D. W. Lee T. H. Brennan T. A. , “Physician Reporting Compared with Medical-Record Review to Identify Adverse Medical Events,” Annals of Internal Medicine 119 (1993): 370–76. In general, this research supports the HMPS researchers' judgment that reviewing a very large sample of hospital records is an acceptable way to draw conservative conclusions about the rate of injuries caused by substandard care in a population. Nevertheless, the research questions the reliability of particular judgments made by particular reviewers in particular cases. E.g. Hayward Hofer , “Estimating Hospital Deaths Due to Medical Errors.”

Brennan , supra note 27, “Incidence of Adverse Events,” at 374. See also Localio , supra note 16, “Relation Between Malpractice Claims,” at 249.

The description of the method comes from HMPS, supra note 17, at 5–27 to 5–28.

Since both teams concluded that there was evidence of a medical management injury in only 35 cases, and a single team concluded that there was such evidence in only an additional 34 cases, this means that the teams only considered the question whether there was evidence of negligence in 79 cases. With regard to those 79 cases, the teams disagreed 21 times. This calculation is based on the table in Brennan, supra note 27, “Incidence of Adverse Events,” at 375. The table suggests that the two sets of reviewers agreed in 293 out of 318 cases that there was no negligence, and the text of the article reports “the overall agreement on judgments of negligence was excellent (93 percent).” But the reviewers only considered the presence or absence of substandard care if they first identified a medical management injury. Thus the rate of agreement was 73%, not 93%. Because the researchers did not report the number of times that one reviewer from each team or one review from a single team concluded that there was negligence in the seventy nine medical management injury cases, the true rate of disagreement about the negligence determination is likely to be even higher.

HMPS, supra note 17, at 7–34: “For example, suppose that the following cases not judged to be negligent according to the Study protocols proved in fact to be instances of negligence: The 5 cases with allegations of failure to diagnose and not caught by the MRA screen; the 4 cases in which physicians disagreed on causation; the 1 low-threshold AE for which one reviewer found negligence; the 1 AE with a single, low confidence finding of negligence; and the 6 AEs with a single finding of negligence. The addition of these 17 cases to the 8 with clear evidence of negligence would triple the percentage of claims with findings of negligence from 17% to about 50%.”

Saks , supra note 5, at 709 n. 89. See also O'Neil , supra note 30.

Andrews , supra note 12. See also Andrews L. , Medical Error and Patient Claiming in a Hospital Setting (Chicago, Illinois: American Bar Foundation, 1993) and Andrews L. , “Studying Medical Error in Situ: Implications for Malpractice Law and Policy,” DePaul Law Review 54 (2005): 357–92.

Andrews , supra note 12.

Schimmel E. , “The Hazards of Hospitalization”, Annals of Internal Medicine 60 (1964):100–10; Steel K. Gartman P. Crexcenzi C. Anderson J. , “Iatrogenic Illness on a General Medical Service at a University Hospital,” New England Journal of Medicine 304 (1981): 638–42; Jahnigen D. Hannon C. Laxson L. La Force F. M. , “Iatrogenic Disease in Hospitalized Elderly Veterans,” Journal of American Geriatric Society 30 (1982): 387–90; Chaudry S. I. Olofinboba K. A. Krumholz H. M. , “Detection of Errors by Attending Physicians on a General Medical Service,” Journal of General Internal Medicine 18 (2003): 595–600.

Andrews , supra note 12, at 312

Localio , supra note 16, “Relation between Malpractice Claims,” at 249.

Weiler , supra note 1; Weiler P. C. , Medical Malpractice on Trial (Cambridge, MA: Harvard University Press, 1991).

Brennan , supra note 4.

Wilson , supra note 11.

Runciman , supra note 25, at 379.

Thomas E. J. Studdert D. M. Runciman W. B. Webb R. K. Sexton E. Wilson R. M. Gibberd R. W. Harrison B. T. Brennan T. A. , “A Comparison of Iatrogenic Injury Studies in Australia and the USA I: Context, Methods, Casemix, Population, Patient and Hospital Characteristics,” International Journal for Quality in Health Care 12 (2000): 371–78.

Id., at 376. Note that there are some differences in the methods of the HMPS and the later U.S. study that was directly compared to the Australian study, so the 70% increase is not directly applicable to the HMPS.

Saks, supra note 5, at 716 et seq. This explanation has not received the attention that it deserves. Indeed it was not acknowledged or addressed in the follow-up study of the HMPS hospitalization malpractice claims. Brennan, supra note 4. See also Bovbjerg , supra note 5, at 2166 (noting that the conclusion that malpractice litigation is erratic “arguably stretches the evidence a bit thin”).

HMPS, supra note 17, at 7–32: “When the management occurred at an outpatient facility, screeners looking at the hospital record might have difficulty in spotting the shortcomings in medical management, because the events during hospitalization might be consistent with optimal treatment of the underlying disease (e.g., cancer).”

The HPMS reports, supra note 17, a finding regarding the appropriateness of care only for cases determined to involve a significant likelihood of medical management injury. Thus, there is no finding regarding the appropriateness of care for the twelve cases screened out in the first stage or for the fourteen cases screened out during the second stage. HMPS, supra note 17, at 7–33 report that seven of the fourteen cases that the physician reviewers determined not to involve a medical management injury were based on an alleged failure to diagnose. If any of these seven cases were among the four in which one reviewer determined that there was a medical management injury they would be likely to have been classified by the reviewer as also involving negligence.

These calculations can all be made using the table in Localio, supra note 16, “Relation between Malpractice Claims,” at 248.

Farber H. S. White M. J. , “Medical Malpractice: An Empirical Examination of the Litigation Process,” RAND Journal of Economics 199, no. 22 (1990); Farber H. S. White M. J. , “A Comparison of Formal and Informal Dispute Resolution in Medical Malpractice,” Journal of Legal Studies 23 (1994): 777–806.

E.g. Andrews , supra note 36, “Medical Error and Patient Claiming in a Hospital Setting”; Farber White , supra note 51, “A Comparison of Formal and Informal Dispute.” Cf. Mazor K. M. Simon S. R. Gurwitz J. H. , “Communicating with Patients About Medical Errors: A Review of the Literature,” Archives of Internal Medicine 164 (2004): 1690–1697; Lamb R. M. Studdert D. M. Bohmer R. M. J. Berwick D. M. Brennan T. A. , “Hospital Disclosure Practices: Results of a National Survey: Most Hospitals Disclose Harm to Patients at Least Some of the Time, this 2002 Survey Finds,” Health Affairs, March-April, 2003.

Weiler , supra note 1, at 75.

Calculations from Table 3 in Localio, supra note 16, “Relation between Malpractice Claims,” at 248. See also Mello M. M. Hemenway D. , “Medical Malpractice as an Epidemiological Problem,” Social Science & Medicine 59 (2004): 39–46.

It is worth noting that the HMPS team reported to the State of New York that “the presence of a claim might be an efficient screen for both adverse events and negligence,” because “both negligence and adverse events were dramatically higher among the claimants, as one would expect.” HMPS, supra note 17, at 7–36. This conclusion is inconsistent with later statements in the book published for a popular audience suggesting that the legal system is erratic. Weiler , supra note 1, at 139.

Brennan , supra note 4.

Id. at 1966.

Id. at 1964.

Brennan , supra note 27, “Incidence of Adverse Events,” explained that the number of matched claims increased from the 47 claims discussed earlier in this article to 51 as a result of a second review of hospital records. These additional cases were mentioned in the earlier HMPS reports but not incorporated into the analysis. It does not appear that including those 4 claims in the earlier analysis would have changed the results materially.

Evidence of this intense interest is provided by Anderson R. , “Correspondence,” New England Journal of Medicine 336 (1997):1680–1681, who is identified as an official of a medical malpractice insurance company.

E.g. Keilholz W. B. Murray R. H. , “Insurance Economics and Legal Systems,” Geneva Papers on Risk and Insurance 29 (2004): 1–5.

Brennan , supra note 4, at 1966, Table 2.

This is one of the cases in which the Harvard researchers read the claims file. They report that this case involved a very serious neurological injury from a vascular procedure and that the insurer thought that “the jury would compensate the patient even though the medical care met the expected standard.” Brennan , supra note 4, at 1966. There are reports that neurological injury cases often involve serious differences of opinion regarding the standard of care. See, e.g. Sussman D. , Pupils: An Eye Opening Account of Medical Practice Without Standards (Silver Spring, MD: Bartleby, 2003).

Brennen , supra note 4, 1996 at 1966.

Based on a comparison of Brennan, supra note 4, Table 1 with the reclassification discussion on p.1966.

The “median” is the midpoint – the point at which half of the settlement amounts are higher. Thus a 0 means that more than half of the claims in the first two categories closed without even a write off of the hospital bills. Medians are not affected by the presence of outlier cases.

Exact medians cannot be calculated because the settlement amounts are presented in ranges. For the first category the median is given as $25,000 because the amounts in half of the paid claims were less than $25,000 and half were greater.

Brennan , supra note 4, at 1966 Table 3.

See, e.g., American Medical Association, supra note 8, and Kessler McClellan , supra note 9.

Brennan , supra note 4, at 1965, Table 1. For the logistic regression analysis the 0 to 8 disability scale was reduced to a dichotomous variable of permanent disability, yes or no.

Id. at 1967.

Five of the write off cases were classified as not involving a medical management injury, with the result that their disability score was a 0. Two of the remaining three were classified as not involving negligence. Brennan , supra note 4, at 1965, Table 1.

Saks , supra note 5.

HMPS, supra note 17, at 7–32.

Ogburn P. L. Julian T. M. Booker D. C. Joseph M. S. Butler J. C. Williams P. P. Anderson M. L. Shepard A. C. Ogburn S. L. Preisler W. C. Wood M. D. , “Perinatal Medical Negligence Closed Claims from the St. Paul Company, 1980–1982,” Journal of Reproductive Medicine 53 (1988): 608–11.

Rosenblatt R. A. Hurst A. , “An Analysis of Closed Obstetric Malpractice Claims,” Obstetrics & Gynecology 74 (1989): 710–14. Rosenblatt and Hurst studied obstetric claims closed during 1980–1988 and reported that the defense costs for all claims put into litigation were greater than $1000. At p. 711, Table 1.

Intuitions here point in opposite directions. On the one hand, any seriously contested case would be likely to produce defense costs in excess of $1000. On the other hand there are reasons to expect that there would be payments in a higher percentage of serious birth injury cases than in malpractice cases generally.

Rosenblatt Hurst , supra note 77.

Id. at 712.

Cheney F. W. Posner K. Caplan R. A. Ward R. J. , “Standard of Care and Anesthesia Liability,” JAMA 261 (1989):1599–1603.

Id. at 1601, figure 2. Note that two-thirds of the “appropriate care” cases that the insurance companies paid were cases that involved non-disabling injuries and tiny payments. Because the published article uses percentages rather than absolute numbers, this is difficult, but not impossible, to figure out. Using Figure 1 and the percentages given in the text on page 1601, one can compute that payments were made in 34 appropriate care death cases, 25 appropriate care disabling cases, and 104 appropriate care non-disabling cases. The median payment in the anesthesiologists' non-disabling cases was less than $15,000. All these cases date back to a time before physicians and insurance companies were required to report all medical malpractice payments on behalf of doctors. Since the U.S. Congress enacted that requirement, nuisance payments have become much less common, so we would not expect to find so many paid appropriate care cases today. Waters T. M. Studdert D. M. Brennan T. A. Thomas E. J. Almagor O. Mancewicz M. Budetti P. P. , “Impact of the National Practitioner Data Bank on the Resolution of Malpractice Claims,” Inquiry 40 (2003): 283–94.

Cheney , supra note 81, at 1601, figure 2.

Farber White , supra note 51, “Medical Malpractice: An Empirical Examination.”

Sloan F. A. Hsieh C. R. , “Variability in Medical Malpractice Payments: Is the Compensation Fair?” Law & Society Review 24 (1990): 997.

They did not report the number of claims in the subsample. Apparently the analysis was done for another project. Id. at 1010.

Id. at 1014 and 1019.

Sloan F. A. Githen P. B. Clayton E. W. Hickson G. B. Gentile D. A. Partlett D. F. , Suing for Medical Malpractice (IL: University of Chicago Press, 1993).

Id. at 98–99.

Id. at 100, 105.

Id. at 107.

Id. at 111–113.

Id. at 166–67. Because the published study does not provide exact numbers I cannot be more specific.

Taragin M. I. Willett L. R. Wilczek A. P. Trout R. Carson J. L. , “The Influence of Standard of Care and Severity of Injury on the Resolution of Medical Malpractice Claims,” Annals of Internal Medicine 117 (1992): 780–84.

Id. at 781.

Id. at 784.

Spurr S. J. Howze S. , “The Effect of Care Quality on Medical Malpractice Litigation,” Quarterly Review Economics & Finance 41 (2001): 491–513.

Id. at 496.

Unlike a logistic regression, a Probit analysis is well-suited to dependent variables like settlement amounts in which there are a large number of $0 payments and a large spread among the other payments. Using a logistic regression requires collapsing the range of payments into a yes/no variable. It would be interesting to reanalyze the Brennan, supra note 4, data using a Probit.

Spurr Howze , supra note 97.

Peeples R. Harris C. T. Metzloff T. B. , “The Process of Managing Medical Malpractice Cases: The Role of Standard of Care,” Wake Forest Law Review 37 (2002): 877–902, at 881.

Id. at 886.

Id. at 888.

E.g. Kohn L. T. Corrigan J. M. Donaldson M. S. , eds., To Err is Human: Building a Safer Health System (Washington, DC: National Academy Press, 2000).