For more than 10 years, reports comparing quality of care in hospitals have been disseminated to the public. The most commonly used measure in these reports is hospital mortality rate. Despite the resources devoted to analyzing and disseminating mortality data, little attention has been given to the question of their validity as a quality measure. In this article, the authors synthesize findings from 18 articles identified as providing information relevant to this issue. From this review, the authors find evidence that poor quality care increases patients' risk of mortality and that, on average, quality of care provided in hospitals identified as high—mortality rate outliers is poorer than that provided in low—mortality rate outlier hospitals. Nevertheless, a clear conclusion from these studies is that when used as a measure of quality for individual hospitals, risk-adjusted mortality rates are seriously inaccurate. Publication of hospital mortality rates misinforms the public about hospital quality.
Get full access to this article
View all access options for this article.
References
1.
Berwick, D. M. , and D. L. Wald. 1990. Hospital Leaders' Opinions of the HCFA Mortality Data. Journal of the American Medical Association263:247-249.
2.
Blumberg, M. S.1987. Comments on HCFA Hospital Death Rate Statistical Outliers. Health Services Research21:715-739.
3.
Blumberg, M. S.1991. Biased Estimates of Expected Acute Myocardial Infarction Mortality Using MedisGroups Admission Severity Groups. Journal of the American Medical Association265:2965-2970.
4.
Brook, R. H. , A. Davies-Avery, S. Greenfield, L. J. Harris, R. Lelah, N. E. Solomon, and J. E. Ware. 1977. Assessing the Quality of Medical Care Using Outcome Measures: An Overview of the Method. Medical Care15 (9, Supp.).
5.
California Office of Statewide Health Planning and Development (COSHPD) . 1996. Report of the California Hospital Outcomes Project: Hospital Specific Detailed Statistical Tables. Vol. 2, Technical Appendix. Sacramento, CA: OSHPD.
6.
Campbell, D. T. , and J. C. Stanley. 1963. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally.
7.
Daley, J.1994. Validity of Risk-Adjustment Methods. In Risk Adjustment for Measuring Health Care Outcomes, ed. L. I. Iezzoni; 239-262. Ann Arbor, MI: Health Administration Press.
8.
DesHarnais, S. , J. D. Chesney, R. Wroblewski, S. T. Fleming, and L. F. McMahon. 1988. The Risk-Adjusted Mortality Index. A New Measure of Hospital Performance. Medical Care26:1129-1148.
9.
Donabedian, A.1988. Quality Assessment and Assurance: Unity of Purpose, Diversity of Means. Inquiry25 (1): 173-192.
10.
DuBois, R. W.1989. Hospital Mortality as an Indicator of Quality. In Providing Quality Care: The Challenge to Clinicians, eds. N. Goldfield and D. B. Nash; 107-131. Philadelphia: American College of Physicians.
11.
DuBois, R. W. , W. H. Rogers, J. H. Moxley, D. Draper, and R. H. Brook. 1987. Hospital Inpatient Mortality: Is It a Predictor of Quality?New England Journal of Medicine317:1674-1680.
12.
Ghali, W. A. , A. S. Ash, R. E. Hall, and M. A. Moskowitz. 1997. Statewide Quality Improvement Initiatives and Mortality after Cardiac Surgery. Journal of the American Medical Association277:379-382.
13.
Goldman, R. L.1992. The Reliability of Peer Assessments of Quality of Care. Journal of the American Medical Association267:958-960.
14.
Goss, M.E.W. , and J. I. Reed. 1974. Evaluating the Quality of Hospital Care through Severity-Adjusted Death Rates: Some Pitfalls. Medical Care12:202-213.
15.
Green, J. , and N. Wintfeld. 1995. Report Card on Cardiac Surgeons: Assessing New York State's Approach. New England Journal of Medicine332:1229-1232.
16.
Green, J. , N. Wintfeld, M. Krasner, and C. Wells. 1997. In Search of America's Best Hospitals: The Promise and Reality of Quality Assessment. Journal of the American Medical Association277:1152-1155.
17.
Hadorn, D. C. , E. B. Keeler, W. H. Rogers, and R. H. Brook. 1993. Assessing the Performance of Mortality Prediction Models. Rand Report ISBN: 0-8330-1335-1. Santa Monica, CA: RAND.
18.
Hannan, E. L. , H. Kilburn Jr., M. L. Lindsey, and R. Lewis. 1992. Clinical versus Administrative Data Bases for CABG Surgery: Does It Matter?Medical Care30:892-907.
19.
Hannan, E. L. , H. Kilburn Jr., J. F. O'Donnell, G. Lukacik, and E. P. Shields. 1990. Adult Open Heart Surgery in New York State. Journal of the American Medical Association264:2768-2774.
20.
Hannan, E. L. , H. Kilbum Jr., M. Racz, E. Shields, and M. R. Chassin. 1994. Improving the Outcomes of Coronary Artery Bypass Surgery in New York State. JAMA271:761-766.
21.
Hannan, E. L. , D. Kumar, M. Racz, A. Siu, and M. R. Chassin. 1994. New York's Cardiac Surgery Reporting System: Four Years Later. Annals of Thoracic Surgery58:1852-1857.
22.
Hannan, E. L. , M. J. Racz, J. G. Jollis, and E. D. Peterson. 1997. Using Medicare Claims Data to Assess Provider Quality for CABG Surgery: Does It Work Well Enough?Health Services Research31:569-678.
23.
Hannan, E. L. , and A. A. Yazici. 1988. Critique of the 1987 HCFA Mortality Study Based on New York Data. Albany: New York State Department of Health.
24.
Hartz, A. J. , M. S. Gottlieb, E. M. Kuhn, and A. A. Rimm. 1993. The Relationship between Adjusted Hospital Mortality and the Results of Peer Review. Health Services Research27:767-777.
25.
Hartz, A. J. , and E. M. Kuhn. 1994. Comparing Hospitals That Perform Coronary Artery Bypass Surgery: The Effect of Outcome Measures and Data Sources. American Journal of Public Health84:1609-1614.
26.
Hill, C. A. , K. L. Winfrey, and B. A. Rudolph. 1997. "Best Hospitals:" A Description of the Methodology for the Index of Hospital Quality. Inquiry34:80-90.
27.
Hofer, T. P. , and R. A. Hayward. 1996. Identifying Poor-Quality Hospitals: Can Hospital Mortality Rates Detect Quality Problems for Medical Diagnoses?Medical Care34:737-753.
28.
Iezzoni, L. I. , ed. 1994. Risk Adjustment for Measuring Health Care Outcomes. Ann Arbor, MI: Health Administration Press.
29.
Iezzoni, L. I.1997. The Risks of Risk Adjustment. Journal of the American Medical Association278:1600-1607.
30.
Iezzoni, L. I. , A. S. Ash, M. Shwartz, J. Daley, J. S. Hughes, and Y. D. Mackiernan. 1995. Predicting Who Dies Depends on How Severity Is Measured: Implications for Evaluating Patient Outcomes. Annals of Internal Medicine123:763-770.
31.
Iezzoni, L. I. , M. Shwartz, A. S. Ash, J. S. Hughes, J. Daley, and Y. D. Mackiernan. 1995. Using Severity-Adjusted Stroke Mortality Rates to Judge Hospitals. International Journal for Quality in Health Care7 (2): 81-94.
32.
Iezzoni, L. I. , M. Shwartz, A. S. Ash, J. S. Hughes, J. Daley, and Y. D. Mackiernan. 1996. Severity Measurement Methods and Judging Hospital Death Rates for Pneumonia. Medical Care34:11-28.
33.
Institute of Medicine (IOM) . 1974. Advancing the Quality of Health Care: Key Issues and Fundamental Principles. A Policy Statement by a Committee of the Institute of Medicine. Washington, DC: National Academy of Sciences.
34.
Jacobs, D. M. , and N. D. Jacobs. 1974. The PEP Primer: The JCAH Performance Evaluation Procedure for Auditing and Improving Physician Care. Chicago: Quality Review Center, Joint Commission on Accreditation of Hospitals.
35.
Jencks, S. F.1995. Measuring Quality of Care under Medicare and Medicaid. Health Care Financing Review16 (4): 39-54.
36.
Jencks, S. F. , D. K. Williams, and T. L. Kay. 1988. Assessing Hospital-Associated Deaths from Discharge Data: The Role of Length of Stay and Comorbidities. Journal of the American Medical Association260:2240.
37.
Kahn, K. L. , E. B. Keeler, M. J. Sherwood, W. H. Rodgers, et al.1990. Comparing Outcomes of Care before and after Implementation of the DRG-Based Prospective Payment System. Journal of the American Medical Association264:1984-1988.
38.
Kahn, K. L. , W. H. Rogers, L. V. Rubenstein, M. J. Sherwood, E. J. Reinisch, E. B. Keeler, D. Draper, J. Kosecoff, and R. H. Brook. 1990. Measuring Quality of Care with Explicit Process Criteria before and after Implementation of the DRG-Based Prospective Payment System. Journal of the American Medical Association264:1969-1973.
39.
Keeler, E. B. , K. L. Kahn, D. Draper, M. J. Sherwood, L. V. Rubenstein, E. J. Reinisch, J. Kosecoff, and R. H. Brook. 1990. Changes in Sickness at Admission Following the Introduction of the Prospective Payment System. Journal of the American Medical Association264:1962-1968.
40.
Knaus, W. A. , E. A. Draper, D. P. Wagner, and J. E. Zimmerman. 1986. An Evaluation of Outcome from Intensive Care in Major Medical Centers. Annals of Internal Medicine104:410-418.
41.
Knaus, W. A. , D. P. Wagner, E. A. Draper, J. E. Zimmerman, M. Bergner, P. G. Bastos, C. A. Sirio, D. J. Murphy, T. Lotring, and A. Damiano. 1991. The APACHE III Prognostic System: Risk Prediction of Hospital Mortality for Critically Ill Hospitalized Adults. Chest100:1619.
42.
Landon, B. , Iezoni, L. I., A. S. Ash, M. Shwartz, J. Daley, J. S. Hughes, and Y. D. Mackiernan. 1996. Judging Hospitals by Severity-Adjusted Mortality Rates: The Case of CABG Surgery. Inquiry33:155-166.
43.
Lave, J. R. , and L. B. Lave. 1971. The Extent of Role Differentiation among Hospitals. Health Services Research6:15-38.
44.
Luft, H. S. , and P. S. Romano. 1993. Chance, Continuity, and Change in Hospital Mortality Rates. Journal of the American Medical Association270:331-337.
45.
McAuliffe, W. E.1984. A Validation Theory for Quality Assessment. In Hospital Quality Assurance: Risk Management and Program Evaluation, eds. J. J. Pena, A. N. Haffner, B. Rosen, and D. W. Light; 157-174. Gaithersburg, MD: Aspen.
46.
McClure, W.1985. Buying Right: How to Do It. Business & Health (October): 41-44.
New York State Department of Health, Office of Health Systems Management, Bureau of Healthcare Research (NYSDOH) . Investigation of Quality of Care in Hospitals. Albany, NY: New York State Department of Health.
49.
Omoigui, N. A. , D. P. Miller, K. J. Brown, K. Annan, D. Cosgrove, B. Lytle, F. Loop, and E. J. Topol. 1996. Outmigration for Coronary Bypass Surgery in an Era of Public Dissemination of Clinical Outcomes. Circulation93:27-33.
50.
Park, R. E. , R. H. Brook, J. Kosecoff, J. Keesey, L. Rubenstein, E. Keeler, K. L. Kahn, W. H. Rogers, M. R. Chassin, et al.1990. Explaining Variations in Hospital Death Rates: Randomness, Severity of Illness, Quality of Care. Journal of the American Medical Association264:484-490.
51.
Roemer, M. I. , A. T. Moustafa, and C. E. Hopkins. 1968. A Proposed Hospital Quality Index: Hospital Death Rates Adjusted for Case Severity. Health Services Research3:96-118.
52.
Romano, P. S. , A. Zach, H. S. Luft, J. Rainwater, L. L. Remy, and D. Campa. 1995. The California Hospital Outcomes Project: Using Administrative Data to Compare Hospital Performance. Joint Commission Journal on Quality Improvement21:668-682.
53.
Rosenthal, G. E. , and D. L. Harper. 1994. Cleveland Health Quality Choice: A Model for Community-Based Outcomes Assessment. joint Commission Journal on Quality Improvement20:425-444.
54.
Rosenthal, G. E. , L. Quinn, and D. L. Harper. 1997. Declines in Hospital Mortality Associated with a Regional Initiative to Measure Hospital Performance. American Journal of Medical Quality15:103-112.
55.
Rubenstein, L. V. , K. L. Kahn, E. J. Reinisch, M. J. Sherwood, W. H. Rogers, C. Kamberg, D. Draper, and R. H. Brook. 1990. Changes in Quality of Care for Five Diseases Measured by Implicit Review, 1981-1986. Journal of the American Medical Association264:1974-1979.
56.
Rubin, H. R. , W. H. Rogers, K. L. Kahn, L. V. Rubenstein, and R. H. Brook. 1992. Watching the Doctor Watchers: How Well Do Peer Review Organization Methods Detect Hospital Quality Problems?Journal of the American Medical Association267:2349-2354.
57.
Schneider, E. C. , and A. M. Epstein. 1996. Influence of Cardiac-Surgery Performance Reports on Referral Practices and Access to Care. New England Journal of Medicine335:251-256.
58.
Sisk, J. E. , D. M. Dougherty, P. M. Ehrenhaft, G. Ruby, and B. A. Mitchner. 1990. Assessing Information for Consumers on the Quality of Medical Care. Inquiry27:263-272.
59.
Steen, P. M. , A. C. Brewster, R. C. Bradbury, E. Estabrook, and J. A. Young. 1993. Predicted Probabilities of Hospital Death as a Measure of Admission Severity of Illness. Inquiry30:128.
60.
Thomas, J. W.1991. Validating Risk-Adjusted Outcomes as Measures of Quality of Care in Hospitals. Report prepared for the Minnesota Coalition on Health. Ann Arbor: University of Michigan, School of Public Health, Department of Health Services Management and Policy.
61.
Thomas, J. W. , and T. P. Hofer. Forthcoming. Accuracy of Risk-Adjusted Mortality Rate as a Measure of Hospital Quality of Care. Medical Care.
62.
Thomas, J. W. , J. J. Holloway, and K. E. Guire. 1993. Validating Risk-Adjusted Mortality as an Indicator for Quality of Care. Inquiry30:6-22.
63.
U.S. Congress, General Accounting Office (USCGAO) . 1988. Medicare: Improved Patient Outcome Analyses Could Enhance Quality Assessment. Washington, DC: Government Printing Office.
64.
Vladeck, B. C. , E. J. Goodwin, L. P. Myers, and M. Sinsi. 1988. Consumers and Hospital Use: The HCFA "Death List."Health Affairs7:122-125.
65.
Williams, R. L.1979. Measuring the Effectiveness of Perinatal Medical Care. Medical Care17:95-110.
66.
Young, W. W. , S. Kohler, and J. Kowalski. 1994. PMC Patient Severity Scale: Derivation and Validation. Health Services Research29:367.
67.
Zalkind, D. L.1997. Mortality Rates as an Indicator of Hospital Quality. Hospital & Health Services Administration42:3-15.
