Commentary: Inaccurate Data on the Quality of Care May Do More Harm than Good

Abstract

Recently the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) announced that it would integrate the use of clinical outcomes and other per formance measures into the accreditation process through its new "ORYX" program. This JCAHO initiative represents a significant new development that will include more than 100 different performance measurement systems, most of which are available through commercial firms and outside organizations. However, we see some potential problems with this new initiative. This is because some indicators recommended by JCAHO may be questionable due to the fact they are based on flawed methodologies that could result in biased and confounded data. To illustrate some of the potential adverse effects that could result from using such data to compare health care providers and facilities, we discuss some common problems associated with several widely available performance measurement systems. We then suggest an alternative approach that could potentially avoid many of these problems in the future.

Get full access to this article

View all access options for this article.

References

U.S. General Accounting Office. "Report Cards" are Useful but Significant Issues Need to be Addressed. Washington, DC: General Accounting Office (GAO/HEHS-94-219), September 1994.

U.S. General Accounting Office. Medicare: HCFA Should Release Data to Aid Consumers, Prompt Better HMO Performance. Washington, DC: General Accounting Office (GAO/HEHS-97-23), October 1996.

US. General Accounting Office. Medicaid: States' Efforts to Educate and Enroll Beneficiaries in Managed Care. Washington, DC: General Accounting Office (GAO/HEHS-96-184), September 1996.

Baum MD Claims in ad untrue: Magazine disputes Erlanger's status. Chattanooga Times, October 1996;17.

USNews World Report. America's Best Hospitals. New York : Wiley, 1996.

Hill C. , Winfrey K. 1995 Index of Hospital Quality. Chicago: NORC, June 1995.

Office of Technology Assessment. The Quality ofmedical Care: Information for Consumers. Washington, DC: Office of Technology Assessment (OTA-H-386), 1988.

Boscarino JA Patients' perceptions of the quality of hospital care and hospital occupancy: Are there biases associated with assessing quality care based on patients' perceptions? Int J Qual Health Care 1996;8:467-477.

Boscarino JA The public's perception of quality hospitals. II. Implications for patient surveys. Hospital and Health Services Administration 1992;37:13-35.

10.

Ellwood PM , Lundberg GD Managed care: A work in progress. JAMA 1996 ;276:1083-1086.

11.

Campbell DT Methodology and Epistemology for Social Science: Selected Papers. Chicago: University of Chicago Press, 1988.

12.

Hulley SB , Cummings SR (eds). Designing Clinical Research: An Epidemiologic Approach. Baltimore: Williams & Wilkins, 1988.

13.

Iezzoni LI (ed). Risk Adjustments for Measuring Health Care Outcomes. Ann Arbor, MI: Health Administration Press, 1994.

14.

Jencks SF , Daley J. , Draper D. , et al. Interpreting hospital mortality data: The role of clinical risk adjustment. JAMA 1988;260:3611-3616.

15.

Green J. , Wintfeld N. , Sharkey P. , et al. The importance of severity of illness in assessing hospital mortality. JAMA 1990;263:241-246.

16.

Iezzoni LI , Ash AS , Shwartz M. , et al. Predicting who dies depends on how severity is measured: Implications for evaluating patient outcomes. Ann Intern Med 1995;123:763-770.

17.

Iezzoni LI , Ash AS , Shwartz M. , et al. Judging hospitals by severity-adjusted mortality rates: The influence of the severity-adjustment method. Am J Public Health 1996:86:1379-1387.

18.

Jencks SF , Wilensky GR The health care quality improvement initiative: a new approach to quality assurance in Medicare. JAMA 1992;268:900-903.

19.

Burton TM Cardiac conflict: Heart transplant team is torn by allegations of unwise procedures . The Wall Street Journal, December 1996;12:A1 and A6.

20.

Joint Commission on the Accreditation of Health Care Organizations. Joint Commission announces ORYX: the next evolution in accreditation. News Release. Oakbrook Terrace, IL: Joint Commission on the Accreditation of Health Care Organizations, 1 p.m. (CST), February 18, 1997.

21.

Meinert CL Clinical Trials: Design, Conduct and Analysis. New York : Oxford, 1986.

22.

Stewart AL , Ware JE (eds). Measuring Functioning and Well-Being: The Medical Outcomes Study Approach. Durham, NC: Duke , 1992.

23.

Hannan EL , Kilburn H. O'Donnell JF , et al. Adult open heart surgery in New York State: an analysis of risk factors and hospital mortality. JAMA 1990;264:2768-2774.

24.

Pennsylvania Health Care Cost Containment Council. Focus on Heart Attack in Pennsylvania: The Technical Report—1993, Part A. Harrisburg, PA: Pennsylvania Health Care Cost Containment Council, June 1996.

25.

Raskin IE , Maklan CW Medical treatment effectiveness research: a view from inside the Agency for Health Care Policy and Research. Evaluation and the Health Professions 1991;14:161-186.

26.

Faulkner and Gray. 1996 Clinical Performance Data Sourcebook. New York: Faulkner and Gray, Inc., 1995.

27.

Gallagher J. Invalid patient surveys: not a bargain at any price. J Health Care Marketing 1989;9:69-71.

28.

Warren KS , Mosteller F (eds). Doing More Good than Harm: Evaluation of Health Care Interventions. New York: New York Academy of Sciences, 1993.

29.

Ginzberg E (ed). Health Services Research: Key to Health Policy. Cambridge, MA: Harvard, 1991.

Commentary: Inaccurate Data on the Quality of Care May Do More Harm than Good—An Alternative Approach Is Required

Abstract

Get full access to this article

References