Assessing Surgical Quality Using Administrative and Clinical Data Sets: A Direct Comparison of the University HealthSystem Consortium Clinical Database and the National Surgical Quality Improvement Program Data Set

Abstract

The use of “clinical” versus “administrative” data sets for health care quality assessment continues to be debated. This study directly compares the University HealthSystem Consortium Clinical Database (UHC CDB) and the National Surgical Quality Improvement Program (NSQIP) in terms of their assessment of complications and death for 26 322 surgery patients using analyses of variance, correlation, and multivariable logistic regression. The NSQIP had more variables with significant correlation with outcomes. The NSQIP was better at predicting death (c-index 0.94 vs 0.90, P < .05) and complications (c-index 0.78 vs 0.76, P = .07), especially for higher risk patients. The UHC CDB missed and misclassified several major complications. The data sets are similar in their explanatory power relative to outcomes, but the clinical data set is better, particularly at identifying higher risk patients and specific complications. It should prove more useful for initiating and monitoring clinical process improvements because of more clinically relevant variables. (Am J Med Qual 2009;24:395-402)

Keywords

National Surgical Quality Improvement Program University HealthSystem Consortium surgical quality assessment contextual data quality surgical morbidity and mortality

Get full access to this article

View all access options for this article.

References

Centers for Medicare and Medicaid Services. National health expenditure tables. http://www.cms.hhs.gov/NationalHealthExpendData/downloads/tables.pdf. Accessed April 17, 2008 .

The Leapfrog Group. Our mission. http://www.leapfroggroup.org/about_us/our_mission. Accessed April 15, 2008.

The Joint Commission. Accreditation programs . http://www.jointcommission.org/AccreditationPrograms/Hospitals/ . Accessed April 15, 2008.

Centers for Medicare and Medicaid Services. Physician quality reporting initiative: overview. http://www.cms.hhs.gov/pqri/. Accessed April 15, 2008.

University HealthSystem Consortium. About UHC. http://www.uhc.edu/results_list.asp?folder=WEB/About_UHC/ . Accessed April 15, 2008.

Khuri SF , Daley J. , Henderson W. , et al. The Department of Veterans Affairs’ NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998 ;228:491-507.

Khuri SF , Henderson WG , Daley J. , et al. The patient safety in surgery study: background, study design, and patient populations. J Am Coll Surg. 2007 ;204:1089-1102.

Fink AS , Campbell DA Jr , Mentzer RM Jr , et al. The National Surgical Quality Improvement Program in non-Veterans Administration hospitals: initial demonstration of feasibility. Ann Surg. 2002;236:344-353.

Jollis JG , Ancukiewicz M. , DeLong ER , Pryor DB , Muhlbaier LH , Mark DB Discordance of databases designed for claims payment versus clinical information systems. Implications for outcomes research. Ann Intern Med . 1993;119:844-850.

10.

Green J. , Wintfeld N. How accurate are hospital discharge data for evaluating effectiveness of care ? Med Care. 1993;31: 719-731.

11.

Hannan EL , Racz MJ , Jollis JG , Peterson ED Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough? Health Serv Res. 1997 ;31:659-678.

12.

Parker JP , Li Z. , Damberg CL , Danielsen B. , Carlisle DM Administrative versus clinical data for coronary artery bypass graft surgery report cards: the view from California. Med Care. 2006;44:687-695.

13.

Tang PC , Ralston M. , Arrigotti MF , Qureshi L. , Graham J. Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures. J Am Med Inform Assoc. 2007;14:10-15.

14.

Shahian DM , Silverstein T. , Lovett AF , Wolf RE , Normand SL Comparison of clinical and administrative data sources for hospital coronary artery bypass graft surgery report cards. Circulation. 2007;115:1518-1527.

15.

Aylin P. , Bottle A. , Majeed A. Use of administrative data or clinical databases as predictors of risk of death in hospital: comparison of models. BMJ. 2007;334:1044.

16.

Park HK , Yoon SJ , Ahn HS , et al. Comparison of risk-adjustment models using administrative or clinical data for outcome prediction in patients after myocardial infarction or coronary bypass surgery in Korea. Int J Clin Pract. 2007;61: 1086-1090.

17.

Fry DE , Pine M. , Jordan HS , et al. Combining administrative and clinical data to stratify surgical risk. Ann Surg. 2007;246: 875-885.

18.

Hall BL , Hirbe M. , Waterman B. , Boslaugh S. , Dunagan WC Comparison of mortality risk adjustment using a clinical data algorithm (American College of Surgeons National Surgical Quality Improvement Program) and an administrative data algorithm (Solucient) at the case level within a single institution. J Am Coll Surg. 2007;205:767-777.

19.

Scott IA , Thomson PL , Narasimhan S. Comparing risk-prediction methods using administrative or clinical data in assessing excess in-hospital mortality in patients with acute myocardial infarction . Med J Aust. 2008;188:332-336.

20.

Wang RY , Strong DM Beyond accuracy: what data quality means to data consumers. J Manage Inform Syst. 1996;12(4): 5-34.

21.

Strong DM , Lee YW , Wang RY Data quality in context. Commun ACM. 1997;40(5):103-110.

22.

Garson D. Logistic regression. http://www.2.chass.ncsu.edu/garson/PA765/logistic.htm#estimates. Accessed April 15, 2008 .

23.

Glance LG , Dick AW , Osler TM , Mukamel DB Does date stamping ICD-9-CM codes increase the value of clinical information in administrative data? Health Serv Res. 2006 ;41: 231-251.

24.

Iezzoni LI , Daley J. , Heeren T. , et al. Identifying complications of care using administrative data . Med Care. 1994;32:700-715.

25.

Iezzoni LI , Heeren T. , Foley SM , Daley J. , Hughes J. , Coffman GA Chronic conditions and risk of in-hospital death. Health Serv Res. 1994;29:435-460.

26.

University HealthSystem Consortium. Risk adjustment methodology for the clinical data base. https://cimprod.uhc.edu/pv/include/document/Risk%20Adjustment%20Methodology.pdf . Accessed April 17, 2008.

27.

Averill R. , Goldfield N. , Hughes J. , et al. All patient refined diagnosis related groups (APR-DRGs): methodology overview. http://www.hcup-us.ahrq.gov/db/nation/nis/APR-DRGsV20MethodologyOverviewandBibliography.pdf. Accessed April 17, 2008.

28.

Hosmer D. , Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: Wiley; 2000.

29.

Hanley J. , McNeil B. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983 ;148:839-844.

30.

Davenport DL , Henderson WG , Khuri SF , Mentzer RM Jr. Preoperative risk factors and surgical complexity are more predictive of costs than postoperative complications: a case study using the national surgical quality improvement program (NSQIP) database. Ann Surg. 2005;242:463-468.

31.

Dimick JB , Chen SL , Taheri PA , Henderson WG , Khuri SF , Campbell DA Jr. Hospital costs associated with surgical complications: a report from the private-sector national surgical quality improvement program. J Am Coll Surg . 2004; 199:531-537.

32.

Surgical Care Improvement Project. Preventing surgical complications. http://www.medqic.org/dcs/ContentServer?cid=1122904930422&pagename=Medqic%2FContent%2FParentShellTemplate&parentName=Topic&c=MQParents . Accessed April 17, 2008.