Measuring the quality of health care is becoming increasingly important. Quality is often conceptualized as 3 dimensions of care: structures, processes, and outcomes. Unfortunately, there is little consensus about what should be measured—and how it should be measured—when it comes to measuring processes of care related to the conduct of surgical procedures. This article reviews recent advances in surgical quality of care measurement with particular emphasis on processes of care, and evaluates existing measures of technical and nontechnical surgical skills as measures of quality of care in surgery.
Gøtzsche PC, Nielsen M.Screening for breast cancer with mammography . Cochrane Database Syst Rev.2006;(4):CD001877.
3.
World Health Organization.The World Health Report 2000. Health Systems: Improving Performance. Geneva, Switzerland: World Health Organization ; 2000.
4.
Leidecker JK , Bruno AVIdentifying and using critical success factors . Long Range Planning. 1984 ;17:23-32.
5.
Donabedian A.Evaluating the quality of medical care 1966 . Milbank Q. 2005;83:691-729.
6.
Donabedian A.The quality of medical care. Science. 1978; 200:856-864.
7.
Dimick JB, Upchurch GR Jr.Measuring and improving the quality of care for abdominal aortic aneurysm surgery. Circulation. 2008;117:2534-2541.
8.
Khuri SF, Daley J., Henderson W., et al. The Department of Veterans Affairs’ NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998 ;228:491-507.
9.
Fink AS, Itani KM, Campbell DC Jr.Assessing the quality of surgical care. Surg Clin North Am . 2007;87:837-852.
10.
Birkmeyer JD , Dimick JB, Birkmeyer NJO.Measuring the quality of surgical care: structure, process, or outcomes?J Am Coll Surg. 2004 ;198:626-632.
11.
Itani KMF , Webb ALB , Fink ASApproaches to assessing surgical quality of care. Hosp Physician . 2008;44:29-37.
12.
Hall BL, Hamilton BH, Richards K., Bilimoria KY, Cohen ME, Ko CYDoes surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals . Ann Surg. 2009;250:363-376.
13.
Dimick JB, Welch HG, Birkmeyer JDSurgical mortality as an indicator of hospital quality: the problem with small sample size. JAMA. 2004;292:847-851.
14.
Hewitt M.Institute of Medicine Report: Interpreting the Volume-Outcome Relationship in the Context of Health Care Quality. Washington, DC: Institute of Medicine; 2000.
15.
Luft HS, Bunker J., Enthoven ACShould operations be regionalized? An empirical study of the relation between surgical volume and mortality. New Engl J Med. 1979;301: 1364-1369.
16.
Birkmeyer JD , Finlayson SRG, Tosteson ANA, Sharp SM, Warshaw AL, Fisher ESEffect of hospital volume on in-hospital mortality with pancreaticoduodenectomy. Surgery. 1999 ;125:250-256.
17.
Birkmeyer JD , Siewers AE, Finlayson SRG, et al. Hospital volume and surgical mortality in the United States. N Engl J Med. 2002;346:1128-1137.
18.
Houghton A.Variation in outcome of surgical procedures . Br J Surg. 1994;81:653-660.
19.
Begg CB, Cramer LD, Hoskins WJ, Brennan MFImpact of hospital volume on operative mortality for major cancer surgery . JAMA. 1998;280:1747-1751.
20.
Porter GA, Soskolne CL, Yakimets WW, Newman SCSurgeon-related factors and outcome in rectal cancer. Ann Surg. 1998;227:157-167.
21.
Birkmeyer JD , Warshaw AL, Finlayson SRG, Grove MR, Tosteson ANA.Relationship between hospital volume and late survival after pancreaticoduodenectomy. Surgery. 1999;126:178-183.
22.
Roohan PJ, Bickell NA, Baptiste MS, Therriault GD, Ferrara EP, Siu ALHospital volume differences and five-year survival from breast cancer . Am J Public Health. 1998; 88:454-457.
23.
Gillis CR, Hole DJSurvival outcome of care by specialist surgeons in breast cancer: a study of 3786 patients in west of Scotland. BMJ. 1996;312:145-148.
24.
Ho V., Town RJ, Heslin MJRegionalization versus competition in complex cancer surgery. Health Econ Policy Law. 2007;2(Pt 1):51-71.
25.
O’Connor GT, Plume SK, Olmstead EM, et al. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery . JAMA. 1996;275: 841-846.
26.
Hariharan S. , Dey PK, Moseley HSL, Kumar AY, Gora J.A new tool for measurement of process-based performance of multispecialty tertiary care hospitals . Int J Health Care Qual Assur Inc Leadersh Health Serv. 2004;17:302-312.
27.
Bradley EM, Herrin J., Elbel B., et al. Hospital quality for acute myocardial infarction: correlation among process measures and relationship with short-term mortality. JAMA. 2006;296:72-78.
28.
Mismetti P. , Laporte S., Darmon JY, Buchmüller A., Decousus H.Meta-analysis of low molecular weight heparin in the prevention of venous thromboembolism in general surgery. Br J Surg. 2001;88:913-930.
29.
Nelson RL, Glenny AM, Song F.Antimicrobial prophylaxis for colorectal surgery. Cochrane Database Syst Rev.2009; (1):CD001181.
30.
Dellinger EP , Hausmann SM, Bratzler DW, et al. Hospitals collaborate to decrease surgical site infections. Am J Surg. 2005;190:9-15.
31.
SCIP Project Information. http://qualitynet.org/dcs/ContentServer?siteVersion=null&c=MQParents&pagename=Medqic%2FContent%2FParentShellTemplate&cid=1137346750659&parentName=TopicCat. Accessed August 25, 2009.
32.
Patterson P.Surgical Care Improvement Project: four years later, what’s the status?OR Manager. 2009;25(8):1, 7-9.
33.
POISE Study Group.Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial. Lancet . 2008;371:1839-1847.
34.
Keus F., de Jong JA, Gooszen HG, van Laarhoven CJLaparoscopic versus open cholecystectomy for patients with symptomatic cholecystolithiasis . Cochrane Database Syst Rev. 2006;(4):CD006231.
35.
Peeters KC, Marijnen CA, Nagtegaal ID, et al; Dutch Colorectal Cancer Group.The TME trial after a median follow-up of 6 years: increased local control but no survival benefit in irradiated patients with resectable rectal carcinoma . Ann Surg. 2007;246:693-701.
36.
Mamounas E. , Wieand S., Wolmark N., et al. Comparative efficacy of adjuvant chemotherapy in patients with Dukes’ B versus Dukes’ C colon cancer: results from four National Surgical Adjuvant Breast and Bowel Project adjuvant studies (C-01, C-02, C-03, and C-04). J Clin Oncol. 1999;17:1349-1355.
37.
Hannan EL, Popp AJ, Feustel P., et al. Association of surgical specialty and processes of care with patient outcomes for carotid endarterectomy. Stroke. 2001;32:2890-2897.
38.
Main DS, Cavender TA, Nowels CT, Henderson WG, Fink AS, Khuri SFRelationship of processes and structures of care in general surgery to postoperative outcomes: a qualitative analysis. J Am Coll Surg. 2007;204:1147-1156.
39.
Main DS, Henderson WG, Pratte K., et al. Relationship of processes and structures of care in general surgery to postoperative outcomes: a descriptive analysis. J Am Coll Surg. 2007;204:1157-1165.
40.
Shih A., Schoenbaum SCMeasuring Hospital Performance: The Importance of Process Measures . New York, NY: Commonwealth Fund ; 2007.
41.
Burney RE, Van Harrison R.Continuing education for practicing surgeons. In: Mulholland MW, Doherty GM , eds. Complications in Surgery. Philadelphia , PA: Lippincott, Williams & Wilkins;2007:18-24.
42.
Kao LSThomas EJNavigating towards improved surgical safety using aviation-based strategies. J Surg Res. 2008;145: 327-335.
43.
Reznick R., Regehr G., MacRae H., Martin J., McCulloch W.Testing technical skill via an innovative "bench station" examination . Am J Surg. 1997;173:226-230.
44.
Grantcharov TP, Schulze S., Kristiansen VBThe impact of objective assessment and constructive feedback on improvement of laparoscopic performance in the operating room . Surg Endosc. 2007 ;21:2240-2243.
45.
Swift SE, Carter JFInstitution and validation of an observed structured assessment of technical skills (OSATS) for obstetrics and gynecology residents and faculty . Am J Obstet Gynecol. 2006;195:617-623.
46.
Martin JA, Regehr G., Reznick R., et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84:273-278.
47.
Khan MS, Bann SD, Darzi AW, Butler PEM.Assessing surgical skill using bench station models. Plast Reconstr Surg. 2007;120:793-800.
48.
Bodle JF, Kaufmann SJ, Bisson D., Nathanson B., Binney DMValue and face validity of objective structured assessment of technical skills (OSATS) for work based assessment of surgical skills in obstetrics and gynaecology . Med Teach. 2008; 30:212-216.
49.
Kishore TA, Pedro RN, Monga M., Sweet RMAssessment of validity of an OSATS for cystoscopic and ureteroscopic cognitive and psychomotor skills. J Endourol. 2008;22:2707-2711.
50.
Vassiliou MC , Feldman LS, Andrew CG, et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005;190:107-113.
51.
Datta V., Bann S., Mandalia M.The surgical efficiency score: a feasible, reliable, and valid method of skills assessment. Am J Surg. 2006;192:372-378.
52.
Beard JD, Choksy S., Khan S.Assessment of operative competence during carotid endarterectomy. Br J Surg. 2007;94: 726-730.
53.
Vassiliou MC , Ghitulescu GA, Feldman LS, et al. The MIS-TELS program to measure technical skill in laparoscopic surgery: evidence for reliability . Surg Endosc. 2006;20:744-747.
54.
FLS-SAGES Fundamentals in Laparoscopic Surgery. http://www.flsprogram.org/. Accessed August 26, 2009.
55.
Camarillo DB , Krummel TM, Salisbury JK Jr.Robotic technology in surgery: past, present, and future. Am J Surg. 2004;188(4A Suppl):2S-15S.
56.
Gawande AA, Zinner MJ, Studdert DM, Brennan TAAnalysis of errors reported by surgeons at three teaching hospitals . Surgery. 2003;133:614-621.
57.
Rogers SO Jr , Gawande AA, Kwaan M., et al. Analysis of surgical errors in closed malpractice claims at 4 liability insurers. Surgery . 2006;140:25-33.
58.
Yule S., Flin R., Paterson-Brown S., Maran N.Non-technical skills for surgeons in the operating room: a review of the literature . Surgery. 2006;139:140-149.
59.
GMC-Licensing and Revalidation. http://www.gmc-uk.org/doctors/licensing/revalidation/relicensing.asp. Accessed August 28, 2009.
60.
The Non-Technical Skills for Surgeons (NOTSS) System Handbook v1.2. Structuring observation, rating and feedback for surgeons’ behaviours in the operating theatre. http://www.abdn.ac.uk/iprc/notss
61.
De Leval MR , Carthey J., Wright DJ, Farewell VT, Reason JTHuman factors and cardiac surgery: a multi-centre study. J Thorac Cardiovasc Surg. 2000;119:661-672.
62.
Baldwin PJ, Paisley AM, Paterson-Brown S.Consultant surgeons’ opinions of the skills required of basic surgical trainees. Br J Surg. 1999;186:1078-1082.
63.
Roth EM, Christian CK, Gustafson M., et al. Using field observations as a tool for discovery: analyzing cognitive and collaborative demands in the operating room. Cogn Tech Work. 2004;6:148-157.
64.
Healey AN, Undre S., Vincent CADeveloping observational measures of performance in surgical teams . Qual Saf Heath Care. 2004;13:i33-i40.
65.
Ward M., MacRae H., Schlachta C., et al. Resident self-assessment of operative performance. Am J Surg. 2003;185:521-524.
66.
Satish U., Streufert S., Marhall R., et al. Strategic management simulations is a novel way to measure resident competencies. Am J Surg. 2001;181:557-561.
67.
Dickinson I. , Watters D., Graham I., Montgomery P., Collins J.Guide to the assessment of competence and performance in practising surgeons. ANZ J Surg. 2009;79:198-204.
68.
O’Connor P., Hormann HJ, Flin R., Lodge M., Goeters K.; the JARTEL Group.Developing a method for evaluating crew resource management skills: a European perspective. Int J Aviat Psychol. 2002;12:263-285.
69.
Sevdalis N. , Davis R., Koutantji M., Undre S., Darzi A., Vincent CAReliability of a revised NOTECHS scale for use in surgical teams. Am J Surg. 2008;196:184-190.
70.
Mishra A., Catchpole K., McCulloch P.The Oxford NOTECHS system: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre. Qual Saf Heath Care. 2009;18:104-108.
71.
Yule S., Flin R., Maran N., Rowley D., Youngson G., Paterson-Brown S.Surgeons’ non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system. World J Surg . 2008;32:548-556.
72.
Yule S., Rowley D., Flin R., et al. Experience matters: comparing novice and expert ratings of non-technical skills using the NOTSS system. ANZ J Surg. 2009;79: 154-160.
73.
Daley J., Forbes MG, Young GJ, et al. Validating risk-adjusted surgical outcomes: site visit assessment of process and structure. National VA Surgical Risk Study. J Am Coll Surg. 1997;185:341-351.
74.
Shroyer AL, London MJ, Sethi GK, Marshall G., Grover FL, Hammermeister KERelationships between patient-related risk factors, processes, structures, and outcomes of cardiac surgical care: Conceptual models. Med Care.1995;33:OS26-OS34.
