Abstract
Introduction:
The primary aims of this study were to assess the learning curve effect of robot-assisted radical prostatectomy (RARP) in a large administrative database consisting of multiple U.S. hospitals and surgeons, and to compare the results of RARP with open radical prostatectomy (ORP) from the same settings.
Materials and Methods:
The patient population of study was from the Premier Perspective Database (Premier, Inc., Charlotte, NC) and consisted of 71,312 radical prostatectomies performed at more than 300 U.S. hospitals by up to 3739 surgeons by open or robotic techniques from 2004 to 2010. The key endpoints were surgery time, inpatient length of stay, and overall complications. We compared open versus robotic, results by year of procedures, results by case volume of specific surgeons, and results of open surgery in hospitals with and without a robotic system.
Results:
The mean surgery time was longer for RARP (4.4 hours, standard deviation [SD] 1.7) compared with ORP (3.4 hours, SD 1.5) in the same hospitals (p<0.0001). Inpatient stay was shorter for RARP (2.2 days, SD 1.9) compared with ORP (3.2 days, SD 2.7) in the same hospitals (p<0.0001). The overall complications were less for RARP (10.6%) compared with ORP (15.8%) in the same hospitals, as were transfusion rates. ORP results in hospitals without a robot were not better than ORP with a robot, and pretreatment co-morbidity profiles were similar in all cohorts. Trending of results by year of procedure showed no differences in the three cohorts, but trending of RARP results by surgeon experience showed improvements in surgery time, hospital stay, conversion rates, and complication rates.
Conclusions:
During the initial 7 years of RARP development, outcomes showed decreased hospital stay, complications, and transfusion rates. Learning curve trends for RARP were evident for these endpoints when grouped by surgeon experience, but not by year of surgery.
Introduction
S
We identified the Premier Perspective Database as a potential source for a learning curve study, as it includes a broad range of hospitals and surgeons, 9 –13 and can separately identify the technique of a prostatectomy, unlike many other databases that can only differentiate minimally invasive (robotic or laparoscopic) versus open. We focused our study on the strengths of the administrative database, as derived from the billing coded source data: perioperative parameters, complications, and correlations with surgeon case volume. The dataset is limited, however, to inpatient events immediately following the surgical procedure.
Materials and Methods
The source population was derived from Premier Research Services® (Premier, Inc., Charlotte, NC) Perspective Database, the largest, hospital, clinical, and economic database in the United States, developed to provide outcome quality and resource utilization benchmarking to its subscribers. The service contains inpatient and outpatient hospital-based billing and diagnostic codes from hospital-based clinics from a geographically diverse set of hospitals. The billing data include a date-stamped log of all billed items from procedures, medications, laboratory, diagnostic, and therapeutic services at the individual patient level. To our knowledge, no published studies using this database exist for prostate cancer, although representative studies in thoracic surgery and gynecology, among others, are available, for examples, of questions examined with this resource. 10 –13 This database allows radical prostatectomy (RP) procedures to be further identified as open radical prostatectomy (ORP), laparoscopic radical prostatectomy (LRP), or RARP, and hospitals can be subcategorized as offering RARP during this time frame or not. Surgeons can be identified in the dataset to assess case volume; however, the queries are blind to surgeries performed before the dates of study, and any activity at non-Premier hospitals.
The key endpoints of study feasible from the dataset were surgery time (incision to closure, i.e., excluding anesthesia time), inpatient stay, and complications. Surgery time and inpatient stay were objective data points captured by Premier Perspective. Complication rates were derived via inpatient stay diagnosis codes added to the record that would be consistent with a complication in a patient admitted for elective prostate surgery. Key diagnostic codes were queried again based upon a similar study by Shih et al. 14 yielding the final categories: medical miscellaneous, anastomotic stricture, surgical miscellaneous, respiratory, cardiac, genitourinary, thromboembolic, wound/bleeding, vascular, transfusion, and mortality. The query methods were identical for all surgical modalities. The data do not permit Clavien-style complication grading, 15 but the descriptors of the diagnosis would mostly fall into grades 3–5, that is, major complications. Co-morbidity was assessed by the All Patient Refined Diagnosis Related Groups (APR-DRGs), which is Premier's preferred co-morbidity tool, as it is designed to predict resource demands, rather than mortality. 11
Prior to undertaking this study we obtained IRB approval from the MD Anderson Cancer Center and Tulane University Medical School. We requested a complete data set of all RP procedures from 2004 through 2010 (7 full years of data) that were performed at Premier's source hospitals. From that, we identified four useful cohorts for comparison: (1) ORP performed at any Perspective hospital (ORP-All; n=43,964), (2) ORP performed at a Perspective hospital with a robotic system in place at the same time interval (ORP-robotic hospital [RH]; n=30,124), (3) RARP (n=27,348), and (4) LRP (n=733). Given the objectives of this study and the very small size of the LRP cohort, this group was dropped from further analysis.
It is unknown which robotic system models were used for cases included in this study, nor whether this influences outcomes. For future reference, the company notes the following breakdown in systems sold in the United States: 2004, daVinci=92; 2005, daVinci=281+daVinci S=6; 2006, daVinci=292+daVinci S=129; 2007, daVinci=294+daVinci S=300; 2008, daVinci=279+daVinci S=545; 2009, daVinci=231+daVinci S=582+daVinci Si=214; and 2010, daVinci=160+daVinci S=579+daVinci Si=545. Note however, that as time goes by, previously sold systems may later go out of service.
Data analysis
Raw data from querying the Premier database were transferred directly to an independent statistical services contractor, Axistat, Inc. (San Francisco, CA), for analysis. Continuous variables were compared using a Student's t-test while discrete variables were compared using a chi-squared test. Comparisons for trends over time or surgical case volume used the Jonckheere–Terpstra test. 16 A p-value of <0.05 was considered statistically significant. Analyses were carried out using SAS version 9.2.1 (SAS Institute, Cary, NC).
Key endpoints of study
The data queries can provide objective reports on length of hospital stay, surgery time, and most any complication that results in a new diagnostic code.
Results
We identified 71,312 RP performed at more than 300 hospitals by up to 3739 surgeons by open or robotic technique (excluding LRP as noted). Table 1 displays the relevant demographic and key endpoints for the three focused cohorts—ORP at non-RH, ORP-RH, and RARP. Within the hospitals with a robot, the mean time for ORP was 3.4 hours (standard deviation [SD]=1.5) and for RARP 4.4 hours (SD=1.7; p<0.0001). The mean length of stay for open was 3.4 days (SD=2.7) and for RARP 2.2 days (SD=1.9; p<0.0001). The overall complication rate was 15.8% for open and 10.6% for robotic (p<0.0001). Specific complication rates, shown in Table 2, were favorable to robotic and statistically significant: stricture 4.8% versus 3.2%, respiratory 2.3% versus 1.4%, cardiac 1.5% versus 0.6%, and blood transfusion 11.5% versus 2.3%. In the combined open cohorts with any discharge complications, they were single in 60% and multiple in 40%. The same analysis for robotic was single in 60% and multiple in 40%.
Data are for all radical prostatectomy cases performed at Premier hospitals from 2004 to 2010.
p-Values are for RARP vs ORP (robotic hospitals), RARP vs ORP (non-robotic hospitals), and ORP (robotic hospitals vs non-robotic hospitals). Morbid obesity=BMI >35. Surgery time is the “skin-skin” time; that is, it excludes additional room/anesthesia time before and after the procedure.
SD=standard deviation; APR-DRG=All Patient Refined Diagnosis Related Group (APR-DRG) uses diagnostic, therapeutic, and demographic attributes to classify patients with similar resource demands (minor, moderate, major, and extreme); LOS=length of stay; RARP=robot-assisted radical prostatectomy; ORP=open radical prostatectomy; NA=not applicable.
Data are for all radical prostatectomy cases performed at Premier hospitals from 2004 to 2010.
See the “Discussion” section for further clarification, as “anastomotic strictures” during an inpatient stay likely do not represent complications from the radical prostatectomy.
Misc.=miscellaneous; p-values are for RARP vs ORP (robotic hospitals), RARP vs ORP (non-robotic hospitals), and ORP (robotic hospitals vs non-robotic hospitals).
The separate cohort of ORP at non-RHs is also displayed in Tables 1 and 2 to explore the hypotheses that patient selection and/or quality of surgery may be different at these locations. There were fewer teaching hospitals comprising ORP-non-RH versus ORP-RH—21% versus 39% (p=0.0003). The differences in surgery time and LOS are statistically different but not likely clinically meaningful; however, ORP-non-RH had slightly higher complications at 17.9% versus 15.8% for ORP-RH. The array of specific complications (Table 2) shows a similar pattern of higher complications after ORP than RARP, and the highest transfusion rate at 17.1%. Table 1 also shows a Premier Perspective proprietary measure of co-morbidity called the APR-DRGs. The incidences of patients with minor, moderate, major, extreme, and unknown co-morbidity were statistically equivalent between the three cohorts. Complication rates by year of surgery are shown in Figure 1, and demonstrate a consistent trend for lower complication rates for RARP compared with either ORP cohort; however, there is no visible learning curve trend for high complications in year 1 (2004) followed by a decline.
Overall complication rates at Premier Perspective Hospitals from 2004 to 2010 during initial hospitalization: robotic, open at robotic hospitals, and open at non-robotic hospitals.
Next, we re-examined the key endpoints by surgeon case volume. There were 226 surgeons in this database who had completed 25 or more robotic cases, 132 surgeons who had completed 50 or more robotic cases, and 67 surgeons who had completed 100 or more cases. Table 3 shows the cohorts of robotic cases performed by surgeons in their first six 25 case increments. The conversion rate declined from 1.13% to 0.18%, the time decreased 5.0 to 3.9 hours, and the complications decreased 11.75% to 8.95%. The ARP-DRG measure was highly consistent within these 25 case increments, indicating fairly consistent patient selection by co-morbidity even in the first 25 cases.
We then performed an additional query of the more extreme comparison of surgeons with <25 total RARP cases versus >25 (unlimited case volume for a surgeon, n=3236 cases vs 24,112). In comparing the sixth cohort of 25 cases, versus 25 and up, there were no significant improvements in conversions, operative time, or length of stay; however, total complications improved further to 5.7%—a demonstration of how results become affected by including surgeons who have “exited” the learning curve at some point.
Figure 2 shows another view of the learning curve for RARP, where the more experienced surgeons' outcomes are plotted in multiples of 100 (black lines) versus less-experienced surgeons plotted in multiples of 25 (red line)—both series starting with the surgeons' first cases in the dataset.
Learning curve for surgeons by experience level, grouped into bundles of 25 (red) or 100 (black) cases: conversion rates, complication rates, surgery time, and hospital stay.
Discussion
Administrative databases offer prostate surgery outcomes, investigators, large volumes of cases, and events that are linked to definable billing/diagnostic codes, and require minimal additional expert interpretation. These databases may require a cost to access the data, but the investigators can quickly move to data analysis and interpretation. Therefore, endpoints, such as surgical time, hospital stay, and several Clavien grade 3–5 complications, are well captured with this method, and there is the added benefit of studying large numbers of surgeons and hospital settings. On the other hand, cancer control and quality of life endpoints that require additional follow-up, expert interpretation, and/or patient reported outcomes require a more costly research effort/infrastructure, and most of our knowledge of these outcomes derives from voluntary reporting from high-volume experts and early adopters.
The key questions posed by our study are whether or not learning curve trends can be detected in a large administrative dataset and how RARP outcomes have compared with ORP outcomes in the first 7 years of RARP adoption. For the RARP learning curve question, the data demonstrate improvements in conversion rates, complications, surgery time, and hospital stay (Fig. 2) when looking at surgeon volume. In particular, complication rates start in the 12% range initially, and decline to the 8% range within the first 100 cases, and may decline further in the extended case volume (>100 cases) to under 6%. Surgery time and hospital stay time also improve noticeably in the first 100 cases, and additional improvements beyond 100 cases. By contrast, the year of surgery did not demonstrate noteworthy trends to suggest that our techniques of training and instruction are improving with time. Our experiences living through the era of study, 2004–2010, would suggest to us that this was mostly an era of surgeons learning in the post-graduate career using brief hands-on training courses, a few proctored cases, and then straight to live-case learning. By contrast, learning curve studies moving forward will have a growing influence from surgeons who learned robotics as part of formal surgical residency/fellowship, 17 extended post-graduate training, 18 and/or have access to surgical simulation. 19
Aside from the influence of the learning curve, the available comparisons between RARP and ORP demonstrate equivalent to favorable outcomes for RARP. As patients do not wish to suffer complications under any circumstance, the reduction in RARP complications would add value to their evaluation as to which technique to choose. For example, inpatient mortality rates were the lowest for robotic—0.03% versus 0.09%–0.14% for ORP (Table 2). In a formal cost-effectiveness study, the reduced complications and hospital stay would certainly factor into the total analysis. Currently the cost reporting within the Premier hospital network is not yet uniform enough to add such an analysis to our study, although some authors have included one despite the limitations. 20
Our study has strengths and limitations worth highlighting. The data source includes a large catchment of surgeries from multiple hospitals and surgeons, thereby closely correlating with generalizable results of surgery. The data capture was prospective, and performed/monitored by an independent third party (Premier, Inc.). Premier undertakes an extensive multistep data validation and correction process, which includes more than 95 quality assurance checks. Once the validations are complete, the data are transferred to Premier's warehouse where they are maintained for health service research. In our study, the requested data were delivered directly to a third party analytics firm (Axistat, Inc.) for final production of tables and statistics for the investigators.
The limitations of this study include the lack of cost figures, functional outcomes, pathologic endpoints, and disease-free status, and therefore is only a partial analysis of comparative effectiveness. The other extreme of study strength is a well-conducted single-institution study such as Di Pierro et al. 18 that can capture a full array of endpoints and show RARP with better positive surgical margins, major complications, urinary continence, and erectile function.
As with all non-randomized study designs, there may exist unknown patient selection differences between the open and RARP cohorts. We were particularly concerned with the influences of patient selection and quality of surgery between open surgeries performed at hospitals with versus without a robot. However, we found that the overall complication rate was lower for ORP in robotic versus non-RHs. The co-morbidity assessment was also identical across the cohorts (Table 1). These findings would argue against the conclusion that lower complications in RARP were due to selecting the healthiest patients. It is common knowledge that most surgeons attempting his/her initial RARP cases would select ideal surgical candidates; however, as Table 3 demonstrates in the ARP-DRG ratios, such selection is not even visible in the first 25 cases versus subsequent ones. As noted in the methods, this database is limited to inpatient events, and therefore does not capture additional complications that may occur at different rates in the outpatient setting. In addition, the learning curve analysis may be effected in that open surgery was always assumed to be an established technique in this time period, whereas robotic case learning curves were built on the estimated technology installation date.
It is also noteworthy that although administrative databases may provide large sample sizes of patients and a representative cross-section of surgeons, the data capture is mostly designed for hospital administration clients, rather than clinical outcome research, and in some cases may lack expert refinement in data capture. As an example, we designed our study, requested the data, and began our analysis, only to find anastomotic stricture rates of 3.2%–5.0%. However, for an inpatient data study, this would make little sense, as patients develop strictures weeks to months later, and not in the initial hospital stay. When we looked at each of 26 individual diagnostic codes used to define “anastomotic stricture,” we could only conclude that the events represented a mix of surgeons managing pre-existing strictures at the time of RP, or coders picking up on certain language elements of the documentation and adding these codes in ways that would have been revised by a clinician or researcher. Examples may include a complex bladder neck reconstruction from a large gland/median lobe, or lower urinary tract obstructive symptoms. All other complication categories such as the “vascular” category (23 separate codes) are consistent with inpatient care, such as carotid artery occlusion or major vein thrombosis with embolism. These examples point to possible biases in comparing studies based upon administrative data capture versus physician/researcher-based interpretation of events and capture.
Our study is consistent with several others with respect to comparing open versus robotic prostatectomy. 21,22 Our dataset showed <1% use of LRP in this era, and we can probably assume that the LRP influence is also minimal in other studies from the United States. 23 Hu et al. 23 also showed that minimally invasive radical prostatectomy had fewer complications and shorter hospitalization, but higher anastomotic strictures and salvage therapy rates, unless higher surgeon volume was considered. This also points to the importance of the learning curve and training efforts. Sharma et al. 24 looked at the learning curve of two surgeons for RARP in 500 cases. There was no open cohort, but improvements were seen throughout the series in several endpoints such as operative time. Agarwal et al. 25 looked at comprehensive complications after RARP and found an overall rate of 9.8%. Of 368 complications in 326 patients, 81.3% occurred within 30 days, 4.6% from 31 to 90 days, and 14.1% after 90 days (driven by anastomotic stricture and lymphocele formation). Our study found 10.6% complications during initial inpatient stay, and therefore a comparable comprehensive complication study would show a higher rate, but by comparing a larger population of surgeons versus a single high-volume center.
Conclusions
During the initial 7 years of robotic RP development, an administrative database study of large numbers of patients and surgeons undergoing open and robotic RP demonstrated fewer complications, shorter length of stay, and reduced transfusion rates for robotic cases. Learning curve trends for robotics were evident for these endpoints when grouped by surgeon experience, but not by year of surgery.
Footnotes
Disclosure Statement
Funding for access to the Premier Database, and Axistat services: Intuitive Surgical. Authorship funding (J.W.D. and R.T.): none. Primary authorship: J.W.D. Data analysis: J.W.D. and U.K. Article editing and approval: J.W.D., J.G., U.K., and R.T.