Abstract
Objective:
Medical specialty boards have a Maintenance of Certification (MOC) paradigm whose intention is to ensure high-quality patient care. How the patient experience is affected by physician MOC enrollment/participation is unknown. Our goal was to determine if patient experience is associated with physician board certification and MOC status.
Methods:
We analyzed physician experience and MOC databases to determine the relationships among physicians’ patient experience national percentile rankings and board certification status and MOC enrollment and activity status.
Results:
Board-certified physicians enrolled in MOC did not have statistically significant different patient experience scores compared to board-certified physicians not enrolled in MOC. Mid-career physicians enrolled in MOC had patients more likely to recommend them and reported higher confidence in them. Patients did not perceive physicians participating in MOC patient safety modules as more cautious in providing patient care.
Conclusion:
Although most analyses did not demonstrate significant differences in patient experience scores for physicians actively participating in MOC compared to those not, some differences were noted. Higher provider-specific patient experience scores were noted, particularly for mid-career physicians.
Background
The American Board of Medical Specialties (ABMS) Maintenance of Certification (MOC) program is a multifaceted program that focuses on lifelong learning by physicians who demonstrate that they are keeping pace with the latest advances in their specialty. The ABMS MOC program requires most certified specialists to maintain certification on a continuous and cyclic basis by participating in a 4-part process that focuses on competency assurance or development in 6 core domains adopted by ABMS in 1999 (1). The 4 components of the process focus on knowledge assessment, practice performance, communication skills, and patient safety (1). The MOC program was initiated in 2000, but the pace of recertification accelerated since 2006. As of December 31, 2012, approximately 450 000 board-certified specialists and subspecialists meet MOC requirements (1). State medical licensure boards and third-party payers are increasingly interested in physician participation in MOC.
While the number of specialists engaged in the process continues to grow each year, the program has drawn some criticism and concern from physicians who assert it is too time-consuming, complex, and expensive (2,3). Younger physicians have suggested that the so-called “grandfathers” (generally specialists who were certified before 1990 and received time-unlimited credentials) should also face the rigors of recertification (4).
Overall, the goal of MOC is to improve physician performance and improve patient outcomes (1). There have been studies in several specialties supporting the value of certification and MOC participation in terms of improved patient care (5 –8). Additionally, a few specialties have shown a positive relationship between MOC examination scores and quality of care (9).
Healthcare reform has focused on pursuing high-value healthcare. Healthcare value, strictly speaking, is defined as health outcome per dollar spent (10). However, organizations have proposed a value index that could be defined by the equation: Value equals quality over cost of care (V = Q/C) (11). The denominator—cost—encompasses cost of care over time. The numerator—quality—includes clinical outcomes, patient safety, and patient-reported experiences with care. To assess patient experience, surveys are most commonly administered to focus on asking patients whether or how often they experienced critical aspects of healthcare, including communication with their doctors, understanding their medication instructions, and the coordination of their healthcare needs (12).
In addition to several third-party patient experience survey vendors, the Centers for Medicare & Medicaid Services (CMS) also develop, implement, and administer several different patient experience surveys designed to reliably assess the experiences of a large sample of patients using standardized questions. The surveys focus on matters important to patients and for which patients are the best source of information. These surveys are an integral part of efforts to improve healthcare in the United States and, in some cases, are used in value-based purchasing (pay-for-performance) initiatives. Instead of only paying for the number of services provided, CMS also pays for providing high-quality services that are measured clinically, administratively, and through the use of patient experience care surveys (12). If physician enrollment and active participation in MOC are associated with an increased positive patient experience, this would add to the evidence base for the benefits of the ABMS MOC program.
Given the massive undertaking of MOC, it would be helpful to add a patient experience perspective to the current literature. The goal of this study was to determine if the patient experience, when objectively measured by standardized patient experience scores, is associated with the board certification and MOC status of the physician. Currently there are absent data regarding the relationship between board certification, MOC enrollment status, degree of engagement within MOC, and the patient experience. Studies are needed to evaluate the influence, or lack thereof, of MOC enrollment on the overall patient experience.
Methods
To examine the relationship between board certification, physician MOC status, and the patient experience, we combined 2 information databases maintained by Marshfield Clinic. The project was approved by the Marshfield Clinic Research Foundation Institutional Review Board.
The patient experience database is maintained by Marshfield Clinic Institute for Quality, Innovation & Patient Safety (IQIPS). IQIPS utilizes a third-party vendor (Press Ganey) to provide each physician with a national percentile rank score on each of 13 care provider questions (Table 1). During this study period, Press Ganey randomly selects patients seen by Marshfield Clinic providers and mails to them a paper-based patient experience survey with a return envelope. Press Ganey collates the data for each provider, provides national comparative data, and creates reports of results for the integrated healthcare delivery system. To receive a percentile rank, physicians must have received a minimum of 30 patient experience survey responses during the measurement time interval (every 3 months) (13). The national percentile rank scores are based on the population of care providers in the United States, grouped by specialty type, that use Press Ganey for their patient experience scores (14). Our analysis included only physicians, not midlevel providers.
Thirteen Press Ganey Care Provider Patient Questions.
The MOC database is maintained by the Marshfield Clinic Division of Education Continuing Medical Education section. The database consists of Marshfield Clinic physician self-reported data regarding board certification status, MOC enrollment status (including those enrolled and not enrolled), and completion or participation status with MOC component 3 examinations, MOC self-assessments, MOC practice performance assessments, MOC patient safety modules, and MOC peer surveys.
Combining the aforementioned databases, we connected physicians’ patient experience national percentile rankings to their board certification status, MOC enrollment, and MOC activity status. Physicians were de-identified after merging the patient experience database with the MOC database to perform statistical analysis. Inclusion criteria consisted of physicians in the Marshfield Clinic system who had available patient experience national percentile rank scores in the designated study period and physicians who were also in the MOC database. Exclusion criteria included physicians in the Marshfield Clinic system who did not have available patient experience national percentile rank scores and physicians not in the MOC database. The data sets collected on physicians were in the time period from July 1, 2012, through December 31, 2012.
The MOC activity level was determined by physicians’ responses to four different participation-related questions regarding MOC activities (participation or completion of self-assessment modules, performance in practice assessment projects, patient safety modules, and peer surveys). Physicians were considered to be “enrolled and active” in MOC if they answered yes to participating in any of the 4 activities and considered “enrolled but not active” if they answered no to all 4 participation-related questions.
Physicians were grouped into late career physicians (board certified prior to 1991), mid-career physicians (board certified between 1991 and 2005), and early career physicians (board certified after 2005). A pairwise statistical analysis of the aforementioned comparison groups was performed and t tests were run on the grouped data.
Results
Overall, 396 Marshfield Clinic physicians were included in the study, including 376 board-certified physicians and 20 nonboard-certified physicians. Nonboard-certified physicians were excluded from the analysis. Among the board-certified physicians, 270 physicians (73%) reported enrollment in MOC, and 103 physicians reported not being enrolled in MOC. Board-certified physicians enrolled in MOC did not have statistically significant different overall patient experience scores compared to the board-certified physicians not enrolled in MOC. See Figure 1 for a summary of the physician groupings.
Summary of physician groupings (3 physicians left Maintenance of Certification [MOC] status unanswered, excluding them from statistical analysis. Of 270 physicians, 226 answered all of the participation-related questions regarding MOC activities allowing for analysis).
To determine if a physician’s level of activity in MOC impacted their patient experience scores, a comparison between active and not active MOC-enrolled physicians was performed. There were 121 (53.5%) MOC-enrolled physicians categorized as active in MOC, and 105 categorized as not active in MOC. The physicians active in MOC had significantly better patient experience scores compared to the enrolled physicians not active in MOC on patient’s likelihood of recommending the physician to others (P = .05).
Physicians who participated in self-assessment modules did not have statistically significantly improved patient experience scores and did not score higher in patient confidence (P = .43) compared to physicians who did not perform self-assessment modules. Physicians who participated in MOC quality improvement activities did not have statistically significantly improved patient experience scores compared to physicians who did not participate on any of the 13 Press Ganey Care Provider Patient Questions. Physicians who participated in a patient safety module had patients feel they were statistically significantly more concerned regarding their questions or worries compared to physicians who had not (P = .05), but they did not show statistically significantly improved patient experience scores on patients’ perceptions of the adequacy of physician precautions taken to protect their safety (P = .09).
The MOC-enrolled physicians in each career experience category (late career, mid-career, early career) were compared to nonenrolled MOC physicians in each career experience category. Mid-career MOC-enrolled physicians had statistically significant better patient experience scores compared to mid-career nonenrolled MOC physicians on the likelihood of patients recommending the physician to others (P = .04) and patient confidence in the provider (P = .05). No other statistically significant differences were noted among other career experience comparison groups. See Figure 2 for a summary of the study findings.
Summary of study findings.
Discussion
Physician enrollment in MOC is highly valued by patients according to public polling, with the majority of the public stating they would change physicians if their physician failed to maintain certification (4,15). Likewise, CMS encourages and even incentivizes physicians to enroll and actively participate in MOC. Since 2011, CMS has added a 0.5% Physician Quality Reporting System MOC Program incentive for physicians participating more frequently and more substantially in MOC (15). Does the patient experience improve as a result of physician MOC enrollment and participation? This is not known, since to date, the literature is lacking a patient experience perspective.
In this study, our goal was to determine if there was a patient experience difference for the board-certified, MOC actively enrolled physicians. Overall, we did not find a significant, pervasive patient experience advantage for physicians enrolled and actively participating in MOC. However, several comparisons did reach near significance, and other interesting findings are worth further comment.
Firstly, physicians enrolled and active in MOC had patients reporting they were more likely to recommend them to others, they were more confident in their skills as a physicians, and they felt they received more information about medications compared to the patients of physicians who were enrolled but not active in MOC. Most comparisons neared but did not reach statistical significance, raising the possibility that patients may perceive an improved experience when receiving care from physicians enrolled and active in MOC. It is possible that the increased activity in MOC contributed to the higher patient experience scores in the enrolled and active MOC group.
An inverse relationship exists between years of practice and performance with quality measures (4,5). We performed an analysis to determine if there was a relationship between years in practice, MOC enrollment, and patient experience scores. There were no significant differences when comparing early career physicians or late career physicians. However, mid-career physicians enrolled in MOC were more likely to be recommended to others by their patients and instilled more confidence in their patients. It could be suggested that mid-career physicians enrolled in MOC may encounter a patient experience benefit associated with MOC enrollment.
Interestingly, physicians who participated in a patient safety module were not perceived by their patients to be more cautious in providing patient care. This may call to attention a need to assess the current content and effectiveness of ABMS-approved patient safety modules that physicians complete for MOC credit, if patient perceptions of safety are important to impact. Perhaps this finding highlights physician willingness to perform certain required activities to get through the MOC process, but these steps do not result in higher ratings from patients regarding physician attention to patient safety at the point of care.
The self-assessment modules are knowledge assessments aimed to assess and update physician knowledge. We hypothesized that physicians with greater, more up-to-date knowledge would rate higher in patient confidence. However, in our study, patients did not have significantly higher confidence in physicians who completed self-assessment modules compared to physicians who did not complete self-assessment modules.
National standards of assessing the value of care emphasize the patient’s experience of care. While the patient experience involves factors within and outside the physicians control, the Press Ganey instrument allows assessment of a number of physician factors and comparison to other physicians in the same specialty in the same location, same region, or nationally (13). However, many physicians feel that Press Ganey questionnaires do not accurately reflect the physician performance and/or the overall patient experience (16,17). Patient experience surveys may not be a good measure of overall patient experience, because not all patients are capable of going online to complete a survey or may not have literacy skills required to complete the survey, thus skewing the responder population. It has also been argued that patients who are neutral or satisfied with the visit will not take the time to do the somewhat lengthy rating process, while dissatisfied patients are motivated to voice their dislikes, altering the measured patient experience outcomes (16,17).
The study has limitations. These are data from a single institution. The study was performed on an incomplete sample of the physicians within the Marshfield Clinic system, because not every employed physician had information in both databases. Also, there may be recall bias, as physicians were self-reporting their MOC status and level of participation. Additionally, this study only looked at 6 months of patient experience data, which can change significantly over time.
The focus of MOC is and should be ensuring that high-quality care is delivered to patients. Physicians, who pay to participate in these required programs, will want to see the justification. Likewise, certifying boards, third-party payers, and patient advocates will want to see the outcomes. Does being enrolled and active in MOC make a physician a “better doctor”? If not, what needs to be altered with the requirements or program to reach that desired outcome? Patients state they expect their physicians to maintain certification and, in some cases, choose physicians according to whether or not they are enrolled and maintain certification. Yet in our institution, physicians’ patient experience scores were not significantly different. Should a physician’s participation in MOC significantly impact the patient’s experience of care that the physician delivers? If that is to be one goal, our study suggests the ABMS boards may not yet be offering programs that lead to pervasively improved patient experience. Our study also suggests that passive enrollment in MOC may not be as advantageous as active enrollment in MOC, as physicians who were active demonstrated a patient experience advantage over physicians who were enrolled but not active. Additional studies should include other multispecialty institutions to determine how active participation in MOC impacts their patient experience, if at all. If similar results are found, perhaps the design and intended outcomes of various MOC components and activities should be reviewed, always keeping the patient at the center of consideration.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Erik Stratman- Director, American Board of Dermatology (volunteer position); Jessica Morrell-none
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.