Abstract
Cleveland Clinic’s Imaging Institute implemented a “Commitment to Respect” initiative and survey process in March 2013 with the goal of improving communication and teamwork among employees and, in turn, improving patient satisfaction. Since the rollout of this initiative, we have worked to increase acceptance of the process, improve the survey response, and more fully incorporate results into staff development. Now that we have 4 years of annual data for analysis, we can state based on feedback from caregivers that the process has had a positive effect on relationships between radiologists and frontline clinical staff. The survey identifies behaviors that individuals were not previously aware of, allowing staff members to make changes based on this feedback. Additionally, institute leaders are able to reinforce the respectful behaviors of those scoring well and support the efforts of those whose scores need improvement. Both scenarios are reinforced through the radiologist annual performance review process.
Introduction
As the focus in health care shifts to enhancing patient experience, institutions have begun to seek out innovative ways to improve their performance in this area. Cleveland Clinic has consistently received high scores on the Press Ganey outpatient survey and the Hospital Consumer Assessment of Healthcare Providers and Systems. As a leading health-care provider, the Clinic continues to search for ways to improve patient experience.
Previous research has demonstrated that improvements in employee satisfaction and respect among coworkers can lead to enhanced patient experience (1 –3). To this end, Cleveland Clinic’s Imaging Institute (comprising more than 180 radiologists, nearly 500 technologists, more than 90 nurses, and more than 160 administrative personnel at the main campus and throughout 9 regional hospitals) implemented an initiative (“Commitment to Respect”) 5 years ago to increase respect among coworkers. The goal of this program is to improve communication and teamwork among Imaging employees and, in turn, to improve patient care and satisfaction. Initial results for the survey have been promising, as reported previously (4). We are now working to improve the survey response rate and to more fully incorporate the survey results into staff development.
In this article, we report on our current work with the Commitment to Respect initiative in the hope that our experience will offer insight into other health-care institutions seeking to improve employee satisfaction and patient experience.
Background
The initial process to develop the Commitment to Respect survey has been described previously (4). Briefly, at an Imaging Institute staff retreat in 2011, a panel of nurses, technologists, and administrative staff spoke about their experiences working within the institute. Their comments revealed that these employees’ feelings about their work were greatly affected by the level of respect that they received from institute staff. The Imaging Institute formed a committee to address this problem, ultimately deciding to create a set of guiding principles called Commitment to Respect. These principles were designed to improve communication, enhance teamwork, and improve employee satisfaction among all employees of the Imaging Institute; this in turn was expected to enhance patient experience.
A core team of radiologists and internal and external organizational development consultants developed a survey to gather information from radiologists and their peers about how closely radiologists were adhering to the Respect guidelines and where there was room for improvement. For the initial survey, each radiologist was assessed by 8 to 10 respondents (including other radiologists, nurses, technologists, and administrative staff). Anonymized results from each survey were provided to the section heads, who shared these results with their radiologists in one-on-one meetings.
The survey has now been conducted 4 times annually. Although causality cannot yet be determined, patient experience scores for the institute have improved since the rollout of the Respect survey (Figures 1 –3). Additionally, the Respect scores for radiologists given by nurses have increased with each administration of the survey.
Trend upward in patient experience scores (Press Ganey) at family health centers.
Trend upward in patient experience scores (Press Ganey) at main campus.
Trend upward in employee engagement scores.
As a final step in this process, we created a tool called the Staff Development Template, in which the radiologists can document their clinical, interpersonal, teamwork, and leadership strengths. The tool also allows the radiologists to note their plans for increasing their skills in these areas and to discuss how these improvements will affect their short- and long-term career goals.
Request for Feedback
To ensure that the information we were receiving from the Respect surveys provided actionable feedback to the radiologists, in August 2013, we decided to survey the radiologists about the content and implementation of the Respect survey. To this end, we developed a survey that asked about various aspects of the Commitment to Respect initiative, such as whether the radiologists had received their results, how valuable the feedback they received was, and how the survey could be improved in the future. In sum, respondents were asked 9 questions. The first 6 questions were multiple choice; the remaining 3 solicited open-ended feedback (Table 1).
Survey Questions.
All of the Imaging Institute’s physicians were asked by the department chair to complete the survey by e-mail. Because responses were anonymous, general e-mail reminders were sent to all, with expressions of appreciation for those who had completed the survey.
Results of Feedback Survey
A total of 97 (54%) radiologists completed the survey. Most respondents answered the first 6 questions, and nearly one-third of respondents gave input on the 3 open-ended questions. This lower response rate for open-ended questions is quite common in survey research (5), as answering open-ended questions requires more time and greater effort on the part of the respondent.
The responses reflected a wide range of opinions about the value of the Commitment to Respect initiative. When asked about the overall value of the Respect survey, some respondents answered that they felt there was “no value” in the Respect survey process, whereas others answered that they found the process very valuable. Most respondents felt that the Respect survey was valuable or somewhat valuable (Figure 4).
Responses to survey question regarding how valuable the Commitment to Respect survey results were overall.
Results for the questions about how valuable the feedback would be for radiologists as they interact with caregivers and patients followed a similar pattern. For example, 75% of radiologists believed the feedback was beneficial when interacting with caregivers and 67% when interacting with patients; 71% believed the feedback was beneficial overall. Most respondents said that the feedback was valuable or somewhat valuable. Respondents were slightly more likely to say that the feedback would be valuable in their interactions with other Imaging caregivers than in their interactions with patients (Figure 5).
Responses to survey questions regarding how valuable the Commitment to Respect survey results would be for interactions with other caregivers and patients.
Although most of the respondents said that the feedback from the survey was at least somewhat valuable, only one-third of the respondents stated that they had received results from the 2013 Respect survey. The fact that so many respondents had not received their survey results suggested that we need to make changes to the implementation of the survey.
Analysis of responses to the open-ended comments demonstrated that the respondents wanted to have the opportunity to provide similar ratings for the nurses, technologists, and administrative staff. Respondents also believed that the survey would be more useful if they were evaluated by a greater number of people.
Changes in Respect Survey Based on Feedback
Using these results, we decided to make several key changes to the process before the next annual survey was conducted. These changes were implemented (1) to clarify the purpose of the survey, (2) to determine specifically who was receiving feedback from the Respect survey and who was not and put a process in place to ensure that all radiologists would receive feedback, (3) to ensure that all radiologists would receive a summary of the verbatim comments made by those evaluating them, and (4) to increase the number of respondents to the feedback survey.
First, we added 2 questions to the Respect survey. One question asked whether the radiologist’s interactions with colleagues and employees live up to the ideals of the Commitment to Respect guidelines. We added this question to gauge whether radiologists are adhering overall to the Respect guidelines. The second question asked whether the radiologist has displayed year-over-year progress in adhering to the Respect guidelines. This question was designed to measure whether the radiologists were using the feedback provided to them (ie, specific recommendations from verbatim comments) to enhance the way they interacted with their coworkers and patients.
To address concerns that radiologists were not receiving feedback from the Respect survey, we worked with the survey vendor to ensure that scores would be e-mailed directly to radiologists and their section heads. In the 2013 survey, the sample size for each radiologist was limited to 4 technologists, 2 nurses, 2 desk personnel, and 2 peers. However, not everyone who received a survey answered, so radiologists received feedback from an average of only 7 respondents. This limited sample size created the potential for highly variable feedback; 1 person’s opinion could significantly alter the results. The limited number of respondents also made it unlikely that the respondents chosen accurately reflected the population of people with whom each radiologist worked. To increase the sample size, we decided that 7 representatives from each of the organizational categories described above would receive surveys for each radiologist. Once this change was implemented, each radiologist received an average of 26 responses. In addition to increasing the reliability of the results, this change also addressed the radiologists’ concern that the sample size was too small.
We also modified the survey scoring criteria. In 2013, scoring was based on a 5-point scale (“completely disagree” to “completely agree”). In 2014, the scale was changed to a 4-point scale (“almost never” to “almost always”). We chose to update the scale to ensure that respondents were not consistently picking the neutral option (in this case, “3”), thus providing the radiologists with more definitive feedback. Scores were updated as follows: Exceptional performance: 3.70 to 4.0 Fully meets expectations: 3.30 to 3.69 Meets most expectations: 2.90 to 3.29 Needs improvement: 2.89 and below
The cutoff of 3.29 was developed when we stratified the array of scores across the Institute and identified a clear break from the mean scores resulting in an increased frequency of scores above 3.29. When the scale changes were factored in, the 2014 results were slightly higher than those seen in 2013.
We also incorporated the survey results into the annual performance review for radiologists. With this new step, section heads or department chairs review the results of the survey with each radiologist and provide a summary of the verbatim feedback provided. Those radiologists receiving the top 10% of survey scores are personally recognized by the Institute chair. If the radiologist has an average score that is lower than 3.29, he or she is required to participate in training. Additionally, with the administration of the 2014 survey, the use of the Staff Development Template became mandatory for those who received an average score of less than 3.29. This tool allows the physicians to note their plans for increasing their skills and discuss how these improvements will affect their short- and long-term career goals. For example, some radiologists have worked to improve their skills by attending training classes that focus on interpersonal communication, by undergoing professional coaching, or by receiving midyear feedback through informal Respect surveys administered separately from the annual survey. Radiologists who score below the average score cutoff are also required to meet with their section head or chair during the year to discuss their progress.
Finally, to enhance the usability of the survey feedback, we made several modifications to the survey scorecard. First, to distinguish survey feedback from the radiologist’s self-rating, a new category was created for an average score that excludes the self-rating. This rating allows radiologists to see which specific areas of the Respect guidelines they are missing when excluding how they perceive themselves. Second, to enhance readability of the online scorecard, the design of the scorecard was changed such that scores for each question were laid out horizontally along with the average score for each respondent category. Finally, each respondent category was color coded (green, yellow, or red) to highlight the significance of each score (green, scores of 3.46-4.0; yellow, scores of 3.30-3.45; and red, scores of 3.29 and below; Figure 6).
Design of the scorecard. Each respondent category is color coded to highlight the significance of each score (green, scores of 3.46-4.0; yellow, scores of 3.30-3.45; red, scores of 3.29 and below).
Results of Survey Changes
After adjusting the question scales from 2013 to 2014, we found that the overall average rating for 2014 was approximately 0.07 higher than the rating for 2013. When questions for the 2 years were compared, the 2014 rating for each was higher than the corresponding adjusted score for 2013.
The greatest increases in the average scores from 2013 to 2014 were in the responses to these questions (in descending order): Communicates information that sets appropriate expectations with the patient’s family or significant other (3.70-3.83; difference = 0.13). Communicates information that sets appropriate expectations with his/her patients (3.71-3.82; difference = 0.11). Manages workplace stress in an appropriate way that minimizes the impact on others (3.56-3.66; difference = 0.1). Refrains from gossip, rumors, and malicious talk about others (3.72-3.81; difference = .09).
For nearly every question, the radiologists’ average ratings of themselves were higher than the ratings from any other group, followed by the ratings from peers, nurses, technologists, and office staff (almost always in this order).
Future Plans
Based on our experience with the Commitment to Respect initiative to date, we have developed plans for expanding the survey in the future. First, we will expand the group of people who are evaluated to include managers and administrators (about 120 additional people). Expanding the Commitment to Respect initiative to include administrators ensures that the guidelines are followed by all staff at the Imaging Institute. When radiologists were asked what changes they would like to see to the process, several said that they would like to be able to evaluate other Imaging caregivers (eg, nurses, office staff, technologists) rather than just their peers. Moving forward, we plan to consider this option, as well.
We plan to continue to meet annually with the leaders of the Imaging Institute to discuss how the survey results will be used in the annual performance review process. We will also continuously evaluate the Staff Development Template to ensure that the tool is as helpful as possible. Continuous monitoring of this template will make it easy to determine when we need to update the tool or alter the processes. Finally, we plan to keep Imaging Institute employees updated on the Respect survey process and on our future plans through Imaging Institute town halls and staff meetings.
Conclusion
Now that the Respect survey has been administered several times (annually, 2013-2016), we can state that the entire process has had a positive effect on employee engagement in the Imaging Institute. In keeping with the intent of the process, there have been positive increases in average scores consistent with the Commitment to Respect guidelines that are at the core of the initiative. As intended, the survey identified behaviors that individuals were previously unaware of, enabling staff members to alter their behavior based on this feedback. The Staff Development Template has been received favorably by radiologists, particularly as a way for lower scoring staff members to address their need for improvement.
The survey also provides additional data for each radiologist’s annual performance review. Incorporating these data into the review reinforces the importance of this initiative and leadership’s commitment to it. This, in turn, helps Imaging leadership to reinforce the respectful behaviors of those scoring well and to support the efforts of those who have identified areas in which they can improve.
The design and implementation of the Commitment to Respect initiative and the survey that supports it have required a significant investment of resources, including weekly meetings of the core leadership team, regular town hall meetings to update the staff, and the expense of an external survey partner. Although we cannot currently establish causality between the Commitment to Respect process and increased patient experience scores, patient experience scores have trended in the same direction as Commitment to Respect scores (Figures 1 and 2), as have employee engagement scores (Figure 3). We believe that the positive direction of radiologists’ scores over time (on average), as well as the importance of the Commitment to Respect guidelines in the annual performance review process, has justified our investment.
For those considering the introduction of a similar process at their institution, we have a number of recommendations. Perhaps most importantly, we recommend that you obtain buy-in and continued support from top leadership. This support is essential to successfully implementing and maintaining the program. Similarly, choose a project team that is deeply committed to the initiative. This will ensure that the team members act as advocates for the initiative and put in the time necessary to see it successfully implemented. In the planning stages of the project, seek out expert advice when designing the survey instrument. Because the success of the initiative is measured using the metrics from the survey, it is important that the questions asked on the survey accurately reflect the goals of the initiative. Finally, provide internal or external tools that can be used to coach those who need or would like to increase their scores. The feedback provided, no matter how valuable, will be useless to the recipient if he or she does not have the resources necessary to act on it.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research and/or authorship of this article.