Abstract
Objectives
Online patient forums have become a platform for patient education and advocacy in many areas of medicine. The anonymity provided by such forums may encourage honest, candid responses. Using patient online reviews, this study sought to explore themes that arose from negatively perceived care interactions with American otolaryngologists using the Accreditation Council for Graduate Medical Education (ACGME) competency framework.
Study Design
Qualitative thematic analysis.
Methods
Through an iterative multistep process, a qualitative thematic analysis was conducted on negative reviews (defined as ratings of two or less out of five) of all American otolaryngologists found on a popular online physician-rating website (RateMDs.com).
Results
A systematic search through the RateMDs website revealed 2950 separate comments of negative reviews. Of these negative reviews, 350 were randomly selected for thematic analysis. The predominant themes that emerged aligned closely with the Accreditation Council for Graduate Medical Education (ACGME) competencies, in particularly with professionalism and interprofessional skills and communication.
Conclusions
The negative reviews of American otolaryngologists revealed a number of areas where improvements could be made to quality of care. Patients value evidence-based medicine delivered by compassionate and respectful physicians. Isolating and aligning predominant themes within the ACGME framework proved a productive method to collect and organize pertinent patient feedback and integrate teaching into the post-graduate training and continuing professional development in order to avoid such negatively perceived interactions in the future.
Keywords
Introduction
Physician-rating websites have gained popularity and are frequently used by patients.1,2 Despite the increased popularity of these sites, many physicians view them unfavorably.3,4 Available online platforms provide the public visibility for unfiltered feedback and retroaction of healthcare encounters, often free of charge and protected by anonymity.5-7 Opponents fear defamation, false accusations, and inability to respond in preserving at all cost confidentiality.8-11 Negative feelings aside, 69% of physicians report having checked their online profile at least once. 3 In addition to using online resources to research health topics, patients are using these resources to access information about their physicians.12-16 Such information is easily available as more than 90% of physicians have professional information available online. 17
A recent American study specific to online ratings of otolaryngologists (OTLs) revealed largely positive reviews. 18 Amongst negative narratives, Calixto et al. (2018) observed that content frequently pertained to unsatisfactory bedside manner, wait time, accessibility, clinical staff, and environment. 19 It has long been reported that patient satisfaction is strongly correlated with maintaining the patient-physician relationship, patient adherence, and reported outcomes. A further exploration of negative reviews is essential as it may provide cues for improvements in quality and delivery of care.
Using patient online reviews, this study sought to explore themes that arose from negatively perceived care interactions with American otolaryngologists using the Accreditation Council for Graduate Medical Education (ACGME) competency framework.
Materials and Methods
Due to the public availability of the online comments, ethics approval was waived for this study.
Study Design
This qualitative thematic analysis used a deductive, constant comparative analysis method applied to a forum. The content used for assessment was retrieved from RateMDs, an online patient rating platform, in January 2016.
Search Strategy
Reviews were obtained by selecting “Specialty: Ears Nose and Throat (ENT) doctor” on RateMDs.com for all American states. 20 This well-known English-language online forum was selected given its popularity and accessibility. Patient reviews that were given a negative global score (i.e., two or less out of a five-point Likert scale, with three considered neutral) were included in our study. If a physician received greater than five negative comments, only the most recent five were included.
Data Analysis
Online patient responses fitting the inclusion criteria were extracted from RateMDs.com into Microsoft Excel Workbook (Version 16.22). Each patient’s responses were divided into units of phrases and each phrase was entered into a new row. Using the ACGME competencies as a conceptual framework, a deductive theoretical analysis was completed. Comments were analyzed to develop a coding framework for dominant themes. In this way, emerging themes from the text were translated and categorized into the ACGME core competencies.
Phrases were coded through an iterative multistep process to identify major themes and sub-themes. A random sequence generator was used to establish a list of random list of RateMD comments by Microsoft Excel (Version 16.22) entry number. Three independent reviewers were first assigned the same set of 50 entries to code. These codes were used for training purposes to achieve congruence among reviewers. During training and discussion, the framework underwent numerous iterations. Ambiguities and potential missing themes were identified and discussed between the reviewers to clarify the coding framework until consensus was reached. After this time, a second set of 50 entries was assigned. At this point, the reviewers achieved congruence (kappa range 0.8–1.0 of assigned codes) to indicate high inter-rater reliability. The same investigators then coded entries independently. After 350 entries were coded, saturation was reached. This was determined by the saturation ratio, whereby the number of new and unique themes in a run of five comments was below the threshold (<5%). This signified that saturation was reached and further coding would have been redundant. In addition to the major and sub-themes, thematic analysis was used to assess the indirect emotion presented in the data. Coding for the indirect emotion was developed from common themes including punctuation, acronyms, tone, and sarcasm. The final coding framework can be found in Tables 2 and 3.
Results
Patient reviews were anonymized and recorded in a Microsoft (Microsoft Corporation, Redmond, WA) Excel 2011 database (Version 16.22) in order of global physician rating. A total of 2950 entries were extracted. Comments corresponded to 1455 OTLs, with an average star rating of 1.5.
Percent of Otolaryngologists Included by State.
Summary of Themes and Sub-Themes (ACGME).
Summary of Themes and Sub-Themes (non-ACGME).
Moreover, 8.7% had complaints about the staff and medical learners (“The front desk is horrible. Those ladies clearly do not want to work there.”), 7.6% for time management (“never less than an hour in the waiting room, and then another 20–30 min in the room waiting for the doctor.”), and 5.6% for cost of healthcare (“To my disbelief I was charged over $2,000”).
“Importantly, 12.6% were not satisfied with their physicians behavior:
“This is the rudest man I have ever met in my life. He insulted me, made derogatory and very personal remarks, used profanity and mocked my condition right at the front desk in front of his staff”.
Finally, the emotion coding segment of this study revealed that excessive punctuation and capitalization were used quite frequently in the negative comments to exacerbate comments. These codes accounted for 6.3 and 5.6% of all codes assigned, respectively: “I WOULDN’T LET THIS MAN DO SURGERY ON A CORPSE!!!” “TO SAY THAT NAME WAS EXTREMELY ARROGANT AND PATRONIZING WOULD BE A GROSS UNDERSTATEMENT”
Sarcasm and disrespect (“Not a doctor but a businessman!”) as well as hyperbole (“The absolute worst doctor ever!”) were both encountered in 1.0% of comments.
Discussion
To our knowledge, this was the first study to specifically examine the content of negative online reviews of American otolaryngologists in relation to ACGME competencies. Negative comments translated well onto the ACGME competencies. The most common themes of negative comments, which were related to perceived unhelpfulness, staff/medical learners, and overall behavior. These sub-themes corresponded to the ACGME competencies of patient care, systems-based practice, and professionalism. Excessive punctuation and capitalization were some of the means used to convey dissatisfaction. Trends in the geographic distribution of online patient negative reviews were also noticed.
In translating the themes from the negative comments onto the pre-existing ACGME competencies framework, learners, educators, and practicing physicians may get a new perspective on patient concerns. The alignment with the core competencies may allow for targeted post-graduate and continuing medical education tailored to these specific areas. This analysis of negative reviews revealed a number of areas where improvement could be made to quality of care. The three most frequently identified codes were “unhelpful,” “staff/medical learners,” and “behavior (arrogant/dismissive/lack of empathy).” None of these three related to medical knowledge, but instead stemmed from the patient care, systems-based practice, and professionalism ACGME competencies. This information confirms that patients value evidence-based medicine delivered by compassionate and respectful physicians.
Physician-rating websites have an increasing presence on the Internet and an unspoken presence in the physician’s office. Regardless of the reliability of the data, patients are increasingly visiting these sites for guidance in choosing healthcare professionals and providing feedback on their experiences. Previous research identified differences between ratings and reviews among subspecialties. In general, facial plastics had lower ratings than other otolaryngology and head and neck surgery subspecialties. In addition, associate professors had lower scores than assistant professors and professors. 3 Though the present study did not isolate negative ratings by subspecialty and academic appointment, some of these trends may apply.
Previous research has shown that a majority of physicians check their online ratings. 2 If OTLs are reviewing these comments, there may be a unique opportunity to reflect and learn. It may serve as an opportunity to see how we are viewed in the eyes of patients, the people to whom we are accountable as physicians. This is why, as the medical community strives to endorse the dual role of professional and healer, it is imperative that the aforementioned areas of unmet patient needs be tackled. Some system-based factors such as insurance claim denials, coverage, and overall policies might not be readily and solely solved in an otolaryngology office. However, other areas such as patient care and professionalism can be improved at an individual level, through education and training. Importantly, the assumption is that the feedback provided online is accurate and valuable. As an open access, unregulated forum, this may prove to be a challenge. Interestingly, other systems such as the Patient Advocacy and Co-Worker Observation Reporting Systems were designed to capture and act upon unsolicited complaints and pitfalls encountered during the delivery of care.22,23 Altogether, these online platforms can provide the necessary feedback to promote patient advocacy, safety, and satisfaction.
Overall, this provocative overview of negative patient-clinician interactions may serve as a useful tool to identify initiatives in the otolaryngology head and neck surgery field. However, there are a few limitations to note. Through the anonymity of this process, patients naturally feel comfortable in being overtly open in their responses. Patrons are free to express themselves how they see fit. This may cause patients to use stronger language to describe the interaction, which in turn would inflate the results of this study. Patients with experiences at either end of the spectrum may have been more prone to publishing a comment, adding to the inherent selection bias. Additionally, patient reviews were not actively monitored for modifications of the original publication and rerating of the same otolaryngologist in a separated posting. Due to the anonymity of the postings, it was not possible to account for the possibility of multiple posting from the same individual or related to one incident. Furthermore, it is foreseeable that negative statements might have been observed even within good overall ratings, which would not have been captured in this present study. Future work to explore the common themes across all ratings is warranted. Lastly, based on the volume of entries on this site, we did not cross-check the names of the otolaryngologists on the RateMDs site with the national registry. As such, the list of otolaryngologists may not be entirely complete and other online platforms such as Press Ganey may provide valuable insight and complementary catchment for future research.
Conclusions
Analysis of negative reviews of American otolaryngologists by patients reveals a number of areas where improvements could be made to quality of care. By aligning predominant themes within the ACGME framework, we may be able to integrate teaching into the post-graduate training and continuing professional development areas. Furthermore, this may serve as a template for a self-assessment tool for practicing OTLs. Such targeted teaching may mitigate some of these negatively perceived interactions and enhance the delivery of care to future patients.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethics Approval and Consent to Participate
Our institution does not require ethical approval for reporting publicly available data.
