Abstract
Objective
Patients consider many factors when deciding how to receive medical care. This study used best-worst scaling (BWS), a technique novel to otolaryngology, to quantitatively examine preferences among patients choosing a laryngologist. Our objective was to quantify in a pilot cohort the relative importance patients place on a variety of attributes when seeking a laryngologist.
Study Design
BWS survey.
Setting
Academic voice clinic.
Methods
New patients were recruited to take a computerized BWS survey developed using attributes derived from patient input, expert opinion, and literature review. Attributes were grouped into 4 categories: physician reputation, physician qualifications, hospital-related factors, and other nonclinical factors. Responses were analyzed using multinomial logit regression to determine importance scores and associations with other variables.
Results
Eighty-seven of 93 patients recruited participated (93.5% response rate). Physician qualifications were the most important attributes to patients, with specialty laryngology training receiving the highest importance score (20.8; 95% CI, 20.2 to 21.5; P < .0001). Recommendations from referring physicians (15.6; 95% CI, 14.3 to 16.9) and use of cutting-edge technology (11.9; 95% CI, 10.7 to 13.1) were the second and third most important, respectively. Least important were nonclinical factors, including wait time to get an appointment (4.3; 95% CI, 2.8 to 5.8) and convenience of office location (1.5; 95% CI, 0.9 to 2.1). Just over half of patients (51.2%) reported willingness to wait 4 weeks for an appointment with a laryngologist. Older patients were less concerned with convenience-related factors.
Conclusion
Nonclinical factors were less important to patients than clinical factors, and laryngology-specific training was paramount. Stated preference methodologies can elucidate underlying preferences and help providers make care more patient centered.
Keywords
Many factors are relevant to patients when choosing where and how to receive medical care. These preferences may pertain to structural aspects of care (eg, wait time, cost, convenience) or to outcomes (eg, highly efficacious or low-risk treatments). In otolaryngology, many decisions that patients make are preference sensitive, 1 but there is little existing information about how patients choose a laryngologist. Given that understanding patient preferences has been demonstrated to improve decision making,2-4 it is imperative to rigorously and reproducibly study how patients choose their providers.
Stated preference methodologies have become an accepted means of measuring preferences pertaining to medical goods and services. 5 These include contingent valuation, discrete choice experimentation, and best-worst scaling (BWS). BWS is a method that asks participants to identify the best and worst options among several choices and, with iterative selections, develops rankings of the options. Results from BWS studies on patient preferences allow researchers to understand the relative importance of different components of choices or preferences in a way that traditional Likert scale data cannot. 6
Using BWS, we aimed to understand the relative importance patients place on a variety of attributes when seeking a laryngologist. The indications for consultation are many: complaints related to voicing, swallowing, breathing, and globus are common. For example, 1 in 13 patients presenting to a primary care doctor’s office may have dysphonia, 7 and the lifetime prevalence of globus is approximately 22%. 8 Patients seeking subspecialty care may present independently or at the request of referring physicians, and they may seek a laryngologist with a specific location, credential, or technology. However, the preferences that underpin these decisions are not well understood. To our knowledge, this is the first study to apply BWS to understand why patients present for laryngology consultation.
Materials and Methods
Survey Attribute Development
Survey development began with a review of previous literature relevant to patient preferences regarding choice of hospitals and practitioners, given that no specific research existed regarding choice of a laryngologist.9-17 A broad list of potentially relevant treatment attributes was combined using these sources. These were organized using a previously published structure that included categories for physician, hospital, and nonclinical factors. 18 Further investigation of these attributes was then undertaken using published guidelines in the field of stated preference research.19-22 A pilot group of 10 patients was purposively selected to represent the population of the laryngology clinic. This group engaged in semistructured interviews about what they considered the most important attributes associated with care delivery in the laryngology clinic. These were conducted by research staff, one-on-one. These patients were then shown the list of treatment attributes generated in the prior literature review and asked to comment. Comments as were scrutinized by 2 expert reviewers for themes. These themes were grouped into broad categories. Overlapping attributes were consolidated. After this, 13 attributes remained in the survey ( Table 1 ), based on (1) patient interviews, (2) expert opinion, and (3) literature review. This set was tested on 10 additional patients to ensure comprehension.
List of 13 Attributes Selected for Survey.
Study approval was obtained from the Massachusetts Eye & Ear Institutional Review Board.
Survey Design
We developed a 3-part web-based survey. One component related to gathering demographic information. A second component asked multiple-choice questions about care delivery attributes, including how long the patient had waited for an appointment, willingness to wait for an appointment, distance traveled to the appointment, chief complaint, and a rating of how bothersome their complaint was. These questions acted as comparators for the BWS exercise to assess for reliability.
The final component consisted of a BWS exercise. We evaluated various stated preference elicitation formats and selected BWS because our goal was to assess prioritization of preferences in a simple format; BWS is a well-accepted, efficient, and easily understood type of state preference research.23,24 BWS asks patients to pick the “best” and “worst” option from a list of attributes and then repeat this for multiple different lists. 6 The questions for this task are created a priori using a balanced incomplete block design. This design is generated to show each participant every attribute an equal number of times (“balanced”) over a fixed number (herein, 12) of questions. Not all attributes were included in every questions (“incomplete block”). One thousand potential designs were computed (Lighthouse Studio); the most balanced design was chosen. Question order was randomized for each participant.
An example of a choice question from this study is shown in Figure 1 . Each participant was asked to pick the “most important” and “least important” attribute from a selection of 5 attributes drawn from the 13-attribute list previously generated; this task would then be completed 11 more times with different attributes listed. In this way, each attribute could be methodically compared to other attributes in a best/worst format. Each selection gives several pieces of ordinal data. For example, if a participant selects attribute “A” as the best and attribute “E” as the worst, we know that the participant values A > B, A > C, A > D, and A > E, as well as B > E, C > E, and D > E. Thus, the participants’ preferences for attributes emerge through segmented best/worst choice tasks.
Example of a best-worst scaling choice question.
Participants
A convenience sample of adults seeking care at an outpatient laryngology clinic at a specialty otolaryngology and ophthalmology hospital were eligible to participate. Patients presenting on Thursdays over a period of 6 months were asked to participate. Inclusion criteria included new patients presenting for laryngology-related complaints, including those related to voice, swallowing, airway concerns, globus, and benign and malignant growths. Patients excluded were those who were established patients within the practice; those without English proficiency, as the survey was given in English; those without the cognitive or motor capabilities to complete the computerized survey; children (<18 years old); and those who elected not to participate. All eligible patients were privately approached at the end of the initial clinical visit by a research assistant (V.F.). Those who agreed to participate then undertook the computerized survey at a workstation compliant with hospital privacy policies. Surveys were taken by the patient alone, but a research assistant was available throughout for any questions.
Quality Control and Statistical Analysis
Several measures of quality control were imposed on the BWS data. First, total time to complete the survey was recorded. Those who took less than 3 minutes to fill out the survey were excluded; this time threshold was judged by the researchers to be the fastest a person could accurately fill out this survey. Second, root likelihood (RLH) was calculated for each participant. This is a measure of how well a modeled solution fits data, with 1.0 being the best possible and the worst being 1/n, where n is the number of choices available in the task; herein n = 5, and 0.2 is the worst RLH. Those having an RLH less than 0.227 were excluded. 25 This was determined by calculating importance scores for 300 randomly responding simulated participants and using the upper 95% RLH as the cutoff. This process allowed us to eliminate random responders with 95% confidence. Finally, patients who did not log answers for the BWS exercise were excluded.
Analysis of the data was performed using several programs. Descriptive statistics (percentages, means, and standard deviations) were performed using Microsoft Excel. Regressions, including multinomial logit modeling, were performed using JMP 14 Pro and Lighthouse Studio to determine importance scores and their associations with sex (using unpaired t tests) and age (using Pearson correlation). Importance scores suggest how much each attribute has upon choice, and they are calculated for each individual by rescaling the coefficients of the multinomial logit model so that the sum of all scores is 100. Thus, importance scores may range from 0 (least important) to 100 (most important) scale. These scores were then averaged across the population. Confidence intervals (at 95%) were calculated for each importance score at the group level.
Results
Ninety-three patients were approached, and 3 declined to participate. Of the 90 patients who agreed to participate, 3 were excluded based on incomplete responses to the BWS questionnaire; none were excluded on the basis of time to complete <3 minutes or RLH criteria. Demographic characteristics of the population are shown in Table 2 . Most (55%) respondents were female, and the mean age was 55. The sample was largely white (85.1%), college educated (70.1%), and married (62.1%). Approximately 12.5% of the population self-identified as singers. Most chief complaints related to voice (36.8%), although breathing, swallowing, and globus sensation were all prevalent. Approximately 54% of patients rated their complaint as “very” or “extremely” bothersome. Median completion time for the survey was 8.7 minutes (range, 4.1-24.7 minutes).
Basic and Demographic Respondent Data (N = 87). a
Values are presented as number (%) unless otherwise indicated.
The importance of each attribute is shown in Figure 2 , with 95% confidence intervals indicating the precision around each estimated importance score. A doctor with specialized training in laryngology was deemed the most important factor in choice of a laryngologist, with the highest importance score (20.8; 95% CI, 20.2-21.5; P < .0001). Recommendations from referring physicians (15.6; 95% CI, 14.3-16.9) and use of cutting-edge technology (11.9; 95% CI, 10.7-13.1) were the second and third most important, respectively. Less important were nonclinical factors, including wait time to get an appointment (4.3; 95% CI, 2.8-5.8), convenience of office location (1.5; 95% CI, 0.9-2.1), and online reviews (1.0; 95% CI, 0.7-1.3). There were no significant sex differences in importance scores. As participants’ age increased, they were more likely to value hospital reputation (P = .027), less likely to value a convenient location (P = .0005), less likely to care about wait time (P = .010), and less likely to care about insurance coverage (P = .0093).
Importance scores for attributes.
Average scores across each category (physician reputation, physician qualification, hospital-related factors, and nonclinical factors) demonstrated that there was a significant difference between categories overall (P < .0001), with physician qualifications ranking most highly (average score, 10.4; 95% CI, 9.7-11.2), followed by hospital-related factors (average score, 8.6; 95% CI, 7.9-9.3). Physician reputation (average score, 6.4; 95% CI, 5.6-7.1) and nonclinical factors (average score, 5.1; 95% CI, 4.3-5.8) were significantly lower than the other categories but not statistically different from one another (P = .074).
Table 3 illustrates responses to the other non-BWS preference questions. Most (51.2%) patients were willing to wait 3 to 4 weeks for a laryngology appointment, whereas the majority (68.6%) had a 2-week or less wait time. Those participants who indicated less willingness to wait for an appointment on the stratified analysis had correspondingly higher importance scores placed on wait time as measured in the BWS exercise (P = .026), indicating reliability of the BWS exercises for assessing responses relative to wait time. The plurality of patients (37.2%) indicated that they would prefer a doctor with at least 6 to 10 years of experience; responses to this question correlated with BWS values pertaining to number of years the doctor has been in practice (P = .031), indicating reliability of the BWS exercises for assessing responses relative to doctor experience.
Other Preference Questions (N = 86).
Discussion
Among a convenience sample of adult patients at an academic laryngology clinic, there are clear preferences for certain features related to choosing a laryngologist. Participants viewed the specialty qualifications of the physician as most important. In addition, there was a strong preference for hospital factors, including availability of advanced equipment, specialty focus of the hospital, and reputation of the hospital. Several of the nonclinical factors included on patient satisfaction surveys and online review sites—specifically, convenience of the office location and wait time for an appointment—were less important, on aggregate, relative to the other attributes. There was a correlation between these nonclinical factors’ importance scores and the more traditional multiple-choice questions, indicating reliability of BWS preferences. Certainly, for any individual patient, there may be a stronger or weaker preference for these factors; future larger studies could attempt to further segment this population into preference phenotypes (eg, “convenience seeking,”“reputation focused”) to more adequately target patients with health care services.
While it should not be surprising to otolaryngologists that patients have preferences related to care, the relative importance of various factors can have a significant impact on how patients choose physicians and how physicians deliver care. Indeed, research in orthopedics, plastic surgery, surgical oncology, dermatology, and primary care has used traditional surveys, discrete choice experiments, and more recently BWS to study how patients choose their surgeons, physicians, and hospital systems.10,13,14,18,26-28 Our results generally align with a BWS study by Ejaz et al 18 evaluating how patients select cancer surgeons, which revealed that physician qualifications and hospital factors are more important than nonclinical factors. Interestingly, the most important factor in that study was surgeon caseload. This was not the case in our results, which demonstrated that caseload was the seventh most important of 13 factors, perhaps reflecting the often medical (rather than surgical) aspects of laryngology. Our results align with a systematic review of the main factors and sources of information patients use to choose their surgeon, in which the authors found that surgeon reputation and competency were the most valued individual attributes. 28
Although the physician’s specialty qualifications were of the most importance to our study population, there was variability between the attributes within the rest of the physician qualification category (years of experience and number of surgeries performed), as has been found in previous research. While specialized training in laryngology was the most important attribute overall, the number of years of practice and the number of surgeries performed were much less important for patients. This is similar to what Waltzman et al 26 found in their conjoint analysis of patient preferences for choosing a plastic surgeon, in that the number of years a surgeon has been in practice was only the fifth most important attribute, behind board certification, method of referral, distance from patient’s home, and office decor. 5 The authors hypothesized that board certification may place enough confidence in the patient’s view of the surgeon’s training that the qualification outweighs the number of years of training. This is contrary to Abghari et al, 27 who found that most patients prefer surgeons with 11 to 20 years of experience. In our study population, a plurality (45.5%) of patients preferred a doctor with greater than 5 years of experience, aligning more with patient preferences when choosing an orthopedic surgeon rather than a plastic surgeon. This may have implications for young faculty just establishing a practice, but it should be noted that other factors were even more important in this pilot cohort. Finally, it has been demonstrated that there is a paucity of accurate provider-specific information available through online review sites.29,30 In our data, this appears to be quite unimportant to respondents.
Prior literature has found that convenience of care is an important factor for patients when choosing a primary care physician. 13 Our results, however, revealed relatively low importance scores associated with office location and wait time. One possible reason for this discrepancy could be the relative frequency of office visits to a primary care physician vs a subspecialist, which could influence how important nonclinical factors are for patients. In another study on consumer selection of a primary care physician, McGlone et al 14 showed that convenience matters more for those with lower levels of education; they suggest that transportation issues may drive this. A high percentage of our group was college educated and may not have had these logistical concerns.
While there are few preference studies in otolaryngology relating to either the structural or outcome-related aspects of care, this information is important. 31 Doctors and surgeons are prone to “preference misdiagnosis,” which can lead to suboptimal decision making.1,32 While this study is limited in scope to what drives patients to see a laryngologist, it provides insight into what patients prioritize during their evaluation. This can directly help physicians tailor access and practice patterns for patient satisfaction. However, the broader field of preference assessment can go beyond these structural aspects of care; this research is a stepping stone to further investigations of what patients prefer for treatment of their specific medical condition. We can imagine a scenario in which preference assessment undertaken during the workup of ear, nose, and throat conditions (eg, thyroid nodules or laryngeal cancer) helps guide treatment choices made together by physicians and patients. Fortunately, quantitative, econometric preference assessment with BWS is relatively easy and efficient and avoids many of the problems and biases associated with simple Likert scales and rating exercises.33,34
The study has limitations. We explicitly evaluated a convenience sample of patients presenting to a single outpatient laryngology clinic. This is a nonrandom sample and cannot be considered a reflection of the general population. Nevertheless, pilot studies such as this can provide in-depth preference data on the selected population, regardless of that population’s composition. Those seeking care at a tertiary referral center may attract a proportion of patients from around the region who are not concerned with wait time to get an appointment or convenience of the office location. They may also prioritize aspects of physicians that are less important to other patient subgroups. In addition, some patients may have been referred to a laryngologist without actually seeking this (eg, a patient following instructions from their primary care physician); nonetheless, patients with decision-making capacity have preferences that can be measured, regardless of how they have arrived in our office. As can be seen from our sample’s demographic characteristics, most patients were white and college educated, making it difficult to determine if our key outcomes differed based on demographic characteristics. Second, the sample size is not large enough to provide power to assess subgroups’ preferences. Future iterations beyond this pilot study should incorporate larger, more diverse, and perhaps randomized samples of patients seeking laryngology care.
Conclusion
This is the first study to our knowledge using best-worst scaling to identify the relative importance of factors patients consider when choosing a laryngologist. Physician specialty training and recommendations from referring physicians were the most important factors, while nonclinical factors such as wait time to get an appointment, convenience of office location, and online reviews were the least important. This information deepens health care providers’ knowledge of the patient decision-making process. Stated preference methodologies, including BWS, can help practitioners elucidate underlying preferences and make care more patient centered.