Abstract
Background:
Although surgical management of a first-time anterior shoulder dislocation (FTASD) can reduce the risk of recurrent dislocation, other treatment characteristics, costs, and outcomes are important to patients considering treatment options. While patient preferences, such as those elicited by conjoint analysis, have been shown to be important in medical decision-making, the magnitudes or effects of patient preferences in treating an FTASD are unknown.
Purpose:
To test a novel shared decision-making tool after sustained FTASD. Specifically measured were the following: (1) importance of aspects of operative versus nonoperative treatment, (2) respondents’ agreement with results generated by the tool, (3) willingness to share these results with physicians, and (4) association of results with choice of treatment after FTASD.
Study Design:
Cross-sectional study; Level of evidence, 3.
Methods:
A tool was designed and tested using members of Amazon Mechanical Turk, an online panel. The tool included an adaptive conjoint analysis exercise, a method to understand individuals’ perceived importance of the following attributes of treatment: (1) chance of recurrent dislocation, (2) cost, (3) short-term limits on shoulder motion, (4) limits on participation in high-risk activities, and (5) duration of physical therapy. Respondents then chose between operative and nonoperative treatment for hypothetical shoulder dislocation.
Results:
Overall, 374 of 501 (75%) respondents met the inclusion criteria, of which most were young, active males; one-third reported prior dislocation. From the conjoint analysis, the importance of recurrent dislocation and cost of treatment were the most important attributes. A substantial majority agreed with the tool’s ability to generate representative preferences and indicated that they would share these preferences with their physician. Importance of recurrence proved significantly predictive of respondents’ treatment choices, independent of sex or age; however, activity level was important to previous dislocators. A total of 125 (55%) males and 33 (23%) females chose surgery after FTASD, as did 37% of previous dislocators compared with 45% of nondislocators.
Conclusion:
When given thorough information about the risks and benefits, respondents had strong preferences for operative treatment after an FTASD. Respondents agreed with the survey results and wanted to share the information with providers. Recurrence was the most important attribute and played a role in decisions about treatment.
The incidence of shoulder dislocation varies in different populations, with substantially increased incidence in young males and athletes. 20 –22,25,33 Treatment of first-time anterior shoulder dislocation (FTASD) can be operative or nonoperative, with the goal of reducing the risk of recurrent dislocation. 12,19,29,30
Although it is known that younger patients have high rates of dislocation, it is unclear which specific patients will develop instability after FTASD, making operative intervention after FTASD controversial. In young, active adults, lower rates of recurrent dislocation and increased rates of return to sport are reported with operative stabilization. 6,13,16,19,30 Other analyses, however, have shown no significant difference in rates of recurrent dislocation among active and nonactive cohorts treated nonoperatively. 12,29 Furthermore, decision modeling suggests that operative treatment after FTASD may confer higher probabilities of future shoulder stability irrespective of pre- or postdislocation activity levels. 24
Operative stabilization, however, requires patients to forgo months of contact sports and overhead shoulder activities. Information provided by clinical experts for this study suggests this break period is often significantly longer for operative than nonoperative treatment, and for the in-season athlete or active individual, this may be particularly burdensome. Therefore, patient-specific circumstances and motivations must also be considered. 26
One approach to eliciting information on an individual’s preferences is conjoint analysis. Long a mainstay in marketing research, conjoint analysis provides an objective measure of the relative importance among features of a given product or service. 31 For example, when buying a car, different models have different combinations of attributes, including cost, fuel economy, reliability, and warranty. One consumer may feel fuel economy and cost are more important while another may value reliability and warranty over cost. Adaptive conjoint analysis (ACA) measures consumer-level estimates of the relative importance of attributes associated with various product or treatment options. Conjoint analysis has been increasingly applied to health care decision-making 2 –4,9,23,32 in settings as diverse as balancing the benefits and side effects of medications 11 and exploring treatment options in scoliosis bracing 5 and knee osteoarthritis. 10
After FTASD, operative and nonoperative treatment options vary along several attributes, including the risk of recurrent dislocation and cost. The ACA exercise outlined below uses responses to survey questions to measure the relative importance of these attributes to each person. This proof-of-concept study thereby aims to create a novel tool to measure respondent preferences relevant to the selection of operative versus nonoperative management of FTASD and to facilitate efficient, evidence-based, shared decision-making. The study seeks to measure the (1) importance of aspects of treatment after FTASD, (2) respondents’ agreement with results generated by the tool, (3) respondents’ willingness to share these results with their physicians, and (4) association of results with choice of treatment after FTASD.
Methods
Survey Construction and Development of Conjoint Analysis Exercise
A survey was created (see the Appendix) using Sawtooth Software SSI Web (Sawtooth Software) and included an ACA exercise to elicit individuals’ relative preferences for each of several scenarios that vary based on a fixed set of attributes. Attributes, which are crucial to design of conjoint analyses, are features of treatment alternatives that are important to patients or stakeholders. 14
Utilizing processes defined elsewhere for best practices in conjoint analysis, 2 a panel of experts in conjoint analysis, clinical research, and 2 board-certified orthopaedic surgeons fellowship-trained in sports medicine selected the attributes relevant to the clinical management of FTASD. Reducing the risk of recurrent dislocation and time out of and return to sports are often cited as benefits of surgery and therefore made appropriate attributes. 15,16 Additional attributes were selected to represent treatment-associated features, including limits on arm movement and duration of physical therapy. Also included was out-of-pocket costs incurred by patients, as cost sharing is an increasingly important aspect of patient decision-making.
The levels for each attribute were chosen to span the range of possible clinical scenarios relevant to operative and nonoperative treatment of FTASD (Table 1). The levels representing the likelihood of a repeat shoulder dislocation were based on ranges reported in previous studies. 29 For limitations on arm movement, the levels represented the period of arm immobilization with operative management, an intermediate level of limited arm movement, and no limitations. Likewise, the levels pertaining to the period over which patients would be restricted from participation in contact sports and durations of physical therapy were based on clinical experience for patients undergoing surgical and nonsurgical management of FTASD. These levels are similar to those reported in the literature. 13,15,16 Levels for out-of-pocket costs were chosen to be reflective of average marketplace deductibles and representative of patients with and without insurance deductibles while recognizing the wide variability in costs inherent in individual treatment.
Attributes and Levels for Treatment of FTASD a
a FTASD, first-time anterior shoulder dislocation.
After the initial selection of attributes and corresponding levels, 5 additional sports medicine orthopaedic surgeons reviewed and confirmed that the attributes represented the key characteristics that they believe patients should consider when selecting a management strategy for FTASD and that the levels represented realistic ranges for the large majority of FTASD cases.
The ACA exercise was constructed by first presenting detailed attribute descriptions and then by gathering individuals’ preliminary importance ratings on each attribute. Next, as shown in Figure 1, combinations of different levels of attributes are created and placed side-by-side as hypothetical situations, asking the individual to rate their preference for one relative to the other. These pairs are customized for each individual to efficiently gather relative preferences for each attribute. 14 Respondents were asked to rate their preferences for 10 pairs of alternatives. To make the task easier, 2 attributes were shown in the first 2 pairs. An additional attribute was added for every 2 tasks completed until 5 attributes were shown for each alternative and 10 pairs of alternative scenarios were completed, as shown in Figure 1. The tool, via software algorithm, then estimates the relative importance of each attribute based on that respondent’s preference ratings for the 10 pair tasks.
Example of rating question in adaptive conjoint analysis with 5 attributes.
Measures and Outcomes
The survey and ACA exercise were designed to measure (1) the relative importance of attributes of treatment after FTASD, (2) respondents’ acceptance and accuracy of preference measurements generated by the tool, (3) choice of treatment for FTASD as associated with these generated preferences, and (4) willingness to share these preference results with their physicians.
To measure the relative importance of attributes, as described, the tool generated values for each of the 5 attributes for each individual respondent. A unique relative importance distribution was generated for each individual based on answers in the ACA exercise, similar to the averaged example shown in Figure 2.
Mean importance weights, all respondents. Error bars represent 95% confidence intervals.
To measure accuracy of generated importance values, respondents were shown a graphical representation of their individual relative importance values (again, similar to Figure 2). Respondents then indicated the degree to which the generated preference weights represented their preferences on a 5-point Likert-type scale from “very well” to “very poorly.”
To measure the relationship between their preferences and their treatment choices, respondents were then asked to choose “surgery” or “no surgery,” as shown in Table 2. The levels for each attribute were selected to approximate the clinical differences between operative and nonoperative management, assuming that individuals would be responsible for a US$1000 greater cost if they chose operative management. All respondents saw the same operative and nonoperative alternatives, with the exception of recurrence risk with nonoperative treatment, which has been shown to depend on age and sex of the individual, from up to 80% for an 18-year-old male to less than 15% for a female 35 years or older. 29 For this attribute, the respondent’s age and sex were used to estimate an individualized risk of recurrence in the ensuing 2 years, with values taken from Robinson et al 29 and rounded to the nearest 5% for ease of comprehension, as shown in Table 3.
Choice Between Surgery and No Surgery as Presented to Respondents
Age and Percent Risk of Recurrent Dislocation in the 2 Years After Nonoperative Treatment for FTASD a
a Data from Robinson et al 29 and rounded to nearest 5%. FTASD, first-time anterior shoulder dislocation.
To measure the willingness or desire to share preference results with providers, respondents were asked to imagine that they were seeking treatment for shoulder dislocation and were asked whether they would want to share the preference weights generated by the ACA exercise with their physician.
To characterize the respondent population, the survey collected demographics (age, sex, education level, and insurance status), activity level, and most frequently played sports. Additionally, respondents were asked about feelings toward surgery, marking the degree to which they were concerned about the following on a 5-point Likert-type scale from “not concerned” to “extremely concerned”: the pain or risk of complications of surgery in general, needing to take time off work or school to undergo surgery, having surgical scars, and requiring assistance after surgery.
Study Population
The survey was administered to participants of Amazon Mechanical Turk (AMT; Amazon.com, Inc), an online marketplace where anonymous users can complete surveys or other small tasks in exchange for compensation. Respondents were recruited by a post on the AMT forum, which requested participation of adults older than 18 years who were physically active and may have had a shoulder injury or dislocation. Respondents were paid $1.25 for completing the survey.
Exclusion criteria were designed to ensure data quality from AMT respondents. Respondents who were among the fastest 20% of individuals who completed the survey were identified. These individuals may have sped through the survey without reading attribute descriptions or instructions or chose responses without careful consideration of the alternatives. Respondents with suspected inconsistent answers to ACA questions were identified using the R 2 “fit” value as generated by the conjoint analysis. R 2 “fit” values for a respondent’s utilities measures how consistent a given user applied their preferences in the conjoint exercise. Low values suggest that answers were chosen haphazardly or that a given user may have been unable to sufficiently understand and complete the exercise. Respondents in the lowest 20th percentile of ACA fit and fastest 20% of completion times were excluded.
Shoulder dislocation was described in detail in the survey, including information to help distinguish true shoulder dislocation from subluxation or other shoulder injuries. Thereafter, respondents reporting shoulder dislocation were asked about the circumstances surrounding their first dislocation. Understanding that dislocation is a universally painful occurrence, respondent data were excluded if they reported having a prior shoulder dislocation and reported that they did not need pain medication or other treatment for pain.
Demographic data were obtained in the survey, including age, sex, participation in sports in general and activities deemed high risk of dislocation including contact or overhead sports, feelings toward surgery in general, education, income level, and health insurance status.
The use of an anonymous AMT Worker ID for each completed survey allowed for cross-referencing with earlier rounds of testing this survey and others at the authors’ institution. 7 To avoid the same individuals contributing survey data in multiple rounds of testing and to exclude individuals who may provide purposefully inaccurate information to participate (and receive payment for) the survey, respondents were excluded if they participated in a previous round of testing of this survey or another survey under development at our institution that targeted a mutually exclusive population (>50 years old, smokers).
Statistical Analysis
Descriptive statistics are reported for the study population. Exclusion criteria were applied, and the ACA R 2 fit value, as provided by Sawtooth Software for each individual, was examined to explore differences in the predicted values for each attribute. Responses to the ACA exercise for each individual were transformed by Sawtooth Software into utility weights for each attribute and level and then normalized to calculate relative importance values for that individual. Responses to questions gauging concern about aspects of surgery in general were combined, and an average score indicating “very” or “extremely” concerned determined aversion. In addition to analysis of the entire sample, subgroup analysis was performed for those reporting dislocation without prior surgery and those reporting no dislocation. Choice of operative versus nonoperative treatment by age, sex, and history of prior dislocation was compared using chi-square tests. Regression was performed using JMP Pro (SAS Institute). Logistic regression was performed to model choice of treatment: operative versus nonoperative management. Included in the covariate analysis were demographic variables, preference weights as determined by the ACA exercise, and stated aversion to surgery. The risk of recurrent dislocation with nonoperative treatment, known to have significant age and sex variability, 29 was included in logistic regression to compare the effects of the recurrence rate as shown to respondents. A critical value of α = 0.05 was chosen for all statistical tests.
Results
As shown in Table 4, 501 respondents completed the survey. Sixty-five (13%) respondents completed the survey in the fastest 20% while also in the 20% lowest consistency on ACA questions (average ACA R 2 fit, 198). Seventeen (3%) reported no pain with dislocation, and 36 (7%) had completed a previous survey during testing or were found in survey testing of other surveys that suggested falsifying of personal information.
Demographics and Shoulder Dislocation Characteristics
a Percent represents sex of full sample.
b Percent of those with previous dislocation.
Hereafter, analysis was performed with all exclusion criteria applied, leaving 374 (75%) respondents who took an average of 13 minutes 37 seconds to complete the survey. As shown in Table 4, most respondents were male (61%), younger than 30 years (68%), and active (89%), especially in sports that carry a high risk of dislocation such as football, lacrosse, hockey, basketball, swimming, or weightlifting (76%). The vast majority (99%) had graduated from high school. Most reported having current health insurance (82%). Nearly one-third (32%) reported a prior shoulder dislocation, and several reported undergoing surgery to stabilize their shoulder. More than 70% of those reporting dislocations had only a single dislocation. The most common injury mechanism for males was contact sports (57%) and for females was noncontact sports (33%). After dislocating, 61% visited an emergency department, and 10% reported no contact with a physician regarding their dislocation.
Importance of Attributes
The relative importance of the 5 attributes was computed for individuals and averaged over subsets based on demographic or other criteria. The preference weights for each attribute averaged over all respondents are shown in Figure 2. Chance of recurrence and out-of-pocket costs proved the most important attributes.
Agreement With Generated Results
When shown the importance of attributes that the tool generated (Figure 2) based on their answers to conjoint analysis questions, 87% of respondents reported that these represented their preferences “well” or “very well.”
Choice of Treatment
Respondent choices of operative or nonoperative treatment are shown in Table 5. Because the risk of recurrence with nonoperative treatment shown to respondents was varied according to the respondent’s age and sex, the findings were stratified by sex, age group, and prior dislocation. The total choosing operative treatment was 158 (42%), with 125 (55%) males and 33 (23%) females choosing operative treatment. Persons younger than 30 years preferred operative treatment at higher rates compared with those older than 30 years (48% vs 30%, P = .001). Respondents generally chose operative treatment at different rates based on sex and age, though not based on whether they had previously sustained dislocation.
Respondents Choosing Operative Treatment by Age and Sex a
a Respondents answering “unsure” about dislocation were not included in subgroup analysis. Based on sex, treatment choices were significantly different in all age groups and dislocation status (P < .05), except for ≥30 and prior dislocation. Based on age, treatment choices were significantly different for younger than 29 versus 30 or older in the total sample (P = .01) and those without dislocation (P = .01) but not in those with prior dislocation (P = .11). Based on dislocation status, treatment choices were not significantly different in the entire sample, between males and females, or between age groups.
Variables such as age group, sex, aversion to surgery, participation in high-risk activities, and preference values were included in the logistic regression to model the choice of operative treatment at US$1000 versus nonoperative treatment. Age, sex, and importance of recurrent dislocation proved significant predictors of treatment choice for the entire sample (Table 6). When stratified by prior dislocation, significant variables included sex and participation in high-risk activities. However, when adjusting for nonoperative risk of recurrent dislocation for age and sex, only the individual’s importance of recurrent dislocation, as determined by the ACA exercise, remained significant for the entire sample (Table 7). For those without prior dislocation, nonoperative recurrence risk and sex were significant, while in those with prior dislocation, participation in high-risk activities remained significant even with the consideration of nonoperative recurrence risk.
Logistic Regression Modeling of Covariates Favoring Choice of Surgery a
a Boldfaced values indicate significant values.
Logistic Regression Modeling of Covariates Favoring Choice of Surgery, Controlling for Recurrence Risk a
a Boldfaced values indicate significant values.
Willingness to Share Information
Nearly 9 of 10 (89%) respondents reported that they would be willing to share their personal preferences generated by this survey with their physician.
Discussion
This study serves as a first step toward development of a tool to measure preferences about features that are relevant to choosing a treatment for shoulder dislocation. It successfully measured data along our primary outcomes: Respondents agreed with the survey’s assessment of their preferences, preferences were predictive of treatment choice in the aggregated sample, and respondents wanted to share this information with their provider. By acknowledging operative management of FTASD is a preference-sensitive intervention, this study suggests such a tool may facilitate shared decision-making.
When viewing personal preferences derived by ACA, respondents felt that the conjoint exercise performed well. Furthermore, an overwhelming majority wished to share their preference information with their doctor. This positive feedback and data quality suggest the survey was well received and understood, which echoes findings from other surveys that include interactive or thought-provoking exercises surrounding treatment choice. 8,17,27 Utilizing a similar survey tool in a clinical environment, such as during or prior to the visit with the physician, may therefore be possible without excessive time or resource demands.
Risk of recurrence emerged as the most important attribute among those considered after FTASD. Respondents to this survey at an individual level, however, differed in the importance of this factor, thus supporting a patient-centered approach to management of FTASD. After recurrence, cost was the next most important attribute. While it is widely known that cost is an important concern to patients, discussions about cost in practice may not be occurring with the frequency or depth to adequately address patient concerns. 1 Among other attributes, limitations on motion of the arm, avoiding high-risk activities, and duration of physical therapy had lower importance.
Forty-two percent of respondents chose surgery after a hypothetical FTASD, with 37% of those with prior dislocation and 45% without prior dislocation. Significant age and sex variation was present in choice of treatment in each subgroup; however, notably, those reporting prior dislocation and those without chose treatment at similar rates. Surgical utilization rates after FTASD are not well described, but some data suggest that approximately one-third of patients will undergo surgery after repeat dislocation, 21 similar to our subgroup who had sustained dislocation. Respondents utilizing our tool chose surgery at similar rates to those described after dislocation, which suggests that our tool may serve as an acceptable novel platform for clinical testing.
Subset evaluation of those reporting dislocation showed low surgical utilization (5/119), though this pilot study was not designed to fully explore this small cohort. Further research in patients with shoulder dislocation is necessary to corroborate these findings and explore the reasons patients may be motivated or reluctant to undergo surgery.
Several factors examined in this study were predictive of choice of surgery after FTASD. The importance of recurrence was significantly predictive of choosing surgery, as were the demographic factors of younger age and male sex. However, when controlling for the wide variation in risk of recurrence shown for different age groups and by sex in logistic regression, only the importance of recurrence remained a significant predictor of treatment choice overall. These findings have profound implications for shared decision-making between patients and providers. The reduction in probability of recurrent dislocation, which patients report as most important on average and which literature deems the most important outcome of treatment, is shown to be more important than other aspects of treatment. Since variables predicting treatment choice were different in those who had sustained dislocation, further study is warranted in that target subgroup. Furthermore, the nonoperative recurrence risk shown in the survey was highly predictive, suggesting a role for presenting individualized evidence-based recommendations to the patient. Overall, to align with an individual’s preferences, providers may benefit from obtaining preference information from patients considering treatment after FTASD.
While the study sample focused on higher risk individuals—those more likely to be young, male, active, and relatively well educated compared with the general population—the sample was appropriate for the study, as a younger, male-dominated sample is representative of patients with FTASD. Moving forward, however, the next step would be to replicate this study in a sample with clinically confirmed diagnosis of FTASD.
There are several limitations of our survey. First, injury and treatment for our respondents is hypothetical, and despite attempts to describe the situation, real patients will no doubt behave differently. Arguably, however, treatment decisions made immediately after a painful dislocation may distort preferences, and preference measurement prior to an injury may better reflect stable long-term preferences. Second, the cost of treatment in our survey is imagined, and individuals’ choices may vary if requiring actual personal expense. Cost of operative treatment will also likely vary between individuals and insurance plans, as will nonoperative costs of physical therapy copays. However, the purpose of the cost attribute is not to reflect exact costs but to estimate the general importance of out-of-pocket cost and to obtain utility values in other attributes that can be converted to dollar amounts to communicate relative value.
Additional limitations include the timing of the decision, which may be important for an in-season athlete. 26 There also may be inconsistent application of the current literature by orthopaedic providers, which favors more aggressive treatment in certain populations. 12,13,16,24 Finally, despite our attempts to describe dislocation in detail, participants may have reported dislocation despite having other shoulder injuries such as subluxation.
Several limitations are noted concerning the use of conjoint analysis as well. Although a panel of experts was convened to establish important attributes in FTASD, patient input was informal during survey development. Long-term sequelae of shoulder instability were also not included. Hovelius and Saeboe 18 describe increased radiographic evidence of arthropathy after follow-up of dislocation, although verification of these findings and their clinical significance is lacking. Nevertheless, patients with FTASD may have preferences about distant or future risks that may be important determinants of surgical or nonsurgical management. Additional concerns relevant to survey research or development of decision aids include the impact of attribute descriptions, framing of questions, and design of the conjoint exercise. 28
Conclusion
Treatment of shoulder dislocation presents an opportunity to explore patient preferences objectively. As individual patient circumstances vary widely in terms of activity levels, participation in sport or other activities, and demographic characteristics, patient preferences may play a role in decisions about treatment. This study tests a preliminary model for collecting patient preferences about shoulder dislocation and treatment and serves as a foundation for future research.
Footnotes
Acknowledgment
The authors would like to thank Carolyn A. Hutyra (Duke University Health System, Department of Orthopaedic Surgery, Durham, NC) for her assistance in the manuscript review and preparation.
One or more of the authors has declared the following potential conflict of interest or source of funding: This study was funded internally by Duke University.
Ethical approval for this study was waived by the Duke University Institutional Review Board for Clinical Investigations (Pro00049685).