Abstract
Highlights
Both observer- and self-rated measures overestimate and inadequately capture the quality of shared decision making for children with medical complexity.
Better provider-driven elicitation of family preferences and values is needed.
Assessment of decisional self-efficacy is often missed.
Shared decision making (SDM) is a communication process that may improve care quality for children with medical complexity (CMC) by helping patients and caregivers make informed decisions aligned with their preferences and values.1,2 SDM occurs when patients, caregivers, and providers share their knowledge, preferences, and values to reach treatment plan agreement. 3 Key components of SDM include defining the problem, presenting options, and elucidating patient and provider preferences. 4 CMC receive overall poorer quality SDM than children without medical complexity do. 5 One contributing factor may be that CMC have multiple comorbid conditions that make treatment outcomes difficult to predict. 6 Caregivers and providers may struggle to have high-quality discussions about treatment risks and benefits in the face of such uncertainty. Overall, little is known about what may contribute to poorer quality SDM in CMC.
Surgical treatment decision making for neuromuscular scoliosis (NMS) is a prototypical decision faced by many CMC that could be facilitated by SDM. 7 NMS affects 50% to 75% of mobility-limited children. 8 Progression results in chronic pain, mobility impairment, and respiratory compromise. 8 Surgical correction with spinal fusion can reduce symptoms and stop progression but has high costs and high complication rates of 17% to 40%, including death.9–17 Nonsurgical management with supportive equipment and therapies can alleviate pain and preserve mobility but does not slow curve progression. Thus, children may still require surgery at a later date. Families who choose surgery must consider whether earlier (proactive) or later (reactive) surgery is best, which weighs surgical risk against symptom severity. 18
The objective of this study was to improve our understanding of the current state of SDM in surgical treatment decision making for NMS. In addition, this study evaluated how the various measures of SDM relate to one another and identified opportunities where decision support may improve SDM for NMS treatment decision making.
Methods
We conducted a convergent mixed-methods study of video-recorded clinical encounters and postencounter surveys about children with NMS, their caregivers, and their providers. Participants were recruited from Primary Children’s Hospital, a quaternary children’s hospital, between September 1, 2020, and July 27, 2022. The University of Utah Institutional Review Board approved the study, with participants providing informed consent.
Study Population
We recruited children 8 to 21 y old with NMS presenting to the orthopedic surgery clinic to discuss treatment for NMS, their English-speaking caregivers, and providers. We included children up to 21 y of age based on pediatric Medicaid coverage. We excluded 1) children <8 y old as surgery at a younger age may indicate atypical disease, 2) children with Duchenne’s muscular dystrophy as alternative treatments exist that may alter NMS progression, and 3) children with prior growing rod procedure (i.e., MAGEC or VEPTR) as those procedures often bridge to spinal fusion, so the decision is predetermined.19–21 We recruited all surgical providers who treat NMS (2 orthopedic surgeons, 1 neurosurgeon). Orthopedic providers served as the primary provider for these decisions, so the neurosurgeon did not complete postencounter surveys. The patient–caregiver–provider triad had to consent and assent (if able) to participate in the study. We planned to enroll 10 patient–caregiver dyads per orthopedist for balanced representation of providers. However, we experienced slow recruitment from pauses in elective surgeries and reductions in patient volume during the COVID-19 pandemic. Thus, our aim to recruit 10 participants per provider became infeasible due to time and resource constraints.
Data Collection
During parent consent procedures, study team members explained that the purpose of the study was to understand parents’ experiences in caring for a child with NMS to inform the team on potential ways they could improve experiences for families of children with NMS in the future. Parent participants were not aware that the focus of the study was on the quality of SDM. In addition, recordings were made using small digital video and audio recorders that were clamped to the handles of clinic cupboards above the eye level of standing participants and placed on clinic countertops, respectively. Collectively, these procedures aimed to minimize the potential for a Hawthorne effect shifting their behavior, although studies have shown that Hawthorne effects dissipate quickly when participants become engrossed in the task at hand.22,23 Provider participants were aware the study focused on SDM as this was disclosed to gain provider buy-in for conducting the study in their clinic.
After giving documented informed consent, participants completed an enrollment survey of sociodemographics, child medical history (caregiver), and clinical practice (providers). Clinic visits were video recorded. Immediately after the visit, providers completed a brief pen-and-paper postencounter survey, ideally before seeing another patient but at the latest by the end of the day. Caregivers completed an online postencounter survey via the institution’s Research Electronic Data Capture system. 24 If multiple caregivers were present for the visit, we asked them to self-identify who would complete the survey. Due to the COVID-19 pandemic, each visit was limited to a maximum of 2 caregivers. Caregivers received a $50 gift card upon study completion, and providers received $5 gift cards for each postencounter survey completed. For treatment decisions that spanned 2 visits, the subsequent visit was also video recorded with postencounter surveys deferred until after the second visit. To discern whether the decision would span another visit, parent or provider participants were asked by the study team immediately after the clinic visit whether a decision was made about the child’s NMS treatment that day.
Outcomes
To gain a more comprehensive understanding of SDM in NMS, we measured the quality of SDM in 2 ways: observer rated through video coding and self-reported through corresponding caregiver and provider surveys. Although most studies on SDM in pediatrics use validated survey outcome measures, these measures are broadly considered to have ceiling effects, potentially due to social desirability bias.5,25,26 Moreover, only a limited number of pediatric SDM measures exist, with most being adaptations of SDM measures developed in adults, thus potentially being inadequate measures of pediatric-specific elements of SDM such as evolving patient autonomy and triadic decision making. 27 Measures using observer ratings have not been applied broadly to pediatrics.28,29 Whether observer-based measures correlate to self-reported measures is unknown in this setting.
We assessed observer-rated quality of SDM by applying the DEEP-SDM coding scheme to the video-recorded visits. 29 DEEP-SDM contains 11 binary outcomes representing behaviors within SDM (e.g., providing a treatment rationale, weighing risks and benefits, assessing decision self-efficacy) observed during encounters. Who initiates each behavior is also documented (i.e., child, caregiver, or provider). As multiple turns of conversation occur throughout a visit, designating who initiated each behavior was not mutually exclusive. Encounters were holistically assessed for degree of decision sharing using a Likert scale (1 = doctor alone to 9 = patient alone). We selected DEEP-SDM over other observer-rated SDM measures due to its ability to analyze decision making across multiple encounters or prolonged lengths of time and to assess patient, caregiver, and provider behaviors, as other measures primarily focus on the provider.
To measure child participation, we created a de novo child-participation Likert scale (1 = no participation to 8 = asks own questions). Capturing child participation could improve our understanding of the degree of child involvement in decisions, as many children with NMS have an underlying neurodevelopmental impairment that may preclude their active participation in decision making.
We measured self-reported quality of SDM via validated survey outcome measures SDM-Q-9 (caregiver) and SDM-Q-Doc (provider), which have been used extensively to evaluate SDM.30,31 Both measures include 9 items on a 6-level Likert scale (0 = completely disagree to 5 = completely agree), with items oriented toward assessing provider behavior. Each item describes distinct elements of SDM (e.g., defining treatment options, reaching decision agreement). A mean composite score is transformed to a 0 to 100 scale, with higher numbers indicating higher quality SDM. We replaced 2 free-text items about the visit’s chief complaint and the decision with a question about treatment option selected: surgery, no surgery, undecided, or other. See Table 1 for all DEEP-SDM and SDM-Q-9/SDM-Q-Doc items and how they correspond to one another.
Comparison of Subdomains Between Shared Decision Making Outcome Measures
Covariate
We measured caregiver decisional conflict via the Decisional Conflict Scale, a 16-item validated measure with a 5-point Likert scale (0 = strongly agree to 4 = strongly disagree). A mean composite outcome score is transformed to a scale of 0 = no decisional conflict to 100 = extremely high conflict. The scale includes 5 subdomains, calculated similarly, that measure feeling: uncertain, informed, clear in their decision-related values, supported, and effective decision.
Analysis
Based on the priority-sequence model for mixed-methods triangulation, our analysis prioritized DEEP-SDM scores given the known ceiling effects of survey measures. 32 Two independent coders experienced in qualitative analysis (J.L.L. and A.Z.) applied the DEEP-SDM to each visit with a composite score for decisions that spanned 2 encounters, kappa = 0.78. One coder (J.L.L.) had a clinical background and expertise in SDM for CMC, and the other had a nonmedical scientific background with limited experience with SDM and CMC (A.Z.). Both were new to DEEP-SDM and underwent training with a creator of DEEP-SDM (M.L.C.) that included collective review of the first 3 videos. Serial coding reviews occurred to reconcile differences. We conducted additional inductive analysis to identify themes in decision making for NMS using grounded theory to provide additional context about SDM during NMS surgical decisions. 33 We developed preliminary codes by first open-coding an initial set of videos then discussing initial codes to create a preliminary codebook. Each coder then independently coded each video. We convened serial coding reviews to reconcile differences and to add, modify, or remove codes. We reviewed codes and analyzed excerpts through memos and recoding to synthesize themes. We finalized themes and deductively sorted themes by DEEP-SDM domains through team meetings. We achieved thematic saturation after 12 encounters. For encounters in which multiple caregivers were present, caregivers were scored collectively on the DEEP-SDM rather than scoring each individual caregiver separately.
We compared and contrasted SDM and decisional conflict survey data with DEEP-SDM scores and looked for patterns. We assessed interrater reliability between caregiver and provider surveys via a prevalence-adjusted and bias-adjusted kappa (PABAK). 34 We selected PABAK post hoc over interclass correlation coefficients because provider survey ratings clustered primarily in the upper range of the rating scale. We dichotomized survey responses as 0 to 3 and 4 to 5. We also conducted a post hoc explanatory mixed-methods analysis of whether SDM-Q-9 and SDM-Q-Doc item ratings matched corresponding DEEP-SDM subdomain scores from Table 1. For this analysis, we dichotomized SDM-Q-9 and SDM-Q-Doc ratings for each item in Table 1 to bias toward the null: 0 = completely disagree to 1 = strongly disagree indicated participant perception that an SDM behavior did not occur and 2 = somewhat disagree to 5 = completely agree indicated that it did. We performed exploratory analyses of all outcomes based on sociodemographics, treatment selected, and decisional conflict composite and subdomain scores.
Results
For the 18 child-caregiver-provider triads recruited, patients had a mean age of 14.8 y (standard deviation [SD] 3.6 y), with 15 (83%) identifying as non-Hispanic White, 10 (56%) with technology dependence, and 12 (67%) with neurodevelopmental impairment. Caregivers had a mean age of 42.1 y (SD 5.2 y), with 17 (94%) identifying as White and 15 (83%) with at least some college education. The mean decisional conflict score was 31 (interquartile range [IQR] 17–36], indicating low decisional conflict (see Table 2). The n = 3 providers comprised all surgeons who provide spinal fusion to treat NMS. All providers were male with 10 to 34 y of practice since residency.
Participant Characteristics
CHIP, Children’s Health Insurance Program; IQR, interquartile range; SD, standard deviation.
Only 2 participants had decisions span 2 clinic visits. Within clinic visits, decision-related discussions lasted a median of 17.6 min (IQR 10.1–24.1 min). Providers led discussions for 12.7 min (IQR 9–16.5 min), while caregivers led for 6.2 min (IQR 3.2–10.1 min). Patients led discussions for 0.5 min (IQR 0–1.8 min).
For the DEEP-SDM, triads achieved a median of 10 (IQR 9–10) behaviors (see Figure 1). Only 1 triad achieved all 11 DEEP-SDM components. All encounters discussed the treatment rationale, identified provider preferences and values, confirmed caregiver understanding, and planned for follow-up; most touched on caregiver preferences and values (17/18). Providers initiated most behaviors except for confirming caregiver understanding and eliciting caregiver preferences and values. The least prevalent behaviors were defining treatment options (8/18) and assessing caregiver decision self-efficacy (4/18). Children infrequently fulfilled DEEP-SDM criteria, primarily participating by asking questions that demonstrated partial understanding. The median score for patient participation was 3 (IQR 2–7), indicating the child vocalized a response sometimes appropriately, sometimes not within the context of the conversation. The median score for degree of decision sharing was 2.5 (IQR 2–5.25), indicating decisions were predominantly provider led.
Quality of shared decision based on DEEP-SDM coding scheme subdomains.
For survey outcome measures, the median score of the SDM-Q-9 was 76 (IQR 66–94), and the SDM-Q-Doc score was 81 (IQR 73–94). The PABAK between caregiver and provider ratings varied highly across items (see Table 3). For “making clear a decision needs to be made” and “wanting to know how caregivers wanted to be involved in decision-making,” PABAK = 0 (CI 0–0.35). For “asking caregiver’s treatment preference,” PABAK = 0 (CI 0–0.67). The highest PABAK of 0.6 (CI 0.04–0.91) occurred with “reaching agreement on how to proceed.” Conversely, the unique survey item of “treatment selected” had perfect agreement.
Interrater Reliability and Crosstab Scores of SDM-Q-9 and SDM-Q-Doc
SDM, shared decision making.
We inductively synthesized 9 themes from the visits. While most themes cut across multiple DEEP-SDM domains, we organized them under the most prominent domain (Table 4 provides exemplar quotes). Four themes fit under the domain of treatment rationale: 1) Literature is cited to inform caregivers about the rationale behind each treatment option. 2) Pain from sitting intolerance is a reason to do surgery. 3) Provider opinion is used to frame the decision for caregivers. 4) Gestures are used to communicate changes to a patient’s function that occur as NMS progresses, to communicate uncertainty, or to demonstrate surgical techniques such as halo traction or rod insertion. Three themes fit under provider preferences and values: 1) Providers used examples of other patients to justify their recommendations. 2) Providers often recommended the degree of decision urgency, particularly whether by deferring surgery the family would miss a window of opportunity for surgery. 3) Discussions about tradeoffs often concluded with a provider recommendation. Two themes fit under outcome expectations: 1) Tradeoffs focused on outcome expectations (e.g., improving sitting tolerance versus preserving mobility; generally, improving quality of life versus avoiding risks, like death). 2) Comorbidities add uncertainty to expected treatment outcomes.
Decision-Making Themes Sorted by DEEP-SDM Subdomain a
Spoken by provider unless otherwise indicated.
Explanatory mixed-methods analyses demonstrated no discernable patterns between those with highly discrepant versus concordant scores between parent and provider ratings in “wanting to know how caregivers wanted to be involved in decision-making.” However, for “asking caregiver’s treatment preference,” caregivers in encounters with more concordant scores made explicit statements about their preference for the selected treatment options, including brief statements such as, “I agree.” This did not occur in encounters in which providers scored the item much higher than caregivers did. For encounters in which caregivers scored the item much higher than the provider did, the providers asked about the caregiver’s values or preferences. Those conversations ranged from asking about general decision-making values to probing about caregiver preferences for a particular treatment option. These encounters also contained more explicit discussions of caregivers being in control of decision making. For example, one provider stated, We’re not committed to anything, but it’s good to have the discussion, start thinking about it, thinking about different options. And even securing times, even showing up on the day of, doesn’t commit you to anything. The only time you’re committed is once the surgery is over. (Encounter 518)
In comparing SDM-Q-9 and SDM-Q-Doc ratings to DEEP-SDM scores, only 3 caregiver–provider pairs rated all survey items with scores that corresponded well to the DEEP-SDM score. Caregivers were correct a median of 67% (IQR 50%–85%) and providers 71% (IQR 54%–87%). No patterns in outcomes and interrater reliability occurred based on sociodemographics, treatment selected, or decisional conflict.
Discussion
At first glance, the quality of SDM appears to be high overall in our study population. However, the poor interrater reliability in SDM-Q-9 and SDM-Q-Doc and discrepancy with observer ratings suggests that the high scores of SDM may not indicate true high-quality SDM. While DEEP-SDM scores were high, providers infrequently assessed caregiver understanding and elicited caregiver preferences and values. Notably, children minimally participated in decision-making conversations. In thematic analyses, discussions about comorbidity-related uncertainty focused on treatment outcomes but not about how comorbidities should be a consideration during treatment option selection. This contrasts starkly with the practice of multidisciplinary provider surgical co-management teams, which focus on presurgical planning and decision making.35,36
The discordance in caregiver, provider, and observer ratings of SDM suggests highly varied perceptions of the encounter despite overall high SDM scores, with different perceptions of challenges within the encounter. The items with greatest interrater variability were identifying a decision is needed, identifying caregivers’ preferred decision-making role, and eliciting caregivers’ treatment preferences. Caregivers and providers may have rated items highly for different reasons. Providers may have rated highly due to the Dunning–Kruger effect, whereby growing recognition of SDM causes providers to believe their SDM skills are better than in reality, especially as providers were aware the study focused on SDM. Social desirability bias may also have caused higher provider ratings because they believe they are supposed to value SDM. Caregivers may rate SDM highly due to recall bias. Compared with providers, caregivers gave higher ratings for the provider asking for caregivers’ treatment preferences when providers stated that the decision was ultimately the caregiver’s, even though providers did not explicitly ask for the caregiver’s preferences. Statements about decision ownership may be conflated with elicitation of caregivers’ treatment preference because the provider is deferring to the caregiver. Alternatively, the relief of being told that a high-risk surgery is not mandatory for their child may lead caregivers to rate their provider more positively. Social desirability bias may drive caregivers to rate their provider highly because of their established relationships with providers to monitor NMS longitudinally. 26
Our results are consistent with prior studies that found inconsistencies between the observer-rated and participant-rated quality of SDM that suggest issues with congruency across measures.37–39 Alternative explanations for the discrepancies in our measures of SDM quality include 1) that the quality of SDM measurement by the tool may be poor, although the survey measures have been well-validated, and 2) that participants may have different perceptions of SDM for the same encounter and that these perceptions may also differ from those made by an objective observer. Other studies have described discordance in patient- and provider-reported decision items, including whether a decision was even made, discordance between positively reported communication experiences and presence of SDM, and self-reported scores of SDM and actual SDM skills observed.40–42 Future work should consider how to integrate multiple perspectives into the assessment of SDM and whether a global measure of SDM quality is possible when perceptions differ significantly.
Caregivers overwhelmingly initiated discussions about their preferences and values and confirmed their own understanding. This matches the findings from another study that found that provider elicitation of preferences and values often were implicit. 43 While most caregiver participants initiated these conversations, this may not be true for all populations, including those from historically marginalized communities, with limited health literacy, or with less established relationships with the provider.43–45 Pediatric providers have identified a need to improve their skills in eliciting preferences and values. 46 Thus, research should focus on how to support providers to elicit patient and caregiver preferences and values. Values clarification exercises are structured tools to help patients and caregivers identify what is most important to them for decision making without extending visit length. 47 Family-centered goal setting is another way to understand caregivers’ values globally. 48 Provider communication training can also improve communication with patients from historically marginalized communities. 49
The assessment of caregiver ability to adhere to the decision (decisional self-efficacy) rarely occurred during the encounters. However, per the DEEP-SDM, high-quality SDM should include whether a family has means to adhere to a treatment plan. For NMS, these discussions could include short- and long-term surgical recovery, mobility changes after surgery, and limitations of adaptive equipment modifications without surgery. Notably, there remains disagreement within the literature about whether decisional self-efficacy is a part of SDM. 50
Providers should also more explicitly identify available treatment options for NMS including nonsurgical supportive care even if they do not halt NMS progression. Caregivers in our study desired greater discussion about alternatives, which rarely occurred in our study. This aligns with expert recommendations for the provision of SDM in pediatrics. 51 Even explaining the lack of reasonable alternatives may bolster caregiver decision-related knowledge. Particularly for CMC, wherein many decisions are complex and lack a strong evidence base from clinical trials, having a thorough understanding of various treatment options and the levels of evidence behind them may improve caregiver decision-related knowledge. 51 Including an assessment of the degree to which the level of evidence was discussed may be especially valuable when assessing the quality of SDM for CMC and pediatrics in general.
Most patients were minimally involved in decision making despite being central to the decision. In pediatrics, the challenges of patient–caregiver–provider triadic decision making are amplified by intergenerational divides and varied and changing cognitive abilities of children.52,53 In NMS, underlying neurodevelopmental disorders further complicate decision making. 54 However, caregivers of CMC can often discern their child’s preferences even without verbal communication. 55 Within NMS, caregivers who could identify their child’s preferences consistently chose treatment options aligned with the child’s preference, even when it was against the provider’s preference. 56 Current SDM measures do not distinguish between parent and patient preferences and values, but both are important in pediatric SDM. There is no expert consensus on how to involve children in SDM and who determines how much a child should be involved. 51 In addition, SDM measures tend to measure the same domains equally across all decision makers, which may assume that the roles served by each participant in SDM should be the same. However, in pediatrics, certain elements may be more important than others for the patient compared with the parent. For example, understanding a child’s values and preferences may be more important than having the child participate actively in planning for decision follow-up. Further complicating matters is the evolving nature in which a child may be involved in various elements, such as decision self-efficacy, which may be less important in a developmentally younger child than in an older child or one who can help provide self-care activities. These nuances are not currently captured in SDM measures but may be critical for facilitating triadic SDM, which is paramount for SDM in pediatrics.
Our study has several limitations. The single-center design may limit generalizability to other populations of children with NMS as participants were English speakers and primarily identified as White race, reflecting local demographics. No validated measures to measure pediatric-specific SDM exist. Thus, aspects unique to pediatrics including the shifting level of autonomy and participation of patients may be insufficiently captured by the measures we used. In addition, the DEEP-SDM is not validated, resulting in unknown measurement quality, and is not developed specifically for pediatrics. 57 Nonetheless, it is the best available instrument for the purposes of this study. Moreover, our evaluation of SDM is limited to what occurred in the clinic visit and does not include elements of SDM that may occur outside of the clinic visit, such as discussions with other caregivers and providers in decision making, independent research, and child involvement outside of a clinical encounter, which may be a daunting place for children to freely express themselves. Future studies should explore how experiences differ with language and racial diversity. Recruitment difficulties from the COVID-19 pandemic limited the sample size.
Conclusion
Initially, the quality of SDM appears to be high overall in our study population. However, poor interrater reliability suggests that high SDM scores may not indicate true high-quality SDM and that caregivers and providers often perceived SDM differently. Developing support for providers to facilitate values clarification and confirm caregiver understanding could improve SDM in this population.
Footnotes
Acknowledgements
We would like to thank the patients, caregivers, and providers who participated in this study and generously shared their expertise and experiences with us. An earlier version of this study was presented as an oral abstract at the Pediatric Academic Societies 2022 conference.
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Dr. Andras owns stock in Eli Lilly, serves as a consultant/paid presenter for Medtronic and Nuvasive, and receives publishing royalties from Orthobullets and royalties from Orthopediatrics. The remaining authors have no financial disclosures or conflicts of interest to disclose. An earlier version of this study was presented at the PAS 2022 Meeting. The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided in part by the Primary Children’s Hospital Early Career Development Award and in part by the National Center for Advancing Translational Science of the National Institutes of Health under award number UM1TR004409. Dr. Lin received support from the National Institute of Arthritis and Musculoskeletal and Skin Diseases of the National Institutes of Health under award number K23AR079778. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Ethical Considerations
The University of Utah Institutional Review Board approved the study.
Patient Consent
All caregiver participants provided informed consent. Patient participants provided assent or consent if greater than 18 y old when able. Otherwise, surrogate consent was provided by the caregiver. All provider participants provided informed consent.
Consent for Publication
Not applicable.
ORCID iDs
Data Availability
Due to the personally identifiable nature of the data, they are not publicly available. Requests for data, analytic methods, and study materials will be considered on an individual basis.