Abstract
Many new decision aids are developed while aspects of existing decision aids could also be useful, leading to a sub-optimal use of resources. To support treatment decision-making in prostate cancer patients, a pre-existing evidence-based Canadian decision aid was adjusted to Dutch clinical setting. After analyses of the original decision aid and routines in Dutch prostate cancer care, adjustments to the decision aid structure and content were made. Subsequent usability testing (N = 11) resulted in 212 comments. Care providers mainly provided feedback on medical content, and patients commented most on usability and summary layout. All participants reported that the decision aid was comprehensible and well-structured and would recommend decision aid use. After usability testing, final adjustments to the decision aid were made. The presented methods could be useful for cultural adaptation of pre-existing tools into other languages and settings, ensuring optimal usage of previous scientific and practical efforts and allowing for a global, incremental decision aid development process.
Keywords
Background
Decision aids (DAs) are tools designed to support the process of shared decision-making (SDM) between patients and their clinician.1,2 DAs can have multiple formats (e.g. leaflets, website), but should at least create choice awareness, offer balanced information and stimulate patients to consider their preferences. 3 In general, DAs are associated with increased knowledge, more accurate risk perceptions and more conservative treatment preferences. 4 The International Patient Decision Aids Standards (IPDAS) provide DA developers consensus-based criteria to ensure DA quality. 5 To help DA developers, a checklist is available that includes nine categories to which the DA should comply (e.g. Provide sufficient information about the decision and using high-quality evidence). 6
A particularly fruitful area for the application of DAs is prostate cancer (Pca) care. Pca is the most common cancer in men in the Western world. 7 Pca treatment guidelines do not indicate a single superior treatment option but recommend a shared treatment decision between clinician and patient. 8 However, selecting the best suiting treatment from the available alternatives can be a burden for many patients. The process involves careful consideration of the risks and benefits of multiple treatments and weighing this against preferences and personal characteristics. Decision-making is further complicated by sub-optimal information provision and a possible misinterpretation of patient preferences by clinicians, which emphasizes the potential benefits from DAs in Pca care.9–11
Recent reviews of Pca DA trials concluded that current Pca DAs provide good quality information and help to increase patients’ knowledge.12,13 Despite improved information provision, current DAs do not guarantee that SDM takes place. Moreover, content, format and presentation of Pca information within DAs varied substantially, with many failing to comply to all components of the IPDAS criteria.12,13 The most identified shortcomings consisted of not including physicians and patients during DA development, a lack of balanced information on all options and the absence of explanation about the evidence used in the DA. 13 Rather than resolving these issues with current tools, often new tools are developed elsewhere. This further increases the variety and number of available tools, though routine use in clinical practice of these tools remains limited. 14
As many care providers articulated the need for a suitable Dutch DA, we built an interactive website based on an existing evidence-based online Canadian DA, developed by Feldman-Stewart and colleagues15–19 as a starting point for further development in the Dutch situation. This article describes the development process of the DA and usability evaluation in Dutch clinical setting.
Methods
The DA development process and usability testing among relevant user groups consisted of six stages and was based on the model described by Kushniruk. 20 This model describes the typical system development starting from initial analysis, prototype development and evaluation, but allows for more input and changes in every development step compared to more traditional methodologies that have a fixed order of steps. Each stage was worked on by a multidisciplinary development team of urologists, psychologists and engineers (N = 6). This section will discuss the stages in the development process, and the final DA as outcome is described in the “Results” section.
Stage 1: translating the pre-existing DA
The background and validation of the existing Canadian DA has been described thoroughly, with particular focus on the information needs of Pca patients when making a treatment decision.15–19 The validity of all topics covered by the original DA for Dutch patients was also confirmed by an earlier cross-country comparison (including The Netherlands) of information needs in Pca patients. 21 Therefore, all content from the original DA was translated from English to Dutch.
Stage 2: evaluating Dutch clinical routine
To investigate typical conversation flow in consultations about Pca treatment decision-making, all non-clinicians within the development team observed consultations between patients and urologist in the outpatient clinic of the initiating hospital. In addition to these observations of actual consultations, role playing was used to emphasize the steps clinicians usually take in treatment decision-making consultations with a patient. Role playing was performed by the two clinicians involved in the development team, with one of them simulating the patient role. Other members from the development team observed with special focus on the structure of the simulated consultations.
Stage 3: DA re-design
Following the observations from stage 2, the original DA was re-designed to fit with typical conversation flow as observed in stage 2. Moreover, the translated textual content from stage 1 was further adjusted to comply with Dutch and European treatment guidelines. All content was re-written according to standards for creating web-based text to ensure readability and comprehensibility for all literacy groups (e.g. maximum of 10–15 words per sentence and 5–10 sentences per paragraph, clear headings and active phrasing).22,23 Readability and comprehensibility was later assessed by an expert in medical communication from the initiating hospital, who was not involved in the further development of the DA.
Stage 4: development of explicit value clarification exercises
For use in patient DAs, IPDAS defines that value clarification exercises (VCEs) should “help patients to clarify and communicate the personal value of options,” in order to ultimately increase congruence between personal preferences and the selected treatment option. 24 However, without clear design guidelines for VCEs, a variety of exercises have been developed with little knowledge about which features actually work best.25–28 A recent review suggests that VCEs should at least include trade-offs between option attributes in order to encourage value congruent decision-making. 29 Therefore, from all topics covered in the DA, those topics that differentiate between treatments were selected to create explicit VCEs. To present these topics as a trade-off, statements were presented in such way that an answer to each statement was related to a (type of) treatment. VCEs were developed within the development team and reviewed from the perspective of the disciplines present in the development team (urology, psychology, engineering design). After consensus by the development team, VCEs were added to the DA. The content and phrasing of the VCEs were further evaluated during usability testing.
Stage 5: usability testing
After completion of the first version of the adjusted DA, a usability test was conducted among patients and care providers (N = 11).
Setting and participants
Participants for usability testing were recruited in the initiating hospital in the southern region of The Netherlands, by the clinicians from the development team. Four urologists (not involved in the DA development), two oncology nurses, one radiation oncologist and four Pca patients with recent experience in Pca treatment decision-making agreed to participate in usability testing. All patients were between 55 and 65 years of age and within 6 months of Pca diagnosis. Patients with experience in the decision situation were selected because they were expected to be better able to imagine the situation of just having received a Pca diagnosis. 30 IPDAS therefore also requires that DA testing is performed by experienced patients. 6 Care providers were included in this usability test to ensure the DA content and usability would match their usual routines and their experiences with patients facing Pca treatment decisions. Also, care providers’ review during development is required by IPDAS. 6 All care providers included in usability testing were affiliated to the initiating hospital, but not involved in any other stage of DA development. Care providers ages ranged from 35 to 60 years and all had a minimum of 5 years of experience in their current position. All participants were instructed to use the DA from the perspective of a patient diagnosed with low-risk Pca and eligible for all four treatments covered in the DA (active surveillance, surgery, brachytherapy and external beam radiotherapy). No specific further usage instructions were given in order to let participants use the DA as naturalistically as possible.
Participants were asked to think aloud when navigating through the DA and to mention every remark or difficulty they encountered during DA usage. This procedure is commonly used to investigate human–computer interactions and has been applied before for DA usability testing as well. 31 The usability test was run in two simultaneous sessions in the outpatient clinic of the initiating hospital, with two observers from the development team present in each session. The observers monitored whether the participants’ verbalization matched their DA usage (e.g. saying navigation was easily accompanied by clicking on the correct buttons). As the DA only consists of a limited number of steps, if any action was not verbalized by the participant, a clarifying question was asked to the participant. During DA usage, participants did not receive further feedback or other instructions from the observers. Each participant was given 30 min to use the DA followed by a 15-min semi-structured interview. The goal of the interview was to reflect on DA usage in addition to the comments made while using the DA. Interviews are commonly added to think-aloud procedures to ensure that the most important aspects have been covered during the usability test and to reduce the risk of bias in the interpretation of participants’ verbalizations. 32 The interview covered five questions asked to all participants: (1) “What were your expectations upfront?” (2) “What is your first impression of the DA?” (3) “Was the information understandable and useful?” (4) “What were positive aspects?” and (5) “What can be improved?” Only patients were then asked the following: (1) “Would you recommend this to other patients?” and (2) “What feeling does the DA gave you?” Care providers were asked whether they would offer this DA to patients. Participants were then thanked for their participation and received a bottle of wine as token of appreciation for participating.
Measures and analysis
As a first step, all notes from all sessions and observers were combined and labeled as either general comments about the DA or related to a specific section of the DA. All comments were then further categorized to Usability, Layout, Language, Content, Amount, Values Clarification or DA Summary. Next, the accuracy and urgency of all comments were discussed by the development team to determine the implications for DA adjustments. If consensus was reached on the need for changes, this led to final adjustments in the DA.
Stage 6: final adjustments
Usability testing resulted in final adjustments to the DA (described in “Results” section). Finally, the DA was evaluated for compliance with the IPDAS criteria. 6
Results
DA
Stage 1 resulted in a plain text translation of the original Canadian DA on a prototype website. From the observations of conversation flow in clinical practice (stage 2), it was learned that following diagnosis clinicians often do not go into detail about all treatment options immediately. If eligible for active surveillance, treatment options are first presented as a consideration between active surveillance and curative treatment, before curative treatment options are discussed in more detail. In order to tailor the DA to this typical conversation flow during consultation, the DA was designed into four steps. Table 1 provides an overview of all topics covered in DA steps 1–3.
Content covered in Dutch DA.
PSA: prostate-specific antigen; DA: decision aid.
DA step 1: general Pca information
This introducing step provides background information about Pca in general. The anatomy of the prostate and the commonly used terms prostate-specific antigen (PSA) and Gleason are explained.
DA step 2: active surveillance versus curative treatment
The pros and cons of not treating immediately are compared to (immediate) curative treatment (Table 1). Specific treatment characteristics are not yet discussed in detail. Step 2 ends with VCEs on topics that require trade-offs between curative treatment and AS (Table 2).
DA value clarification exercises (VCEs).
DA: decision aid.
DA step 3: surgery versus radiotherapy
If patients are still undecided or have a preference for curative treatment following step 2, they continue to step 3. This step explains the difference between surgery and radiotherapy in more detail (Table 1). An example page from this step is provided in Figure 1. Patients who already prefer AS after step 2 are allowed to skip this step. Step 3 ends with VCEs on topics that differentiate between surgery and radiotherapy (Table 2). If patients already indicated a preference for AS in step 2, continuing with step 3 is optional.
Screen from DA step 3: information about active treatments.
DA step 4: summary
An overview of how many topics have been read and the responses to VCEs are provided in a printable summary at the end of the DA (Figure 2). This summary can be taken by the patient to the next consultation with the clinician in order to further facilitate SDM. Alternatively, the summary can be accessed online during consultation.
DA summary page.
Usability testing
Usability testing resulted in 212 usability and content comments. Care providers mainly reported feedback on the specific radiotherapy-related content, a need for more descriptive notes to accompany the illustrations and risk representations. Patients mainly reported usability remarks and comments about the DA summary section. All participants reported that the writing style was comprehensible and that the DA structure and navigation were clear. A summary of the results from the think-aloud procedure and interview results are presented in Table 3. In addition to the usability items, all care providers (100%) indicated that they would offer the DA to patients, and all patients (100%) indicated that they would recommend the DA to other patients. After discussion of the results in the development team, three main adjustments to the final DA were made: (1) accompanying legends were added, (2) radiotherapy content was adjusted and (3) the DA summary section was simplified. The final version of the DA complied to all IPDAS criteria 6 (Table 4).
DA usability test summary.
DA: decision aid; PSA: prostate-specific antigen.
IPDASi v3 checklist.
DST: decision support technology.
Discussion
This article describes the development of a Dutch Pca treatment DA, based on an evidence-based Canadian Pca treatment DA and the subsequent usability testing among relevant user groups. Results of usability testing show that the DA was evaluated positively by patients and care providers and both groups would recommend use of the DA in clinical practice. The described development method could be useful for adaptation of other pre-existing and validated tools to different cultural or local circumstances.
Development of DAs is an effortful process and usually involves multiple rounds of assessing needs, required content and preferred structure among patients and care providers. 33 An important benefit of the proposed model of adapting a pre-existing tool is that these steps are already taken. For the current DA, the content was previously validated,15–19 and a cross-cultural comparison also confirmed importance of the included topics to Dutch patients. 21
The availability of validated content made it possible to focus more on the fit between DA structure and typical conversation flow in routine clinical practice. Many DAs have been developed for use independent from the consultation, 34 which may have led to a suboptimal fit between conversation flow during consultation and DA structure. A known barrier related to limited DA uptake in clinical practice is that clinicians often find DAs impractical to use or that other consultation-specific factors limit structural DA implementation. 35 Therefore, additional observations of clinical consultations and role playing took place and identified a two-step approach in discussing Pca treatment alternatives with patients. Instead of offering four alternative treatments simultaneously, a first step contains choosing between active surveillance and curative treatment and a second step discusses curative treatments in more detail. By also transferring this two-step approach from consultation into the DA, it is expected that patients experience a more natural fit between consultation and DA usage. Moreover, the DA provides direct support to the clinician’s explanation.
To further improve facilitation of SDM, we added two features to the DA. First, VCEs were developed and added to the DA. Second, the DA ends with a printable summary of preferences and responses to the VCEs that the patient can bring to his clinician for discussion. The summary provides the clinician with insight on what matters most to the patient and to what extent the patient has formed a preference or is still undecided. The following consultation and additional decisional support (e.g. consultations with nurses, radiotherapists) can then be adjusted accordingly. A cluster randomized controlled trial (RCT) is in progress to evaluate the efficacy of this DA and to test whether decision outcomes align with patients’ preferences and values. 36
Literature reports mixed findings from using VCEs in DAs and provide no clear guidelines for VCE design.25,26,28,37,38 However, there are indications to assume that the benefit of VCEs emerge after the decision is made and that VCE design should at least incorporate trade-offs between treatment attributes.19,29 In the absence of design guidelines, further development of the VCE within the current DA was based on consensus within the development team. However, future research should look into the effectiveness of the VCE features used in this DA.
A specific aspect that needs to be investigated further is the labeling of VCE outcomes. For the current DA, the development team decided to label VCE outcomes with corresponding treatments. With a strong initial treatment preference (pre-DA), it could be that labeling may lead to patients seeking confirmation of their initial preference rather than achieving actual preference elicitation or misinterpreting information. 39 However, for clarity reasons, we believed that the VCEs should have labeled outcomes to make patients aware of the consequence of their preference (e.g. when valuing incontinence worse than bowel problems, a patient should place radiation therapy over surgery on this topic). With this insight, the responses to the VCE contribute to the construction of an informed treatment preference. We expect that labeled VCEs support this process better compared to unlabeled items. To gain more understanding on the development and usage of VCEs, more studies are needed to investigate VCE effectiveness and optimal presentation formats.
A potential limitation of the current development and usability test was the relatively small sample used in usability testing (N = 11). However, our sample included all relevant user groups: patients, urologists, nurses and a radiotherapist. All participants in usability testing consented on the usability and acceptability of the DA to a point where it seemed saturation was reached, and it was not expected additional participants would have resulted in new insights. The point of saturation in qualitative research is often reached within 6–12 participants. 40
Conclusion
The newly developed Dutch Pca treatment DA was evaluated positively by patients and care providers, both groups would recommend DA usage to others. Patients consented on easy usability and care providers confirmed the accuracy of the provided information. Adapting an existing tool to a (culturally) different setting and adjusting it to local circumstances seems a useful alternative to an entirely new development process. This could free resources to focus on other important aspects like DA implementation.
Practice implications
The process of developing and testing the DA as described in this article could be applied to the (cultural) adaptation of other pre-existing tools to different languages and clinical settings. As it enhances focus on usability and fit with clinical practice, it could be a fruitful step to improve implementation of DAs in routine clinical care.
Footnotes
Acknowledgements
The authors thank Dr Feldman-Stewart and her team at Queen’s Cancer Research Institute in Ontario, Canada, for sharing their original decision aid for further development in The Netherlands. The development of the DA in The Netherlands was funded by the Delectus Foundation. The authors thank all patients and care providers for participating in the usability test. Finally, the authors thank the Communications Department of the Elisabeth-TweeSteden Hospital for revising all textual content of the decision aid.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship and/or publication of this article.