Abstract
Clinicians face challenges in deciding which older patients with dementia to report to transportation administrators. This study used a qualitative thematic analysis to understand the utility and limitations of implementing a computer-based Driving in Dementia Decision Tool in clinical practice. Thirteen physicians and eight nurse practitioners participated in an interview to discuss their experience using the tool. While many participants felt the tool provided a useful ‘virtual second opinion’, specialist physicians felt that the tool did not add value to their clinical practice. Barriers to using the Driving in Dementia Decision Tool included lack of integration with electronic medical records and inability to capture certain contextual nuances. Opinions varied about the impact of the tool on the relationship of clinicians with patients and their families. The Driving in Dementia Decision Tool was judged most useful by nurse practitioners and least useful by specialist physicians. This work highlights the importance of tailoring knowledge translation interventions to particular practices.
Introduction
Dementia is associated with substantial deterioration in driving skills, which likely has a significant impact on road safety.1–3 While many persons with mild dementia may be safe to drive until their dementia progresses further,4,5 many patients with dementia, especially men, continue to drive for years.6,7 Cessation of driving in older adults is associated with depression, social isolation, and mortality.8–10 Physicians are often reluctant to report patients with dementia and other medical conditions that affect driving to transportation authorities, even in jurisdictions with mandatory reporting legislation.11–13 This reluctance has been attributed, in part, to the impact of reporting on the doctor–patient relationship and uncertainty about how the condition affects actual driving skills.14–19 Older physicians, male physicians, and those with higher self-reported levels of discomfort with medical uncertainty have been found to be more likely to address driving during appointments with persons with dementia.12,20 Furthermore, in our earlier work with hypothetical cases, we found that physicians vary greatly in the proportion of patients with dementia whom they report to transportation administrators, with rates ranging from 1.6 to 78.3 per cent in one study, showing substantial nonadherence to the strictly-worded mandatory reporting legislation in a Canadian province. 12
Qualitative research on the driving cessation process in dementia has highlighted limitations of planning abilities in people with dementia as their insight and other cognitive skills decline.21–25 Healthcare providers have been identified as key stakeholders who can increase the understanding of patients and caregivers about the link between dementia and deterioration of driving skills,21,25,26 address the attendant emotional losses, and engage in practical planning.23,24,27,28 There have been a limited number of intervention approaches aimed at facilitating the process of driving cessation among older adults with dementia26,29–32 with various designs, target audiences, results, and limitations.
Our group developed a computer-based Driving in Dementia Decision Tool (DD-DT) which incorporates a decision algorithm to help physicians meet the challenge of deciding whether to report drivers with mild dementia or mild cognitive impairment (MCI) to transportation administrators, and to develop a harmonized, fair and transparent approach, and enhance adherence to the legislation. 33 The healthcare provider views a training video, then uses the tool at the point of contact with the patient and caregiver. The clinician logs onto a secure web page to enter the patient’s cognitive diagnosis, driving history including motor vehicle collisions, concern about driving safety, behavioural disturbances, level of independence in daily activities, and basic cognitive testing results. The DD-DT produces a recommendation either to report, not to report, or in cases where the algorithm yielded no suggestion, a ‘no consensus’ output. The clinician also receives an information package for the patient and caregiver, with resources including how to start a conversation about driving safety, warning signs related to declining driving safety, an insert about grief, a list of approved driver testing centres, a list of driving alternatives in the patient’s region, and contact information for the local Alzheimer Society.
The present study aimed to understand the following: how this decision tool best fits into clinical practice for physicians and nurse practitioners in a region with mandatory reporting of drivers with medical conditions, why and how they integrate it into care, the utility and limitations of the tool, and its impact on therapeutic relationships. We conducted a qualitative thematic analysis of interviews with physicians and nurse practitioners.
Materials and methods
Recruitment of physician participants
Sixty-nine physicians were recruited for a study of the DD-DT through emails issued from national physician organizations, by direct contact from the investigators, and by advertisement in journals and at a primary care conference. To be eligible for the study, physicians needed to meet the following criteria: practice in Ontario, Canada; see up to 12 new patients per year with mild dementia; agree to watch a training video on how to use the DD-DT; agree to enter anonymized patient information into the online tool pages; have access to a printer where they could produce a summary of the data entered for each patient; not have recent exposure to the earlier study 20 that informed the online tool recommendations; and not have an immediate colleague enrolled in the study. This qualitative study was done alongside a quantitative randomized controlled trial (RCT) of the intervention for physicians. 34 While the main users of the tool in the RCT were intended to be primary care physicians, we encountered challenges recruiting primary care physicians who would have adequate patient volumes to use the tool sufficiently during the time frame of the study, and subsequently expanded our participant group to include more specialist physicians. Thirty-five physicians were randomized to use the DD-DT, and 34 were randomized to the control condition; the trial ran from 9 September 2014 to 29 January 2016. All physicians who were randomized to the DD-DT group were invited to participate in an interview for the present study via email invitation, and 13 agreed to participate in this study and were enrolled.
Recruitment of nurse practitioner participants
Nurse practitioners were recruited to participate in an open implementation trial of the tool via emails issued by the Nurse Practitioners’ Association of Ontario. To be eligible for the study, nurse practitioners needed to meet the following criteria: practice in Ontario, Canada; agree to watch a training video on how to use the DD-DT; agree to enter anonymized patient information into the online tool pages; and agree to participate in a post-trial telephone interview. Forty-five nurse practitioners expressed interest in the study; however, 20 of those did not reply to further correspondence or withdrew their interest in the study; 25 were screened, and 14 were enrolled.
Legislative context
All participants were in Ontario, Canada, a jurisdiction with legislation broadly worded as ‘Every medical practitioner shall report to the Registrar … every person … condition that may make it dangerous to operate a motor vehicle’. 35 This legislation only applied to the physicians, and not the nurse practitioners.
Intervention
The DD-DT is a multi-faceted, computerized decision support tool that was developed based on systematic review of the literature and clinical practice guidelines, expert opinion from an earlier Delphi study, 20 as well as qualitative interviews and focus groups with physicians, transportation administrators, and caregivers of former drivers with dementia. 33 The tool integrates clinician inputs about the patient’s clinical and driving history, as well as cognitive performance, and it produces a recommendation for reporting to transportation administrators. The recommendations were either to make these reports or not make these reports, but in some instances, the algorithm provided a ‘no consensus’ statement (i.e. the tool neither supported nor refuted making a report to transportation administrators). If applicable, the recommendation for reporting is translated into a customized reporting form for the transportation authority, and additional print resources are provided for the patient and caregiver. 33
Data collection
Semistructured telephone interviews were conducted by one of the co-authors (L.R.), who was not involved in the tool design and had no vested interests in the success or failure of the project. All interviews were digitally audio-recorded and transcribed verbatim by a professional transcriptionist. Transcripts were checked against the sound files for accuracy by L.R. and corrected where necessary. The interviews covered participants’ reasons for participating in the study, as well as how and when they used the tool. Participants were asked about the value added by the tool to their clinical encounters, the user-friendliness of the tool, its limitations, and its impact on the clinician–patient relationship. Study information provided to participants emphasized that evaluation of the tool was independent of the study team and that their identities would not be revealed. Participants were reminded of this just prior to commencing interviews and were encouraged to provide their frank assessment of the tool. Interview guides are included in the Supplemental Appendix. All participants provided informed consent and received a small cash compensation for completing the interview. This study was approved by the Research Ethics Board of Sunnybrook Health Sciences Centre (REB #269-2013) and conforms to all recognized standards regarding conduct of research.
Analysis
A descriptive qualitative approach36,37 was taken as this was considered most appropriate in the context of this applied health services study. HyperResearch 38 software for qualitative data management was used to code for both anticipated and emergent themes. The coding framework was developed by incorporating key concepts identified through discussion within the study team, a priori concepts based on questions in the interview guides, and unanticipated concepts emerging from the data. Open, axial, and selective coding informed the thematic analysis which employed the method of constant comparison and included searches for disconfirming evidence (divergent cases).38,39 The range of responses across datasets was suggestive of thematic saturation. The software was used as an aid to data coding and management, allowing the qualitative research analyst (L.R.) to apply codes to blocks of text in order that thematic reports could be generated across the entire dataset.
Results
The participants and interviews
All thirteen physicians who agreed to participate were interviewed by L.R. between 26 October and 25 November 2015. The physician interview sample included four males and nine females, who had been in practice between 1 and 30 years. Eight of the physicians described themselves as family physicians, two with a specialized focus on geriatric medicine; the others were specialist physicians, including three geriatricians, one neurologist, and one geriatric psychiatrist. Physician participants practised in a variety of settings including family health teams, group practices, hospital-based speciality clinics, community hospitals, and a walk-in clinic. Geographic locations included large and small cities as well as smaller communities in Ontario, Canada (see Table 1). Physician participants used the tool a median of 4.0 times, ranging from zero to 12 times. The tool recommendations to physicians included 19 recommendations to report the patient, 6 recommendations not to report the patient, and 27 no-consensus recommendations. Physician interviews lasted an average of approximately 26 min. Two of the physicians interviewed had not actually used the tool; however, we included their data to shed light on barriers which prevented them from using the tool.
Participants’ location, type of practice, and number of tool uses at time of interview.
NP: nurse practitioners.
A range of population density is given to preserve confidentiality.
Fourteen nurse practitioners were enrolled in the open trial of the DD-DT, which ran from 14 March to 5 June 2017. One of them withdrew before using the tool, and 10 of the remaining 13 used the tool. Eight of these 10 participants were interviewed by L.R. between 8 June and 6 July 2017. Two nurse practitioners were unavailable to complete the interview. One male and seven female nurse practitioners completed an interview; they had been in practice between 3 and 28 years and worked in diverse settings such as family health teams, geriatric clinics in hospitals and in the community, and memory clinics in rural and urban areas in Ontario, Canada (see Table 1). They used the tool a median of 2.5 times, ranging from one to 12 times. The tool recommendations to this group included 14 recommendations to report the patient, 5 recommendations not to report the patient, and 12 no-consensus recommendations. Nurse practitioner interviews lasted an average of approximately 22 min.
Additional illustrative quotes from the interviews are presented in Table 2.
Illustrative quotes from interviews.
Quotes from family physicians are identified ‘FP’, nurse practitioners ‘NP’, and specialist physicians ‘SP’. Shaded areas of the table represent areas of concern or criticism about the use of the tool.
Participants’ reasons for wanting a tool
A common experience which emerged in interviews was that encounters with patients over the driving issue were emotionally fraught, and frequently posed a risk to the therapeutic relationship. This imparted a sense of immediacy to participants’ interest in the tool. Participants also described the need for a more objective, consistent, and standardized way of dealing with driving assessment, potentially making it easier for patients to accept the loss of their licence when this was deemed appropriate.
How participants used the tool
While study guidelines requested that the tool be used with the patient and/or family member present during the assessment, the majority of participants used the tool after the patient encounter. The central reason for this was feeling that it was insensitive to be dealing with a computer rather than focusing entirely on the patient. A family physician commented on using the tool with the patients ‘rather than on them’ [MD03-FP], and a nurse practitioner indicated that it ‘often paved the way for more fruitful discussion’ [NP03]. Similarly, a specialist physician used the tool to teach family practice residents and found it effective in encouraging students to think more holistically about driving assessments.
Nurse practitioners frequently reported using the tool between segments of lengthy patient assessments, and they generally looked for ways to incorporate the tool into their established practice patterns. Physicians typically used the tool on an ad hoc basis both in cases when they were confident in their independent assessment of the patient’s driving ability and when there was uncertainty: I used it when I felt there was a dilemma. And, unfortunately, that was probably reflected in the results. A lot of the times there was no consensus. And that was kind of frustrating but, at least it allowed me to reflect and to highlight some of the different points on the decision process. And so sometimes even that would just give me a little bit of clarity. [MD01-SP]
Another experience that emerged in interviews was the impact of workflow issues as a factor that influenced use of the tool. Some who undertook comprehensive geriatric assessments were concerned about adding another element to an already lengthy consultation. In addition, complex geriatric assessments could also unfold over a series of encounters, and some clinicians reported finding it difficult to fit the tool into practice because the driving issue was intertwined with other aspects of the patient’s functioning.
Alignment of the tool recommendations with clinical assessment
Overall, physician and nurse practitioners found that the tool often aligned with their independent clinical assessments, and that this supported the face validity of the tool. Other physician participants, however, did not attach the same value to receiving a recommendation that aligned with their own assessment. For these clinicians, the process was redundant rather than a source of reassurance. Participants also described instances where the recommendation generated by the tool diverged from their clinical judgement. In some cases, the tool indicated ‘report’ but the participant chose not to do so because they felt that the risk to safety was low and due to contextual factors other than cognitive impairment, or they felt that reporting would alienate the patient.
In several cases, the tool did not recommend that a report be made to transportation administrators, but the physician participants chose to report because they felt that there were subtle indications of risk not captured by the tool. In general, physicians indicated that they would err on the side of caution regardless of the recommendation generated by the tool, and would either report the patient or refer them for a specialized road test if they had any doubts about the patient’s fitness to drive. In contrast, no nurse practitioners mentioned that they would refer the patient to be reported when the tool did not recommend doing so.
Some physician specialist participants found the ‘no-consensus’ recommendation frustrating because it did not contribute to their thinking about the case. Other participants (including many nurse practitioners) felt the no-consensus decision validated their perception of complexity of the clinical scenario.
Perceived value added by the tool
Participants identified several ways in which the tool could add value to their practice, including validating the clinician’s thinking, providing a virtual second opinion, and particularly for nurse practitioners, functioning as an additional voice in multidisciplinary team discussions. A widely shared perception was that the tool added value in the form of consistency and standardization of the driving assessment process in persons with MCI and mild dementia. Several participants commented on the perceived value of having something more objective than the clinician’s opinion to offer patients: It makes it seem like it’s not just me saying ‘I think you should stop driving.’ [NP04]
Family physicians and nurse practitioners were generally more enthusiastic than specialist physicians about the potential of the tool to add value to their practice. They emphasized the wide range of issues they had to deal with and the long intervals that might elapse between driving assessments. They generally appreciated the tool’s evidence-based approach to assessment of an issue that was emotionally complex, highly consequential to patients, and often required them to make judgement calls. Using the tool helped de-personalize the assessment process so that patients and their families were less prone to perceive the recommendations as arbitrary and subjective. In contrast, specialist physicians consistently pointed out that their current approach to driving assessment was more detailed and nuanced than what the tool had to offer. For this reason, the specialist physicians generally felt it had limited potential to add value to their current practice.
Limitations of the tool
Participants emphasized the importance of dynamics which were not assessed by the tool, such as contextual factors and nuances of the clinical encounter that no algorithm could take into account. For example, the fact that patients’ family members might be saying one thing but communicating another through their body language or general demeanour was considered to be important: So the pure algorithm would have said increased risk, maybe suggest an on-road driving test. But all the vibes in the room are saying, please send the letter. So, there’s nuances which the computer is never going to have access to. [MD05-FP]
In addition, several physicians pointed out the need to interpret rather than input totals on cognitive test scores, and to incorporate information on where points were lost on cognitive testing, the impact of baseline intellect, education, language skills, and the clinical context. Other aspects of the tool that participants found problematic were the need to log in to a website each time they used the tool, and the lack of interface with their electronic medical records system.
Impact of the DD-DT on therapeutic relationship
Since many participants used the tool subsequent to their meeting with the patient, many felt that the tool had no discernible impact on the therapeutic relationship because patients and caregivers were unaware that the tool had been used. Several participants pointed out that while some patients or family members experienced the tool as adding credibility or objectivity, others were mistrustful of decisions seemingly made by a computer. Some participants mentioned that the tool could be beneficial for families who had concerns about their relative’s driving capacity.
Incorporating the tool into future practice
Specialist physicians generally did not foresee themselves incorporating the tool into their future practice because they felt their current approach to driving assessment in patients with MCI and mild dementia was already more sophisticated than the tool: I do a very, very comprehensive assessment that I’m positive no family doctor would ever have time for. So, maybe a tool is more useful in that setting compared to, you know, my assessment where I have a lot of the information. [MD12-SP]
Some physicians were concerned about the additional time required to use the tool and mentioned barriers such as sharing assessments in team-based care, separate computers requiring separate log-ins, and the need for access to external resources such as specialized occupational therapy-based on-road testing.
Family physicians were generally more receptive to the idea of incorporating the tool into their routine and did not experience it as unmanageably time-consuming, especially if it could be integrated with their electronic medical records. In contrast, nurse practitioners highlighted the goodness-of-fit between the tool and their established workflow and patterns of practice. Longer patient visits and the fact that nurse practitioners mostly worked as part of multidisciplinary teams were two factors that were felt to encourage easy incorporation of the tool.
Discussion
This qualitative evaluation explored the experiences of 13 physician and 8 nurse practitioner users of the DD-DT. The DD-DT appeared to hold potential value for family physicians, nurse practitioners, or others who deal with driving assessment in relation to MCI or mild dementia infrequently. While the specialists who participated in the study generally did not see the tool as adding substantial value to their own practices, they acknowledged its potential to improve driving assessment in family medicine. Family physicians and nurse practitioners were generally more enthusiastic about the value of the tool in their clinical practice, emphasizing the consistent, standardized approach, and evidence-informed decision-making as particular strengths of the tool.
A common theme which emerged was the use of the tool as a virtual second opinion. One of the key challenges in translating guidelines on driving in dementia into clinical practice is the limitations of the literature on which these guidelines are based. 40 The DD-DT was developed by incorporating expert clinical opinion, and this feature of the tool was appreciated particularly by the family physicians and nurse practitioners. Specialist physicians, on the contrary, may use intuitive and implicit heuristics and algorithms, 40 making them less likely to appreciate the value of a knowledge translation tool such as the DD-DT. In Ontario, the fact that physicians are legally required to report patients with medical illness that may affect driving, whereas other health practitioners are not, may explain some of the differences between the physician and nurse practitioner participants’ evaluation of the tool. All of the nurse practitioners in this study indicated that if they felt a patient shouldn’t be driving, they would communicate this to a physician, who would then initiate the report if they agreed with the nurse practitioner’s assessment.
Based on how the DD-DT was used by this group, there was no clear positive or negative impact on clinicians’ relationships with patients and their families. Similarly, in our quantitative evaluation of the RCT of the DD-DT, we found no difference in physicians’ perception of their relationship with the patient or caregiver based on whether they were in the intervention or in the control group. 34
An unanticipated use of the tool was as a teaching resource, although a cautionary note was sounded with regard to over-reliance on tools in lieu of clinical judgement, especially by less experienced clinicians. Another finding was that uptake of the tool will depend not only on the sophistication of the algorithm generating the tool recommendations but also on logistics and workflow issues specific to various healthcare providers. The variety of clinical settings and routines in which the tool was used in this study suggests that flexibility in its adaptation will be important going forward. Specifically, important barriers to wide implementation of the tool were identified, including the need to log onto an external portal, the lack of integration between the tool and existing electronic medical records, and aspects of clinical encounters that no algorithm can easily capture, such as nuances of communication, body language, and behaviour observed by clinicians. Some participants used the tool selectively in their most difficult cases: This may have increased the proportion of ‘no-consensus’ recommendations by the tool and decreased participants’ perceptions of the tool’s usefulness.
The DD-DT is one more resource that clinicians can use in a variety of settings, to initiate discussions about driving cessation and provide objective evidence supporting a decision to report the patient to driving administrators, rather than appearing to rely primarily on the clinician’s opinion. Our results show that the tool was used, at least by some participants, as a way of facilitating communication with patients and caregivers. When such a tool is used in jurisdictions with mandatory reporting, the healthcare practitioners have no choice but to assess and report on safety to drive, but the tool can ideally facilitate early conversations about driving for those who do not yet need to be reported. When such a tool is used in jurisdictions without mandatory reporting legislation, the role of the healthcare practitioner as an advocate and the tool as a conversation starter can be further enhanced. To our knowledge, our intervention is the first to undergo a qualitative analysis to elucidate the facilitators and barriers to widespread implementation of such a tool.
There were several limitations of this study. First, while specialist physicians were not the primary intended users of the tool and while specialist physicians were more critical of the tool in this study, we confirmed that the tool may be more useful to family physicians and other non-specialists who conduct driving assessments infrequently. Second, many participants had a limited number of uses of the tool. A larger pool of experience may have yielded additional insights. Third, we were unable to assess the influence of participant characteristics (e.g. gender, years in practice, practice settings) on users’ experience with the tool. Fourth, we did not collect data on whether or not the users made the patient and family member aware that the tool was being used. In keeping with the work of Epstein and Street, 41 future work on the DD-DT should embrace the values of patient-centred care by examining patients’ and caregivers’ response to the use of the tool, their assessment of the tool’s impact on communication during office visits, and the impact on shared decision-making regarding driving safety and driving cessation. Fifth, more nurse practitioners practised in rural regions, so differences in views about the tool between nurse practitioners and physicians may be related to practice setting rather than discipline. Future work would be needed to delineate differences between these groups of clinicians and their approach to the use of such a tool given their differences in training, expectations, and patient relationships, particularly since many nurse practitioners work in multidisciplinary teams, often with physicians. Finally, we did not study the use of this tool by other health practitioners involved in driving assessments, such as occupational therapists and neuropsychologists.
The DD-DT was viewed least favourably by specialist physicians and most favourably by nurse practitioners. Specialists felt the DD-DT fits best in family practice where such assessments arise only intermittently, and some of the family physicians agreed with this. We identified many different ways that participants in practice used the DD-DT. ‘One size fits all’ clearly will not work for a knowledge translation intervention that addresses an issue as complex as driving assessments in dementia. Our findings highlight the importance of tailoring interventions to particular types of clinical practice, incorporating information about the intended user’s clinical training and discipline, the frequency with which this clinical question arises, geography, and practicalities of electronic records and office set-up.
Supplemental Material
Virtual_Second_Opinion_Appendix_-_Revised_submitted_to_HIJ_2019-02-28 – Supplemental material for A virtual second opinion: Acceptability of a computer-based decision tool to assess older drivers with dementia
Supplemental material, Virtual_Second_Opinion_Appendix_-_Revised_submitted_to_HIJ_2019-02-28 for A virtual second opinion: Acceptability of a computer-based decision tool to assess older drivers with dementia by Mark J Rapoport, Carla Zucchero Sarracini, Benoit M Mulsant, Dallas P Seitz, Frank Molnar, Gary Naglie, Nathan Herrmann and Linda Rozmovits in Health Informatics Journal
Footnotes
Author’s note
some healthcare practitioners feel that a computer-based online tool has value in helping them make decisions regarding driving safety in patients with mild cognitive impairment and mild dementia. uptake of the knowledge translation intervention tested here is heavily influenced by workflow issues across healthcare disciplines. knowledge translation interventions must be tailored to the type of clinical practice of the intended user.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by Canadian Institutes of Health Research Grant # KAL129896.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
