Abstract
While eHealth technologies such as web-based interventions, mobile apps, and virtual reality have the potential to be of added value for forensic mental healthcare, there is a gap between this potential and the current situation in practice. The goal of this study was to identify recommendations to bridge this gap. In total, 21 semi-structured interviews and 89 questionnaires were conducted in a Dutch forensic mental healthcare sample consisting of professionals, patients, and eHealth experts. Based on the broad range of identified recommendations, it can be concluded that attention should be paid to the characteristics of professionals, patients, technology, and the organization throughout the development, implementation and evaluation of eHealth.
Introduction
Over the last few years, the use of eHealth technologies in the treatment of forensic psychiatric in- and outpatients has received increased attention. eHealth is not merely the use of technology to improve health, well-being and healthcare, but also incorporates a change of perspectives, ideas, attitudes and ways of working within the healthcare system (Eysenbach, 2001; van Gemert-Pijnen et al., 2018a). eHealth technologies such as web-based interventions, virtual reality or mobile apps have the potential to improve the quality and efficiency of forensic mental healthcare. However, until now, there has been little research on the actual effects and benefits of eHealth in forensic mental healthcare (Kip et al., 2018). Research on eHealth in mental healthcare in general has shown multiple actual and potential advantages, such as: an effectiveness comparable to face-to-face treatment, increased efficiency, an increase in patients’ autonomy, and the delivery of healthcare independent of time and place (Andersson et al., 2014; Andersson & Cuijpers, 2009; Carlbring et al., 2018; Cuijpers et al., 2010; Rochlen et al., 2004; van Gemert-Pijnen et al., 2018b). However, these positive findings cannot simply be generalized and applied to forensic mental healthcare due to its unique nature of transpiring at the intersect between psychiatry and law. Forensic mental healthcare deals with the relationship between assessment and treatment of mental illness and the criminality of people whose behavior has led, or could lead, to offending (Arboleda-Florez, 2006; Mullen, 2000). Since knowledge about this particular healthcare field is limited, there is an urgent need for more research on eHealth technologies in forensic mental healthcare.
A recent review synthesized information about different types of technologies in treatment of offenders (Kip et al., 2018). This review showed that different types of technologies can have different advantages for forensic mental healthcare. In research and practice, most attention is currently being paid to web-based interventions, in which treatment, or parts of treatment, are delivered online. A main advantage of this type of technology is that its content and form can be adapted to the needs of individual patients, for example, by providing information either via video or written text. By means of such tailored eHealth interventions, a better fit between technology and patient can be achieved (Batastini et al., 2016; Cunningham et al., 2012; Levesque et al., 2012). In this way, the predominant one-size-fits-all approach toward eHealth interventions can be overcome in the broad and diverse forensic mental healthcare sector, which is characterized by a large variety of patients, treatments and judicial statuses (Fielenbach et al., 2019; Whitaker et al., 2006). Another technology that has been receiving much attention the last few years is virtual reality (VR). VR has the possibility to address multiple senses such as sound and vision, instead of thinking and talking, which is often the case in cognitive behavioral therapy (Cornet et al., 2019; Fromberger et al., 2014; Gooch & Living, 2004; Hodge et al., 2015; Klein Tuente et al., 2018; Smeijers & Koole, 2019). This focus on “doing” instead of “thinking” can address issues related to functional illiteracy, which is relatively high in offenders (Kip et al., 2019a). Another form of technology identified in the review was serious games. The idea behind this type of technology is to make eHealth interventions more engaging by adding persuasive elements or gamification (Kip et al., 2018, 2019a; Smeijers & Koole, 2019). This approach is expected to have a positive impact on treatment motivation, which is often low in forensic psychiatric patients (Drieschner & Boomsma, 2008; Greenberg et al., 2007). Finally, relatively new technologies such as wearables can be used to collect data about heart rate or skin conductance (Cornet et al., 2017). This type of data can yield new insights for patients, supporting their self-management as well as providing new information for their therapists, which can, in turn, positively impact the quality of treatment.
However, most of these advantages remain only as potential benefits. There are few studies on effectiveness, and there is a large gap between the current use of eHealth in forensic practice and its expectations based on its potential (Bierbooms et al., 2015). Several reasons for this gap can be identified. Amongst other things, not all therapists and patients have a positive attitude toward technology. For example, many therapists are not enthusiastic about using web-based interventions in a “blended” way with their face-to-face treatment (Kip et al., 2020). Also, there is not much knowledge on the potential positive or negative impact of technology on treatment, which can be a barrier for uptake by either management or professionals. Furthermore, technology can have negative unintended consequences, such as the elicitation of unnecessary negative emotions when using VR, of which not much is known. Finally, protocols and regulations specifically focused on eHealth are minimal, which can hinder its integration into clinical practice (Kip et al., 2018). While there are multiple recommendations from the literature on how to overcome these barriers and achieve the technologies’ potential benefits, these recommendations are abstract and not directly usable in forensic practice (Kip et al., 2018). Additionally, there is a risk that recommendations from the literature are outdated since it often takes several years before articles on eHealth technology are published (Hekler et al., 2016). The current study, therefore, focusses on identifying recommendations to overcome barriers and optimally benefit from the possibilities of eHealth technology. These recommendations derive from multiple perspectives within forensic mental healthcare practice – professionals, patients, and eHealth experts.
Methods
Design and setting
The current study is part of a larger project that aimed to (1) analyze the current state of technology in Dutch forensic mental healthcare; (2) map benefits and barriers of 12 different types of technology; and (3) identify recommendations to achieve the benefits and overcome the barriers of eHealth in forensic mental healthcare, as determined by professionals, patients, and experts. The current paper focuses on the third goal. For readers interested in the outcomes of the first two goals, we refer to the Dutch report of the complete project (Kip et al., 2018, 2019b) and to the identified benefits and barriers that were generated in the project, which can be found in Appendix A and B, respectively. The project was coordinated by a project team, which included a forensic nurse, policy manager, researcher and the authors of the current paper. All the conclusions and recommendations in the project were checked by a psychologist, a forensic nurse and a forensic psychiatric patient, to ensure that these were in line with the experiences from practice.
The study described in this paper applies a qualitative multi-method approach in which data from two different sources are triangulated. By combining two methods, a more comprehensive picture of the recommendations can be painted (Flick, 2004; Van Staa & Evers, 2010). The methods used were an interview study and an online questionnaire. The interview study aimed to gain insight into the participants’ recommendations to optimize the use of technology in forensic mental healthcare and to overcome potential barriers by discussing the current state of affairs of technology in forensic mental healthcare. The questionnaire aimed to also gain insight into recommendations by means of asking participants to evaluate the potential of 12 different types of technologies. By using a questionnaire, a larger and more diverse set of respondents could be reached, while the interviews provided more in-depth information. Ethical approval for both studies was obtained from the Ethical Committee from the Faculty of Behavioral, Management and Social Sciences (BMS) of the University of Twente (18807).
Interviews
Participants
Interviews were conducted with three categories of participants: professionals working in forensic mental healthcare, current or former forensic psychiatric patients, and experts on technology in forensic mental healthcare. Inclusion criteria were that the participants needed to have experience with eHealth and had to have knowledge about or experience in forensic mental healthcare. The rationale for the recruitment strategy was based on the purpose of this study to identify recommendations that seamlessly fit the context of the end-users of eHealth in forensic mental healthcare, specifically professionals and patients. In addition, experts on technology in forensic mental healthcare were deemed able to reflect on this issue from a broader perspective. In this way, in-depth contextual knowledge on technology in forensic mental healthcare and more generic knowledge on eHealth were combined to complete a comprehensive picture of recommendations.
A combination of purposive and convenience sampling was used to recruit participants from these three categories. The project team identified a list with names of potential participants belonging to either the professional, patient or expert category via individual networks of the project members. Additionally, snowball sampling with included participants was used to identify participants that the project team might have overlooked. In selecting participants to include in the interview study, the project team paid attention to the range of participants to ensure that multiple perspectives were included in the final sample, such as a distinction between in- and outpatient care and experience in different types of forensic organizations.
In total, 21 participants were interviewed. Of those participants, five were researchers, four were patients, four were innovation or eHealth experts or consultants, three were psychologists, three were forensic nurses, one was a policy advisor, and one was a former patient working as an “experience expert” in clinical practice. Participants worked or were treated at 13 different organizations: seven forensic hospitals, three universities, one company and two government institutions. Our sample of 21 participants was based on practical feasibility and data saturation. Based on the five-month duration of the interview study and the availability of resources, a number of 20–25 interviews was anticipated. Data saturation was checked based on the appearance of new codes in the interviews. In the final five interviews no new main codes appeared, indicating that data saturation on a thematic level had been reached.
Materials and procedure
The semi-structured, in-depth interviews were conducted at a location of the participant’s choice and averaged 50 minutes in length. To start, the researcher explained the study and an informed consent was signed. The interview scheme (see Appendix C) consisted of five parts. First, background information such as demographics and experience with working with technology was discussed. Second, a list with technologies was presented to the participant: virtual reality (VR), videoconferencing, web-based modules, mobile apps, wearables, serious games, and social media and fora. This list was based on the aforementioned literature review (Kip et al., 2018) and the knowledge of the interdisciplinary project team. A short description of each technology was provided, and participants had the opportunity to ask questions if any technology was unfamiliar to them. The interviewer asked the participant whether he or she had any additions to this list, after which the participant could indicate which of these technologies he or she would like to discuss in more detail. In the third part of the interview, the participant was asked about benefits, barriers and recommendations of this selected technology. In the fourth part, the remaining technologies were discussed more briefly, and again attention was paid to benefits, barriers and recommendations. Lastly, the participant was asked for their most important recommendations and final remarks. Throughout the entire interview, probes were used to invite the participants to elaborate on their answers.
Analysis
The transcripts were transcribed verbatim and analyzed by means of a combination of inductive and deductive coding (Boeije, 2014). The coding schemes for the recommendations were based on those developed for a systematic review with a similar topic (Kip et al., 2018). This existing coding scheme was used by two researchers (HK & KO) to start the coding process of the first five interviews. Throughout this process, codes were rephrased, combined, or added to optimally fit the content of the transcripts. This resulted in an adapted version of the existing coding scheme in which several codes were rephrased or redefined, and multiple new codes were added. Next, the remaining 16 transcripts were coded with this adapted version of the coding scheme by one researcher, during which minor changes to the coding scheme were constantly made. In case of doubt, codes were discussed with the second researcher. No new codes could be identified in the final five interviews and no further changes to the definitions were necessary, which indicated that theoretical data saturation was reached and that no additional interviews were required. Finally, the quotes used in this paper were translated to English and rephrased by a native speaker.
Questionnaire
Participants
The main target group of the questionnaire consisted of professionals such as therapists or policy managers working in Dutch forensic mental healthcare, but the questionnaire was designed in such a way that patients or people with relatively little knowledge on forensic mental healthcare could also participate. The link to the questionnaire was distributed in several ways. The questionnaire was presented at a national conference on forensic care via a presentation and, consequently, cards with the questionnaire link were distributed to attendees of the conference. This link was also posted on a well-known national platform for forensic mental healthcare, and members of the project team distributed the link throughout their networks. In this way, all professionals working in Dutch forensic mental healthcare had the opportunity to participate in the study.
In total, 118 people started the questionnaire, of which 76 people fully completed it (64%). Of the 118 participants, 89 (75%) gave their opinion on at least one technology, and thus were included in data analyses. Of these 89 participants, 61% were female, and 39% were male. The average age was 40.36 (SD = 12.44). Of the 100 people who indicated their profession or function, 78 worked in forensic mental healthcare, of which 57 worked as a professional for patients, and the remaining 21 had a supportive or policy-related function. Furthermore, 13 participants were researchers and 7 were professionals working in another branch of mental healthcare. Finally, a student and software developer participated. Overall, these participants were positive about technology in forensic mental healthcare; their average score on a scale ranging from 1 to 5 was 4.00 (SD = 0.71).
Materials and procedure
The questionnaire started with a short explanation, informed consent and several questions on socio-demographic information. In order to support participants of the questionnaire to propose a broad range of recommendations as well as to gain insight into their preferences, an overview of different types of technology that can be used in forensic mental healthcare was created. Since the goal of this overview was to give participants an idea of the broad range of the possibilities of technology, this overview of technologies was not exhaustive nor mutually exclusive. Consequently, for each provided technology, one unique quality was considered central. In Table 1, an overview of the technologies and a summary of their definition is provided. In the questionnaire, the 12 technologies, a brief explanation and a picture to illustrate each one were presented to the participant in randomized order. The participants were asked to indicate on a Likert scale ranging from 1 to 5 to what extent they found each technology promising for forensic mental healthcare. Next, the participants had the possibility to make remarks about benefits, barriers or recommendations for each technology. The questionnaire ended with questions regarding the participant’s general remarks or recommendations for technology in forensic mental healthcare.
Analysis
The quantitative data of the Likert scales in the questionnaire were analyzed via descriptive statistics. Average scores and standard deviations for each technology were calculated. The qualitative answers to the open-ended questions were coded deductively by two researchers (KO & HK), by means of the coding scheme that was used to code the data of the previous 21 interviews. This coding scheme was used to ensure integration of the data of both research methods of this paper: the interviews and questionnaires. In total, 86 fragments in which recommendations were mentioned were coded. Since the existing codes fit the content of these fragments, no changes to the codes or their definitions were required.
Results
Technologies for forensic mental healthcare
Participants indicated for each of the 12 technologies (see Table 1) how promising they found that specific technology on a 5-point Likert scale. In Table 2, the average score for each technology is provided.
An overview of the technologies and a summary of their definitions in the questionnaire provided to the participants.
Types of technologies with the average score that the participants of the questionnaire (n = 89) gave regarding the extent to which they found the technology promising for forensic mental healthcare on a Likert scale of 1 to 5.
Recommendations provided by participants
In Table 3, all codes that were identified in the interviews and qualitative data of the questionnaire are provided, including the total and relative number of interviews and questionnaires in which the code was identified. In the remainder of the “Results” section, these codes are defined, further explained and illustrated by means of quotes.
The main codes, sub-codes and the number of interviews and questionnaires they were found in.
Development process
The recommendations within this main code all focus on the importance of iterative, participatory development of eHealth technology. All sub-codes, their definitions and an illustrative quote to illustrate the code are provided in Table 4. Participants indicated that the main goal of such a development process was to achieve a fit between the technology, users and context. The importance of developing personalized technology was emphasized to ensure its adaptation to the diverse, broad forensic patient population. Therefore, active involvement of stakeholders, such as patients and therapists, throughout the entire development process was advised. Furthermore, participants stated that development never stops; technologies should be updated to ensure the optimization of the aforementioned fit with the patients, therapists and context. Finally, collaboration at a national level between organizations and IT developers was recommended. Instead of multiple small initiatives operating with a nominal chance of success, by joining forces, such collaborative projects might have a better opportunity to create sustainable eHealth technologies.
The sub-codes, definitions and an illustrative quote for the main code “development process.”
Implementation in forensic organizations
The following recommendations focus on what is required for successful implementation of technology in organizations that offer forensic mental healthcare. Implementation refers to activities that are conducted to ensure the technology’s dissemination, adoption, and long-term use in practice. The accompanying sub-codes can be found in Table 5. Participants highlighted the importance of integrating technology into the existing situation in order to prevent the technology from being overlooked and seen as a separate, stand-alone tool, which is currently often the case. This might require re-arranging the way care is delivered and organized. Furthermore, since many therapists and managers do not have enough knowledge of the possibilities of technology, there should be more activities – such as courses, team meetings, websites or ambassadors – to disseminate technological knowledge. Participants also emphasized the importance of meeting basic prerequisites, such as a good internet connection and/or suitable hardware. The financial side of implementation was mentioned as well, for example, through the creation of business cases and keeping long-term financing of technology in mind. Finally, as in development, interdisciplinary collaboration on a national level between different types of stakeholders was deemed essential for successful implementation.
The sub-codes, definitions and an illustrative quote for the main code ‘implementation in forensic organizations’.
Evaluation
The recommendations within this main code focus on the use of evaluation studies to investigate to what extent a technology: reaches its goals, adds value to forensic mental healthcare, and further improves the current technology. As can be seen in Table 6, participants indicated that it was not only important to conduct more research to determine if eHealth works, but also to better understand why and for whom the technology works. Suitable and innovative research methods are fractional factorial designs or log data analyses. To illustrate, web-based modules might work better for patients with higher literacy and reflective skills, while VR might be most effective for patients with low educational levels and aggression regulation problems. Furthermore, besides effectiveness, participants also stated that the evaluation should focus on whether a technology is a reliable and a valid method to measure certain behaviors, biases or other phenomena. Finally, participants indicated that research needs to show if the use of eHealth results in decreased costs and more efficient healthcare, especially since this type of information was deemed important for management and healthcare insurance companies.
The sub-codes, definitions and an illustrative quote for the main code “evaluation.”
Guidelines and standards
The following recommendations focus on the importance of creating clear guidelines, standards or protocols for the use of technology in treatment that assist with avoiding (technical) problems and legal matters. The sub-codes for this topic are provided in Table 7. To ensure that therapists are aware of the numerous possibilities, participants indicated that there should be an overview of available eHealth technologies, for example, by means of an online database. Furthermore, there was a need for more guidelines on ethical dilemmas, such as what can be shown to sexual offenders in VR. Participants also indicated that there should be more protocols to support therapists in integrating eHealth technology into their treatment. The importance of clear and suitable legislation was raised as well. Participants indicated the need for more clarity and guidelines on topics such as privacy, data security and ownership of data.
The sub-codes, definitions and an illustrative quote for the main code “guidelines and standards.”
Readiness of user and technology
The following recommendations aim to ensure that a technology can be used successfully and faultlessly by professionals and forensic patients. The sub-codes belonging to this main code can be found in Table 8. Participants pointed out the importance of improving the skills of therapists via, for example, training or intervision. Training should not only focus on technical knowledge, but also on attitudes toward technology and skills that are required for embedding eHealth technology into treatment, such as discussing the technology with patients, motivating patients, and integrating data into follow-up conversations. The way technology is designed was also deemed important. For example, participants indicated that many web-based interventions contain too much text and are hard to navigate. Aligned with this, participants also stated that technology should not have too many hard- or software errors since these types of problems can hinder usage.
The sub-codes, definitions and an illustrative quote for the main code “readiness of user and technology.”
Improving technology
The following set of recommendations focuses on several ways to improve the effectiveness and/or use of technologies. The sub-codes belonging to this main code are defined and illustrated in Table 9. Participants pointed out that technology should be constantly improved and recommended that, in order to achieve this, it is important to have up-to-date knowledge about innovations elsewhere, such as artificial intelligence or robotics. Trying out combinations of technologies was also recommended, such as virtual reality and neurofeedback. This underlines the importance of a broader, more holistic view of technology. Participants suggested that another way to further improve eHealth technology was to ground these technologies in existing, evidence-based theories from different domains. Furthermore, adapting existing technologies to fit specific types of forensic psychiatric patients was recommended, since this is more efficient than creating an entirely new technology.
The sub-codes, definitions and an illustrative quote for the main code “improving technology.”
Discussion
The current study has provided an overview of professionals’, patients’, researchers’ and managers’ recommendations to overcome barriers toward and to optimally benefit from eHealth technologies in forensic mental healthcare. Identified recommendations were related to development, implementation, and evaluation of eHealth in practice; the importance of guidelines and standards; the facilitation of working with technology; and constant improvement of technology. In the remainder of this paper, promising directions related to professionals, patients, the technology, the context, and scientific research are discussed. These recommendations along with this multi-perspective, holistic reflection are essential in view of the general ambition to create sustainable innovations in forensic mental healthcare (Sarkis et al., 2010). It adds to an understanding of how to use the potential of an innovation for systemic change within organizations on multiple levels (van Gemert-Pijnen et al., 2018b).
The professional
A large share of the identified recommendations revolved around the key role of the professional in enabling successful use of technology in forensic mental healthcare. In order to improve the current situation, participants indicated that more training of professionals is necessary and that training should not just focus on technical skills. This recommendation is in line with studies on eHealth in general (Barakat et al., 2013; Chan & Kaufman, 2011; Feijt et al., 2018; Norman, 2011). To successfully use eHealth technologies, education would also have to focus on, for example, developing a positive attitude toward eHealth, creating a sense of ownership for maintaining up-to-date knowledge, improving skills to discuss collected data with patients, and gaining knowledge on how to support patients in their use of eHealth technologies (Barakat et al., 2013; Donovan et al., 2015). Subsequently, the use of technology by practitioners might require more than upgrading skills; in fact, the introduction of technology can implicate in practice a change in people’s professional roles. A therapist’s role might shift from taking the lead in treatment to a more supportive role that includes: giving feedback on assignments in a web-based module, creating environments to allow a patient to individually practice with social skills in VR, or supporting a patient in drawing conclusions on data collected by a wearable to gain insight into what triggers them. However, not much is known about this topic yet, so subsequent research should focus on the changing role of the professional. These types of studies could answer questions on the necessary skills, knowledge and attitudes of forensic professionals, and how they can be translated into education programs or training (Bowen, 2016), as other research has previously determined for nursing in general (Booth, 2006). Furthermore, recent research showed that psychologists differ in the type of perceived drivers and barriers toward eMental health (Feijt et al., 2018). This implies that there are multiple categories of therapists that have different levels of eHealth-skills and -attitudes. New research can focus on these differences in skills and preferences, and the necessity to develop tailored training programs for different types of professionals.
The patient
The current study points out the importance of a good fit between the technology and the forensic psychiatric patient who is using it. Participants indicated that patients often do not possess the required skills for using technologies. For example, in case of web-based modules, many patients were said to have difficulty reading and writing and lack the motivation to individually work on assignments at home, which appears to be a major barrier hindering the use of these types of interventions (Kip et al., 2020). Consequently, developers of eHealth interventions should take the skill level and preferences of patients into account to prevent a mismatch between the technology and patient. Such a mismatch might result in high levels of non-adherence (Abd-Alrazaq et al., 2019). On the one hand, it can be beneficial to further investigate how patients can be better supported in using these mostly text-based apps or websites, which are used a great deal in current practice. This might be done by adding more options for personalization - for example, offering information as videos instead of text or with different literacy levels, or by training therapists to better support patients in using a technology (van der Vaart et al., 2014; Wentzel et al., 2016). On the other hand, organizations might have to invest more in technologies of which the characteristics might better fit the forensic psychiatric patient population. An example of such a technology is virtual reality, which was seen as most promising by participants of this study. An important advantage of interactive VR for forensic mental healthcare is that it offers the possibility to practice specific behavior in a realistic way instead of merely discussing it (Cornet et al., 2019; Klein Tuente et al., 2018; Smeijers & Koole, 2019). Wearables are another example of a technology that focuses less on cognitive reflection. These wearables, such as smartwatches, can collect physiological data such as heart rate variability or skin conductance, which can be used to gain more insights into a patient’s physiological arousal and its potential causes (Cornet et al., 2017; de Looff et al., 2019). However, while both technologies are receiving an increasing amount of attention (Cornet et al., 2017; 2019; Kip et al., 2019a), more studies into their effectiveness and efficiency are required to draw conclusions about whether these types of technologies actually fit forensic patients and are of added value for their healthcare.
The technology
With regard to technology itself, personalization appeared to be important, mostly because of the different needs and characteristics of the diverse forensic psychiatric patient population. Despite the widespread opinion that a one-size-fits-all approach is not suitable for forensic mental healthcare interventions (Cornet et al., 2019; Fielenbach et al., 2019; Kip et al., 2018; Whitaker et al., 2006), many eHealth interventions are still used as a fixed tool instead of an adaptive set of possible interventions (Kip et al., 2018). In order to ensure that a technology seamlessly fits the needs of patients and therapists, participants indicated that a multi-method, participatory development process is pivotal. The relevance of a thorough development process regarding eHealth has indeed been stressed in many other studies (Hekler et al., 2016; Michie et al., 2017; Mohr et al., 2014; Patrick et al., 2016; van Gemert-Pijnen et al., 2011). Within such a process, stakeholders should be involved as co-creators right from the start when the first ideas arise, instead of asking them for feedback on ideas that have already been developed by researchers or developers (DeSmet et al., 2016; Scaife et al., 1997; Yip et al., 2013). To support this fit between the development of a technology and the users’ context, characteristics and needs, continuous formative evaluations of ideas and products are of great importance (Mohr et al., 2014; Patrick et al., 2016). This iterative nature of development with multiple formative evaluation cycles was also mentioned by participants in the current study. Potential suitable methods are semi-structured interviews, questionnaires or usability tests (Kip & van Gemert-Pijnen, 2018). However, more knowledge needs to be gathered about suitable participatory development methods for personalized eHealth technologies in the unique and complex forensic psychiatric context. This can for example be achieved by encouraging publications on development studies in which authors critically reflect on the methods used, or by means of an interview or questionnaire study with researchers and developers with experience on this topic.
The context
When looking at the context in which technology is used, participants emphasized the important role of the forensic organization in the technology’s implementation. Implementation of eHealth technologies should be improved to ensure successful and sustainable use, but this was seen as a complicated activity. This is in consonance with research on eHealth in general; a good implementation process boosts the success of eHealth, but is a complex exercise (Greenhalgh et al., 2017; Pieterse et al., 2018). In line with this complexity, participants provided a broad range of recommendations for improving implementation in organizations by, for example: investing more in technologies, better integrating eHealth into organizational structures, ensuring that all employees are aware of and educated in eHealth, and joining forces with other organizations to share knowledge and costs. The diversity of the recommendations given by the participants can be explained by the broad, holistic nature of implementation. Multiple studies indicate that during eHealth implementation, attention should be paid to factors from multiple domains that relate to the users, organization and the intervention (Damschroder et al., 2009; Greenhalgh et al., 2004, 2017). Consequently, organizations might need to invest more in multilevel implementation plans and activities that focus on these different domains. Also, to systematically plan and guide implementation processes, organizations can use existing implementation frameworks that are suitable for eHealth implementation, such as the non-adoption, abandonment, scale-up, spread and sustainability (NASSS) framework (Greenhalgh et al., 2017) or the consolidated framework for implementation research (CFIR; Damschroder et al., 2009). Research that applies these models to practice should be conducted in order to test the suitability of these models for eHealth implementation in forensic organizations and to increase knowledge on implementation strategies that fit forensic mental healthcare.
Research
The importance of thorough research was intertwined in many recommendations identified in this study. As mentioned before, research can be used to gain more insight into development and implementation methods that are suitable in forensic mental healthcare. Because of this setting’s aforementioned unique characteristics, it is probable that strategies that were successful in other settings, such as in hospitals or regular mental healthcare facilities, cannot simply be copied-and-pasted into forensic mental healthcare (Kip et al., 2019a). Furthermore, outcomes of this study clearly indicated a need for more knowledge on the effectiveness of eHealth interventions, which is in line with the conclusions of a review on this topic (Kip et al., 2018). Several participants indicated that, while a randomized controlled trial (RCT) can be very useful, other types of evaluation methods might be more suitable for eHealth evaluation in context, from a practical and methodological point of view. RCTs often require major investments of time and money, which is not always feasible (Rauwerdink et al., 2019). Also, RCTs do not provide much insight into what elements of an intervention works best for which type of patient and why (Collins & Kugler, 2018; Sieverink et al., 2017). Examples of evaluation approaches that might better address these issues are: (fractional) factorial designs to gain insight into effective components of an intervention (Collins & Kugler, 2018; Kelders et al., 2015); single case experimental designs in which in-depth analyses are conducted on single subjects (Dallery et al., 2013); RCTs in combination with log data analyses to provide insight into usage patterns and potential relationships of usage with effectiveness (Sieverink et al., 2018); realist evaluations to determine what works, how, in which conditions and for whom (Pawson et al., 2005); or mixed-methods approaches in which quantitative data is integrated with qualitative findings to provide more insight into reasons for effectiveness (Lilford et al., 2009). However, to our knowledge, none of these approaches have been used to study eHealth interventions in forensic mental healthcare, so researchers might consider applying other types of designs instead of or in combination with an RCT.
Limitations
While this study offers a broad view of recommendations to improve the use of eHealth in forensic mental healthcare, several limitations can be identified. For the interview study, a combination of purposive and convenience sampling was used. This might have resulted in a bias, in which only participants with an overly positive opinion might have been included. To account for this, the interview scheme was created in such a way to also elaborately account for a critical perspective by discussing the disadvantages and barriers. Also, since 21 participants were interviewed, it was not possible to create a representative overview of all existing opinions and recommendations on eHealth. However, the goal of this study was not to provide an exhaustive, representative overview of all existing opinions in clinical practice, but to gain insight into the most important and urgent recommendations for improving the use of eHealth technologies within forensic mental healthcare. Also, while many different participants were included, theoretical saturation on an abstract level occurred after approximately 15 interviews since no new main codes were identified. Additional interviews would have probably yielded new examples and more variation within codes, but would not have resulted in new codes or major changes to the definitions of codes. Consequently, conducting more interviews would most likely have not changed the conclusions of this study.
With regard to the questionnaire, sampling took place on a national level and the goal was to reach all professionals working in forensic mental healthcare via online networks. However, it is impossible to assess whether this was actually the case. A selection bias might have occurred. It might be possible that only participants that were relatively positive about the possibilities of eHealth took part in the study, which might have influenced the validity of the results. However, the results from the questionnaire showed that multiple participants were not positive about eHealth, which shows that not only participants with a positive opinion participated. Consequently, we have attempted to interpret the data of this study with care in order to avoid generalizing the results.
Another limitation is that the organization of Dutch forensic healthcare differs from that of other countries (van Marle, 2000), which might mean that several findings are not applicable to other countries. However, many of the current findings are, on an abstract level, comparable to those of the aforementioned recent systematic review on eHealth in treatment of offenders in which studies from multiple countries were included (Kip et al., 2018). The recommendations of the current study are fairly abstract and thus are not ready-to-use guidelines, which means they should be further specified when applied in specific settings or countries.
Conclusion
Overall, this study made clear that, with regard to the use of eHealth technologies in forensic mental healthcare, attention should be paid to professionals’ and patients’ perspectives, the technological characteristics and the organization throughout the eHealth technology’s development, implementation and evaluation processes. Such a holistic approach in which multiple perspectives are integrated is necessary to ensure that eHealth is of actual added value for forensic mental healthcare and not merely used for the sake of using technology. Consequently, an interdisciplinary, collaborative approach is necessary in order to optimally benefit from the potential of different types of technology. Developers, therapists, patients, researchers, financers, managers and other stakeholders should closely collaborate and learn from each other. In order to ensure that eHealth is of actual added value for forensic mental healthcare, people need to cross the borders of their own discipline in order to integrate knowledge and experiences.
Footnotes
Acknowledgements
We would like to thank the members of the project team for their valuable contributions to the project: Saskia Kelders, University of Twente; Dirk Dijkslag, Transfore; Bianca Roelofsen, GGZ Noord-Holland-Noord; Ron Voorhuis, Transfore; Monique van Haaren, GGzE; Tanja van der Gulik, GGZ Noord-Holland Noord. No potential conflict of interest was reported by the authors.
Conflict of interest
We have no conflicts of interest to disclose.
Appendix A
Appendix B
Appendix C
Note: this interview scheme is the version that was used for interviews with professionals working in forensic mental healthcare. The interview scheme, mostly Part 1, was adapted when used with patients, researchers or other types of participants.
These questions are mainly used as background information for the project team. Later, we will further elaborate on different types of technologies and its benefits and barriers. What is your current function? And what are your main activities? With which organization(s) are you professionally involved? What is your age? How many years have you been working in forensic mental healthcare? What is the role of technology in your professional activities? Currently and previously? What is your general opinion on the use of technology in treatment in forensic mental healthcare on a scale of 1 to 10? Why?
In this second part we will first discuss types of technologies that are used or might be used in forensic mental healthcare. The focus is really on the types of technologies; we will not be discussing specific interventions. First, I will show technologies that were identified in our systematic literature study. After that, I will ask you whether any technologies are missing, or if there are any unclarities.
The following technologies were presented to the participants by means of cards that were put on the table by the interviewer for the sake of providing overview. Web-based modules Video-conferencing Games Virtual reality Social media and for a Wearables (e.g. a smartwatch) Is it clear what is meant by each technology? If not, on which ones would you like some additional explanation? Do you have any additions to this list of technologies? Are you using specific types of technologies in your treatment of patients? If so, which ones?
We will now continue with the next part of this interview, where we will discuss one technology in-depth. Amongst other things, I’ll ask you about the benefits, barriers, and what is needed to further benefit from the potential of this technology. After that, we will pay attention to the other types of technologies. Can you choose one technology where you want or can tell the most about? It does not necessarily have to be a technology with which you have a lot of experience. First, can you indicate what the advantages or benefits of this technology are, or could be, for forensic mental healthcare? Note: the interviewer first lets the participant talk, after that – if necessary – more detailed questions are asked: What are advantages or benefits for patients? What are advantages or benefits for professionals? What are advantages or benefits for the quality or efficiency of care? What are disadvantages or barriers of this technology for forensic mental healthcare? What are disadvantages or barriers for patients? What are disadvantages or barriers for professionals? What are disadvantages or barriers for the quality or efficiency of care? What is required to use this technology successfully in forensic mental healthcare? Note: the interviewer first lets the participant talk, after that – if necessary – more detailed questions are asked: Patients Professionals Management of organizations Practical requirements Government Research
We’ve now zoomed in on one technology to make it a bit more concrete. We will now again look at all technologies that are written on these cards here on the table, and discuss them a bit shorter. Again, I’ll ask for advantages, barriers and recommendations, but we’ll discuss them a bit less in-depth. With which technology do you want to start? Can you indicate the barriers for this technology? Which barriers do you see for this technology? What is needed to successfully use this technology in forensic mental healthcare? Note: this continues until all technologies have been discussed; the researcher ensures that this part does not take up too much time.
We’ve now arrived at the last part of this interview. If you look at all these technologies, which one do you find most promising or useful for forensic mental healthcare? You can choose more than one. Why do you find this technology/these technologies promising or useful? Are there any other things you want to say on technology in forensic mental healthcare that are important according to you, and have not been discussed yet?