A holistic perspective on continuing care for substance use and dependence: Results and implications from an in-depth study of a Norwegian continuing care establishment

Abstract

This article explores, systematically and in depth, users’ perceptions of participating in a Norwegian non-profit establishment that provides a continuing care programme for substance use and dependence. Identified results are linked to a holistic system perspective, where human, technology, and organisation (HTO), as well as external environment, are viewed as intertwined. At the establishment level, i.e., where the continuing care programme is delivered, we find that a clear holistic and user-oriented profile – comprising combined interventions including physical and social activities – can create a safe and stable environment that exerts a positive mental and physical influence on the user and thereby promotes abstinence from substances. However, our results suggest that the internal environment needs to connect more strongly with the external environment, such as a substance-free network, close family, and working life. At the establishment level, we conclude that there is a need to develop an explicit strategy and practice for collaborating with the external environment, built on systemisation and application of individual users’ insights into the design of the current interventions. Further research should explore the presence and absence of interplays between elements of human, technology, and organisation and the external environment, and the associated consequences for intervention processes and users’ health outcomes. Our holistic system model, empirically informed by data from a Norwegian context, can represent a starting point for such endeavours. The holistic system model also constitutes an original and novel contribution to research on continuing care interventions.

Keywords

continuing care electronic content analysis holistic system perspectives qualitative study substance use and dependence system modelling

Aligning with this article's focus on continuing care interventions related to substance use and dependence, we elaborate on Norwegian and international studies that document results or outcomes of interventions. By continuing care, we imply the period of lower intensity treatment that follows a more intensive initial treatment within or outside an institution (McKay, 2009, 2021).

Costello et al. (2019), Bergman et al. (2015), Grella et al. (2010) and Sterling et al. (2009) found that participation in 12-step activities/groups in combination with other factors (outpatient support, professional treatment, and continuing care services) produces positive outcomes in the form of reduction in substance use, abstinence, and percentage of days with abstinence (PDA) as well as improvements in mental health (MH) and chemical dependency (CD). In other words, the studies documented positive results of traditional continuing care services, and especially of several or combined interventions. The latter element of several or combined interventions corresponds with the findings of Klein et al. (2012), who revealed a significant relationship between substance use outcomes (lower consumption) and the number of modules accessed in the computerised recovery support program MORE (My Ongoing Recovery Experience) in the year following treatment. Positive results of combined factors involving computer software are also seen in Rose et al. (2015), who found that the fully automated continuing care program ATIVR (Alcohol Therapeutic Interactive Voice Response) contributed to maintaining improvements achieved through other interventions and treatments. Furthermore, McKay et al. (2013) and McKay (2009) found that long-term continuing care interventions provide better results, such as abstinence from substances and heavy alcohol use. The significance of an intervention's duration corresponds with Dahl et al. (2003), who identified that the chance of success increases for younger women and long-term attachment to continuing care. Further supportive of an intervention of longer duration, Nilsen and Mydland (2008) found that the continuing care service “Enter”, comprised of four workshops offering participants work practice (cycle and art shops, kitchen/cafe, and voluntary work), helps to increase participants’ ability to maintain social ties. This is achieved by working purposefully towards creating a safe community where the participants can develop a new identity through the experience of belonging and social mastery. Finally, McKay (2009) and Strunz et al. (2015) revealed a connection between continuing care interventions that are culturally adapted and actively aimed at the user and the likelihood of positive results.

The positive impression above is somewhat in contrast to the results reported by McKay et al. (2011), who found non-significant results of telephone-based interventions after 24 months, though effects had been present during months 1–18. Furthermore, Nesvåg and McKay (2018) identified that digital interventions, delivered via mobile applications or a combination of desktop/web and mobile technology, do not consistently contribute to a reduction of substance use or achievement of other rehabilitation goals such as better health and life situation. These findings contrast with the positive results of both traditional and digital continuing care services described above. This contrast is enhanced by several national studies on continuing care services. Although Bergsgard and Lie (2012) and Dyrstad and Ose (2014) documented a comprehensive municipal continuing care service (with emphasis on interdisciplinary collaboration, variation in user services, etc.) with individual adaptions to different user needs (younger users, active substance users, users in drug-free housing, etc.), they, as well as Actis (2016), described a persistent pressure on the need for municipal housing and 24-hour care housing in combination with deficient allocation of suitable housing by municipalities. Moreover, Dyrstad and Ose (2014) and Actis (2016) identified weaknesses in the municipal services related to offers of low-threshold activity and social activities, work training and work, rehabilitation, and more.

In summary, the studies link positive results of continuing care interventions, such as abstinence and substance use reduction, entering work/school, and a strengthened social ability, to the focus on several or combined interventions actively aimed at the user and the emphasis on long-term and culturally adapted interventions. The studies connect negative results to deficient or absent interventions; for the national studies associated with a lack of scope of offers along the parameters housing, activities, and work, and for the international studies associated with a lack of effect of telephone-based interventions over time. In other words, continuing care initiatives have a complex nature and consist of several factors that can have both positive and negative effects. Highlighting this complexity, McKay (2021) points out that existing research shows smaller or moderate effects of continuing care interventions among young and old, and in particular interventions aimed at patients with a higher risk of relapse and interventions of longer duration and more active design. The continuing care establishment EV (anonymised) explored in this article (see next section) applies interventions of a longer duration and active design.

Besides adding to the existing knowledge on continuing care interventions, at an establishment level, the main contribution of our article lies in the application of a holistic system model to capture the complexity of users’ everyday lives in the continuing care establishment EV (see later theory section). We have not been able to identify the application of a holistic system model in the existing literature on continuing care interventions, making our article an original and novel contribution to the field.

Context and article aim

The interview study and content analysis presented in this article were part of the Norwegian research project “Municipal continuing care at Jæren built on design and effect studies”, funded by the Regional Research Fund (RFF) Western Norway. The project explored the non-profit continuing care establishment EV. Among several project goals, the interview study presented in this article was aimed at mapping users’ perceptions of EV as well as identifying corresponding implications for the development of continuing care interventions. The overall purpose of the project was to explore how continuing care services can be improved from a user perspective.

The continuing care establishment EV is a key partner in an overarching and nationally funded pilot project NSJ (anonymised), which is being implemented by several municipalities. Specifically, NSJ tests a new method for municipal follow-up after treatment at an institution or stay in prison for individuals struggling with substance use and dependence. The new method includes a 12-month continuing care programme from EV, comprising farm work, work training, physical training, group therapy, and socialisation. The 12-month programme is divided into three phases: a three-month “mapping phase” focused on identifying the user's work capacity, challenges, economic situation, resources, and more; a three-month “integration phase” emphasising the user's ability to cope with depression, develop resources and responsibilities, attend physical training, focus on personal goals, and so forth; and a six-month “transition phase” targeting the user's contact with potential employers or schools, working life training, school attendance, and more. The 12-month programme and associated activities seek to strengthen the user's mental health and ability to cope with life, through high-intensive training, meaningful tasks, responsibilities, and close follow-up. The NSJ project places a strong emphasis on achieving a dynamic and close cooperation between the municipalities and EV regarding interventions and follow-up solutions. This cooperation includes central actors involved with substance users, such as the Labour and Welfare Administration (NAV), mental health and addiction units, specialist healthcare services including outpatient treatment, and the Norwegian Correctional Service including prisons.

Further, on the project background and in the overall Norwegian context, many of those who become substance-free at an institution fall back into intoxication after they become residents in the municipalities. Nordfjærn (2011) found that the risk of relapse, understood as an increase in substance use following a more or less substance-free phase or period, is very high in the first months after treatment, and especially among young adults aged 18–25 years. Rambøll (2012) emphasised that weaknesses in the municipalities’ ability to address substance use among young people can be explained by challenges in coordinating and cooperating across municipal agencies, lack of municipal efforts spent on gaining an overview of drug use among young people, and a lack of sufficient municipal resources and capacities to establish quality interventions. The situational picture in Norway when it comes to work on substance use is further characterised by a persistent need for municipal housing and 24-hour care housing, where in particular municipal follow-up regarding the allocation of suitable housing is deficient, which can be considered paradoxical given that housing measures are emphasised by municipalities (Actis, 2016; Bergsgard & Lie, 2012; Dyrstad & Ose, 2014).

By means of an interview study, the aim of this article is to explore systematically and in depth, users’ perceptions of participating in the Norwegian continuing care establishment EV, including from a holistic system perspective (see section next). The interview study and associated analyses have been structured according to the following factors, informed by literature and of interest to the overall research project: user orientation and holistic design of interventions, physical training and coping, source of livelihood/economy, housing and work, meaningful activity including social networks, offers and functioning, and substance use including the degree of abstinence. See the methods section later for more details on the analytical approach selected for this study.

Theoretical perspective: A holistic system model

This article is intended for several reader groups, from researchers and managers to users of continuing care interventions, suggesting a pragmatic theoretical approach. However, the perspective must be able to capture the complex everyday lives that users experience at the establishment level. Thus, a starting point for our theoretical approach is the notion that people inside and outside of everyday work, are part of a larger dynamic system consisting of other people, organisations, technologies, and the external environment (politics, laws, regulations, media, and so forth).

We are now talking about a perspective focused on the specific interplays or interactions that take place between three distinctive and at the same time closely intertwined subsystems: human (H), technology (T), and organisation (O) (Karltun et al., 2017). This is the HTO perspective, which has been applied across a wide number of contexts and sectors, and over time it has developed from a focus on safety to also embracing system performance and health aspects (Berglund & Karltun, 2007; Eklund, 2003; Grote et al., 2000; Karltun, 2011). For example, within health research one talks about how new technology and more functions reduce user-friendliness and make everyday work more complicated for health professionals, which implies that technology is perceived as an obstacle rather than as an aid (Reiman & Oedewald, 2006; Sittig et al., 2005).

The individual subsystems in the HTO perspective are comprehensively elaborated in literature (see, among others, Daniellou, 2001; Mumford, 2006; Porras & Robertson, 1992; Robbins, 1990), but can be explained as follows: the H subsystem consists of human physical, cognitive, psychosocial, and social aspects (which affect interaction); the T subsystem represents the technology itself and includes the use of different types of objects, procedures and methods; while the O subsystem encompasses formal organisation and informal social structures. Karltun et al. (2017) point out that the subsystems T and O include the internal environment, in systems theory terminology, which refers to the physical environment including light, noise, vibration, temperature, available space, air quality, and more factors. However, in relation to the three subsystems, HTO literature does not mention the external environment found in systems theory. Specifically, an external environment incorporates macro-level factors outside an organisation and related to society, economics, and politics (see Carayon et al., 2006; Carayon et al., 2009; Carayon et al., 2014; Holden et al., 2013; Høyland et al., 2019). Given that the external environment represents larger conditions that can affect all aspects of our holistic system model, outlined in Figure 1 below, the external environment is included as an aspect outside of the H, T, and O subsystems as well as intervention processes and health state/outcome (indicated by the dotted square in Figure 1).

Figure 1.

A holistic system model comprising mutually influencing HTO elements/subsystems, intervention processes (user’s work and activities in the continuing care establishment) and resulting health states/outcomes.

Consequently, our holistic system model (Figure 1) incorporates the subsystems H (individual/user), T (technical tools and equipment), and O (continuing care interventions, social structures, internal environment) as well as the external environment surrounding and influencing the subsystems. The thick two-way arrows to the left of the model indicate mutually influencing relationships or interactions between the three subsystems. In turn, the HTO subsystems influence “intervention processes” and “health states/outcomes” elements in the model. Specifically, with regard to the intervention processes in the model, the HTO subsystems can affect the content and quality of activities that the users participate in to achieve given goals such as freedom from substance use and normalisation of everyday life. This is illustrated by the one-directional thick arrow that goes from the individual subsystems to the intervention processes in the middle of the model. The result or outcome of the intervention processes is that users experience a greater or lesser degree of improvement or deterioration in their own health (right-hand part of the model). Moreover, the thin arrows that run from both the intervention processes and health state/outcomes in the model represent feedback loops that can improve the HTO subsystems, such as current organisation (e.g., type and design of activities) and technical tools (e.g., training equipment such as a bike).

Note that the holistic system model, as represented in Figure 1 with regard to the external environment, processes, and outcomes, is inspired by the System Engineering Initiative for Patient Safety (SEIPS) model developed by Carayon et al. (2006, 2009, 2014). The SEIPS model has a practice-oriented nature in that it seeks a deeper understanding of the “sharp-end” of an organisation (Carayon et al., 2015), making the model suited to exploring different types of work complexity (see Høyland et al., 2019). As stated in the introduction section, our holistic system approach and model uniquely contributes to research on continuing care interventions, including through the ability to capture the complexity of users’ everyday lives at a continuing care establishment level.

Methods

We conducted a total of 17 semi-structured in-depth interviews face to face with participants (users) in the Norwegian non-profit continuing care establishment EV. Audio recording was used during the interviews, with subsequent full transcription and anonymisation of all sensitive information. An interview guide was used centred on mapping users’ backgrounds as well as perceptions of participating in the continuing care establishment EV (study aim). The guide included both open-ended and specific questions derived from a systematic literature review of national and international studies on the effects of continuing care interventions (see Høyland et al., 2020). Questions included: “Have you participated in other continuing care interventions, and how were these compared to EV?”; “How is the collaboration between EV, the municipality, and you?”; “How does EV improve your health and ability to stay away from substance use”; “What is positive and negative about EV?”. The interview guide was tested in two pilot interviews, resulting in minor adjustments. The interviews were mainly conducted at EV, at the participants’ request.

We applied a systematic content analysis of the interviews, aimed at identifying patterns and underlying themes in the qualitative data material (Høyland, 2018; Høyland et al., 2019; Høyland et al., 2018; Leech & Onwuegbuzie, 2007). Specifically, the empirical material was analysed using the computer-assisted qualitative data analysis software (CAQDAS) QSR NVivo 11 (QSR International, Melbourne, Australia). CAQDAS helps researchers to operate systematically at the research design level, including the analysis process, strengthening reliability (De Ruyter & Scholl, 1998). The improvements in systematics include dynamic and instant access to various components of the data analysis, such as nodes (data codes) and sources (transcribed interviews) in QSR NVivo's terminology, as well as an increased overview and ability to identify patterns and connections in the data (Talanquer, 2014). CAQDAS also facilitates “creative management of multiple data sources and enables researchers to make visible their methodological processes for a more ‘trustworthy’ study” (Ryan, 2009, p. 159). The implication being that researchers should strive to provide a detailed account of steps in the analysis process (Kapoulas & Mitic, 2012), as we outline next in relation to this interview study.

The content analysis was structured according to the following parameters, informed by literature and of interest to the overall research project: user orientation and holistic design of interventions (K1); physical training and coping (K2); source of livelihood/economy, housing and work (K3); meaningful activity including social networks, offers and functioning (K4); and substance use including the degree of abstinence (K5). These parameters represent coding keys (K1–K5) in QSR NVivo, where meaning units/nodes in the transcribed interviews are identified and organised according to these keys (see Tables 1 to 5 later). A meaning unit/node conveys our (the researchers’) condensed understanding of one or more text segments in the interview transcription that address the users’ perceptions of the continuing care establishment EV.

Table 1.

Results of coding process; N1–NX = nodes, AN1–ANX = aggregated nodes. Sources refer to the number of interviews that support nodes and theme; references refer to the number of identified text segments.

Theme 1: EV has developed a clear holistic and dedicated user orientation (K1, K3, K5)
Nodes	Descriptions	Sources	References
AN1	EV is a community of training, work, and socialisation that promotes the individual's importance and follow-up	3	3
AN2	EV captures and flexibly facilitates changes in the user's needs such as goals and motivation	2	2
AN4	EV combines both a home and a job, characterised by a culture of help and informal but clear goals set through conversations and adapted to the user	2	3
AN5	EV users help and motivate others in the same situation and create a non-judgmental and constructive unit	2	2
AN6	EV offers dedicated and continuous follow-up as well as an apartment and help with finances, networking, and work	5	7
AN11	EV contributes with training and work that keeps the user active and focused away from substance use, which in turn provides opportunities for later schooling	2	2
AN13	EV's contact or collaboration with working life has resulted in a job for the user as well as contact with his own family	2	2
N13	EV's focus on creating job opportunities, the training regime, and contact with one's own family promotes freedom from substance use	1	1

Table 2.

Theme 2: Strengthening external contact and marketing of EV as well as formalising goals (K1–K3)
Nodes	Descriptions	Sources	References
N1	EV should strengthen marketing so that awareness of the continuing care establishment increases	1	1
N2	EV should strengthen the user's awareness of goals, for example by setting goals and sub-goals in writing	1	1
N4	When the EV cycling training goal is reached, the meaning of the training is reduced, and a vulnerable vacuum arises	1	1
AN12	EV lacks a strong awareness of and direct link to the labour market that is of concern to practice training and avoiding the feeling of “emptiness” or void caused by breaks in activities and goals	2	2
N7	EV needs a stronger focus on the user's plan after reaching goals, such as getting involved in charity and jobs	1	1

Table 3.

Results of coding process; AN1–ANX = aggregated nodes. Sources refer to the number of interviews that support nodes and theme; references refer to the number of identified text segments.

Theme 3: Safe and stable EV conditions promote self-confidence and freedom from substance use (K4)
Nodes	Description	Sources	References
AN7	EV's focus on training, responsibility, fresh air, animals, and people who see you, is suitable for people with substance addiction, anxiety, routine problems, etc.	4	4
AN15	EV is a community that promotes self-insight, contact rather than escape from emotions, and where you share common goals and help each other	5	6
AN16	EV is about helping, being seen, showing understanding and patience, and building self-confidence, self-insight, and security	7	10

Table 4.

Theme 4: Substance-free network, economy, family, and self-confidence needed to stay clean (K3, K5)
Nodes	Description	Sources	References
N8	External physical conditions such as work, apartment, and economy are subordinate to the desire for stronger self-confidence and avoiding falling back into substance use	1	1
AN13	EV's contact or collaboration with working life has resulted in a job for the user as well as contact with his own family	2	2
N11	EV provides meaning and security at the same time as one must have economy and a substance-free network including family outside EV	1	1

Table 5.

Theme 5: The importance of physical training and mastery – especially on a mental level (K2)
Nodes	Description	Sources	References
N3	EV cycling training is mentally demanding but strengthening to quality of life such as sleep, food intake, and profits	1	1
AN8	EV cycling training motivates through goals, visible progress, and a professional support system as well as providing a sense of intoxication	3	4
AN9	EV training through cycling and running breaks with the comfort zone and fills an internal void as well as meeting the need to spend energy	2	3
AN10	EV training provides not only physical but mental mastery including strengthened self-esteem, a feeling of well-being, anxiety relief, and improved socialisation	9	11

As for the specific analytical steps, the first author performed a systematic content analysis of the transcribed interview material (17 interviews, 268 A4 pages) using QSR NVivo and a split-screen 43-inch monitor setup. In the first step, the 17 transcribed interviews (in .docx format) were imported into QSR NVivo and marked “document sources” by the software. In the second step, the first author opened one source at a time and used the “undock all” function to organise the source content (the transcription/text itself) on the right-hand part of the screen, while the “nodes” overview in NVivo was placed on the left-hand side of the screen. By organising the work area and desktop in this way, the researcher achieves a detailed and instantly accessible overview of both the transcribed interview (document source) and the individual nodes, throughout the entire coding process. In the third step, the first author read each individual source document with the aim of identifying and summarising via nodes text segments that convey the user's perceptions of the continuing care establishment EV on the selected parameters/coding keys K1–K5. This produced several nodes (N1–N14), aggregated nodes (AN1–AN20) representing opinions summarised from the nodes (N1–N14), and overall themes/main patterns (Themes 1–5) summarising opinions from all nodes including the aggregated ones. Having conducted all the interviews, the second author reviewed the coding results including summaries, and concluded that the coding resonated with her own impression of the interviews. The third author also reviewed the coding results, including summaries, which increased awareness of nuances in how to present the user's rich insight into positive and negative aspects of the continuing care establishment EV, including through quotations. By reviewing the coding results across several researchers, i.e., analytical triangulation, we improved the validity of the coding process (Denzin, 1978; Patton, 1990). The review also facilitated consensus between the researchers/authors, by resolving minor nuances and differences in the understanding of the coding process (Bradley et al., 2007; Hruschka et al., 2004). The outcome of the coding process is presented in the results section below including Tables 1 to 5.

Ethical aspects of the research project

Substance users are a vulnerable group. Thus, we informed participants thoroughly about the research project, sought explicit project and study consent, and ensured that interview data were handled and stored securely as well as being anonymised. However, ethics is also about maintaining the highest standards of research, sincere and credible reporting of procedures and results as well as crediting co-authors and other contributors such as funding bodies and study participants (Comitas, 2000; Hobbs, 1968; Van Deventer, 2009). In this article, these aspects are covered through acknowledgments of key actors and contributors, accurate methodological descriptions of the interview study, a detailed and transparent presentation of results from the interview study, and critical reflections on the strengths and limitations of the interview study (see discussion section). The research project, focused on exploring how continuing care services can be improved from a user perspective, was approved by the Norwegian Centre for Research Data (NSD) in 2018 (reference 61516/BGH/LR).

Results

In this section, we present results from a systematic content analysis of in-depth interviews with 17 participants or users in the continuing care establishment EV. The results are organised in tabular form to strengthen the reader's overview. We identify and describe a total of five themes/main patterns (Themes 1–5), with supporting quotations, and the themes are connected to the coding keys (K1–K5) as described in the methods section.

Table 1 suggests that the continuing care establishment EV has developed a clear holistic and dedicated user orientation with emphasis on facilitating farm work, physical training, socialisation, individual goals and needs, and contact with working life (Theme 1). More specifically, respondents describe that EV captures and flexibly facilitates changes in the user's needs, such as goals and motivation. The degree of individual adaptation is found in several respondents’ impressions that EV offers dedicated and continuous follow-up as well as apartments and help with finances, networking, and work. Furthermore, the respondents express that EV creates a community of training, work, and socialisation that promotes the individual's importance and follow-up. The experience of community is also made visible in respondents’ descriptions of EV combining both a home and a job, with a help culture and informal but clear goals set through conversations and adapted to the user as well as in descriptions that EV users help and motivate others in the same situation and create a non-judgmental and constructive unit. The following respondent quotations are supportive of Theme 1:

It is the community […] These are people who really care. People who believe in you. And treat you like a proper person and not just like a drug addict or… Because you notice quickly… it is hard to describe… the love then. (Interview/Respondent #12)

You can live at home in your own apartment, or you can live here [at EV] in an apartment while you get to train on the other things. They help you to get your finances in place, get yourself a network, get yourself a meaningful job when you are leaving EV. This makes this transition [to society] a little less steep. (Interview/Respondent #3)

[EV] sees you. If it is not going great for a couple of weeks, they will see it and ask you about it. If there is something they can do, they will help you with it. If there is something you want to do that you like, they are there to support you or talk about it. (Interview/Respondent #6)

Table 2 shows that respondents call for a stronger focus on and contact with external actors and working life, marketing, raising awareness of EV, and formalising clearer goals (Theme 2). Specifically, one respondent describes that the EV establishment should strengthen marketing so that awareness of EV increases. Several respondents express that EV should strengthen the user's goal awareness, for example by formulating goals and sub-goals in writing. Similarly, several respondents express that EV lacks a strong awareness of and direct link to working life, in terms of practical training and avoiding users feeling an “emptiness” due to breaks in activities and goals. Specifically, and related to the physical training regime, one respondent expresses that when the goal of the cycling training is reached, the meaning of the training is reduced, which in turn creates a vulnerable vacuum for the user. Furthermore, on goal awareness at EV, one respondent describes that EV needs a stronger focus on the user's plan after achieving goals, such as getting involved in charity and jobs. Several respondent quotations support Theme 2:

Writing it down is important to me. Writing down such goals and sub-goals. Why I am here. Yes, I will have a job, I will train and aim to achieve it. When I came into treatment, I had such goals. You fall a little out of the path, so then it can be good to know that you have done the things you are supposed to, you also need this and that to get the job. Then you always have something to look forward to. A carrot. (Interview/Respondent #7)

One thing that would have been good is a closer dialogue with the labour market. That there is a link right out there. That you have a clearer plan for the program here. That not only the goal is to cycle Trondheim-Oslo. Yes, what is afterward then? What then? That there must be no breaks in between things, because … leisure is the worst thing there is. A weekend … when I have a long weekend I start to “jump the walls”. (Interview/Respondent #1)

We have to market ourselves in a better way and there has to be … something has to happen there. Because it is too good an offer for people not to use it, in my opinion. And of course, we can get better at absolutely everything. (Interview/Respondent #3)

The socialisation element, described in relation to Theme 1, is also made visible by the fact that the continuing care establishment EV is based on social cohesion and humanity (a see-and-be-seen culture), which, in combination with a “healthy approach” (drug-free environment, focus on training and work), creates a safe and stable environment with activities and coping opportunities; a combination that promotes self-confidence and freedom from substance use (Theme 3). Illustrative of this, many respondents describe that EV is about helping, being seen, showing understanding and patience, and building self-confidence, self-insight, and security. The importance of being seen and creating health-promoting interventions is also evident in the respondents’ impression that EV focuses on exercise, responsibility, fresh air, animals, and people who see you, and is suitable for people with substance use addiction, anxiety, routine problems, and more. The connection between a supportive culture and self-confidence in Theme 3 also appears in the respondents’ view that EV is a community that promotes self-insight, contact rather than escape from emotions, and where one shares common goals and helps one another. The following respondent quotations are supportive of Theme 3:

I think [EV is] especially for those who have bad nerves. It does not matter if it is because of the addiction or if it is something they already have. For people who have anxiety in general, this is a wonderful place because you have fresh air, animals, and people around you who are wonderful and good at talking to you. Yes, so people who need to learn routines again. It is really for everyone. (Interview/Respondent #6)

[EV is about] getting some responsibility … getting in some routines then. You get up at a certain time and you do something sensible during the day. Not least the training. The training is one of the most important things for me. (Interview/Respondent #11)

[EV] was a good place to find yourself again. And be with people who are struggling similarly, that we help each other. At least when you are used to just substance users around you, [i.e.,] your friends and acquaintances. To get here where everyone shares the same goal. So, it was good for me during that period. (Interview/Respondent #2)

Regarding conditions that can facilitate freedom from substance use, as identified in Theme 3, respondents note that a drug-free network, economy, and a family outside EV are required for “staying clean” (Theme 4). Specifically, although EV contributes in several ways to staying substance free, such as providing security, stability, and coping opportunities including physical training as well as a focus on creating job opportunities, one respondent states that economy and a substance-free network including family outside the EV establishment is necessary to stay free from substance use. At the same time, this respondent perceived external physical conditions, such as work, housing, and economy, as being less important than the desire to build a stronger self-confidence and avoiding falling back into substance use. The significance of family contact also resonates with respondents’ descriptions that contact or collaboration with working life, facilitated by EV, has provided the user a job as well as closer contact with his own family. Several respondent quotations support Theme 4:

I want to be more confident in myself. Better self-esteem, better self-image. I want my life to be stable, without drugs. Because there are not so many things I need, such as job, economy, apartment. Such external things. I need to take away those relapses into substance use. That is what is ruining my life. (Interview/Respondent #3)

[EV] has given me a meaningful everyday life. I think that is alpha and omega if you are going to do something. You must have a safe place to live. You must have an economy so that you can … that you can live with. You must have a network and you must have a meaningful everyday life and a meaningful leisure time. (Interview/Respondent #3)

Had it not been for them [EV], I probably would not have had contact with my daughter or anything. Now my daughter visits me whenever she wants to. (Interview/Respondent #10)

Table 5 highlights the importance of physical training and mastery, especially mentally, by (i) breaking with the usual pattern/comfort zone and filling a void; (ii) creating an addictive feeling of well-being, anxiety reduction, and increased socialisation; and (iii) strengthening the quality of life such as sleep, food intake, and energy levels (Theme 5). Many respondents express that physical training at EV facilitates both physical and mental mastery, including strengthened self-esteem, a feeling of well-being, anxiety relief, and improved socialisation. The interplay between physical and mental training is reflected in a respondent's view of cycling training as mentally demanding but strengthening to quality of life such as sleep, food intake, and energy levels. Furthermore, respondents experience the need to satisfy a goal or a rush. For example, some respondents experience EV cycling training as motivating through goals, visible progress, and a feeling of intoxication, while others describe that cycling and running are useful in breaking out of the comfort zone and in filling an internal void as well as meeting the need to spend energy. The following respondent quotations are supportive of Theme 5:

I have probably never run a race in my whole life, and I train to ride a downhill race next year. I cannot fully explain what it is, but it is about getting out of my comfort zone. I did that. There are no days off when you need to get hold of drugs. There is no Sunday rest or Friday cosiness. You kind of have to sort it out yourself. (Interview/Respondent #4)

I have cycled since … like always. When I was in prison, I had a bicycle wheel that I sat and spun on. I kind of cycled to build up energy … so it has been a tool that I have always used. It is super ingenious. So, taking it to the next level [at EV] and get that support and get your own bike and get all that equipment and help set goals and cycle Trondheim-Oslo, i.e., all these big things. It was just right. (Interview/Respondent #12)

Exercising is underestimated in so many ways … If someone is severely depressed, if you get them to exercise, then I am 100% sure that their quality of life will be better. Substance users, psychiatry … it helps against everything. You sleep better, you eat better, you feel better, you get extra energy. I cannot think of anything negative about exercising. Nothing. Of course, it is not always fun, in bad weather and so forth. But I have never returned [from exercising] and thought that I should never have done this. (Interview/Respondent #3)

Discussion

In this section, we connect our results to the holistic system perspective and model introduced in the theory section. As illustrated in Figure 2, interplays take place between the HTO subsystems and the external environment, which in turn affects the users’ ability to take part in and carry out activities (intervention processes) at the continuing care establishment, aimed at the overall goal of achieving a substance-free and normalised everyday life or situation (health outcomes) (Carayon et al., 2015; Carayon et al., 2006; Carayon et al., 2009; Carayon et al., 2014; Karltun et al., 2017). Our results indicate that there are multiple positive conditions of the HTO subsystems that contribute to reduced substance use and abstinence. This includes conditions identified in existing literature, such as several or combined interventions, user-oriented design, and interventions actively aimed at the user (see the introduction section). Specifically, we find that a continuing care establishment with a clear holistic and user-oriented profile comprising combined interventions including physical (farm work, training) and social (a community, socialisation) activities (Subsystems O and T), can create a safe and stable environment that exerts a positive mental and physical influence on the user (Subsystem H) facilitative of abstinence from substances (health outcome).

Figure 2.

A holistic system model that incorporates insights from the interview study.

However, our results suggest that the internal environment needs to connect more strongly with the external environment, such as a substance-free network, close family, and working life, which resonates with the inadequate scope of intervention offers identified in existing literature (see the introduction section). This introduces the feedback loops of the holistic system model (Figure 2), aimed at improving the state of and interaction between human and organisation subsystems as well as the external environment. Illustrative of this, at a continuing care establishment level, the impression of a clear holistic and user-oriented profile at EV can be improved further by raising awareness of EV's existence including through external actors and working life collaboration. One way of achieving this collaboration is to incorporate elements of charity or voluntarily work in current activities (Subsystem O), or as a separate activity, to form a stronger tie to working life (External environment). The working life connection is of particular concern to work practice training and avoiding the feeling of “emptiness” or vacuum caused by breaks in activities and goals (Subsystem H). Interview respondents suggested that the vacuum can be counteracted to a certain degree by maintaining training exercises such as cycling (Subsystem T). A larger counteracting effect on the perception of vacuum can likely be achieved by combining activities involving charity or voluntary work with other activities such as physical training exercises. Thus, we recommend that interacting with working life should become an explicit goal in the user's plan at the continuing care establishment and be integrated into existing interventions.

Overall, becoming aware of, systemising, and incorporating insights into users’ positive as well as negative experiences at the continuing care establishment level, can be a key to improving the current design of HTO subsystems and the associated intervention processes and health outcomes. In our case (EV), this was exemplified by the working life connection (external environment) being included in the current organisation of activities (Subsystem O), thereby filling the user's perceived goal vacuum (Subsystem H), somewhat compensated for by the current exercises/cycle training (Subsystem T). An approach of actively systemising and utilising users’ insights into the design of interventions resonates with McKay (2021), who argues that high-quality guidelines for continuing care need to account for information on adapting continuing care over time at the individual level to achieve optimal outcomes. Further supportive of systematically engaging users in intervention design, Lenaerts et al. (2014) reviewed studies of continuing care interventions for patients with alcohol use disorders, and observed better outcomes associated with interventions actively involving the patient.

Strengths and limitations of the interview study

Qualitative studies contribute with unique in-depth insight, which considers that the world is socially constructed and consists of subjective experiences and understandings of reality that change over time and social contexts. In this complexity, it is the researcher's task to describe, interpret, and understand the meanings that humans attribute to their existence and the world, to deepen and develop our understanding of the complex nature of continuing care interventions (Cutcliffe & McKenna, 1999; Dew, 2007; Wadel, 1990). This complexity is seen in users’ perceptions of how participating in the continuing care establishment EV affects their own substance use, life situation, and overall needs. The detail richness of qualitative studies complements the “positivist” focus in quantitative studies on number-based categorisations and effect measurements (Miles & Huberman, 1994). Moreover, a strength of our interview study lies in the application of a systematic methodological approach, comprising analytical triangulation and validation through coding and consensus-seeking across multiple researchers (Bradley et al., 2007; Denzin, 1978; Hruschka et al., 2004; Patton, 1990).

A classic limitation of qualitative research relates to the extrapolation and generalisation of findings (Erlandson et al., 1993; Lincoln & Guba, 1985). Generalisability implies that a researcher's observational notes or interview data in one case, for a given context, time, and people, can also be found in other cases, i.e., across contexts, times, and people, which appears problematic given the socially constructed and complex reality. However, limited generalisability can be achieved in qualitative research data, in the sense that the deep and in-depth nature of qualitative research can reveal higher-level concepts and theories with potential applicability in other contexts (Glaser, 2002; Misco, 2007; Morse, 2004), such as our holistic system model informed by empirical insights/themes from the interview study (Figure 2). Thus, our model may be explored and developed further in other continuing care intervention contexts, especially if the context shares similarities with the Norwegian setting (see the context and article aim section).

Conclusion

We conclude by outlining the main implications of our study for the development of user-oriented continuing care interventions. First, at the establishment level, there is a need to develop an explicit strategy and practice for collaborating with the external environment, built on systemisation and application of individual users’ insights into the design of the current interventions. Second, from our original and novel holistic system perspective, further national and international research should explore and map the presence and absence of interplays between the three (HTO) subsystems and the external environment, and the associated consequences for intervention processes and health states/outcomes. Our holistic system model, empirically informed by data from a Norwegian context, can represent a starting point for such endeavours.

Footnotes

Acknowledgments

We would like to thank the study participants from the continuing care establishment EV for their valuable contributions. Furthermore, we would like to thank our project partners the municipalities of Klepp, Hå, Time, and Gjesdal, the design agency Inventas AS, and the establishment EV, in addition to the project's advisory group who provided input on our research. The study was funded by the Regional Research Fund Western Norway. The funding source has not been involved in any phases of the research.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Regional Research Fund Western Norway

ORCID iD

Sindre Aske Høyland

References

Actis. (2016). Erfaringsundersøkelse blant ansatte i rusbehandlingen [Experience survey among employees in substance use treatment] (note 2:2016). Actis.

Berglund

Karltun

(2007). Human, technological and organizational aspects influencing the production scheduling process. International Journal of Production Economics, 110(1–2), 160–174. https://doi.org/10.1016/j.ijpe.2007.02.024

Bergman

B. G.

Hoeppner

B. B.

Nelson

L. M.

Slaymaker

Kelly

J. F.

(2015). The effects of continuing care on emerging adult outcomes following residential addiction treatment. Drug and Alcohol Dependence, 153, 207–214. https://doi.org/10.1016/j.drugalcdep.2015.05.017

Bergsgard

N. A.,

Lie

(2012). Evaluering av tilbudet til rusavhengige i Hamar [Evaluation of offers to substance users in Hamar]. (International Research Institute of Stavanger AS (IRIS) report 2012/296.).

Bradley

E. H.

Curry

L. A.

Devers

K. J.

(2007). Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Services Research, 42(4), 1758–1772. https://doi.org/10.1111/j.1475-6773.2006.00684.x

Carayon

Hancock

Leveson

Noy

Sznelwar

van Hootegem

(2015). Advancing a sociotechnical systems approach to workplace safety: Developing the conceptual framework. Ergonomics, 58(4), 548–564. https://doi.org/10.1080/00140139.2015.1015623

Carayon

Hundt

A. S.

Karsh

B. T.

Gurses

A. P.

Alvarado

C. J.

Smith

Brennan

P. F.

(2006). Work system design for patient safety: The SEIPS model. Quality in Health Care, 15(1), i50–i58. https://doi.org/10.1136/qshc.2005.015842

Carayon

Smith

Hundt

A. S.

Kuruchittham

(2009). Implementation of an electronic health records system in a small clinic: The viewpoint of clinic staff. Behaviour & Information Technology, 28(1), 5–20. https://doi.org/10.1080/01449290701628178

Carayon

Wetterneck

T. B.

Rivera-Rodriguez

A. J.

Hundt

A. S.

Hoonakker

Holden

Gurses

A. P.

(2014). Human factors systems approach to healthcare quality and patient safety. Applied Ergonomics, 45(1), 14–25. https://doi.org/10.1016/j.apergo.2013.04.023

10.

Comitas

(2000). Ethics in anthropology: Dilemmas and conundrums. Annals New York Academy of Sciences, 925(1), 196–210. https://doi.org/10.1111/j.1749-6632.2000.tb05591.x

11.

Costello

M. J.

Remers

MacKillop

Sousa

Ropp

Roth

Weiss

Rush

(2019). Effects of 12-step mutual support and professional outpatient services on short-term substance use outcomes among adults who received inpatient treatment. Addictive Behaviors, 98, Article 106055. https://doi.org/10.1016/j.addbeh.2019.106055

12.

Cutcliffe

J. R.

McKenna

H. P.

(1999). Establishing the credibility of qualitative research findings: The plot thickens. Journal of Advanced Nursing, 30(2), 374–380. https://doi.org/10.1046/j.1365-2648.1999.01090.x

13.

Dahl

Johansen

G. H. L.

Ramsdal

(2003). Evaluering av ettervernet i Fredrikstad: Faglig rapport [Evaluation of continuing care in Fredrikstad: Professional report]. (Report 2003:3). Østfold University College.

14.

Daniellou

(2001). Epistemological issues about ergonomics and human factors. In Karwowski

(Ed.), International encyclopaedia of ergonomics and human factors, part 1 (pp. 43–46). Taylor & Francis.

15.

De Ruyter

Scholl

(1998). Positioning qualitative market research: Reflections from theory and practice. Qualitative Market Research, 1(1), 7–14. https://doi.org/10.1108/13522759810197550

16.

Denzin

N. K.

(1978). Sociological methods. McGraw-Hill.

17.

Dew

(2007). A health researcher’s guide to qualitative methodologies. Australian and New Zealand Journal of Public Health, 31(5), 433–437. https://doi.org/10.1111/j.1753-6405.2007.00114.x

18.

Dyrstad

Ose

S. O.

(2014). Kommunalt rusarbeid og innlemming av statlig tilskudd i den kommunale rammen [Work on municipal substance use and incorporation of public subsidy within the municipal frame]. SINTEF.

19.

Eklund

(2003). An extended framework for humans, technology and organization in interaction. In Luczak

Zink

K. J.

(Eds.), Human factors in organizational design and management – VII. Re-designing work and macroergonomics: Future perspectives and challenges (pp. 47–54). IEA Press.

20.

Erlandson

D. A.

Harris

E. L.

Skipper

B. L.

Allan

S. D.

(1993). Doing naturalistic inquiry: A guide to methods. Sage.

21.

Glaser

B. G.

(2002). Conceptualization: On theory and theorizing using grounded theory. International Journal of Qualitative Methods, 1(2), 23–38. https://doi.org/10.1177/160940690200100203

22.

Grella

C. E.

Stein

J. A.

Weisner

Chi

Moos

(2010). Predictors of longitudinal substance use and mental health outcomes for patients in two integrated service delivery systems. Drug and Alcohol Dependence, 110(1–2), 92–100. https://doi.org/10.1016/j.drugalcdep.2010.02.013

23.

Grote

Ryser

Wäfler

Windischer

Weik

(2000). KOMPASS: A method for complementary function allocation in automated work systems. International Journal of Human-Computer Studies, 52(2), 267–287. https://doi.org/10.1006/ijhc.1999.0289

24.

Hobbs

(1968). Ethics: Ethical issues in the social sciences. In Sills

D. L.

(Ed.), International encyclopedia of the social sciences (Vol. 5). The Macmillan Company & The Free Press.

25.

Holden

R. J.

Carayon

Gurses

A. P.

Hoonakker

Hundt

A. S.

Ozok

A. A.

Rivera-Rodriguez

A. J.

(2013). SEIPS 2.0: A human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics, 56(11), 1669–1686. https://doi.org/10.1080/00140139.2013.838643

26.

Høyland

S. A.

(2018). Exploring and modeling the societal safety and societal security concepts: A systematic review, empirical study and key implications. Safety Science, 110, 7–22. https://doi.org/10.1016/j.ssci.2017.10.019

27.

Høyland

S. A.

Gressgård

L. J.

Hansen

Holte

K. A.

(2019). Exploring multiple working arrangements in Norwegian engineering, procurement, and construction industry from a middle manager and supervisor perspective: A socio-technical system perspective. Applied Ergonomics, 76, 73–81. https://doi.org/10.1016/j.apergo.2018.12.005

28.

Høyland

S. A.

Skotnes

R. Ø.

Holte

K. A.

(2018). An empirical exploration of the presence of HRO safety principles across the health care sector and construction industry in Norway. Safety Science, 107, 161–172. https://doi.org/10.1016/j.ssci.2017.07.003

29.

Høyland, S.A., Shuchert, A., Mamen, A. (2020). Nasjonale og internasjonale effekter av ettervern innen rusbruk og avhengighet – Forskningsresultater og implikasjoner for et helhetlig og brukerorientert kommunalt ettervernstilbud. NORCE Norwegian Research Centre AS report no. 22 [in Norwegian].

30.

Hruschka

D. J.

Schwartz

St John

D. C.

Picone-Decaro

Jenkins

R. A.

Carey

J. W.

(2004). Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods, 16(3), 307–331. https://doi.org/10.1177/1525822X04266540

31.

Kapoulas

Mitic

(2012). Understanding challenges of qualitative research: Rhetorical issues and reality traps. Qualitative Market Research, 15(4), 354–368. https://doi.org/10.1108/13522751211257051

32.

Karltun, A. (2011). Developing HTO systems thinking for organizational and technological change. In: Proceedings of 10th International Symposium on Human Factors in Organizational Design and Management. Grahamstown, South Africa, April 4-6, 2011.

33.

Karltun

Berglund

Eklund

(2017). HTO: A complementary ergonomics approach. Applied Ergonomics, 59(Part A), 182–190. https://doi.org/10.1016/j.apergo.2016.08.024

34.

Klein

A. A.

Slaymaker

V. J.

Dugosh

K. L.

McKay

J. R.

(2012). Computerized continuing care support for alcohol and drug dependence: A preliminary analysis of usage and outcomes. Journal of Substance Abuse Treatment, 42(1), 25–34. https://doi.org/10.1016/j.jsat.2011.07.002

35.

Leech

N. L.

Onwuegbuzie

A. J.

(2007). An array of qualitative data analysis tools: A call for data analysis triangulation. School Psychology Quarterly, 22(4), 557–584. https://doi.org/10.1037/1045-3830.22.4.557

36.

Lenaerts

Mathei

Matthys

Zeeuws

Pas

Anderson

Aertgeerts

(2014). Continuing care for patients with alcohol use disorders: A systematic review. Drug & Alcohol Dependence, 135, 9–21. https://doi.org/10.1016/j.drugalcdep.2013.10.030

37.

Lincoln

Y. S.

Guba

E. G.

(1985). Naturalistic inquiry. Sage Publications.

38.

McKay

J. R.

(2009). Continuing care research: What we have learned and where we are going. Journal of Substance Abuse Treatment, 36(2), 131–145. https://doi.org/10.1016/j.jsat.2008.10.004

39.

McKay

J. R.

(2021). Impact of continuing care on recovery from substance use disorder. Alcohol Research: Current Reviews, 41(1), Article 01. https://doi.org/10.35946/arcr.v41.1.01

40.

McKay

J. R.

Van Horn

D. H.

Lynch

K. G.

Ivey

Cary

M. S.

Drapkin

M. L.

Coviello

D. M.

Plebani

J. G.

(2013). An adaptive approach for identifying cocaine dependent patients who benefit from extended continuing care. Journal of Consulting and Clinical Psychology, 81(6), 1063–1073. https://doi.org/10.1037/a0034265

41.

McKay

J. R.

Van Horn

Oslin

D. W.

Ivey

Drapkin

M. L.

Coviello

D. M.

Lynch

K. G.

(2011). Extended telephone-based continuing care for alcohol dependence: 24-month outcomes and subgroup analyses. Addiction, 106(10), 1760–1769. https://doi.org/10.1111/j.1360-0443.2011.03483.x

42.

Miles

M. B.

Huberman

A. M.

(1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Sage Publications.

43.

Misco

(2007). The frustrations of reader generalizability and grounded theory: Alternative considerations for transferability. Journal of Research Practice, 3(1), 1–11.

44.

Morse

J. M.

(2004). Constructing qualitatively derived theory: Concept construction and concept typologies. Qualitative Health Research, 14(10), 1387–1395. https://doi.org/10.1177/1049732304269676

45.

Mumford

(2006). The story of socio-technical design: Reflections on its successes, failures and potential. Information Systems Journal, 16(4), 317–342. https://doi.org/10.1111/j.1365-2575.2006.00221.x

46.

Nesvåg

McKay

J. R.

(2018). Feasibility and effects of digital interventions to support people in recovery from substance use disorders: Systematic review. Journal of Medical Internet Research, 20(8), e255. https://doi.org/10.2196/jmir.9873

47.

Nilsen

A. C.

Mydland

T. S.

(2008). Evaluering av prosjektet Enter. Et ettervernstilbud ved Kirkens Bymisjon Kristiansand [Evaluation of the project Enter. A continuing care offering by The Church City Mission] (Project report nr. 1). Agderforskning.

48.

Nordfjærn

(2011). Risikofaktorer ved tilbakefall etter rusbehandling [Relapse risk factors after substance use treatment]. Rusfag, 3(1), 5–11.

49.

Patton

M. Q.

(1990). Qualitative evaluation and research methods. Sage Publications.

50.

Porras

J. I.

Robertson

P. J.

(1992). Organizational development: Theory, practice, and research. In Dunette

M. D.

Hough

L. M.

(Eds.), Handbook of industrial and organizational psychology (pp. 719–822). Consulting Psychologist Press Inc.

51.

Rambøll. (2012). Evaluering av tilskudd til kommunalt rusarbeid [Evaluation of public subsidy to municipal work on substance use]. Rambøll Management Consulting, report to the Norwegian Directorate of Health, July 2012. International Research Institute of Stavanger AS (IRIS) report 2012/296.

52.

Reiman

Oedewald

(2006). Organizational culture and social construction of safety in industrial organizations. In Svenson

Salo

Skjerve

A. B.

Reiman

Oedewald

(Eds.), Nordic perspectives on safety management in high reliability organizations: Theory and applications (pp. 115–130). NKS Secretariat.

53.

Robbins

S. P.

(1990). Organization theory, structure, design and applications (3rd ed). Prentice Hall.

54.

Rose

G. L.

Skelly

J. M.

Badger

G. J.

Ferraro

T. A.

Helzer

J. E.

(2015). Efficacy of automated telephone continuing care following outpatient therapy for alcohol dependence. Addictive Behaviors, 41, 223–231. https://doi.org/10.1016/j.addbeh.2014.10.022

55.

Ryan

M. E.

(2009). Making visible the coding process: Using qualitative data software in a post-structural study. Issues in Educational Research, 19(2), 142–161.

56.

Sittig

D. F.

Krall

Kaalaas-Sittig

Ash

J. S.

(2005). Emotional aspects of computer-based provider order entry: A qualitative study. Journal of the American Medical Informatics Association, 12(5), 561–567. https://doi.org/10.1197/jamia.M1711

57.

Sterling

Chi

Campbell

Weisner

(2009). Three-year chemical dependency and mental health treatment outcomes among adolescents: The role of continuing care. Alcoholism: Clinical and Experimental Research, 33(8), 1417–1429. https://doi.org/10.1111/j.1530-0277.2009.00972.x

58.

Strunz

Jungerman

Kinyua

Frew

(2015). Evaluation of an assertive continuing care program for Hispanic adolescents. Global Journal of Health Science, 7(5), 106–116. https://doi.org/10.5539/gjhs.v7n5p106

59.

Talanquer

(2014). Using qualitative analysis software to facilitate qualitative data analysis. In Bunce

Cole

(Eds.), Tools of chemistry education research. ACS Symposium Series (pp. 83–95). ACS.

60.

Van Deventer

J. P.

(2009). Ethical considerations during human centred overt and covert research. Quality & Quantity, 43(1), 45–57. https://doi.org/10.1007/s11135-006-9069-8

61.

Wadel

(1990). Den samfunnsvitenskapelige konstruksjonen av virkeligheten [The social construction of reality]. SEEK A/S.