Sage Journals: Discover world-class research

Abstract

Objective

Depression is a major challenge for many people and societies, also accelerated by the COVID-19 pandemic. Around 300 million people worldwide are affected by a depressive disorder. Health insurances encourage patients to use applications (“apps”) to tackle, for example, mild and moderate depression. The advantages of these depression apps are well understood, but their dissemination and adoption rates among intended users leave room for improvement. To enhance their uptake, we provide a holistic view regarding critical design aspects of depression apps and develop design principles (DPs) for depression apps.

Methods

We use a qualitative approach with 58 semistructured interviews between March 2021 and August 2022 with different stakeholders (healthcare providers, patients with mild and moderate depression, app developers and operators, and health insurance companies) to derive our DPs to provide a holistic view of the crucial design factors of depression apps. We then evaluated our DPs derived from four individuals from the health sector in terms of appropriateness and completeness.

Results

Following this approach, we examine seven meta-requirements (MRs) like “quality” and “simplicity” to deduce 16 corresponding DPs for depression apps. Our prescriptive design knowledge enables us to carve out interrelations and contributes to the literature on depression app design knowledge. Generally, our MRs and DPs derived were evaluated positively.

Conclusions

Our research ignites a structured discussion among stakeholders in the healthcare sector, for example, scientists and practitioners, about the importance of specific design aspects of depression apps and the overall development process from planning to dissemination.

Keywords

Depression applications success design principles stakeholders qualitative research

Introduction

About 300 million people worldwide are affected by a depressive disorder.¹ Accordingly, depression is a major challenge for many people, affecting their lives, social participation, and quality of life.^2,3 The COVID-19 pandemic has further increased the global burden. As a result, the prevalence of depression has increased by 28% on average.⁴ One problem with managing depressive disorders is the long waiting time for treatment. Because of insufficient capacity, patients seeking help have to wait many weeks for an appointment for an initial consultation with a psychotherapist. In Germany, for example, the waiting time is about 3 weeks until the initial consultation and another 20 weeks until the start of therapy.⁵ This also requires asking different therapists for appointments, which can burden individuals.⁶

Against this backdrop, health insurance today encourages using and deploying Information Technology (IT) to tackle mild and moderate depression. Patients are encouraged to use applications (“apps”) to support—or in some cases to replace—face-to-face depression therapies to provide leverage effects and to bridge long waiting times.⁷ These self-help interventions are delivered through mobile and web apps on patients’ smartphones, tablets, and computers. Unless the advantages and functionalities of these depression apps (For the case of clarity, we use the collective term “depression apps” or “apps for depression” to include all web and mobile apps that deliver healthcare interventions for mild or moderate depression through a web and/or a mobile app.) are understood from a theoretical point of view,^8,9 the use and deployment of such apps, for example, in Germany, leave room for improvement.¹⁰ Characteristics, contents, similarities, and differences, that is, design patterns, of depression apps have already been studied,⁹ for example, through taxonomic approaches,¹¹ acceptance research,¹² or market observations.¹³ Depression apps are equipped with various functions such as self-monitoring of health data, disease detection, or remote monitoring systems.^9,14 However, an interdisciplinary, holistic view, including all stakeholders’ opinions regarding the design and structure of depression apps that could enhance the dissemination, is still missing and is proposed as a research avenue, for example, by Mayer et al.¹⁵ Also, the pre-requisite for successful mobile medical app development is the integration of developers and medical professionals. It can lead to more appropriate digital solutions that better fit the stakeholders’ needs.¹⁴ Those stakeholders include patients, healthcare professionals, health insurance, and developers since they are involved in the overall lifecycle of depression apps, that is, from their development to their dissemination to their usage.

Research regarding indication-specific, that is, depression-specific, design principles (DPs) is scarce. Kenny et al.¹⁶ establish specific DPs for developing health apps. These principles are for apps that elderly patients with age-related macular degeneration can use. Based on previous research, they derive four DPs, that is, the appropriate interface for the target group or the integration of the application into the routine of patients and physicians. However, they are not tailored to the depression-specific context. On a more general view, Mathews et al.¹⁷ focus on the validation of digital health solutions and provide a framework to guide the evolution of general health apps. Also, Murray et al.¹⁸ design an evaluation strategy to support the development and deployment of digital health apps. Their research concerns the quality and effectiveness of digital health implications rather than specific design features. Most recently, a study by Wierenga et al.¹⁹ examined specific design aspects of mobile apps for peripartum depression. Their mixed-methods approach reveals that fewer survey moments, more reminders, and a need for more unique content compared to commercially available apps are crucial for app design in the special case of peripartum depression.

To sum up, past research was mainly concerned with the characteristics and functionalities of depression apps with different methodologies like randomized control trails and focused, for example, on acceptance²⁰ or medical²¹ results on dedicated stakeholder groups. However, to our knowledge, studies would need to incorporate the views of various stakeholder groups and synthesize them into DPs, since the literature agrees that this would be beneficial. We have not found such a study so far. It is well known that design features are important to improve app adoption and encourage usage in a healthcare context.²²

Consequently, we systematically examine crucial design factors, that is, meta-requirements (MRs) from interviews with relevant stakeholders, and identify a representative set of the most important DPs for depression apps. Based on 58 stakeholder interviews, the prescriptive collected design knowledge enables us to deduce design patterns, an established strategy in Information Systems Research. It carves interrelations within a design phenomenon and contributes to depression apps’ design knowledge production.^23,24,25,26 Practitioners, for example, project managers or developers, can use our derived DPs as actionable, straightforward advice, and focus on what is most important to consider for developing and disseminating depression apps successfully. As a result, this can reduce development costs and lower the complexity of the overall development process.^27,28,29 This process can be made more stakeholder-centric since it aims to develop depression apps that suit the needs and interests of the intended stakeholders. Based on these motivations, we address the research question (RQ):

“What are the critical design principles for successful depression app development?”

To answer our RQ, we follow the well-established Design Science Research (DSR) methodology proposed by Peffers et al.³⁰ Our problem-centered initiation procedure examines important DPs, that is, artifacts for depression apps that can help deliver better services to tackle mild and moderate depression. To construct DPs, we use qualitative data, qualitative content analysis techniques,²⁹ and recent DP construction guidelines.²⁴ We discussed our DPs with scientists and practitioners to evaluate and iterate our findings according to the different evaluation criteria^31,32 and examine the usefulness of our artifacts.

The rest of this article is structured as follows: first, we provide theoretical foundations on apps for depression and the importance of their design. Then, we describe our methodology and data collection. We present and discuss our MRs and DPs subsequently and discuss implications and contributions for theory and practice. Lastly, we provide limitations, future research directions, and concluding remarks.

Methods

We deduce theoretically grounded and empirically examined MRs, DPs, and their interrelations to guide the design and development of depression apps as the ultimate goal of our research. We build on the structured approach of DSR studies proposed by Peffers et al.³⁰ We use a problem-centered initiation as an entry point in our research inquiry. The definition of the problem and its importance are shown in the first section of this article and further elaborated in our theoretical foundation section. The motivation for this article arises from the observation that despite the great burden of depressive disorders, there is low use of apps for depression. This reveals major problems and unused opportunities in the (non-)usage of these apps. Consequently, the RQ conceptualizes crucial design factors to develop a useful and successful solution, that is, an app for depression. In the second step, according to Peffers et al.,³⁰ we define the objectives of the solution. Our MRs and DPs can be used to develop and establish depression apps more successfully. DPs can be interpreted as meta-artifacts that belong to the type of theory for design and action.³³ As meta-artifacts, they enable the categorization of design knowledge and make it reusable in different applications.²⁴ In our case, DPs enable knowledge reuse for, for example, developers, potential investors, or project management to allow efficient and successful IT-based services for depression through stakeholder-oriented apps.²⁴ Also, DPs are intended to improve the quality of already existing apps.²⁴ Led by current research on the functionalities of depression apps and e-health and m-health apps, we initiated the next stage of the DSR methodology process model.

To examine MRs and construct crucial DPs, we use a qualitative approach within the third stage of the procedure model by Peffers et al.³⁰ Interviews with different stakeholders will help to derive our DPs to provide a holistic view of the crucial design factors of depression apps. Existing and well-established procedures for DP generation guide our DPs deduction. According to Gregor et al.,²⁴ a complete design principle specifies the goal, context, and mechanism (including rationale, if appropriate). In addition, it considers the role of the stakeholders in the relationships between these elements, which will be discussed in later sections. Our interviews were conducted with an interview guideline with four main question blocks, namely (a) (non-)successful functionalities, (b) design elements of depression apps, (c) depression apps’ general relevance for a healthcare system, and (d) an outlook on the topic. Our question blocks were inspired by examining related literature on depression apps, their design aspects and functionalities, and possible acceptance factors for such artifacts’ (non-)usage. With the usage of (semi-)open questions within the interview guidelines, we avoid potential response biases in the interview situations.³⁴ Also, this literature review allowed us to go into the field with basic knowledge about the research area.³⁵ Interview guidelines were slightly adapted for the specific stakeholder group since practitioners and patients with mild or moderate depression show different levels of knowledge on the topic. The interview guidelines were pretested with (post-)doctoral students regarding the understandability of the questions. We recruited interview partners through our research and professional networks, e-mail, or relevant social media. Interviews were conducted online and/or face-to-face (in Lower-Saxony, Germany) with healthcare providers and health insurance companies. Patients with mild and moderate depression were interviewed within the locations of the Hanover Medical School in Hanover, Lower-Saxony, Germany. App developers were interviewed online with Webex. Our data collection occurred between March 2021 and August 2022 and was saturated as no new topics emerged from new interview situations. In total, we conducted 58 (n = 58) interviews among 21 (n = 21) healthcare providers that can be divided into four subgroups: patients with mild or moderate depression (n = 19), app developers and operators (n = 10), and health insurance companies (n = 8). With our purposive sampling technique to avoid potential sampling biases, we made sure that we received meaningful responses to answer our RQ.³⁴ With this composition of the collection of different stakeholders’ opinions, we received a holistic view of the chances and challenges that arise with depression apps. Table 1 summarizes our data collection and composition:

Table 1.

Data collection and composition.

Stakeholder group	Subgroup	Number of interviews
Healthcare providers		21
	Psychotherapists	7
	Psychiatrists	5
	Family doctors	5
	Cardiologists	4
Patients with mild and moderate depression		19
App developers and operators		10
Health insurance companies		8
	Total	58

Significant values are in bold.

Interviews lasted around 30 to 120 minutes. The author's team conducted them in German or English and fully transcribed them afterward. Our transcripts serve as primary data for the construction of the DPs. We analyzed the transcripts with MAXQDA 2022 and made use of an iterative descriptive coding process, as suggested by Myers,³⁶ to identify MRs (“patterns”) and DPs (“characteristics within those patterns”) from the data. Descriptive coding has been used several times in related contexts, for example, DSR-specific contexts.³⁷ Initially, we used the question blocks from our interview guidelines as the first orientation to reduce the complexity of the transcripts and the content. Then, with this descriptive coding procedure, we ran a structured analysis within those question blocks regarding the qualitative data's most important design factors, characteristics, and possibly other important conditions. For example, we found several text passages through descriptive coding that were concerned with the “content and structure” of the depression app. Within this “content and structure” MR, we found arguments regarding “gamification” elements as potential DP. We used the identified coding examples from this DP “gamification” as inspiration to formulate a description of the DP “gamification” within the MR “content and structure” according to the logic of Gregor et al.²⁴ for the formulation of DPs. We structured our DPs according to the components of the aim of the DP, the context, the mechanism, and the rationale behind the respective DP.²⁴ Also, we took a closer look at the DPs that the different stakeholder groups named.

After identifying the first set of MRs and DPs, we initiated the evaluation stage, as suggested by Peffers et al.³⁰ of our research. We evaluated our initial MRs and DPs in light of several evaluation criteria based on existing seminal literature on DSR artifacts and DPs evaluation.^{31,32,38,39,40,41} As a result, we oriented our evaluation interviews to important criteria like accessibility, importance, and insightfulness, as well as guidance and usefulness. As we identified researchers and practitioners as the main stakeholder groups for our findings derived, we invited individuals from those. We ensured these were not included in the first data collection phase to avoid response biases. As a result, we were able to conduct four evaluation situations. MRs and DPs were sent in advance for the preparation of the interviews. They lasted around 45 minutes, were held online and in German, and conducted in June and July 2023. We wrote extensive interview protocols that serve as primary information for a potential evaluation and revision of our MRs and DPs. We refined our MRs and DPs according to the collected statements derived. Interview guidelines, the coding scheme with anchor examples, and evaluation questions can be found in the appendix of this article.

Results

We identified seven distinct MRs (MR1–MR7) and 16 corresponding DPs (DP1–DP16). While some DPs show some clear tendency as a “stand-alone” principle to guide the process of the development of a depression app, some of the DPs are connected to two MRs and show interconnections. We will present and discuss the results in light of the existing literature. Furthermore, we analyze some multistakeholder-specific considerations. Figure 1 shows the set of MR, DPs, and descriptions. Also, we show whether the DP was explicitly named by developers (D), patients (P), healthcare professionals (HCP), and health insurances (HI) to provide a holistic view of the opinions of the stakeholder groups.

Figure 1.

Meta-requirements and design principles for depression apps.

Content and structure

Based on MR1, we identified the first DPs regarding the content and structure of depression apps. Control and monitoring have been extensively studied and investigated as important characteristics of depression apps.^9,14 Our interviews have confirmed this statement. Consequentially apps for depression should include monitoring and control features, for example, a diary function or observation by therapists, to enable the detection of the deterioration patient's condition (DP1). An often-mentioned important design factor is the ability to tailor the treatment to the patient's needs, as one of our interviewed patients (P5) mentioned:

“I would find that important because I always say that it's not appendicitis, and the symptoms and the people are so different. I would find that important that you could pick out something that affects you or where you need support. […] I don't have a lack of drive or anything, and you have to shape that very individually [in the depression app], I think.”

This can be achieved by individualization, that is, selecting therapeutic modules according to a therapist's recommendation (DP2). The literature has identified that technology directly influences the patient–physician partnership because of a (partial) lack of personal contact.⁴² Also, missing individualization has been identified as a crucial failure factor for depression apps from the view of the patients and should be taken seriously regarding the development process.⁴³ Within DP2, the rationale is to offer the patients customized therapeutic treatment and not to lose the important therapist–patient connection. Psychotherapists need to control and monitor activities by having access to patients’ inputs, which allows for recognizing, for example, deteriorations in the patient's condition. In the medical professional's view, depression apps should only be used to supplement face-to-face personal contact, not as a stand-alone therapy. They would rather be used as an addition to the traditional therapy, as one of our interview partners (HCP16) acknowledged:

“I could imagine, if I use it [in my therapy], that I would always pick up on how the patient uses it and whether they use it. So, I would already accompany [the application] and not simply separate it, so to speak.”

Lastly, to create a more enjoyable user experience, incorporating gamification features, for example, providing a mascot or virtual companion, is helpful and should be considered by app operators (DP3). The wish and need for gamification elements within depression apps were also identified by previous studies, that is, by Mayer et al.¹⁵ However, offering gamification elements within depression apps should always be treated in the light of the seriousness of the product and the illness, that is, depression. Focusing on a stakeholder perspective regarding MR1, we observed a clear tendency from all included interview groups toward individualization elements. Only developers and patients have named gamification elements as important factors.

Simplification

A central aspect emphasized by all interviewees is illustrated in MR2, which aims to make usage as simple and low threshold as possible for the target group. Within the interview, it became apparent that, in particular, developers and patients provided many examples of the three DPs connected to this MR. There is consensus among all stakeholder groups on the importance of this MR. To minimize frustration during use, patients should be able to navigate the app intuitively. This statement is somewhat consistent with previous literature on the design of depression apps, showing that confusing menu structures hamper the acceptance and usage of such apps.^44,45 Within DP4, concerned about operability and usability, we found several examples from patients on how they consider a depression app usable and operable. Here, patients liked having an easy language, limited choice options, glossaries and search functions, and download options to provide treatments offline. This is especially important to use the app time-independently.⁴⁶ Based on these statements and examples, we formulate the following DP4: provide intuitive navigation features, for example, implementation of easy language, a keyword directory, and a categorized menu with limited choice options to minimize frustration. Also, different stakeholder groups with different technical know-how must be considered. One of our interviewees marked (P4):

“Interviewer: Okay. And now, […] what difficulties do you see in the independent use of such a depression app if a therapist doesn't accompany it?

Patient 4: My technical know-how [I: yeah]. In any case, the technical know-how, the fear, if I download it now and do something wrong, press somewhere that my data is freely available. And often, I find these incredibly long explanations about apps are so cumbersome that I leave it there.”

Our interviews brought several statements regarding appropriate user interfaces for depression apps. We received numerous potential design aspects from developers and patients to achieve an appealing user interface. Here, developers put effort into subtle colors and illustrations, while patients emphasized easy-to-read fonts. Especially the last has been identified as crucial for the success of depression apps.⁴⁴ While the examples are diverse among the stakeholder groups, subtle colors are important for patients and developers. As a result, we formulate the following DP5 regarding appropriate use interfaces: an appealing user interface that supports effortless use should be designed by, that is, using subtle colors, easy-to-read fonts, and illustrations (DP5). Simplicity and low usage thresholds can be reached by providing easy access. Here, all stakeholder groups argued that testing options for the depression app are an important characteristic of depression apps, as is stated in the literature.⁴⁷ Test trails provided by responsible operators are seen as useful to make depression apps prominent in terms of what they can offer. In addition, technical support (named by patients and developers) and possible mouse-over functions (named by patients) within the app are important design aspects. To sum up, we subsumed the following DP6: provide supplemental information, technical support, and testing options to simplify the registration and starting process for users (DP6).

Supportive resources

This simultaneously provides a partial aspect to ensure supportive resources like mouse-over functions and has some aspects of simplicity (MR3). However, supportive resources also include design features and functionalities like emergency buttons and the provision of emergency numbers that our interview partners have named. This emergency management feature is crucial in line with the literature, for example, Schuster et al.⁴⁷ Also, providing personal contact in acute crisis situations has been named important by healthcare professionals and insurance companies. These are extended by the ability to respond to emergency situations of users adequately. Consequentially, support services should be provided to respond to users’ emergency situations, such as an emergency button, emergency phone numbers, and personal support from a doctor or therapist (DP7). DP7 is closely connected to DP8, an interaction function with other affected people that can encourage using the app in crises. We align with the literature here and confirm findings that a lack of the possibility to contact other affected people hinders the dissemination of depression apps.^48,49 However, this statement does not hold for all patients, as we heard it from interview partner P3:

“So, for me personally, now not so important. […] not because I'm not a social person, not a person who doesn't like to talk to people, but when I talk to other people, I wouldn't want to talk about the depression app [through it].”

As a result, allowing the user to connect with other affected individuals to promote social interaction and encourage each other (DP8) completes the fulfillment of MR3. Also, it goes hand in hand with the following MR4.

Reassurance

A lack of drive and poor motivation are characteristic symptoms of depression sufferers. Patients typically experience difficulty establishing continuous use of treatment programs. Interventions in depression treatment should be designed to reassure the user (MR4). Besides the connection opportunity in acute crisis, depression app providers should provide some general interaction features for the intended users with praise and feedback. This was also named explicitly by interviewed patients as well as developers. Literature also shows that such functionalities and design features can foster the motivation to use depression apps.⁵⁰ A developer argued that motivation from the app to the users could be useful, but should deliver seriousness to the user based on their sufferings (D8):

“One idea would also be to see we know quite a lot about the psychopathological profile of the users, that we don't just say we're going to make one solution for everyone, but that we also personalize it a bit. The people where we notice, ok, they have good self-regulation abilities, they get fewer reminders, because they [the remainders] are just annoying anyway, and the people who have difficulties, they get a little bit more and maybe also get a little bit of cheerleading from the app and a little bit more positive reinforcement and praise and ‘Hey, great, you've done a course!.’”

To sum up, we identify that, in addition to implementing social interactions in, for example, forums, receiving direct feedback and praise when using the app, for example, upon reaching a milestone, also provides increased motivation for the user (DP9).

Security and privacy protection

Generally, security and privacy protection are very important while using health data. Still, the processing of sensitive health data, in particular, is an aspect that is a key issue for all interview partners (MR5). As one of the interviewed developers (D1) marked, a lack of security can lead to crucial data breaches. However, as D1 noted, even the best security measures can fail and not claim for 100% guarantee against data breaches, and data losses through attacks by hackers. Yet, in any case, depression apps should be up-to-date in terms of security and privacy protection standards:

“Yes, so of course, it's an important topic, especially in depression. I can also think of a negative example: A hacker attack on a Finnish depression online therapy. The hackers blackmailed the patients who used the online therapy with personal data. So, data security and data protection are one thing. You do everything you can, but a professional can always get at the data somehow. Another thing is, of course, if you leave a door open, for example, or have old software and it is therefore no longer secure, then it's your fault.”

This is not surprising since data protection and security issues have been extensively studied as crucial for the success of mental health apps⁵¹ and depression apps in particular.^10,52 We formulate the following DP: To establish the user's trust in the app provided, the user must be made aware of the measures that have been taken to protect their health data (DP10).

Quality

Furthermore, ensuring quality is essential for a reliable product, that is, a successful depression app (MR6). We explored three related DPs within this MR. To achieve a holistic and comprehensive solution and thus a high-quality product, an interdisciplinary approach is recommended by developers, patients, and healthcare professionals whom we interviewed. This includes not only developers but also therapeutic and medical expertise as well as patients of the target group in the design and further development of the app (DP11). As described in the related literature about the development of depression apps, multistakeholder inclusion within the development process is useful and appreciated.^53,54 This holistic approach enables the design of an app whose efficacy can be proven scientifically, that is, through clinical studies. Evidence-based proof has been intentionally identified as a crucial aspect of using depression apps, especially by healthcare professionals.⁵⁵ Our examination aligns with this since developers, healthcare professionals, and insurance companies named this issue. The proof of efficacy generates credibility and respectability among users and should therefore be considered within the development process (DP12). The chosen multistakeholder perspective, in combination with the condition of proven efficiency, enables the needs-oriented functional development called for in the literature.⁵²

To summarize the fulfillment of all-important quality requirements, widely recognized certifications should be supplied as a quality criterion, showing the user that the product was developed according to recognized standards (DP13). Some developers that we interviewed were aware of this certification and proved the efficacy of their offerings through scientific studies, for example, D6:

“I think the most important thing is that we have carried out studies on our program, there are currently three studies on the depression program, and they are investigating how effective the programs are, and I think this is the only way to go out confidently and say “we know that our program works”.”

Many interviewees noted that a key reason for the low uptake of depression apps is the lack of knowledge among patients and therapists, which will be tackled by MR7.

Framework conditions

There is a need for framework conditions that must be considered within the overall development process to make depression apps attractive to potential users and enhance their dissemination (MR7). As previous literature notes, the dissemination and knowledge of depression apps are restricted among healthcare providers.⁴⁸ The possibility of getting a clear representation of the quality of the intervention through a certification supports finding a suitable app among the large supply available and is therefore connected to MR6. However, since many therapists are unfamiliar with depression apps, training, and onboarding activities can help to increase distribution (DP14). From the interviews with healthcare professionals and health insurances, it emerged that another critical factor affecting the uptake of depression apps is the cost incurred by the user. This issue is also named by past literature on depression apps focusing on costs.⁴³ Establishing reimbursement opportunities, for example, through contracts with health insurance, as in Germany, allows wide access for patients (DP15). As one of the interview partners from the insurance sector noted, different cost models for depression apps could be useful and implemented (HI3):

“And the second point, which I also signaled a bit of disappointment about earlier, is that something like pay-for-use would also be possible for digital applications. Saying, no, I don't pay for downloading the app, but I pay for using the app, and if the thing has ten lessons, then the payment amount also breaks down into ten partial payments, each of which is called up when working on these ten lessons in the app. This would all be possible, would not involve a large administrative procedure, would be more in line with the uses and applications of the apps, and would also resolve a few stomach-aches on the subject of remuneration and costs.”

The cost aspect is also relevant on the side of the practitioners. The additional time and effort they spend to get used to new treatment methods must be adequately compensated (DP16). Note that potential developers are somewhat restricted in controlling the compensation by, for example, insurances. However, this overall issue should be considered in the cost of the development process of depression apps.

Multistakeholder-specific considerations

As shown, different DPs were mentioned by the various stakeholders. Table 2 shows an overview of the DPs and their articulation by our interviewed stakeholder groups. DPs identified are sorted according to their mentions.

Table 2.

Design principles sorted according to the mentions per stakeholder group.

		D	P	HCP	HI	Mentioned by X stakeholder groups
DP2	Individualization	x	x	x	x	4
DP4	Operability and usability	x	x	x	x	4
DP5	Appealing user interface	x	x	x	x	4
DP6	Low threshold	x	x	x	x	4
DP7	Crisis intervention	x	x	x	x	4
DP10	Data security	x	x	x	x	4
DP1	Monitoring and control	x	x		x	3
DP11	Involvement of multiple stakeholders	x	x	x		3
DP12	Proof of efficacy	x		x	x	3
DP14	Spread of information	x		x	x	3
DP15	Reimbursement	x	x		x	3
DP16	Remuneration	x		x	x	3
DP3	Gamification	x	x			2
DP9	Praise and feedback	x	x			2
DP8	Interaction		x			1
DP13	Certification	x				1
		15	12	10	11

D: developers; DP: design principle; HCP: healthcare professionals; HI: health insurances; P: patients.

Significant values are in bold.

Some observations can be made from this multistakeholder perspective and overview: The developers name most DPs (15), followed by patients (12), health insurances (11), and healthcare providers (10) out of our total number of 16 DPs. It can be assumed that developers have several ideas and imaginations on what seems crucial for designing depression apps. Healthcare providers, in contrast, follow the opinions of the majority and are included in DPs that are named by at least three or four stakeholder groups.

We identified congruent assessments among all stakeholder groups of several design factors: there is agreement that individualization (DP2), all DPs regarding simplicity (DP4, DP5, and DP6), crisis interventions and management (DP7), and data security (DP10) must be considered as crucial. It can be assumed that those DPs show high importance and must be considered in any way within the overall development process of apps for depression.

DPs that are named by three of four stakeholder groups (DP1, DP11, DP12, DP14, DP15, and DP16) are diverse in their composition of stakeholder groups. However, we observed here a centered inclusion of opinions by healthcare providers. Furthermore, only developers and patients, not health insurance companies and service providers, attach importance to gamification (DP3) and praise and feedback (DP9).

Two DPs were named by only one stakeholder group: these are DP8 “Interaction,” which was named exclusively by the patients, and DP13 “Certification,” which was named exclusively by developers. It should be noted that because a DP is named by one stakeholder group only, this does not necessarily mean it is unimportant for other groups. It can be assumed that different stakeholder groups understand design aspects under different terms or cannot formalize them if not explicitly asked.⁵⁶ For example, it became apparent that patients understand certification in terms of the quality of the app for depression. Consequentially, their statements, in this case, are connected to the overall (medical) quality of the contents provided by the app. While developers, healthcare providers, or health insurance companies do not place particular emphasis on the ability of patients to interact with each other through the app, this aspect may be a critical feature for patients to use. However, these single-stakeholder opinions highlight the need to consider a multistakeholder perspective in developing depression apps to receive a holistic overview of the phenomenon.

To sum up, each DP mentioned in Table 2, even if it is mentioned by all stakeholder groups or only one, can impact the acceptance and success of the app and, thus, the overall implementation process. The service portfolio of the depression apps heavily depends on the interests, that is, business model, of the developers, and healthcare providers that mainly want to offer appropriate services to patients and healthcare professionals. This issue of potential criticalities between those DPs was further explored through our conducted evaluation phase in this research.

Discussions

We evaluated our MRs and DPs that we identified before with four individuals. In total, three researchers, in the following denominated as evaluation partners 1 to 3 (EP1 to EP3) and one practitioner (EP4), were interviewed and asked their opinions about our research output. EP1 is a postdoctoral researcher in the e-health and m-health sector. EP2 and EP3 are research assistants at two German universities concerned about medical management, technologies, and m-health apps. All three scientists have a strong fit for the research topic and have published several peer-reviewed conference and journal articles concerned with m-health and/or depression apps. EP4 is a psychotherapist with a strong connection to the health insurance sector in Germany. Also, EP4 is closely connected to the app developer industry in the medical sector and apps for depression in particular. Statements regarding different evaluation criteria follow the procedural logic of Iivari et al.³² for reusability of DPs, namely accessibility, importance, novelty and insightfulness, actability and guidance, and effectiveness and the evaluation criteria by Prat et al.,³¹ namely ease of use, consistency, or completeness. In the appendix, we present our evaluation questions for all criteria mentioned.

The presented MRs and DPs have been evaluated as accessible, especially by EP1 and EP3, meaning they are expressed in an understandable language. They recommend a change in the denomination of MR7 “framework conditions” that could be renamed to “access factors.” Also, a merge of MR5 to MR7 could be realized since security aspects play, according to EP1 and EP2, a subordinate role within the overall development process since they are often a pre-requisite within the depression app development and sometimes led by regulatory aspects, as in Germany. EP4 underlined this statement. In addition, a relabeling to “data security“ is considered for MR5 by EP2. As a result, the first pre-requisite (“accessibility”) for the reusability of DPs is fulfilled.^31,32

We then revisited the importance of the listed MRs and DPs regarding their completeness, especially regarding our data's multistakeholder perspective. Here, respondents liked the good overview of the opinions of the different stakeholder groups about important design factors and characteristics of depression apps. This comparison has been evaluated as insightful and novel (EP1, EP2, and EP3).

Scientists argued that the DPs could guide (“provide guidance and actability”, according to Iivari et al.³²) a process for the development and dissemination of a depression app. However, EP1 argues that a holistic view of all DPs could also guide a starting process as a checklist for founding new businesses offering mental healthcare through apps. Following the DPs does not necessarily mean the developed depression app will be automatically successful. This fact is underlined by EP4, who argues that certification is one of the most crucial aspects of the design of depression apps. If the certification is not trustworthy to the user, other aspects play a subordinate role in the usage.

As noted by Iivari et al.,³² effectiveness or usefulness³¹ is sometimes difficult to measure as one of the main indicators for the reusability of DPs. Consequently, we asked our evaluation partners about the potential usefulness of our results and findings. They find the comprehensive overview of the phenomenon, that is, design aspects of depression apps and their importance for specific stakeholder groups (EP1, EP3, and EP4) useful. From a practitioner's perspective, seeing which aspects are named and identified as most important is useful. Practitioners will be motivated to assess and order the DPs for their individual depression app development process (EP2 and EP4). Figure 2 represents the iterated MRs and DPs that became apparent through our evaluation phase.

Figure 2.

Iterated meta-requirement and design principles for depression apps.

As the evaluation shows, our MRs and iterated DPs are important and can be useful for practitioners and academics. Focusing on the problem-centered approach by Peffers et al.,³⁰ our theoretical contribution is a solution for a real-world problem in medicine to develop and offer depression apps that are appealing and cost-efficient for providers and intended users. Targeted app development requires indication-specific consideration and the inclusion of all stakeholders because focusing only on the patient gives a limited view of the care situation. The pre-requisite of successful mobile app development in medicine, that is, depression app development, is the integration of software developers and medical professionals.¹⁴ The involvement of other stakeholder groups in the design of those apps has so far been limited, but it is critical to tailor the health app to the characteristics and needs of the end-user.¹⁰ Also, healthcare professionals’ perspective regarding the app's content and development influences the application's acceptance.¹⁵ In addition, health insurance companies form the interface between healthcare recipients, providers, and insurers and thus influence the adoption of m-health.¹⁵ The question is which specific design factors and aspects can increase the acceptance of health apps from the perspective of stakeholders involved in the overall lifecycle of depression apps. With our multistakeholder perspective and examination, we aid this involvement.

In line with Gregor's logic,³³ we contribute with a theoretical contribution of the “design and action” type. Consequently, our DPs expand the current view on scattered design aspects and characteristics of depression apps that must be conserved within a development process. We contribute to the literature on the successful design and important characteristics of depression apps by providing clear recommendations. Also, our DPs overcame the development process challenges and the motivations in the introduction section. With our holistic multistakeholder perspective, established through extensive qualitative data collection among four stakeholder groups, we examined the differences and similarities of important design aspects between different stakeholders. DPs can be interpreted as nascent design theories since they can provide generalizable insights into other classes of mental health apps.⁵⁷ Two of our evaluation partners (EP2 and EP3) also confirmed this finding. Both interview partners in our evaluation stage reveal that the identified DPs could also be transferred to an extent to another e-health and m-health context. They stated that the DPs could also be transferred to international markets, even if the primary data was collected primarily in a German context. Contrarily, EP3 missed a DP regarding the obligation of providing an advertising-free depression app, which must be considered in several countries, like Germany.

Contributions for practitioners are intuitive and were explicitly mentioned by all our evaluation interview partners: as described, with our DPs, we want to guide an overall development process of depression apps. Responsible stakeholders like IT project managers can use our carefully derived DPs for their development projects to save time and effort in reducing development costs and the complexity of the overall development process.^26,27 In addition, as indicated by EP1, our DPs can guide a grounding process of development companies that want to specialize their offerings within the healthcare sector or in the depression domain in particular. As a result, our findings can help practitioners overcome challenges like the high complexity of the overall process connected to implementing and disseminating digital health solutions.⁵⁸ As marked by our interview partners for evaluation (EP1 and EP4), developers and investors in depression apps must be clear about their own prioritization of the presented DPs. Regarding our RQ, we can claim that we identified the most important DPs, based on the statements of our interview partners.

However, successful depression app development in terms of, for example, development costs can be influenced by many other factors, like the composition of the development team,,⁵⁹ and implicit importance by the developers (as also mentioned by EP4) that may not be reflected through our DPs.

As indicated before, EP3 missed a DP regarding the obligation of providing an advertising-free depression app, which must be considered in several countries, like Germany. While the DPs derived provide a holistic overview, some of them, like data security (DP10) or remuneration (DP16), are influenced by country-specific regulations. The study was conducted in Germany with mostly German-based participants, and reflects potential specifics for Germany or the European Union. It should be noted that the study's findings should be reflected in the corresponding regulations. For example, areas like the United States or China, in which different regulations apply, could have different views on some DPs.

Also, as seen, the DPs derived holistic views and opinions on the topic of relevant design aspects. They do not precise a concrete development process and may be difficult to apply in real-world settings. We have some advice here: the DPs could be complemented by practitioners with forward/backward chaining models²⁸ to first structure them into a sequential behavior and visualize them in a processual manner. Also, the usage of the Persuasive Systems Design Framework⁶⁰ and persuasive systems principles, like reductions or self-monitoring, could be valuable for practitioners to first cluster the DPs derived to special interests in the light of the corresponding business model that the developer wants to achieve. Especially the last has been successfully adapted in the mental health context.⁶¹ This outlined complementation could lead to further complexity reduction for the start of the development process of a depression app. In addition, incorporating the DPs into practitioners’ environments process models in the (mobile) software development domain, for example, agile processes⁶² can help. In any case, the usage and adaptation of the DPs derived are influenced by a set of various internal and external factors that should be considered.

Limitations, further research directions, and concluding remarks

Our research has limitations that open the potential for further research. First, our presented DPs have been evaluated generally according to established evaluation criteria, but have to be tested in real-world setups. This research does not aim to develop a depression app based on the MRs and DPs derived. We received the first statements regarding the ordering and wording of our DPs and MRs. However, our evaluation here mainly focuses on the interview partners’ subjective opinions.

In addition, we did not interview noncare-providing researchers who develop depression diagnoses with advanced technologies like machine learning pipelines. In the past, several researchers have articulated the importance and usefulness of advanced technologies for depression diagnosis and interventions.^63,64 While those technological possibilities were not explicitly mentioned in our material, including this stakeholder group in future research, like interview studies, could enhance a more unified view of important design aspects and technological backbones for depression apps.

Practitioners should use the DPs derived in real-world contexts and test the usefulness of these problem-solving artifacts in more detail. It is also valuable to weigh the importance of the DPs mentioned here or quantify them in terms of money and time resources saved. This procedure would lead to more quantitative insights into the usefulness of specific DPs. At the same time, iterative evaluation is an imminent step toward more appropriate DPs in DSR.^30,65 This can be a fruitful avenue for further research.

Also, our DPs derived could be quantitatively measured and verified with surveys that collect the opinions of patients who suffer from mild or moderate depression. Research like this could examine the importance of the design factors derived in terms of importance and prioritization from a user's perspective. With this promising sequential mixed-method design and triangulation procedure⁶⁶ DPs and MRs derived can be iterated for a wider and better understanding of the phenomenon.

Our DPs are designed for depression apps and have a narrow design perspective. However, some MRs could be used to develop and construct other e-health and m-health solutions with or without focusing on mental disorders.⁵⁷ It could be valuable to investigate the generalizability of our findings to other domains, that is, to apps designed in an anxiety context. Here, interviews and subsequent instantiations, according to the DSR paradigm, could be an interesting first step.

The current research analyzes a set of crucial MRs and DPs for depression apps. We developed the DPs with qualitative data collected from interviews with 58 stakeholders to carve out design aspects and elements that must be considered when developing depression apps successfully in terms of tailored app functionalities and framework conditions. DPs were discussed and evaluated with scientists and practitioners, showing the importance and usefulness of our results. Our research can ignite a structured discussion among scientists and practitioners about the importance of design aspects of depression apps and the overall development process from planning to dissemination. The set of examined and evaluated DPs can lead to well-accepted and high-quality depression apps for patients who suffer from mild or moderate depression and, in the end, to quicker recovery.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076251336276 - Supplemental material for Success by design: A holistic analysis of design principles for depression apps

Supplemental material, sj-docx-1-dhj-10.1177_20552076251336276 for Success by design: A holistic analysis of design principles for depression apps by Oliver Werth, Irene Jankowski, Nina S Müller, Fenja Schulte, Paula Warnemünde-Jagau, Michael H Breitner, Annika Herr and Kai G Kahl in DIGITAL HEALTH

Footnotes

Acknowledgments

The authors would like to thank all the individuals who participated in this study.

ORCID iDs

Oliver Werth

Michael H Breitner

Annika Herr

Kai G Kahl

Ethical considerations

The study was approved by the Ethics Committee of the Hannover Medical School under number 10017_BO_S_2021.

Consent to participate

All probands gave informed written consent before study entry.

Author contributorship/CRediT

OW, NSM, and IJ contributed to conceptualization; OW to methodology; OW, NSM, IJ, PW-J, and FS to investigation, visualization, and validation; OW, NSM, IJ, PW-J, FS, MHB, AH, and KK to writing—original draft; and OW, NSM, IJ, PW-J, FS, MHB, AH, and KGK to writing—review & editing.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Ministry of Science and Culture of the State of Lower Saxony (grant number ZN3751).

Conflicting interest

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: KGK received speaker honoraria by EliLilly, Servier, TAKEDA, Idorsia, Dr Schwabe, Janssen (J&J).

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Supplemental material

Supplemental material for this article is available online.

References

Smith

. Mental health: a world of depression. Nature 2014; 515: 180–181.

Ettman

Abdalla

Cohen

, et al. Prevalence of depression symptoms in US adults before and during the COVID-19 pandemic. JAMA Netw Open 2020; 3: 1–12. doi:https://doi.org/10.1001/jamanetworkopen.2020.19686

Liu

Yang

, et al. Changes in the global burden of depression from 1990 to 2017: findings from the global burden of disease study. J Psychiatr Res 2020; 126: 134–140.

COVID-19 Mental Disorders Collaborators. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet 2021; 398: 1700–1712.

Singer

Maier

Paserat

, et al. Wartezeiten auf einen Psychotherapieplatz vor und nach der Psychotherapiestrukturreform. Psychotherapeut 2022; 67: 176–184.

Hafner

Schönfeld

Tokgöz

, et al. Digital health interventions in depression care—a survey on acceptance from the perspective of patients, their relatives and health professionals. Healthcare (Basel) 2022; 10: 1–16. doi:https://doi.org/10.3390/healthcare10102019

Firth

Minen

, et al. User engagement in mental health apps: a review of measurement, reporting, and validity. Psychiatr Serv 2019; 70: 538–544.

Cuijpers

Noma

Karyotaki

, et al. Effectiveness and acceptability of cognitive behavior therapy delivery formats in adults with depression: a network meta-analysis. JAMA Psychiatry 2019; 76: 700–707.

Mueller

Werth

Koenig

, et al. How is your mood today? a taxonomy-based analysis of apps for depression. In Proceedings of the 28th Americas Conference on Information Systems, 2022, pp.1–10.

10.

Mayer

Gronewold

Alvarez

, et al. Acceptance and expectations of medical experts, students, and patients toward electronic mental health apps: cross-sectional quantitative and qualitative survey study. JMIR Ment Health 2019; 6: e14018.

11.

Nickerson

Varshney

Muntermann

. A method for taxonomy development and its application in information systems. Eur J Inf Syst 2013; 22: 336–359.

12.

Davis FD. Perceived Usefulness. Perceived ease of use, and user acceptance of information technology. MIS Q 1989; 13: 319–340.

13.

Stawarz

Preist

Tallon

, et al. User experience of cognitive behavioral therapy apps for depression: an analysis of app functionality and user reviews. J Med Internet Res 2018; 20: e10120.

14.

Lienhard

Legner

. Principles in the design of mobile medical apps: guidance for those who care. In Proceedings of the 13th International Conference on Wirtschaftsinformatik, pp.1066–1080.

15.

Mayer

Hummel

Oetjen

, et al. User experience and acceptance of patients and healthy adults testing a personalized self-management app for depression: a nonrandomized mixed-methods feasibility study. Digital Health 2022; 8: 1–17. doi:https://doi.org/10.1177/20552076221091353

16.

Lienhard

Legner

. Principles in the design of mobile medical apps: guidance for those who care. In Proceedings of the 13th International Conference on Wirtschaftsinformatik, pp.1–15.

17.

Mathews

McShea

Hanley

, et al. Digital health: A path to validation. NPJ Digital Med 2019; 2: 1–9. doi:https://doi.org/10.1038/s41746-019-0111-3

18.

Murray

Hekler

Andersson

, et al. Evaluating digital health interventions: key questions and approaches. Am J Prev Med 2016; 51: 843–851.

19.

Wierenga

Pagoni

Skalkidou

, et al. Dropping out of a peripartum depression mHealth study: participants’ motives and suggestions for improvement. BMC Med Res Methodol 2025; 25: 6.

20.

Grüneberg

Bäuerle

Karunakaran

, et al. Medical students’ acceptance of tailored e-mental health apps to foster their mental health: cross-sectional study. JMIR Med Educ 2025; 11: e58183.

21.

Funk

Kopf-Beck

Takano

, et al. Can an app designed to reduce repetitive negative thinking decrease depression and anxiety in young people? Results from a randomized controlled prevention trial. J Behav Ther Exp Psychiatry 2025; 87: 102014.

22.

Wang

. Influencing factors of acceptance and use behavior of mobile health application users: systematic review. Healthcare (Basel) 2021; 9: 1–13. doi:https://doi.org/10.3390/healthcare9030357

23.

van Aken

. Management research based on the paradigm of the design sciences: the quest for field-tested and grounded technological rules. J Manage Stud 2004; 41: 219–246.

24.

Gregor

Kruse

Seidel

. Research perspectives: the anatomy of a design principle. J Assoc Inf Syst 2020; 21: 1622–1652.

25.

Spohrer

Fallon

Hoehle

, et al.

Designing effective mobile health apps: does combining behavior change techniques really create synergies?

J Manage Inf Syst 2021; 38: 517–545.

26.

Wimble

Leroy

. Health information technology: promise and progress. Health Syst 2018; 7: 161–165.

27.

Windeler

Maruping

Venkatesh

. Technical systems development risk factors: the role of empowering leadership in lowering developers’ stress. Inf Syst Res 2017; 28: 775–796.

28.

Kruse

Purao

Seidel

. How designers use design principles: design behaviors and application modes. J Assoc Inf Syst 2022; 23: 1235–1270.

29.

Adjerid

Adler-Milstein

Angst

. Reducing medicare spending through electronic health information exchange: the role of incentives and exchange maturity. Inf Syst Res 2018; 29: 341–361.

30.

Peffers

Tuunanen

Rothenberger

, et al. A design science research methodology for information systems research. J Manage Inf Syst 2007; 24: 45–77.

31.

Prat

Comyn-Wattiau

Akoka

. A taxonomy of evaluation methods for information systems artifacts. J Manage Inf Syst 2015; 32: 229–267.

32.

Iivari

Perlt

Hansen

, et al. A proposal for minimum reusability evaluation of design principles. Eur J Inf Syst 2021; 30: 286–303.

33.

Gregor. The nature of theory in information systems. MIS Q 2006; 30: 611–642.

34.

Johnson

Adkins

Chauvin

. A review of the quality indicators of rigor in qualitative research. Am J Pharm Educ 2020; 84: 7120.

35.

Giles

King

Lacey

. The timing of the literature review in grounded theory research: an open mind versus an empty head. Adv Nurs Sci. 2013; 36: 29–40.

36.

Myers

. Qualitative research in business and management. 3rd ed. Los Angeles: Sage, 2020.

37.

Rothe

Wessel

Barquet

. Accumulating design knowledge: a mechanisms-based approach. J Assoc Inf Syst. 2020; 21: 771–810.

38.

Marschak

. Economic information, decision and prediction: selected essays: volume III. Dordrecht: Springer, 1974.

39.

March

Smith

. Design and natural science research on information technology. Decis Support Syst 1995; 15: 251–266.

40.

Venable

Pries-Heje

Baskerville

, et al. A comprehensive framework for evaluation in design science research. In: Hutchison

Kanade

Kittler

(eds) Design science research in information systems. Advances in theory and practice. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp.423–438.

41.

Sonnenberg

vom Brocke

, et al. Evaluations in the science of the artificial—reconsidering the build-evaluate pattern in design science research. In: Hutchison

Kanade

Kittler

(eds) Design science research in information systems. Advances in theory and Practice. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp.381–397.

42.

Liu

Guo

. The effects of participating in a physician-driven online health community in managing chronic disease: evidence from two natural experiments. MIS Q 2020; 44: 391–419.

43.

Lattie

Adkins

Winquist

, et al. Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review. J Med Internet Res 2019; 21: e12869.

44.

Collins

DAJ

Harvey

Lavender

, et al. A pilot evaluation of a smartphone application for workplace depression. Int J Environ Res Public Health 2020; 17: 1–14. doi:https://doi.org/10.3390/ijerph 17186753

45.

Dederichs

Weber

Pischke

, et al. Exploring medical students’ views on digital mental health interventions: a qualitative study. Internet Interv 2021; 25: 100398.

46.

Schneider

Sarrami Foroushani

Grime

, et al. Acceptability of online self-help to people with depression: users’ views of MoodGYM versus informational websites. J Med Internet Res 2014; 16: 1–14. doi:https://doi.org/10.2196/jmir.2871

47.

Porras-Segovia

Díaz-Oliván

Gutiérrez-Rojas

, et al.

Apps for depression: are they ready to work?

Curr Psychiatry Rep 2020; 22: 1–9. doi:https://doi.org/10.1007/s11920- 020-1134-9

48.

Patoz

M-C

Hidalgo-Mazzei

Blanc

, et al. Patient and physician perspectives of a smartphone application for depression: a qualitative study. BMC Psychiatry 2021; 21: 1–12.

49.

Espinosa

Carrasco

Moessner

, et al. Acceptability study of “Ascenso": an online program for monitoring and supporting patients with depression in Chile. Telemed J E Health 2016; 22: 577–583.

50.

van der Vaart

Witting

Riper

, et al. Blending online therapy into regular face-to-face therapy for depression: content, ratio and preconditions according to patients and therapists using a Delphi study. BMC Psych. 2014; 14: 1–10.

51.

Martinez-Martin

Kreitmair

. Ethical issues for direct-to-consumer digital psychotherapy apps: addressing accountability, data protection, and consent. JMIR Ment Health 2018; 5: 32.

52.

Stiles-Shields

Montague

Lattie

, et al. What might get in the way: barriers to the use of apps for depression. Digital Health 2017; 3: 1–11. doi:https://doi.org/10.1177/2055207617713827

53.

O'Neill

Brady

RRW

. Colorectal smartphone apps: opportunities and risks. Colorectal Dis 2012; 14: 530–534.

54.

Lingg

Lütschg

. Health system stakeholders’ perspective on the role of mobile health and its adoption in the Swiss health system: qualitative study. JMIR Mhealth Uhealth 2020; 8: e17315.

55.

Brantnell

Woodford

Baraldi

, et al. Views of implementers and nonimplementers of internet-administered cognitive behavioral therapy for depression and anxiety: survey of primary care decision makers in Sweden. J Med Internet Res 2020; 22: e18033.

56.

Burnay

Jureta

Faulkner

. What stakeholders will or will not say: a theoretical and empirical study of topic importance in requirements engineering elicitation interviews. Inf Syst 2014; 46: 61–81.

57.

Gregor

Hevner

. Positioning and presenting design science research for maximum impact. MIS Q 2013; 37: 337–355.

58.

Kulkov

Ivanova-Gongne

Bertello

, et al. Technology entrepreneurship in healthcare: challenges and opportunities for value creation. J Innov Knowl 2023; 8: 100365.

59.

Standing

Guilfoyle

Lin

, et al. The attribution of success and failure in IT projects. Ind Manag Data Syst 2006; 106: 1148–1165.

60.

Oinas-Kukkonen

Harjumaa

. Persuasive systems design: key issues, process model, and system features. CAIS 2009; 24: 485–500.

61.

Guracho

Thomas

Almutairi

, et al. Persuasive system design features for mobile mental health applications. In: Baghaei

Ali

Win

, et al. (eds) Persuasive technology. Cham: Springer Nature Switzerland, 2024, pp.116–132.

62.

Kokol

. Agile software development in healthcare: a synthetic scoping review. Appl Sci 2022; 12: 9462.

63.

Kim

Han

, et al. Machine-learning model for predicting depression in second-hand smokers in cross-sectional data using the Korea National Health and Nutrition Examination Survey. Digital Health 2024; 10: 1–14.

64.

Pinto

Parente

. Comprehensive review of depression detection techniques based on machine learning approach. Soft Comput 2024; 28: 10701–10725.

65.

Schoormann

Möller

Chandra Kruse

, et al. BAUSTEIN—a design tool for configuring and representing design research. Inf Syst J 2024; 34: 1871–1901.

66.

Venkatesh

Brown

Bala

. Bridging the qualitative-quantitative divide: guidelines for conducting mixed methods research in information systems. MIS Q 2013; 37: 21–54.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.26 MB