Towards Achieving Mental Health Equity for the Underserved Population: Evaluating the Potential of Mobile Apps

Abstract

The underserved population—i.e., individuals who identify themselves as non-female, non-heterosexual, or non-White—have long faced inequities when accessing traditional clinic-based mental health services. Individuals from the underserved population tend to use and benefit from these services less than their better-served counterparts (i.e., those who identify themselves as female, heterosexual, and White). This study investigates whether similar inequities exist within the context of mobile apps specifically designed to provide self-support and peer-support. To this end, we conduct an empirical analysis using longitudinal user-level data collected from a mental health mobile app (MHMA). Our results indicate that: (i) in contrast to the traditional clinic-based mental health services, in a MHMA setting, users from the underserved population engage with the app services and derive similar benefits as much as their better-served counterparts; and (ii) there is a positive relationship between app usage frequency and the mental condition of app users. Our post-hoc analysis uncovers that the MHMA promotes equitable usage and benefit for users: (i) from the underserved population via self-management functions that enable self-support; and (ii) from the better-served population via online community functions that facilitate peer-support. These findings suggest that MHMAs have the potential to achieve equity through both user engagement and the benefits derived from such engagement. We conclude by discussing the implications for mobile app firms, policymakers, and organizations in their efforts to achieve mental health equity for the underserved population.

1. Introduction

Affecting more than 700 million people around the world (WEF, 2019), mental illnesses ¹ have become a leading cause of disability for decades (Murray et al., 2018). In the U.S. alone, mental illnesses are estimated to generate a $193.2 billion loss in annual earnings (NAMI, 2021a). The COVID-19 pandemic has further exacerbated the already-severe mental illnesses due to fear and stress related to COVID-19 death tolls, working from home, reduced social contact, and unemployment (Moreno et al., 2020; Wang et al., 2020). Recognizing that preventing mental illnesses is essential for sustainable development on a global scale, the United Nations has officially established Goal 3.4, aiming to reduce mental illness-related mortality by one-third before 2030 (United Nations, 2020a). Similarly, the United Nations has responded to the mental health challenges posed by the COVID-19 pandemic by issuing comprehensive guidance (United Nations, 2020b).

Despite the increasing global efforts to address mental illnesses, surprisingly, only around 40% of adults with any mental illness used mental health services between 2013 and 2018 (SAMHSA, 2019) mainly for two reasons. First, on the supply-side, mental health services suffer from insufficient resources (e.g., shortages of psychiatrists and mental health clinics, and limited insurance coverage) (Mnookin, 2016). Second, on the demand-side, unlike the patients with physical health conditions (e.g., patients with flu or cancer), individuals with mental health conditions often fail to recognize or acknowledge their illnesses due to self-stigma and/or social-stigma associated with seeking professional help at mental health clinics (Eisenberg et al., 2009; Masuda et al., 2009; 2012). Collectively, these observations indicate a significant gap between the supply and demand of mental healthcare delivery, with existing resources for mental health services being constrained and underutilized.

Furthermore, it is concerning that the highlighted gap in mental health service usage varies significantly across populations. The gap widens disproportionately for the underserved population characterized by their socio-demographic characteristics of race-ethnicity and sexual orientation. ² Individuals from this population are often concentrated in underdeveloped regions, face socio-economic disadvantages (Chow et al., 2003), and/or experience discriminations (Thornicroft, 2008). Consequently, essential services, including mental health services, are not equally provided to these groups, making them encounter difficulties in accessing these services due to the supply-side challenges. For instance, in 2021 in the U.S., among adults with any mental illness, 52.4% of Whites received mental health services, compared with 39.4% of African Americans, 36.1% of Hispanics or Latinos, and 25.4% of Asians (NAMI, 2021a). Similarly, lesbian, gay, bisexual, and transgender (LGBT) individuals are less likely to access mental healthcare services compared to the non-LGBT population (NAMI, 2021a; Semlyen et al., 2016). These inequities extend beyond service usage and also persist in the benefits derived from such services. For instance, African-American patients are more likely to discontinue clinic-based treatments prematurely compared to White patients (Fleck et al., 2005; Mongelli et al., 2020; Satcher, 2001), suggesting that the underserved population is also less likely to benefit from mental health services even when they use them.

Interestingly, unlike other service domains, the definition of the underserved population within mental healthcare goes beyond sub-populations characterized by their race-ethnicity or sexual orientation, and also includes the male sub-population. The mental healthcare literature consistently demonstrates that males (including White males and heterosexual males) are worse off with respect to their mental health indicators, exhibiting higher rates of suicide, disorders, substance use, and violence compared to their female counterparts (Berger et al., 2005; Davis et al., 2023; Michael, 2017; Rice et al., 2018; SAMHSA, 2023; Smith, 2022; Wong et al., 2017). This arises because compared to females, males: (i) have more difficulty in recognizing emotional issues; and (ii) have greater self- and social-stigma associated with seeking mental health support as they perceive help-seeking as a threat to their masculinity (Berger et al., 2005). Subsequently, even though mental health services are equally available across gender groups, males face difficulties in utilizing these services due to the demand-side challenges stemming from social norms. In the U.S., for instance, among adults with any mental illness, the rate of mental health service usage is 51.2% for females but only 37.4% for males (NAMI, 2021a). Similarly, among White adults with any mental illness in the U.S., the rate of mental health service usage between 2015 and 2019 was 54.5% for White females but only 39.7% for White males (SAMHSA, 2021). Considering these empirical insights and aligning with the mental healthcare literature, this paper refers to individuals as belonging to the underserved population if they identify themselves with one or more of the following socio-demographic characteristics: (i) Gender (i.e., male and other non-female gender identities (Berger et al., 2005, Eisenberg et al., 2009)); (ii) Sexual orientation (i.e., homosexual and other non-heterosexual identities (Hegland and Nelson, 2002; NAMI, 2021a; Plöderl and Tremblay, 2015)); and (iii) Race-ethnicity (i.e., African, Asian, Hispanic/Latino and other non-White identities (Masuda et al., 2009)). Conversely, the better-served population refers to individuals who identify as female, heterosexual, and White (Terlizzi and Norris, 2021).

Given the significant supply and demand gap, as well as the existing inequities, various stakeholders in the mental healthcare ecosystem are actively seeking solutions to ensure equitable mental health services (Agic, 2019; Kirmayer and Jarvis, 2019). One solution that has gained traction is the use of mental health mobile apps (MHMAs). In recent years, there has been a proliferation of MHMAs offering a range of mental health services to individuals with mental health needs (Donker et al., 2013; Moreno et al., 2020). The demand for MHMAs has been on the rise as people increasingly feel comfortable and secure seeking support for their mental health needs through mobile apps (Gray et al., 2005; Rickwood et al., 2007). Initially, early versions of MHMAs (e.g., DBT Field Coach, CBT MobilWork) were primarily developed for clinical use, empowering mental health professionals to prepare patients for treatments, deliver interventions (e.g., therapies) via the apps, and engage in follow up with patients post-treatment (Ancis, 2020; Price et al., 2014). More recent versions of MHMAs (e.g., I Am Sober, Sanvello, and Happify) were designed for non-clinical use by individuals, providing self-support and/or peer-support mental health services (Dorwart, 2023). In these MHMAs, mental health professionals do not play a direct role. Instead, app users can: (i) partake in self-support by utilizing app functions such as self-management, skill-training, and symptom tracking, which do not require interactions with other users (NIH, 2019); and/or (ii) engage in peer-support with other app users by utilizing app functions that facilitate informational support, emotional support, and companionship (Yan and Tan, 2014). ³ Consequently, unlike the traditional clinic-based mental health services that are typically accessed when patients seek treatment from professionals in clinical settings, mental health services within MHMAs are accessed as app users engage with self-support and peer-support functions available to them. Despite the emergence of such apps, to the best of our knowledge, no study has examined whether the well-documented inequity across the underserved and better-served populations in the traditional clinic-based mental healthcare setting also exists within the context of MHMAs designed for providing self-support and peer-support. Towards addressing this gap, our study aims to investigate the following research questions:

(i)

(Equity in Usage) Do users from the underserved population use MHMAs less than users from the better-served population?

(ii)

(Benefit of Mobile Apps Usage) Does MHMA usage improve mental condition of the users of MHMAs?

(iii)

(Equity in Benefit) Do users from the underserved population benefit from MHMA usage less than users from the better-served population?

We investigate our research questions using data obtained from an MHMA, hereafter called Hope (pseudonym). Launched by a mobile app start-up company in 2015, Hope is a free platform available for iOS or Android smartphone users. It offers two distinct functions for self-support and peer-support purposes:

• (i)

  The self-reflection function enables users to evaluate their own mental condition, facilitating self-support by enhancing their awareness of personal mental health needs, all without the need to interact with other users.

• (ii)

  The online community function enables app users to freely interact with other app users (peers) by sharing their feelings, seeking and/or providing informational and emotional support. This function fosters peer-support among users and requires active engagement with other users. ⁴

Our data include extensive app activity records for the two functions, along with data on socio-demographic characteristics, covering 1843 app users from May 2015 to February 2018. Among those users, 62% belong to the underserved population whereas the remaining 38% belong to the better-served population. With this dataset, we can monitor an individual user's app activities and observe how their mental condition evolves over time.

Using this longitudinal dataset, we evaluate the equity of usage by comparing the app usage frequency between users from the underserved population and users from the better-served population. We measure the benefit derived from app usage by examining the relationship between a user's mental condition at a specific time and their prior app usage frequency. Introducing an indicator that distinguishes users from the underserved population versus the better-served population as a variable that moderates this relationship, we evaluate the equity in benefits experienced by these two populations. From a methodological standpoint, we assess equity by examining statistical equivalence in usage and benefit between users from the underserved population and users from the better-served population.

Our study makes the following contributions to the literature on mental healthcare operations management. First, we find that the well-documented inequities in professional mental health service usage between the underserved and better-served populations (Alegría et al., 2002; Aneshensel, 2009; Berger et al., 2005; NAMI, 2021a; Plöderl and Tremblay, 2015) are unlikely to exist in the context of MHMA designed to provide self-support and peer-support. As such, the app usage among users from the underserved population is statistically equivalent to that of users from the better-served population. Second, MHMAs are likely to complement professional mental health services, as we find a positive association between app usage frequency and users’ mental condition. For instance, doubling app usage frequency for users with the median self-reflection score (i.e., the measure we adopt for users’ mental condition) in our study sample is estimated to improve their self-reflection score by 4.69%. Third, concerning equity in benefit, we find that users from the underserved population are at least as likely as users from the better-served population to benefit from using Hope. This suggests that MHMAs are likely to promote equity not only in usage but also in the benefits derived from such usage.

Next, we conduct post-hoc analysis to examine: (i) how the two major functions of Hope—i.e., the self-reflection function for self-support, and the online community function for peer-support—contribute to our main results; and (ii) whether the main results are more nuanced across users from various underserved sub-populations. Regarding the two major functions, we find that our main results hold true for the usage of each function. However, users from the underserved population are marginally more inclined to use the self-reflection function than users from the better-served population. This implies that the benefit of Hope is primarily realized through the self-reflection function for users from the underserved population, whereas for users from the better-served population, it is realized more through the online community function. This highlights the importance for app developers to consider the distinct value proposition of each function, since MHMAs can effectively cater to the awareness of mental health support needs for users from the underserved population and facilitate peer-support for users from the better-served population. Regarding heterogeneity across users from different underserved sub-populations, our results indicate that MHMAs are particularly beneficial for female users from the underserved population (e.g., female African-American, female homosexual, etc.). Compared to users from other underserved sub-populations, these female app users not only use the app more frequently but also derive greater benefits from the app usage. This suggests that MHMAs can be particularly valuable in societies where violence and discrimination against females from the underserved population are prevalent. For other underserved sub-populations, while some (e.g., users with other gender identity, other sexual orientation, and White underserved) use Hope significantly more than the better-served population, equity in benefit is present across all underserved sub-populations.

Our results highlight the potential of MHMAs designed for self-support and peer-support to offer equitable services to individuals with mental health needs from the underserved population. This contribution aligns with the calls for research in Production and Operations Management on identifying emerging technologies that prioritize inclusion and equity for the underserved population (Kalkanci et al., 2019) and social platforms that enhance healthcare operations and promote inclusive healthcare (Cohen et al., 2022; Qiu et al., 2021). The results have operational implications for stakeholders within the mental healthcare ecosystem. In particular, our findings suggest that to ensure equitable delivery of services, (i) MHMA firms should develop apps that include a variety of self-support and peer-support functions; and (ii) organizations (e.g., United Nations, World Health Organization, governments, firms) should expand the delivery of mental health services via MHMAs to “hard-to-reach” patient population.

The remainder of the paper is organized as follows. In Section 2, we present a review of the relevant literature and develop the study hypotheses. In Section 3, we discuss the empirical setting, data, and variables. In Section 4, we present the econometric model specification and report the main results of the empirical analysis. In Section 5, we present the post-hoc analysis results to delve into the more nuanced implications of MHMAs. Finally, in Section 6, we conclude with an overview of the key study findings, practical implications, and future research directions.

2. Literature Review and Hypothesis Development

Below, we review the relevant literature and develop study hypotheses pertaining to the equity of usage and benefit of MHMAs.

2.1 Mental Healthcare Operations Management Literature

In contrast to the well-established body of literature on physical healthcare operations management, the literature on mental healthcare operations management is still at a nascent stage and is comprised of only a few published studies. Among these studies, Yan and Tan (2014) find that social support exchange within a mental health online community can improve individuals’ mental condition. In a follow-up study, Yan et al. (2019) demonstrate that treatment experiences shared by the online community members can influence how a member perceives her own mental health treatment. Zepeda and Sinha (2016) find that enhancing quality and affordability can particularly benefit the underserved population with socio-economic disadvantages. In a recent study, Li et al. (2021) find that eHealth platforms can enable mental health professionals to better-schedule follow-up visits for patients. Although these studies make valuable contributions to the literature on mental healthcare operations management, we are not aware of any study addressing the inequities associated with socio-demographic characteristics of individuals with mental health needs. To advance this emerging stream of literature, our study focuses on the inequities in usage and benefits of mental health services and examines the potential of MHMAs to provide equitable mental health services for the underserved population.

Inequities in mental health service usage and benefits can be attributed to several factors: First, there are significant variations across populations with different socio-demographic characteristics in terms of affordability, accessibility, and awareness (i.e., 3As that are strongly associated with the consumption of healthcare services (Kohnke et al., 2017; Sinha and Kohnke, 2009)) of mental health services (Berger et al., 2005; Beyer, 2020; Saxena et al., 2007; Sussman et al., 1987). For example, compared to the Whites, African-Americans and other minority race-ethnic sub-populations face challenges in affording and accessing mental health services due to lower income and education levels, limited insurance coverage, and geographical barriers (Beyer, 2020). Furthermore, African-Americans exhibit lower awareness regarding the potential need for such services, often considering symptoms of mental illness as normal aspects of their daily life (Sussman et al., 1987). Similarly, males are less likely than females to be aware of their mental health needs, primarily due to the greater difficulties males experience in recognizing and acknowledging their emotional problems (Berger et al., 2005; Davis et al., 2023).

The second contributing factor is the presence of self-stigma and social-stigma associated with seeking care for mental health conditions, which distinguishes it from seeking care for physical health conditions (Masuda et al., 2012; Saxena et al., 2007). Self-stigma refers to the fear of self-disclosing distressing or potentially embarrassing personal information, whereas social-stigma refers to the societal stereotype that viewing individuals seeking mental health services as unpredictable, permanently damaged, incompetent, or threatening (Masuda et al., 2009, 2012). Such stigmas are particularly prominent within the underserved population. For instance, compared to the White individuals, African-American, Asian, and Hispanic/Latino individuals are less likely to acknowledge their mental health needs (Eisenberg et al., 2009; Lipson et al., 2018; Masuda et al., 2009; 2012). Additionally, even if they acknowledge their needs, they have less trust in mental health services (Cooper et al., 2003; Eisenberg et al., 2009). Similarly, compared to female individuals, male individuals are more inclined to perceive mental health treatment as undermining their social power and control (Berger et al., 2005; Davis et al., 2023). Thus, they are less likely to seek mental health services and openly discuss their emotional conditions during treatment (Good et al., 1989; Good and Wood, 1995).

Third, the process of diagnosing and treating mental health conditions primarily relies on verbal communication, making seamless communication between mental health patients and professionals crucial for effective delivery of mental health services (Alegría et al., 2002; Nguyen and Reardon, 2013). Thus, mental health patients often prefer consulting with a health professional who shares similar socio-demographics characteristics such as gender, sexual orientation, or race-ethnicity in order to establish rapport and enhance understanding (Cooper et al., 2003). Otherwise, a socio-demographic mismatch between patients and mental healthcare providers can contribute to inequities in care. The underserved population is more likely to experience such mismatches than the better-served population because there are far fewer mental healthcare professionals from the underserved population than from the better-served population. For instance, in 2015, only 14% of clinical psychologists in the U.S. were Asian, Hispanic, or African-American, whereas the majority were White (Lin et al., 2018).

Finally, discriminatory practices, including denial of care or unfair diagnosis and treatment, can also contribute to inequities in mental health services. For instance, despite a significantly higher mental illness rate of 44.1% among sexual minorities (i.e., lesbian, gay, bisexual, transgender, and questioning individuals) compared to the national average rate of 20.6%, they are at least twice more likely to be denied mental health services than their better-served counterparts (i.e., heterosexual individuals) due to discrimination (NAMI, 2021a). Such discrimination can foster long-term mistrust within the underserved population, which hinders their willingness to seek support. For instance, African-Americans exhibit a tendency to rely less on mental health services due to historical discrimination and mistreatment (Diala et al., 2000), opting to address their mental health issues independently rather than seeking professional help (Alegría et al., 2002).

2.2 Hypothesis Development

We contend that MHMAs have the potential to mitigate the inequities observed in traditional modes of seeking mental health support. First, MHMAs are often free or have low-cost subscription plans. In addition, access to MHMAs is not restricted by geographical, temporal, or physical constraints (Stephens-Reicher et al., 2011), eliminating the commute time and costs for their users. These features make MHMAs more affordable and accessible than in-person mental health services delivered by professionals. Second, MHMAs offer the benefit of anonymity, creating a psychologically safe environment for app users. This allows users who are hesitant or uncomfortable discussing their mental health conditions to freely share their issues with their peers or seek mental health-related information without disclosing personal details such as gender, race-ethnicity, or sexual orientation (Ancis, 2020), thereby minimizing the self-stigma and social-stigma. Third, MHMAs can be designed to support various languages and user interfaces, facilitating effective conversations among users. This design feature allows individuals with similar socio-demographic characteristics and mental health concerns to connect with peers, reducing socio-demographic mismatches on the platform. Lastly, by implementing design features like active monitoring of online communities by human moderators or artificial intelligence algorithms to detect and address discriminatory, harassing, or abusive content, MHMAs can foster high quality interactions, while minimizing the risk of discrimination (Wadden et al., 2021) (i.e., a potentially significant downside that other online platforms such as social media entail (Ancis, 2020)). Overall, these features are likely to motivate individuals, irrespective of their socio-demographic characteristics, to seek mental health support more frequently through MHMAs compared to in-person mental health services delivered by professionals. Because smartphone ownership rates are similar between the underserved and better-served populations (e.g., 85% of whites, 83% of African-Americans, and 85% of Hispanics/Latinos own smartphones in 2021 (Pew Research Center, 2021)) and people with different socio-demographic characteristics show comparable interest in utilizing MHMAs (Anderson-Lewis et al., 2018; Lipschitz et al., 2020; Samarasekera, 2022), we anticipate that the opportunities for accessing mental health support via MHMAs apply equally to both populations. Therefore, we posit the following hypothesis:

Hypothesis 1(H1): The usage of a MHMA is equivalent between users from the underserved population and users from the better-served population.

MHMAs also offer several advantages that can contribute to improved mental condition for users. First, most MHMAs include functions that enable users to track their mental condition over time using visual trend plots. These tools can alert users to any downward trends or significant fluctuations that may indicate a deterioration in their mental condition. By receiving automated alerts, users are likely motivated to seek mental health support expeditiously. Studies have shown that individuals who seek mental health support early are more likely to benefit from the support than individuals who delay seeking support (McGorry et al., 2006; McGorry and van Os, 2013). Additionally, MHMAs facilitate peer-support by enabling users to connect with individuals experiencing similar mental health conditions. Specifically, through MHMAs, peers can learn from each other's experiences, offer immediate feedback, and exchange coping strategies (Yan and Tan, 2014). In conclusion, MHMA usage has the potential to help users significantly reduce loneliness, enhance stress management, and improve depression management (Ancis, 2020), leading to an improved mental condition (Donker et al., 2013). Thus, we posit the following hypothesis:

Hypothesis 2(H2): As the usage of a MHMA increases, users’ mental condition improves.

Figure 1.

Screenshot of Hope, a mental health mobile app (MHMA). (a) Self-reflection function. (b) Online community function.

We infer from our review of the relevant literature that improving the affordability of mental health care within clinic-based services yields the greatest benefits in terms of improving mental condition, particularly for socio-economically disadvantaged communities (Zepeda and Sinha, 2016). Additionally, peer-support has been shown to offer greater mental health benefits to individuals facing higher levels of stress, such as minorities defined by their sexual-orientation (Meyer, 2003) or race-ethnicity (Aneshensel, 2009). Along these lines, we contend that MHMAs, by virtue of their affordability and ability to facilitate peer-support among users, are also likely to contribute towards enhanced mental health benefits, particularly for the underserved population. In fact, the positive impact of MHMAs may even be more pronounced, as certain underserved sub-populations (e.g., males or African-Americans) tend to express their mental health concerns more openly within a mobile app (virtual) environment (McCall et al., 2021; Ritterband, 2021), which is fundamental to realizing the benefits of mental health services (Pennebaker, 1999). As a result, MHMAs can enable individuals from the underserved population to benefit from app usage at least as much as individuals from the better-served population. Thus, we hypothesize:

Hypothesis 3(H3): As the usage of a MHMA increases, the mental condition of users from the underserved population improves at least as much as that of users from the better-served population.

3. Research Design

3.1 Empirical Setting

In this study, we collaborate with a mobile app startup company to understand whether MHMAs can offer equitable self-support and peer-support across populations with different socio-demographic identities. In 2015, the company launched its MHMA, Hope, for public users. Hope was not designed or marketed to target any specific population. Rather, it was accessible by anyone with a smartphone. The Hope user community consists of users who are non-professionals (i.e., Hope is peer-based) and seek mental health support. Hope provides its users with two primary functions for self-support and peer-support:

(i)

The self-reflection function. This function allows users to self-assess their overall mental condition by responding to seven questions in Hope's Mental Condition Survey (refer to Online Appendix B for the survey questions). Once registered on Hope, users can use this function as frequently as they desire. This feature enables users to not only evaluate their current mental condition when needed, but also track the recorded results over time through a visual chart. This chart helps users understand the variations in their mental condition. Figure 1(a) demonstrates an example of the user interface and visual chart of this function.

(ii)

The online community function. This function establishes an anonymous online community within the app. Here, app users can connect and freely engage in peer-support at any time without the need to disclose their physical identities, minimizing the risk of users feeling compelled to present themselves differently for any reason. Within this online community, users can anonymously post to seek or provide help, share their feelings and personal stories, and interact with other users by responding to their posts. Figure 1(b) provides an example of user posts and replies within this function.

3.2 Data

Our study dataset is comprised of user-level socio-demographic and app activity data from 1935 Hope users between May 2015 (i.e., the launch-month of the app) and February 2018. Hope records information on the socio-demographic characteristics of each user during the registration process. In addition, each time a user uses the self-reflection function or interacts with the online community function, Hope generates an activity record with detailed information such as the timestamp, user id, self-reflection scores, or the post/reply texts. Our sample consists of, on average, 5.91 self-reflection records, 1.08 posts, and 7.60 replies per user.

Using these records, we construct an unbalanced panel in which each user has one observation for every Hope's Mental Condition Survey completed using the self-reflection function. Hence, a row in the panel data includes variables for user i who uses the self-reflection function at time j. An important consideration when analyzing mobile app data is user retention. It is likely that some users that have been on a mobile app for a long period will have more data points than others who have been in the app for a shorter period. This discrepancy can introduce sampling bias, as users with more data points will be overrepresented in the sample. To address this issue and mitigate the sampling bias, we remove 92 outliers from the dataset, resulting in a sample of 1843 users. These outliers represent users whose cumulative self-reflection function usage exceeds the 95th percentile (i.e., users who completed more than 22 Mental Condition Surveys and thus are overrepresented in the sample). Note that in this setting, a user must complete at least one Hope's Mental Condition Survey to be included in the panel. Among the 1843 users, 1688 meet this criterion. Thus, for simplicity, we initially conduct our analysis using app activity data from 1688 users. Later, in a different specification in Online Appendix E.1, we consider all 1843 users to evaluate the robustness of our results. Table 1 presents the distribution of the 1843 users with respect to their socio-demographic identities including gender, sexual orientation, and race-ethnicity.

Table 1.

Comparison of socio-demographic characteristics of app users in the study sample.

		Gender			Sexual orientation			Race-ethnicity
	Total	Female	Male	Other	Heterosexual	Homosexual	Other	White	African	Asian	Latino	Other
N	1843	1270	487	86	1235	153	455	1502	67	122	131	21
%	100%	68.91%	26.42%	4.67%	67.01%	8.30%	24.69%	81.50%	3.64%	6.62%	7.11%	1.13%

3.3 Variable Definitions and Measurements

For each observation in our panel, we construct the following variables.

3.3.1 Outcome Variables

Consistent with our hypotheses, we consider two outcome variables:

SRS_ij (Self-Reflection Score) represents a user's mental condition and is operationalized as the score that resulted from the Hope's Mental Condition Survey completed by user i at time j. The survey includes seven questions, each measured on a continuous scale ranging from 0 to 100. Therefore, we construct SRS as a latent variable using those questions. Since these questions are designed and implemented by the partner company, the scale items do not align perfectly with existing scales in the literature. It is important to note, however, that these questions conceptually capture the same underlying constructs as the widely used PHQ-9 (Patient Health Questionnaire with 9 items) and GAD-7 (Generalized Anxiety Disorder scale with 7 items) scales, which are self-administered depression or anxiety disorder diagnostic instruments in mental health clinical practice. To evaluate the reliability of SRS, we conduct an exploratory factor analysis and a confirmatory factor analysis. The results are presented in Online Appendix C. The factor analysis indicates that the theoretical model between the latent variable and the seven scale items fits the data, and the scales are reliable. Thus, we operationalize the latent variable SRS as the weighted average of the seven survey items, where the weights are the standardized loadings in the confirmatory factor analysis.

As an additional step, we conducted a pilot study to assess the face validity of SRS. In this study, we recruited 61 college students at a public research university in the mid-western United States and asked the participants to first answer both PHQ-9 and GAD-7 questionnaires, and then complete Hope's Mental Condition Survey. As detailed in Online Appendix D, we found that SRS is strongly correlated with both PHQ-9 and GAD-7 scales, providing face validity for SRS in measuring the mental condition of Hope users.

AppUsage_ij indicates the frequency of app usage and is operationalized as the natural logarithm of the number of times the app functions (i.e., both the self-reflection function and the online community function) are used by user i within 15 days prior to the Mental Condition Survey completed at time j (excluding the app usage at time j). ⁵ For example, a user who completes the survey twice, writes three posts, and responds to other's posts ⁶ four times within the last 15 days (i.e., AppUsage_ij= log(1 + 2 + 3 + 4)) is considered to have a higher frequency of app usage than a user who completes the survey once, writes two posts, and responds to others’ posts three times within the same time-period (i.e., AppUsage_ij = log(1 + 1 + 2 + 3)). We measure AppUsage_ij by strictly excluding the app usage on the day the Mental Health Condition Survey is completed to ensure that there is no simultaneity between SRS_ij and AppUsage_ij (i.e., akin to using lagged variables). We select the threshold of 15 days for the following reason. The effects of a single mental healthcare session such as therapy tend to diminish over time, necessitating multiple sessions for effective treatment (Austin, 2022; Thompson, 2022). Therefore, mental healthcare typically follows a treatment course involving weekly, bi-weekly, or monthly sessions, depending on the severity of symptoms, nature of issues, and therapy types (Solara Mental Health, 2023). By setting the AppUsage threshold at 15 days, we align with the bi-weekly frequency of mental healthcare delivery. Notably, this choice is consistent with our empirical mobile app setting, as 98.8% of app usages before taking a Mental Condition Survey in our panel occur within 15 days prior to taking that survey. Furthermore, we define AppUsage using different time-periods and validate the robustness of our findings in Online Appendix E.2.

3.3.2 Key Independent Variable

The key independent variable in our study is Underserved_i , which indicates whether user i belongs to the underserved population (Underserved_i= 1) or the better-served population (Underserved_i= 0). Consistent with the literature documenting inequities in mental health services (Berger et al., 2005; Masuda et al., 2009; NAMI, 2021a), we define a user as belonging to the underserved population if they do not identify as a female-heterosexual-White individual. Conversely, female-heterosexual-White users are categorized as representing the better-served population. ⁷

3.3.3 Control Variables

We use several control variables that are potentially related to the outcome variables:

Age_ij represents the age of user i at time j and is used to control for the potential impact of technology usage patterns across different age groups on mobile app usage (Pew Research Center, 2021) and mental condition (SAMHSA, 2022).

InitialSRS_i is the first Self-Reflection Score recorded for user i. We use each user's InitialSRS as the proxy for their initial mental condition when starting using the app. Users with better initial mental condition are likely to have higher app usage frequency. We control for this variable to rule out this potential impact of users’ mental condition before joining the app on their usage behavior.

Tenure_ij indicates a user's membership length and is measured as the number of days between user i's app registration day and time j when the Mental Condition Survey is completed.

AppRemindFrequency_i indicates the number of weekly notifications received by user i. This count variable captures the feature where users have the option to set, between 1 and 7, how many times a week Hope should send a notification to their smartphones to remind users that they can engage with the app.

Year-month fixed effect controls for any common trend and seasonality that might be related to app usage or mental condition. We operationalize this fixed effect using year-month indicators.

Table 2.

Descriptive statistics and the correlation matrix.

	Underserved = 0		Underserved = 1		Correlation matrix
Variable	Mean	SD	Mean	SD	1	2	3	4	5	6	7
1. AppUsage	1.21	1.09	1.38	1.10	1.00
2. SRS	41.42	30.05	40.87	28.36	0.02†	1.00
3. Age	23.70	8.15	23.37	7.82	−0.07***	0.12***	1.00
4. InitialSRS	39.57	28.95	33.88	25.48	−0.10***	0.70***	0.19***	1.00
5. Tenure	22.94	50.28	24.41	58.45	−0.02*	0.14***	0.08***	0.17***	1.00
6. AppRemindFrequency	3.28	2.73	3.94	2.84	0.19***	−0.05***	−0.12***	−0.13***	−0.01	1.00
7. Underserved	-	-	-	-	0.07***	−0.01	−0.02†	−0.10***	0.01	0.11***	1.00
Number of users	648		1040
Number of observations	2628		4814

Note: (i) *** p < 0.001, ** p < 0.01, * p < 0.05, † p < 0.1. (ii) The AppUsage and Tenure variables in this table are repeated observations for users measured at the moment of a specific SRS record and are not the final app usage for users or the tenure measured at the last app activity.

Table 2 presents descriptive statistics and the correlation matrix for the variables used in the study for users from the underserved population (Underserved = 1) and users from the better-served population (Underserved = 0). With a variance inflation factor score mean of 1.05 and a range between 1.02 and 1.09, below the rule-of-thumb cut-off of ten, there is little evidence of multi-collinearity.

4. Empirical Analysis

In this section, we introduce our econometric model specification and present the estimation results.

4.1 Model Specification

In the healthcare literature, the patient outcome is a pivotal outcome variable, holding equal importance for app developers who seek to establish the credibility of their apps towards improving mental condition of app users through usage. Therefore, in our study, we are particularly interested in whether there exist any differences between users from the underserved and better-served populations in: (i) AppUsage and (ii) SRS as a result of AppUsage. This implies that AppUsage is expected to influence SRS, while SRS is not anticipated to affect AppUsage. Following this assumption ⁸ , we specify our econometric model as a recursive system of equations:

[App Usage Equation]

$$\begin{array}{rl}AppUsag{e}_{ij}& ={\alpha }_0+{\alpha }_{1}Underserve{d}_i+{\mathit{U}\mathit{s}\mathit{e}\mathit{r}\mathit{C}\mathit{o}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{l}\mathit{s}}_{\mathit{i}\mathit{j}}{\mathit{\alpha }}_2\\ & +{\mathit{T}\mathit{i}\mathit{m}\mathit{e}\mathit{C}\mathit{o}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{l}\mathit{s}}_{\mathit{i}\mathit{j}}{\mathit{\alpha }}_3+u_i+{\epsilon }_{ij}\end{array}$$

(1)

[SRS (Self-Reflection Score) Equation]:

$$\begin{array}{rl}SR{S}_{ij}& ={\beta }_0+{\beta }_{1}AppUsag{e}_{ij}+{\beta }_{2}Underserve{d}_i\\ & +{\beta }_{3}AppUsag{e}_{ij}\times Underserve{d}_i+{\mathit{U}\mathit{s}\mathit{e}\mathit{r}\mathit{C}\mathit{o}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{l}\mathit{s}}_{\mathit{i}\mathit{j}}{\mathit{\beta }}_{\mathbf{4}}\\ & +{\mathit{T}\mathit{i}\mathit{m}\mathit{e}\mathit{C}\mathit{o}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{l}\mathit{s}}_{\mathit{i}\mathit{j}}{\mathit{\beta }}_{\mathbf{5}}+v_i+{\tau }_{ij}\end{array}$$

(2)

where ${\epsilon }_{ij}$ and ${\tau }_{ij}$ are random error terms. UserControls include Age, InitialSRS, Tenure, and AppRemindFrequency. TimeControls include year-month fixed effects. Lastly, $u_i$ and $v_i$ denote the random effects for user i. Since our interest is to identify the impact of the time-invariant variable Underserved on outcome variables, we specify our model as a random effect model. Later in Section 4.3, we estimate our model using a fixed effect specification and find that our results related to SRS still hold. Note that, in our specification, AppUsage is a mediator and Underserved moderates the relationship between AppUsage and SRS, making our model a moderated mediation model. The coefficient of interest to test H1 is ${\alpha }_1$ in Equation 1. Similarly, the coefficients of interest to test H2 and H3 are ${\beta }_1$ and ${\beta }_3$ in Equation 2, respectively. We specify Equations 1 and 2 as linear models.

4.2 Estimation Results

We estimate our econometric model using maximum likelihood estimation and present the results in Table 3. Model 1 demonstrates the estimation of Equation 2 (the SRS equation) with only control variables. Model 2 builds on Model 1 by adding AppUsage into the SRS equation. Model 3 further enhances Model 2 by including the mediation model for AppUsage, estimating both Equations 1 and 2 simultaneously without the interaction term in Equation 2. Lastly, Model 4 represents the moderated mediation model, derived from Model 3 by adding the interaction term AppUsage × Underserved into the SRS equation.

Table 3.

Main analysis estimation results.

	Model 1	Model 2	Model 3	Model 4
APP USAGE EQUATION
Underserved			0.06	0.06
			(0.04)	(0.04)
Age			−0.01**	−0.01**
			(0.00)	(0.00)
InitialSRS			−0.00	−0.00
			(0.00)	(0.00)
Tenure			−0.00	−0.00
			(0.00)	(0.00)
AppRemindFrequency			0.07***	0.07***
			(0.01)	(0.01)
SELF-REFLECTION SCORE (SRS) EQUATION
AppUsage		2.59***	2.59***	1.91**
		(0.39)	(0.39)	(0.72)
Underserved	1.57*	1.39†	1.39†	0.57
	(0.72)	(0.72)	(0.72)	(0.60)
AppUsage × Underserved				1.01
				(0.85)
Age	0.00	0.02	0.02	0.02
	(0.03)	(0.03)	(0.03)	(0.03)
InitialSRS	0.81***	0.81***	0.81***	0.81***
	(0.02)	(0.02)	(0.02)	(0.02)
Tenure	0.01	0.01	0.01	0.01
	(0.01)	(0.01)	(0.01)	(0.01)
AppRemindFrequency	0.27†	0.08	0.08	0.09
	(0.14)	(0.14)	(0.14)	(0.14)
AIC	63739.42	63612.48	82825.99	82823.51
BIC	64009.1	63889.07	83372.26	83376.71
Likelihood ratio test	-	128.94***	-	4.47*
Number of users	1688	1688	1688	1688
Number of observations	7442	7442	7442	7442

Note: (i) Robust standard errors clustered at the user level in parentheses. (ii) *** p < 0.001, ** p < 0.01, * p < 0.05, † p < 0.1. (iii) User random effects and year-month fixed effects are included in all models.

We examine the significance of the incremental variance due to any added variable in the SRS equation by performing a likelihood ratio test that compares the model with the added variable and the model without the added variable. We use Model 3 to test H1 and H2 because these hypotheses are related to the main effects of Underserved (in the App Usage equation) and AppUsage (in the SRS equation). We test H3 using Model 4 as it includes the interaction term.

With respect to the equity in MHMA usage, the estimated coefficient of Underserved in the App Usage equation in Model 3 ( ${\alpha }_1=0.06,p=0.134$ ) is not statistically significant. Following the recent published literature that establishes racial equity in various contexts through demonstrating statistical insignificance (e.g., Cui et al., 2020; Ganju et al., 2020), this result may suggest equity in app usage between users from the underserved and better-served populations. However, the statistics literature indicates that statistical insignificance can arise due to not only statistical equivalence, but also statistical indeterminance (Tryon, 2001). Therefore, to further examine the source of statistical insignificance in our empirical setting, we conduct a statistical equivalence test (Schuirmann, 1987) by comparing the average app usage between users from the underserved population and the better-served population and find support for statistical equivalence ( $t_1=5.79,$ $p_1=0.00;t_2=2.21,p_2=0.01;$ $\mathrm{\Delta }=0.2{\sigma }_{\overline{AppUsage}}).$ ⁹ Combined with the estimated coefficient ${\alpha }_1$ in Table 3, this result suggests that users from the underserved population use Hope at least as much as the users from the better-served population, supporting H1.

With respect to the benefit of MHMA usage, we find that the estimated coefficient of AppUsage in the SRS equation in Model 3 ( ${\beta }_1=2.59,p<0.001$ ) is positive and statistically significant, supporting H2. Based on the predictive margins of SRS, doubling app usage frequency is associated with an increase of 1.80 unit increase in SRS. For instance, such an increase would correspond to a 4.69% increase in self-assessed mental condition of a user with median SRS in our study sample, testifying the benefit of MHMA usage.

Figure 2.

Estimation results of the user-specific effect of AppUsage on SRS (self-reflection score).

Finally, Model 4 estimation results indicate that the interaction term AppUsage × Underserved ( ${\beta }_3=1.01,p=0.237$ ) is positive but not statistically significant. Note that unlike with the equity in usage in which we seek statistical equivalence between the two groups with respect to the variable AppUsage, the equity in benefit requires statistical equivalence between the two groups with respect to the effect of AppUsage on SRS (i.e., ${\beta }_1$ in Model 3, which is a slope rather than a variable). Thus, to examine the source of the insignificance of the interaction term coefficient, we first re-specify Equation 2 as a random slope model in which the effect of AppUsage on SRS varies between users:

$$\begin{array}{rl}SR{S}_{ij}& ={\beta }_0+{\beta }_{1}AppUsag{e}_{ij}+{\beta }_{2}Underserve{d}_i\\ & +{\mathit{U}\mathit{s}\mathit{e}\mathit{r}\mathit{C}\mathit{o}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{l}\mathit{s}}_{\mathit{i}\mathit{j}}{\mathit{\beta }}_{\mathbf{4}}+{\mathit{T}\mathit{i}\mathit{m}\mathit{e}\mathit{C}\mathit{o}\mathit{n}\mathit{t}\mathit{r}\mathit{o}\mathit{l}\mathit{s}}_{\mathit{i}\mathit{j}}{\mathit{\beta }}_{\mathbf{5}}\\ & +{\zeta }_{0i}+{\zeta }_{1i}AppUsag{e}_{ij}+{\tau }_{ij}\end{array}$$

(3)

where ${\zeta }_{0i}$ denotes the deviation of user i's intercept from the mean intercept of ${\beta }_0$ and ${\zeta }_{1i}$ denotes the deviation of user i's slope for AppUsage from the mean slope of ${\beta }_1$ . With this specification, the effect of AppUsage on SRS for user i is estimated to be ${\beta }_1+{\zeta }_{1i}$ . The significant log-likelihood ratio test statistic (i.e., ${\chi }^2(2)=2833.47$ , p =0.00) obtained from the estimated random slope model (not reported) indicates that the effect of AppUsage on SRS indeed varies between users.

The histogram plots in Figure 2 summarize the estimated $\widehat{{\beta }_1+{\zeta }_{1i}}$ for users from the underserved and better-served populations. Next, similarly to testing H1, we conduct a statistical equivalence test by comparing the estimated $\widehat{{\beta }_1+{\zeta }_{1i}}$ between the two groups of users and find that there is statistical equivalence between them ( $t_1=4.43,$ $p_1<0.001;t_2=3.56,p_2<0.001;$ $\mathrm{\Delta }=0.2{\sigma }_{\widehat{{\beta }_1+{\zeta }_{1i}}}$ ). ¹⁰ Combined with the estimated coefficient ${\beta }_3$ in Table 3, this result suggests that using MHMA improves the mental condition equally for users from the underserved population and users from the better-served population, providing support for H3.

We also conduct several robustness tests and demonstrate in Online Appendix E that our main results are consistent across several model specifications, variable operationalization, and alternative explanations.

5. Post-hoc Analysis

Having established the robustness of our results, we now conduct post-hoc analysis to understand: (i) how the two Hope (MHMA) functions (i.e., the self-reflection and online community functions) contribute to equity in usage and benefit; and (ii) whether this equity holds equally across different underserved sub-populations.

5.1 Self-Reflection Function versus Online Community Function

The two MHMA functions offer distinct types of support to app users. The self-reflection function enables users to monitor their mental condition and increase the awareness of their own mental healthcare needs without interacting with other users. In contrast, the online community function facilitates user interactions and the exchange of peer-support within Hope. It provides a platform for users to connect with one another, offering the opportunity to receive from and provide support to fellow users. Hence, the two app functions can be associated with two different user behaviors, with the self-reflection function emphasizing individual introspection and the online community function emphasizing interaction and peer-support.

To investigate whether there are differences in user behaviors between users from the underserved population and users from the better-served population, we conduct post-hoc analysis. This analysis expands on our main analysis by decomposing the AppUsage variable into two components. In particular, we operationalize SRUsage_ij (Self-Reflection Usage) and OCUsage_ij (Online Community Usage) as the natural logarithm of the number of times the self-reflection function and the online community function, respectively, are used by user i within 15 days prior to the Mental Condition Survey completed at time j. We then replace AppUsage in Equations 1 and 2 with SRUsage and OCUsage, and estimate our main model with each decomposed variable in isolation. Further, there is a potential interplay between SRUsage and OCUsage. Users who engage with the online community through the app may feel happier and more inclined to track their mental condition, leading to increased SRUsage. Similarly, users who frequently use the self-reflection function may be more motivated to engage and assist others, resulting in increased OCUsage. To account for this interdependency, we include the lagged OCUsage variable as a control variable in the SRUsage equation and the lagged SRUsage variable as a control variable in the OCUsageequation.

As is evident in column SRUsage Model 3 in Table 4, we find that users from the underserved population use the self-reflection function marginally more frequently than users from the better-served population ( ${\alpha }_1=0.05,p=0.078$ ). We do not find a similar difference with respect to the usage of the online community function ( ${\alpha }_1=-0.003,p=0.928$ in column OCUsage Model 3). With respect to the impact of the usage of each function on mental condition, we find that the use of both functions is associated with an increase in SRS ( ${\beta }_1=3.58,p<0.001$ in column SRUsage Model 3 vs. ${\beta }_1=1.34,\text{p}=0.005$ in column OCUsage Model 3). However, when we compare these two coefficients in a seemingly unrelated regression model, we find that, for the same frequency of use, the self-reflection function is associated with significantly more increase in SRS than the online community function ( ${\chi }^2=18.18,p<0.001$ ). This suggests that MHMAs are likely to offer value particularly through increasing awareness of users’ mental health needs. When we examine the two functions with respect to the equity in benefit, we find no significant difference between users from the underserved and better-served populations ( ${\beta }_3=1.73,p=0.121$ in column SRUsage Model 4 vs. ${\beta }_3=-0.66,p=0.536$ in column OCUsage Model 4). This suggests that the equity in benefit is present for both functions. However, considering that the coefficients of the two interaction terms have opposite signs, we find in a seemingly unrelated regression model that ${\beta }_3$ in the SRUsage model is significantly greater than ${\beta }_3$ in the OCUsage ( ${\chi }^2=4.77,p=0.03$ ). In other words, users from the underserved population benefit more from using the self-reflection function whereas users from the better-served population benefit more from using the online community function. This suggests that MHMAs are likely to offer value to: (i) users from the underserved population mainly through increasing their awareness of mental health needs and (ii) users from the better-served population more through facilitating peer-support among app users.

Table 4.

Estimation results with decomposed AppUsage variables: SRUsage (Self-Reflection Function Usage) and OCUsage (Online Community Function Usage).

	Decomposed AppUsage
	SRUsage		OCUsage
	Model 3	Model 4	Model 3	Model 4
APP USAGE EQUATION
Underserved	0.05†	0.05†	−0.00	−0.00
	(0.03)	(0.03)	(0.03)	(0.03)
Lagged SRUsage			0.35***	0.35***
			(0.03)	(0.03)
Lagged OCUsage	0.43***	0.43***
	(0.02)	(0.02)
Age	−0.00**	−0.00**	−0.00	−0.00
	(0.00)	(0.00)	(0.00)	(0.00)
InitialSRS	0.00	0.00	−0.00**	−0.00**
	(0.00)	(0.00)	(0.00)	(0.00)
Tenure	−0.00*	−0.00*	−0.00	−0.00
	(0.00)	(0.00)	(0.00)	(0.00)
AppRemindFrequency	0.05***	0.05***	0.00	0.00
	(0.01)	(0.01)	(0.01)	(0.01)
SELF-REFLECTION SCORE (SRS) EQUATION
SRUsage	3.58***	2.41*
	(0.51)	(0.96)
OCUsage			1.34**	1.79†
			(0.47)	(0.91)
Underserved	1.34†	0.22	1.54*	1.75*
	(0.71)	(0.60)	(0.72)	(0.69)
SRUsage × Underserved		1.73
		(1.12)
OCUsage × Underserved				−0.66
				(1.06)
Age	0.02	0.02	0.01	0.00
	(0.03)	(0.03)	(0.03)	(0.03)
InitialSRS	0.81***	0.81***	0.81***	0.81***
	(0.02)	(0.02)	(0.02)	(0.02)
Tenure	0.01	0.01	0.01	0.01
	(0.01)	(0.01)	(0.01)	(0.01)
AppRemindFrequency	0.04	0.04	0.24†	0.24†
	(0.14)	(0.14)	(0.14)	(0.14)
AIC	78225.29	78218.75	78130.36	78131.21
BIC	78778.48	78778.86	78683.55	78691.32
Likelihood ratio test	-	8.54**	-	1.15
Number of users	1688	1688	1688	1688
Number of observations	7442	7442	7442	7442

Note: (i) Robust standard errors clustered at the user level in parentheses. (ii) *** p < 0.001, ** p < 0.01, * p < 0.05, † p < 0.1. (iii) User random effects and year-month fixed effects are included in all models.

In summary, our post-hoc analysis of the two app functions reveals that the variety in mental health support offerings through MHMAs is likely to be key towards ensuring equity in usage and benefits. Such variety allows users from both underserved and better-served populations to leverage each of the two MHMA functions in ways that best align with their specific mental health needs.

5.2 Equity Across Users From Different Underserved Sub-Populations

In our main analysis, we treat all users belonging to the underserved population (e.g., African-American, Asian, Hispanic/Latino, etc.) as similar in terms of the inequities they may face regarding usage and benefit of mental health services, without distinguishing between different socio-demographic characteristics. However, there is anecdotal evidence suggesting that the degree of inequity may vary across different underserved sub-populations. For instance, it is reported that African-American and Asian patients experience more discrimination in mental health services than Hispanic/Latino patients (Horowitz et al., 2019). In order to obtain nuanced insights, we now extend our analysis to examine potential variations across users from different underserved sub-populations within a MHMAsetting.

To identify users from different underserved sub-populations, we use three socio-demographic variables corresponding individually to: gender (male, female, other gender), sexual orientation (heterosexual, homosexual, other sexual orientation), and race-ethnicity (white, african, asian, latino, other race-ethnicicty). In Equations 1 and 2, we substitute the variable Underserved with a categorical variable indicating different underserved sub-populations and the better-served population. For instance, when identifying users from underserved sub-populations based on gender, the categorical variable indicates whether a user belongs to the better-served population (i.e., female, heterosexual, White individual), the female-underserved sub-population (e.g., a female African-American), the male-underserved sub-population, or the other-gender-underserved sub-population (e.g., Hispanic individuals who do not associate themselves with any gender). We handle sexual orientation and race-ethnicity in similar ways. We then estimate our recursive system of equations (i.e., Model 4) with each of the three categorical variables (with users from the better-served population being the reference level) and present the results in Table 5.

Table 5.

Estimation results for different underserved sub-populations.

	Identification of constituent underserved sub-populations
	Gender (Column 1)		Sexual orientation (Column 2)		Race-ethnicity (Column 3)
APP USAGE EQUATION
Socio-demographic characteristics:
	Female-Underserved	0.07	Heterosexual-Underserved	−0.02	White-Underserved	0.08†
		(0.05)		(0.05)		(0.04)
	Male-Underserved	0.02	Homosexual-Underserved	0.06	African-Underserved	0.09
		(0.05)		(0.08)		(0.13)
	Other-Underserved	0.25*	Other-Underserved	0.15**	Asian-Underserved	0.03
		(0.10)		(0.05)		(0.08)
					Latino-Underserved	0.01
						(0.09)
					Other-Underserved	−0.15
						(0.18)
Age		−0.01*		−0.01*		−0.01**
		(0.00)		(0.00)		(0.00)
InitialSRS		−0.00		−0.00		−0.00
		(0.00)		(0.00)		(0.00)
Tenure		0.00		0.00		−0.00
		(0.00)		(0.00)		(0.00)
AppRemindFrequency		0.07***		0.07***		0.07***
		(0.01)		(0.01)		(0.01)
SELF-REFLECTION SCORE (SRS) EQUATION
AppUsage		1.91**		1.91**		1.91**
		(0.72)		(0.72)		(0.72)
Socio-demographic characteristic main effects		Estimated		Estimated		Estimated
AppUsage × Socio-demographic characteristics:
	Female-Underserved	1.90†	Heterosexual-Underserved	1.04	White-Underserved	1.18
		(1.00)		(0.99)		(0.90)
	Male-Underserved	0.18	Homosexual-Underserved	1.47	African-Underserved	1.91
		(0.95)		(1.67)		(1.61)
	Other-Underserved	0.35	Other-Underserved	0.86	Asian-Underserved	−0.33
		(1.62)		(0.98)		(1.83)
					Latino-Underserved	0.65
						(1.68)
					Other-Underserved	−2.49
						(2.57)
Age		0.01		0.02		0.01
		(0.03)		(0.03)		(0.03)
InitialSRS		0.81***		0.81***		0.81***
		(0.02)		(0.02)		(0.02)
Tenure		0.01		0.01		0.01
		(0.01)		(0.01)		(0.01)
AppRemindFrequency		0.10		0.09		0.09
		(0.14)		(0.14)		(0.14)
AIC		82818.76		82826.28		82838.94
BIC		83413.44		83420.96		83475.11
Number of users		1688		1688		1688
Number of observations		7442		7442		7442

Note: (i) Robust standard errors clustered at the user level in parentheses. (ii) *** p < 0.001, ** p < 0.01, * p < 0.05, † p < 0.1. (iii) User random effects and year-month fixed effects are included in all models.

From these results, the following two insights stand out. First, regardless of how we identify underserved sub-populations, we consistently find across all three model estimation results that none of the users from these sub-populations use and benefit from the MHMA significantly less than users from the better-served population. ¹¹ This suggests that the inequity against the underserved population reported in the prior literature for clinic-based mental health services (NAMI, 2021a, Terlizzi and Norris, 2021) is unlikely to be present in the context of MHMAs for users from any of the underserved sub-populations examined. Second, the results reported in the first column of Table 5 indicate that MHMAs might be particularly effective for female users within the underserved population (e.g., female African-American, female homosexual, etc.). The relationship between app usage and self-reflection score is marginally stronger for these users compared to users from the better-served population. We do not find such nuanced variation for the equity in benefit for users from other underserved sub-populations, despite some of them (i.e., users from underserved sub-populations identified by other gender identity, other sexual orientation, and White race-ethnicity) using MHMAs more frequently than users from the better-served population. In summary, our analysis reveals some variations in the usage and benefit of MHMA-enabled mental health support across users from different underserved sub-populations. ¹² Nonetheless, these variations are unlikely to result in inequity in usage and benefit against users from any underserved sub-populations when compared to users from the better-served population.

6. Conclusion

6.1 Overview

In this paper, we evaluate the potential of MHMAs to provide equitable self-support and peer-support mental health services across users from the underserved and better-served populations. It is well-documented in the literature that the traditional clinic-based mental health services cannot be systematically accessed by certain sub-populations based on their socio-demographic characteristics such as gender, sexual orientation, and race-ethnicity, leading to inequities in usage of and benefit from mental health services (NAMI, 2021a, Terlizzi and Norris, 2021). COVID-19 has further highlighted these inequities (Wang et al., 2020). Anecdotal evidence suggests that MHMAs that facilitate self-support and peer-support have the potential to offer equitable mental health support to the underserved population (Aziz et al., 2022; Emerson et al., 2021). Our study examines this potential using longitudinal user-level data on socio-demographic characteristics and app activity data collected from a MHMA. By way of research design, we estimate a recursive system of equations to assess whether equity in MHMA usage and benefit exists between users from the underserved and better-served populations. Further, we investigate: (i) whether users from the underserved and better-served populations use the two MHMA functions—namely, the self-reflection function and the online community function—similarly or differently; and (ii) whether there exists any difference in equity in MHMA usage and benefits from such usage across users from different underserved sub-populations.

6.2 Contributions to the Literature

By way of contributions, our study provides, to the best of our knowledge, the first theoretically-grounded empirical analysis that highlights the potential of MHMAs in delivering equitable mental health services—in the form of self-support and peer-support—to the underserved population. Specifically, we examine equity with respect to two user outcomes, namely the frequency of app usage (i.e., equity in usage) and mental condition (i.e., equity in benefit). With respect to equity in usage, we find that users from the underserved population exhibit MHMA usage frequencies statistically equivalent to those from the better-served population. This suggests that the inequity in clinic-based mental health service usage reported in the literature (Berger et al., 2005; Masuda et al., 2009; NAMI, 2021a) is unlikely in MHMAs designed particularly for self-support and peer-support. In addition, our results indicate that MHMA usage frequency has a positive impact on users’ mental condition. For instance, considering users with a median self-reflection score from our sample, doubling their app usage frequency (i.e., from 0.2/day to 0.4/day) can lead to a 4.69% increase in self-reflection score. With respect to equity in benefit, we find that MHMAs are equally beneficial for users from both underserved and better-served populations, as our results indicate that the association between usage frequency and mental condition is statistically equivalent between the two populations. It is worth noting that these novel insights with regards to equity in mental health support offered by MHMAs are relevant even when we consider specific underserved sub-populations (e.g., African-American underserved, female underserved, homosexual underserved, etc.) in isolation. As such, our post-hoc analysis demonstrates that, when considered in isolation, users from none of the underserved sub-populations experience lower usage and benefit from MHMAs compared to users from the better-served population. Conversely, users from certain underserved sub-populations (e.g., female African-American, female homosexual, etc.) tend to derive greater benefit from MHMAs compared to users from the better-served population.

To shed light on how MHMAs uphold equity in usage and benefit between users from the underserved and better-served populations, we conduct a post-hoc analysis to compare the usage of the app functions (i.e., the self-reflection and the online community functions) between the two groups of users. The results suggest that equity in MHMA usage and benefit is likely to be driven by the heterogeneity in users’ preferences for different app functions. Specifically, we find that users from the underserved population use the self-reflection function marginally more than users from the better-served population. Therefore, users from the underserved population derive benefit from MHMAs more through the use of the self-reflection function, whereas users from the better-served population derive benefit from MHMAs more through the use of the online community function. The self-reflection function enables users to develop self-awareness of their mental health needs and can be used individually without the need for interaction with other users. The online community function allows users to exchange peer-support with each other and requires virtual interactions. Given the distinct nature of these two functions, it is conceivable that MHMAs will likely offer value to the underserved population primarily by enhancing their self-awareness of mental conditions and needs. In contrast, MHMAs will likely offer value to the better-served population primarily by facilitating a psychologically safe environment for peer-support interactions.

6.3 Practical Implications

A significant implication of our study findings is that MHMAs designed to deliver self-support and peer-support mental health services have the potential to address the well-documented inequity across the underserved and better-served populations in the traditional clinic-based mental healthcare settings. As detailed in Online Appendix F, fundamental to this implication is the identification of two types of relationship between the app users and the traditional mental health patient population. Both relationships can shed light on our results and suggest distinct implications of our study findings for stakeholders within the mental healthcare ecosystem.

First, MHMA users can inherently be different from the population seeking traditional clinic-based professional support. Hence, MHMA users can be those who do not seek traditional clinic-based professional support, yet seek peer-support and self-support help on their own in the app. This implies that MHMAs may expand the mental health services (through self-support and peer-support) to a new population segment who otherwise would not seek traditional clinic-based professional support. This explanation highlights the value of MHMAs in providing access to the “hard-to-reach” mental health patient population in the traditional clinic-based mental healthcare delivery settings, which can have several practical implications when combined with our study results. For instance, several global organizations such as United Nations and WHO consider improving access and advancing equity as the core guiding principle for achieving sustainable mental healthcare delivery (World Health Organization, 2013). Our study findings suggest that by way of supporting initiatives directed towards enhancing access and advancing equity, global organizations and local governments should consider promoting and investing in enhancing the use and access to MHMAs within the underserved population, particularly in societies where there is high level of discrimination and violence against underserved females. Next, notwithstanding the potential for significant lost earnings on account of mental health-related disabilities of employees, many organizations are still hesitant to provide mental healthcare benefits and resources to their employees due to high upfront costs (Brodey, 2021). In organizations that do provide such benefits and resources, employees with most needs (e.g., those from the underserved population) tend to avoid using mental health services due to social stigma and fear of losing their jobs (Brodey, 2021), resulting in significant wastage of resources. Our study suggests that MHMAs can be an alternative value-added resource for organizations due to their relatively low upfront costs and the anonymity that MHMAs can offer to employees regardless of their socio-demographic characteristics. By expanding mental health services via MHMAs to employees who might not use mental health benefits in the traditional clinic-based delivery settings otherwise, organizations can enhance employee welfare, reduce wastage of investments in providing mental health benefits, and minimize lost earnings due to mental health-related disabilities.

Second, MHMA users can inherently be the same as the population seeking traditional clinic-based professional support. This implies that the self-support and peer-support functions of MHMAs can complement the delivery of mental health services for those seeking traditional clinic-based professional support. This scenario highlights the value of MHMAs in providing an alternative channel for seeking help and potentially facilitating behavioral changes among underserved population with the following implications. Specifically, our results demonstrate that MHMAs are likely to achieve equity by providing variety in app functions such as self-support and peer-support functions, which can induce different user behaviors. Therefore, mobile app firms should design apps with a variety of functions including both self-support and peer-support. Self-support functions such as Hope's self-reflection function can enable users to track their mental condition and conduct skill-training functions (i.e., to teach users coping or thinking skills) without interacting with other users (NIH, 2019). Peer support functions such as Hope's online community function enable users to seek informational support or emotional support by virtually connecting and interacting with other app users. Next, to ensure safety and effectiveness of the app functions, governmental regulatory agencies such as the United States Food and Drug Administration have formed new divisions to develop protocols and guidelines for mobile app firms. Our study suggests that, to ensure the delivery of equitable mental healthcare services to the underserved population, in addition to safety and effectiveness, regulatory agencies should also consider developing market standards for designing MHMAs that encompass functions aligning with diverse behavioral needs.

All in all, regardless of these two likely relationships between the app users and the traditional mental health patient population, our results highlight that MHMAs may have the potential to address equity in mental healthcare delivery to the underserved population. Understanding how this potential can be realized will require further research. In particular, there is a need to investigate the interaction between MHMAs and the traditional clinic-based mental healthcare setting to shed light on questions such as: (i) how MHMAs can advance equity in mental health service delivery and (ii) whether adopting MHMAs (versus not adopting MHMAs) changes mental health patients’ access to and use of the traditional clinic-based mental health services. Being the first to highlight the potential of MHMAs designed to provide self-support and peer-support services, we believe our study will motivate future inquiries addressing the above two questions that have consequential health, social, and economic implications for populations at large.

6.4 Limitations and Future Research Directions

As with any paper, our paper, too, has limitations. However, notwithstanding the limitations, our paper provides the motivation and lays the groundwork for future research aimed at sustaining equitable mental health service delivery, as discussed above. First, our empirical setting consists of a MHMA operated by a mobile app firm that does not involve mental health professionals. By involving professionals, MHMAs may offer more benefits by enabling them to detect at-risk patients based on their app activities and provide timely interventions and treatments. Yet, MHMAs operated by professionals may also provide users with the feeling of close monitoring, and thus exacerbate their feelings of stigma and inequity. Therefore, future research should: (i) examine how the impact on equity in usage and benefit of MHMAs changes with or without involving professionals; and (ii) identify conditions, including detailed patient characteristics, under which MHMAs should be integrated into the traditional mental health services while continuing to improve equity in usage and benefit.

Second, while we find empirical support for a positive relationship between MHMA usage and a user's mental condition, we do not investigate how mobile app firms can increase MHMA usage. A direction of future research could be to explore the effectiveness of different strategies (e.g., monthly subscription, sending notifications, using certain incentives) to improve usage of both the self-support and peer-support functions.

Third, in addition to mental condition tracking and peer engagement, it is conceivable that MHMAs can be leveraged to offer proven therapies such as cognitive behavioral therapy or acceptance commitment therapy to treat mental health disorders. A direction for future research could be to investigate whether these MHMA-enabled therapies improve mental conditions as well as advance equity and inclusion in care delivery to the underserved population.

Fourth, users from the underserved population account for more than 50% of all users in the MHMA that serves as the empirical setting of our study. This proportion is typically greater than the proportion of patients from the underserved population in the traditional clinic-based mental healthcare setting (NAMI, 2021a) potentially for the following reasons: (i) the underserved population consider MHMAs to be attractive, given their effectiveness in providing access to “hard-to-reach” mental health patients; and/or (ii) the better-served population are likely to be less interested in seeking help via MHMAs. Future research could explore why the underserved population is overrepresented in MHMA settings.

Finally, as with many observational studies on online communities (e.g., Yan et al., 2019; Yan and Tan, 2014), our study relies on self-reported data on socio-demographic characteristics. Since MHMA users do not have to disclose their physical identities because the online communities in MHMA settings are anonymous, they are likely to report their socio-demographic characteristics accurately. Despite this conjecture, future research could explore what factors (e.g., distrust in any online platform, need for role playing, etc.) may motivate (if any) MHMA users to report their socio-demographic characteristics inaccurately.

Supplemental Material

sj-pdf-1-pao-10.1177_10591478231224914 - Supplemental material for Towards Achieving Mental Health Equity for the Underserved Population: Evaluating the Potential of Mobile Apps

Supplemental material, sj-pdf-1-pao-10.1177_10591478231224914 for Towards Achieving Mental Health Equity for the Underserved Population: Evaluating the Potential of Mobile Apps by Yi Tang, Kingshuk K Sinha, Adam Moen and Necati Ertekin in Production and Operations Management