An Evaluation of 5-Year Web Analytics for HeadsUpGuys: A Men’s Depression E-Mental Health Resource

Abstract

The present study reports an evaluation of web analytics, over a 5-year period, for HeadsUpGuys.org, an eHealth resource for men with depression. Google Analytics, Search Console, and Tag Manager were used to monitor user activity over the course of the website’s first 5 years (June 15, 2015–June 15, 2020). Through this period, HeadsUpGuys had a total of 1,665,356 unique users, amounting to 1,948,481 sessions and 3,328,258 page views. Organic traffic accounted for the highest proportion (53.44%; n = 1,041,277) of website sessions. Four of the top 10 Google search queries that brought users to the website related to suicidality. Three countries (United States, United Kingdom, Canada) accounted for almost three-quarters (71.10%; n = 1,385,485) of the site’s traffic. Nearly three-quarters (73.35%; n = 1,429,285) of sessions occurred on a mobile device. The goal conversion rate for the Self Check was 60.27%. The average time on page was 2 min 53 s, with a bounce rate of 65.92%, and an exit rate of 57.20%. The goal conversion rate for the Stress Test was 52.89%. The average time on page was 4 min 8 s, with a bounce rate of 72.40% and an exit rate of 48.88%. The conversion rate for the final goal was 11.53%, indicating that approximately one in 10 visitors to the site had a session of at least 3 min. The findings illustrate the potential of eHealth resources to support men’s mental health and provide a real-world benchmark to help advance the men’s eHealth field.

Keywords

men’s depression suicidality men’s e-mental health website Google Analytics Google Search Console Google Tag Manager

Introduction

Suicide is a leading cause of death of men worldwide (Naghavi, 2019). One of the strongest risk factors for suicide is depression (Crump et al., 2014). Although epidemiological data indicate that women are disproportionately diagnosed with depression (Kuehner, 2017), global prevalence estimates indicate that a substantial number of men are affected by depression (Ferrari et al., 2013). For example, according to findings from the 2017 U.S. National Survey on Drug Use and Health and 2018 U.S. population estimates, approximately 8 million American men suffered from depression (National Institute of Mental Health, 2019; Smith et al., 2018; U.S. Census Bureau, 2013). In addition to being a risk factor for suicide, depression now is the leading cause of disease burden (World Health Organization, 2017).

Men’s high rates of depression and suicide are juxtaposed with their low rates of psychological help-seeking (Oliffe et al., 2019; Rice et al., 2020). Indeed, accumulating evidence attests to men’s reluctance to seek help for mental health concerns (Seidler et al., 2016). Especially worrisome is that help-seeking among men with more severe depression have been reported to be particularly impaired (Rice et al., 2017). Repeatedly implicated in these low levels of mental health help-seeking are traditional masculine ideals, including strength, self-reliance, and stoicism, that frame seeking help as the embodiment of weakness, dependence, and vulnerability (Addis & Mahalik, 2003; Yousaf et al., 2015). Psychological help-seeking has been understood to be in direct conflict with prevailing masculine ideals (Johnson et al., 2012; Seidler et al., 2016). Also implicated in men’s reticence to engage with “in person” mental health care are the fragmented pathways to specialist services (Strike et al., 2006). Referral processes and fee for service issues routinely result in long wait times and also pose barriers to disadvantage many men (especially those in crisis). Concerns about stigma amplifying the effects of being known to need and having accessed mental health care services pose significant barriers to “in person” help-seeking for men who experience depression (Oliffe et al., 2016). Considering the profound social and economic burden of men’s untreated mental illness (Roche et al., 2016; White et al., 2011) and globally high male suicide rates, there is an urgent need to develop and evaluate resources tailored to support men’s mental health (Seidler et al., 2018).

Rapid growth in the area of eHealth represents a new frontier for delivering tailored health interventions to men (Deady et al., 2020; Linardon et al., 2019). Some research has reported that young men in particular have a strong preference for web-based health information and interventions (Ellis et al., 2013; Fridrici & Lohaus, 2009). There has been a wide variety of men’s eHealth programs tailored to address weight loss (Young & Morgan, 2018), smoking cessation (Bottorff et al., 2016), prenatal health education (Mackert et al., 2018), fathering (Da Costa et al., 2017), sexual health (Hirshfield et al., 2019), prostate cancer (Forbes et al., 2019), and health behaviors (Oliffe, Black, et al., 2020). There have been few sustained efforts for developing eHealth programs specifically oriented to men with depression (Fogarty et al., 2017; Murphy et al., 2018), despite reviews concluding that eHealth interventions for depression show promise (Deady et al., 2017; Luo et al., 2020; Massoudi et al., 2019). Addressing this gap, Cheng and colleagues (2020) argue there is a need for further development of targeted eHealth initiatives proactively aimed at men to advance their mental health management and engagement with care service (Rice et al., 2018).

Building on the promise of eHealth resources for providing men anonymity in accessing information about depression, a team of clinicians, researchers, and mental health advocates developed the HeadsUpGuys website (headsupguys.org) (Ogrodniczuk et al., 2018). Following an intensive development process that involved focus groups, surveys, and individual interviews of men with lived experience of depression and suicidality, this free online resource was launched in June 2015, offering men information, practical tips, and guidance for managing and recovering from depression (see Box 1 for a list of the website’s features). The resource provides a male-friendly medium through which to start the help-seeking process (see Figure 1). It was designed to capitalize on men’s desire for independence, autonomy, and preference for self-sufficiency, while also building in messaging to norm and affirm men’s connections to peer and/or professional help. By building a laddered approach that normalizes help-seeking, HeadsUpGuys intended to bridge men’s tendency to self-manage their mental health (Zanchetta et al., 2017) with augmenting and/or directing to more formal care resources. The language used on the site is purposefully workaday, rather than technical/medically-oriented, amid a clear call to action that positions effective self-management as a manly strength and normative value.

Box 1.

Features of the HeadsUpGuys Website.

The HeadsUpGuys website, headsupguys.org , contains the following:
• Information about men’s depression and suicide, including potential risk factors and triggers, as well as common misconceptions about depression among men (e.g., depression is a sign of personal weakness)
• A Self Check screening tool for depression (PHQ-9), including directives for action when the user is provided with the screening score results
• A Stress Test that covers 20 different stressors for which users rate the intensity of the stressor, duration of the stressor, and perceived capacity to manage the stressor (feature added in February 2019)
• Practical tips for self-management, focusing on the topics of sleep, stress management, social life, physical activity, food, and sex and relationships
• Health and crisis lines that men can reach out to for help
• Information about professional services, including psychotherapy, medications, and inpatient services
• Guidance about what to do if a crisis (i.e., heightened suicide risk) arises
• Advice on how to reach out to others, including friends, family, and health professionals
• Testimonials (stories of recovery, practical tips) and YouTube videos from men who have recovered from depression
• Guidance for supporters of men living with depression
• Links to social media outlets (Facebook, Twitter, and Instagram) to encourage user engagement

Note. PHQ-9 = Patient Health Questionnaire-9 (Kroenke et al., 2001).

Figure 1.

Screenshot of HeadsUpGuys Home Page as It Appears on a Desktop Device.

The objective of the present study was to conduct an evaluation of web analytics for HeadsUpGuys over the 5-year period since its launch, the first study of its kind to examine user engagement of a men’s eHealth resource across an extended duration. The evaluation, following the process evaluation model established by Song and colleagues (2018), utilized Google Analytics, Search Console, and Tag Manager data focusing on (a) user engagement, including number of visits, visit duration, bounce rates, most visited pages; (b) traffic sources, including traffic filters, country sources, city sources; and (c) goal conversion (i.e., specific interactivity targets). Findings of real-world usage of web-based mental health resources are important to report, as a recent review of e-interventions in randomized controlled trials revealed that more than half did not state their methods for recording web usage and only 5% reported Google Analytics data (Koneska et al., 2020). Usage statistics of popular online heath resources such as Healthline and WebMD are virtually nonexistent in the research literature. Thus, another objective the study was to share real-world data from HeadsUpGuys as a baseline to map progress over time and afford other eHealth resources a comparator.

Methods

Google Analytics

Google Analytics was used to access website data for HeadsUpGuys from its launch on June 15, 2015, to its 5-year anniversary on June 15, 2020. Data derived from Google Analytics did not contain any personally identifiable information and are presented in aggregate form, making it an accessible and ethical tool for research. Linked to the footer of the HeadsUpGuys website, the “Terms, Conditions, and Privacy” page outlines how information about users is collected, and how visitors can opt-out of data collection.

Before launching, a Google Analytics account was set up, verifying ownership of our domain. JavaScript tracking code created by Google Analytics was then added to the HeadsUpGuys website (loaded on every page). This code permits the collection of various forms of data related to website user behavior. These data include the user’s browser, geographic location, and device type (desktop vs. mobile), along with information about the user’s interactions with the website, including pages visited, length of the session, and channels used to access the platform (e.g., Google search, social media, and email link). This information is then accessible through a real-time, interactive dashboard that can be accessed by logging into the registered account on the Google Analytics website.

The “Demographics and Interests Reports” was enabled in Google Analytics, thus providing additional information about users who are also logged into their Google accounts (Google Chrome Browser, YouTube, Gmail, Chromebook laptop devices, Android mobile devices) when visiting websites. Information about user age and gender was derived in this way. As such, website statistics using age and gender are based on a subset of website visitors.

Google Search Console

Google Search Console was used to access data about search terms used in Google Search that brought users to the HeadsUpGuys website. Google Search Console data are currently available up to 16 months before the date the data are accessed. To access Google Search Console, an account was created, with the domain having already been verified through Google Analytics. Through Google Search Console, information is provided about the terms/phrases (“queries” or “keywords”) users utilized in Google Search before visiting the HeadsUpGuys site. Google Search Console also provides information on queries, clicks, impressions (how many times a website was included in search results presented to the user), click-through rate (how often a site was clicked for each query), and position (where the website was ranked/listed in Google Search results). Google Search Console also provides information on searches by page accessed.

Google Tag Manager

Google Tag Manager was used alongside Google Analytics to track events (important actions on a website), including time spent on the site, when a form is submitted, clicks on links (specified by the website owner), file downloads, and other interactions a user may have with a website. Goal conversion reflects visits where an event (also referred to as goals) occurred. For the present study, Google Tag Manager was used for tracking three conversion goals: Self Check submissions (see Box 2 for a description of the Self Check), Stress Test submissions (see Box 3 for a description of the Stress Test), and sessions to the site that lasted more than 3 min. The HeadsUpGuys Self Check is a web-based interactive version of the Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001), a well-established self-report measure of depressive symptomatology representing the nine Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; American Psychiatric Association, 2013) criteria for major depressive disorder. The PHQ-9 has been validated extensively and is commonly used across research and clinical contexts (Beard et al., 2016; Keum et al., 2018; Kocalevent et al., 2013). The HeadsUpGuys Stress Test is a proprietary tool developed specifically for the website. The purpose of the Stress Test was to get visitors to reflect on diverse aspects of their lives that may be contributing to their depression or may be impacted by their depression. Visitors provided informed consent for the collection of their anonymous Self Check and Stress Test responses. Collection of anonymous Self Check and Stress Test data from visitors was approved by the University of British Columbia Behavioural Research Ethics Board (H13-02811; H17-01334).

Box 2.

HeadsUpGuys Self Check: Over the Last 2 Weeks, How Often Have You Been Bothered by Any of the Following Problems?

	Several days	More than half the days	Nearly every day
1. Little interest or pleasure in doing things	1	2	3
2. Feeling down, depressed, or hopeless	1	2	3
3. Trouble falling or staying asleep, or sleeping too much	1	2	3
4. Feeling tired or having little energy	1	2	3
5. Poor appetite or overeating	1	2	3
6. Feeling bad about yourself—or that you are a failure or have let yourself or your family down	1	2	3
7. Trouble concentrating on things, such as reading the newspaper or watching television	1	2	3
8. Moving or speaking so slowly that other people could have noticed? Or the opposite—being so fidgety or restless that you have been moving around a lot more than usual	1	2	3
9. Thoughts that you would be better off dead or of hurting yourself in some way	1	2	3

Box 3.

HeadsUpGuys Stress Test: List of Stressors Included in the Stress Test.

Personal illness or injury	Difficulties at work
Illness or injury of close family member	Difficulties at school
Loss of social status	Separation or divorce
Difficulties with alcohol, tobacco, cannabis, or other substance use	Issues with sexual performance
Retirement	Death of a pet
Loneliness	Death of a spouse
Lack of purpose or meaning in life	Death of a loved one (family member, close friend)
Loss of employment	Being bullied or cyberbullied
Financial strain	Relationship problems with spouse/romantic partner
Move to a new neighborhood	Birth of a child/adoption of a new child
	Other

Stress Test item ratings:

Over what period has this stressor been affecting you? [duration]

Week Month Several Months Year Multiple Years

How intense is the stress caused by this issue? [intensity]

Minimally stressful 1 2 3 4 5 Extremely stressful

How well are you able to manage this stress? [manageability]

Not managing at all 1 2 3 4 5 Managing very well

User Engagement

Several indicators of user engagement were examined for this study. These included the number of sessions and users, number of returning users, number of pages accessed per session, mean session duration, bounce rate, and most visited pages. We also examined number of sessions by age and gender.

A session refers to a visit by a unique user to the website, including one or more page views or interactions with the site. A user refers to a unique visitor to the site, and can have one or more sessions associated with it. A unique client ID is assigned and added to a cookie in the user’s browser and device when they first visit the site. Returning users were estimated using the number of sessions visited with unique client IDs.

The number of pages per session refers to the number of web pages of the site that the user viewed in a single session. The mean session duration (minutes, seconds) refers to the time the users spent on the website.

A bounce is a session that triggers only a single request to the Google Analytics server. If a user loads a page but does not interact a second time (by viewing another page or triggering an event) within 30 min, this is considered a bounce. Bounce rate is the percentage of single-page sessions the site/page receives.

The most visited pages were observed in terms of their unique page views, average time on page, exit rate (proportion of sessions ending from a given page), and bounce rate.

Traffic Sources

Traffic refers to the number of sessions or users visiting the website. Traffic sources reflect how the user arrived to a website (e.g., search engines, social media, referral from other sites, direct entry). Google Analytics separates traffic sources with several filters, depending on the source of the traffic and/or if the link was tagged with additional information. Top queries (i.e., search terms) and top pages (i.e., pages accessed) for search traffic was also examined.

Other aspects of traffic source include the country and city from which a user accesses the website, and the type of device used (desktop, tablet, mobile).

Goal Conversion

Goal conversation rate reflects the proportion of users who came to the Self Check or Stress Test web pages and completed the corresponding Self Check or Stress Test, or the proportion of website sessions that were 3 min or longer.

Results

Engagement

Through its first 5 years of operation, HeadsUpGuys had a total of 1,665,356 users, amounting to 1,948,481 sessions and 3,328,258 page views (see Figure 2). Figure 3 reports the change in number of sessions over the 5-year period, revealing a progressive increase over that period of time. One in seven visits (14.49%; n = 282,250) was from a returning user. On average, users visited 1.71 pages per session, with an average session duration of 1 min 21 s. The average bounce rate was 71.55%.

Figure 2.

HeadsUpGuys Overview Presented in Google Analytics.

Figure 3.

Sessions by Year.

The top pages by page views are listed in Table 1. Not surprisingly, the home page received the most page views (18.51% of all page views; n = 615,930). The second most viewed page (12.18%; n = 405,340) was the Self Check page, on which users could complete a depression screening tool. Among the top 10 pages, three were blog articles (Five Steps to Overcoming Suicidal Thoughts; Marijuana and Depression; I Never Wanted to Die, I Only Wanted to End My Pain), all with an average time on page more than 3 min, and bounce and exits rates in excess of 85%. The practical tips page (the access point to a range of self-help tips) stood out as having the lowest bounce and exit rates.

Table 1.

Top 10 Pages by Page Views.

	Page	Page views^a	Unique page views^b	Avg. time on page^c	Entrances^d	Bounce rate (%)^e	Exit (%)^f
1.	Homepage	615,930 (18.51%)	512,545	01:54	496,556	47.97	50.48
2.	Self Check	405,340 (12.18%)	355,614	02:53	174,247	65.92	57.20
3.	Five Steps to Overcoming Suicidal Thoughts	250,785 (7.54%)	234,421	03:25	228,336	85.08	85.34
4.	Symptoms	248,547 (7.47%)	223,679	02:12	125,977	73.78	58.42
5.	Marijuana and Depression	135,508 (4.07%)	128,974	05:23	127,811	88.95	92.39
6.	Practical Tips	108,234 (3.25%)	77,045	00:25	8,655	54.05	15.48
7.	Depression in Men	102,956 (3.09%)	84,687	01:11	38,577	47.00	34.10
8.	I Never Wanted to Die, I Only Wanted to End My Pain	100,718 (3.03%)	95,503	04:49	94,890	89.39	90.86
9.	22 Male Athletes Speaking Out About Depression	93,471 (2.81%)	86,010	08:25	85,072	84.24	89.98
10.	Take Action	78,376 (2.35%)	65,026	00:55	8,199	66.12	27.49

Page views: Number of times a page from the website is loaded (or reloaded) in a user’s browser (one user visiting a page multiple times will result in multiple page views). ^b Unique page views: Number of page views by unique users to the site (one user visiting the same page multiple times will result in one unique page view). ^c Avg. Time on Page: The average amount of time a session lasts on a page, before the user switches to another page. ^d Entrances: Number of times a user’s session begins on a page. ^e Bounce rate: The percentage of single-page sessions a page received (the percentage of visits to the site, where a user leaves from the same page they entered on, without visiting another page, or triggering an event such as a form submission). ^f Exit %: The percentage of users who left the website from a page (the last page visited by a user, before they leave the website).

One quarter (25.39%; n = 422,900) of users were logged into a Google-related service/account when visiting HeadsUpGuys, thus providing data about their gender and age. With regard to gender, a little over half of the sessions (52.31%; n = 261,142) were by men who viewed 1.86 pages per session, spent an average time of 1 min 39 s per session, and had an average bounce rate of 67.63%. Just under half (47.69%; n = 238,103) of the sessions were from women who viewed 1.63 pages per session, spent an average time of 1 min 14 s per session, and had an average bounce rate of 75.89%. Concerning age, more than half the users (56.40%; n = 267,376) were under 35 years old (see Table 2). Bounce rate decreased with age, while pages per session and average session duration increased with age.

Table 2.

Sessions by Age.

Age	Sessions^a	% New sessions^b	New users^c	Bounce rate^d	Pages / session^e	Avg. session duration^f	Self Check submission (%)^g
18–24	93,896	85.69	80,459	77.19%	1.52	0:01:10	3.68
25–34	173,480	84.11	145,912	72.99%	1.72	0:01:25	4.12
35–44	86,830	83.88	72,837	68.21%	1.86	0:01:34	4.71
45–54	64,051	84.32	54,009	66.34%	1.94	0:01:40	4.98
55–64	37,634	84.45	31,782	65.44%	2.01	0:01:48	4.78
65+	19,211	85.08	16,344	64.96%	2.02	0:01:51	5.02

Sessions: Number of visits by unique users to the website (each session can include one or more page views or interactions). ^b % of New Sessions: Proportion of sessions by people entering the website for the first time, as opposed to returning visitors. ^c New Users: A unique visitor to the site that doesn’t have any previous sessions associated with it. ^d Bounce rate: The percentage of single-page sessions a page received (the percentage of visits to the site, where a user leaves from the same page they entered on, without visiting another page, or triggering an event such as a form submission). ^e Pages/Session: The number of pages visited within a single session. ^f Avg. Session Duration: The amount of time from when a session is started until the last interaction (event) with the website before the user leaves the site. ^g Self Check Submission: The percentage of sessions that included a Self Check submission.

Traffic Sources

Organic traffic accounted for the highest proportion (53.44%; n = 1,041,277) of all website sessions (see Table 3). Paid search (i.e., from paid ads on search engines such as Google Search Ads) accounted for another quarter (26.60%; n = 518,343) of the sessions, and had the lowest bounce rate of all traffic sources. Traffic via referral (i.e., from links on other websites, such as those that link to us as a resource, or from an online media/news feature), while representing only 3.35% of sessions (n = 65,260), had the highest average time on page, most page views, and second lowest bounce rate.

Table 3.

Traffic by Source.

	Source	Page views^a	Unique page views^b	Avg. time on page^c	Entrances (%)^d	Bounce rate (%)^e
1.	Organic Search	1,041,277 (53.44%)	1.45	0:01:04	87.63	81.15
2.	Paid Search	518,343(26.60%)	1.99	0:01:38	86.57	53.26
3.	Direct	187,676(9.63%)	2.06	0:01:52	77.39	69.48
4.	Social	112,353(5.77%)	1.69	0:01:15	80.68	75.95
5.	Referral	65,260(3.35%)	2.68	0:02:26	75.01	57.43
6.	(Other)	22,625(1.16%)	1.33	0:00:48	86.39	84.82
7.	Display	561(0.03%)	1.48	0:00:39	78.25%	82.53
8.	Email	386(0.02%)	1.87	0:01:43	68.91	70.73

Table 4 presents the top 10 queries from Google searches that brought users to the website. Four of the top 10 queries relate to suicidality. Among these, the query “how to stop suicidal thoughts” had the highest search position (4.58) and the highest click-through rate (20.38%; n = 11,886). On average, the website received 2,045 clicks per month from searches relating to suicidality.

Table 4.

Google Search Traffic: Top Queries Ranked by Clicks.

	Query	Impressions^a	Clicks^b	Click-through rate (%)^c	Position^d
1.	suicidal thoughts	469,137	13,536	2.89	8.57
2.	how to stop suicidal thoughts	58,333	11,886	20.38	4.58
3.	I want to kill myself	449,636	11,782	2.62	7.17
4.	heads up guys	9,386	7,246	77.20	1.04
5.	headsupguys	8,135	6,493	79.82	1.02
6.	weed and depression	24,289	5,387	22.18	2.95
7.	how to deal with depression	303,907	4,869	1.60	9.01
8.	does weed help with depression	26,151	4,684	17.91	3.10
9.	how to cope with depression	144,594	4,335	3.00	7.71
10.	how to deal with suicidal thoughts	23,014	3,358	14.59	5.36

Impressions: Number of times any URL from the site appears in Google Search results, viewed by a user (not including from paid ads). ^b Clicks: Number of clicks on a URL from the site appearing on Google Search results page (not including from paid ads). ^c Click-through rate: The proportion of clicks received per impressions. ^d Position: The average ranking of the website’s URLs for the search terms (with 1 being the first website listed at the top search results).

The top 10 pages of the website that appear in Google searches are reported in Table 5. Corresponding to the above findings regarding search queries, four of the top 10 website pages concerned articles about suicidality. The web page containing the article “Five Steps to Overcoming Suicidal Thoughts” had the highest number of search clicks (20.37%; n = 141,538) and the page containing the article “How to Stop Thinking About Suicide” had the highest click-through rate (10.60%; n = 22,936).

Table 5.

Google Search Traffic: Top Pages Ranked by Clicks.

	Query	Impressions^a	Clicks^b	Click-through rate (%)^c	Position^d
1.	Five Steps to Overcoming Suicidal Thoughts	1,871,085	141,538	7.56	10.79
2.	Marijuana and Depression	1,234,446	118,552	9.60	13.14
3.	I Never Wanted to Die, I Only Wanted to End My Pain	1,499,756	58,162	3.88	12.66
4.	22 Male Athletes Speaking Out About Depression	513,828	49,337	9.60	16.03
5.	I Wanted to Kill Myself, But I Survived	1,304,674	30,228	2.32	9.47
6.	Symptoms	1,618,581	23,409	1.45	14.51
7.	How to Cope With Depression	1,015,110	23,317	2.30	11.52
8.	How to Stop Thinking About Suicide	216,349	22,936	10.60	11.72
9.	Homepage	208,569	19,940	9.56	31.86

Users of the HeadsUpGuys website came from many places across the world, but three countries (United States, United Kingdom, Canada) accounted for nearly three-quarters (71.10%; n = 1,385,485 visitors) of the website traffic (see Table 6). The United States alone provided just under a third (29.53%; n = 575,465 visitors) of traffic to the site.

Table 6.

Traffic by Country.

	Source	Sessions^a	% New sessions^b	New users^c	Bounce rate (%)^d	Pages / session^e	Avg. session duration^f
1.	United States	575,465 (29.53%)	88.18	507,422	79.16	1.49	0:01:04
2.	United Kingdom	414,194 (21.26%)	85.59	354,522	64.20	1.75	0:01:23
3.	Canada	395,826 (20.31%)	79.64	315,235	63.38	2.19	0:01:58
4.	India	101,717 (5.22%)	86.29	87,771	77.21	1.54	0:01:06
5.	Australia	96,792 (4.97%)	87.05	84,260	71.92	1.63	0:01:12
6.	Ireland	49,556 (2.54%)	88.12	43,668	61.25	1.8	0:01:20
7.	Philippines	40,980 (2.10%)	86.94	35,628	72.96	1.58	0:01:19
8.	South Africa	25,879 (1.33%)	88.01	22,775	77.16	1.53	0:01:08
9.	New Zealand	22,068 (1.13%)	87.89	19,395	66.40	1.74	0:01:19
10.	Pakistan	18,519 (0.95%)	85.86	15,900	69.43	1.73	0:01:27
11.	Germany	12,424 (0.64%)	84.45	10,492	77.00	1.63	0:01:24
12.	Malaysia	10,024 (0.51%)	86.06	8,627	84.17	1.31	0:00:56
13.	Netherlands	9,262 (0.48%)	87.84	8,136	77.86	1.56	0:01:05
14.	Nigeria	9,002 (0.46%)	82.60	7,436	76.78	1.5	0:01:25
15.	Singapore	7,998 (0.41%)	86.51	6,919	82.45	1.43	0:01:04
16.	Indonesia	6,198 (0.32%)	84.22	5,220	83.17	1.33	0:01:03
17.	France	5,448 (0.28%)	88.22	4,806	78.52	1.46	0:01:07
18.	Sweden	5,337 (0.27%)	89.28	4,765	83.51	1.36	0:00:51
19.	Mexico	4,943 (0.25%)	84.10	4,157	76.69	1.55	0:01:13
20.	Poland	4,685 (0.24%)	83.78	3,925	72.61	1.68	0:01:27

Figure 4 presents traffic by city. Representing more than a tenth of all website traffic, the top five cities included London, England (4.97%; n = 96,798 sessions); Toronto, Canada (2.28%; n = 44,427 sessions); Vancouver, Canada (2.00%; n = 38,918 sessions); Dublin, Ireland (1.61%; n = 31,437 sessions); and Sydney, Australia (1.32%; n = 25,656 sessions). Users from Vancouver, Canada, had the lowest bounce rate (56.46%), most pages per session (3.05), and the longest average session duration (3 min 10 s).

Figure 4.

Traffic by City

Nearly three-quarters (73.35%; n = 1,429,285) of sessions occurred on a mobile device, with a bounce rate of 72.87%, 1.56 pages per session, and an average session duration of 1 min 9 s. Just under a quarter of sessions came by way of a desktop (22.32%; n = 434,959), with a bounce rate of 68.14%, 2.13 pages per session, and an average session duration of 1 min 54 s. The remainder of sessions occurred on a tablet device (4.32%; n = 84,237), with a bounce rate of 66.86%, 1.99 pages per session, and an average session duration of 1 min 44 s.

Goal Conversion

With regard to goal conversion, the first goal considered Self Check completions; a total of 214,329 Self Checks were completed. The goal conversion rate was 60.27% (n = 214,329 completions from 355,614 unique page views). This represents six in 10 visitors to the Self Check page completing the Self Check. The average time on page was 2 min 53 s, with a bounce rate of 65.92% and an exit rate of 57.20%. Organic search accounted for 88.39% (n = 217,622) of Self Check page views. The top search queries that directed users to the Self Check page included “depression test” (26.82%; n = 66,039), “depression symptoms” (13.00%; n = 32,016), “depression” (8.85%; n = 21,794), “depression quiz” (5.44%; n = 13,405), and “am I depressed” (4.70%; n = 11,580). As reported in Table 3, Self Check submission rates increased with age of website visitors. In the first year of operation, the Self Check received an average of 30 submissions per day; in the fifth year, it received 230 submissions per day. Table 7 presents the distribution of Self Check scores according to the PHQ-9 scoring instructions developed by Kroenke and colleagues (2001), revealing that 78.6% (n = 279,512) of Self Check completions scored above the threshold for moderate depression. For Item 9 (the suicidality item), 57.9% of Self Check completers (n = 124,096) scored above 0, indicating at least some suicidal ideation; with 16.2% (n = 34,721) indicating suicidal ideation nearly every day.

Table 7.

Distribution of Self Check (PHQ-9) Scores (n = 214,329).

Self Check score	N	Percentage (%)
0	1,094	0.51
1–4 Minimal depression	10,198	4.76
5–9 Mild depression	34,625	16.16
10–14 Moderate depression	53,913	25.15
15–19 Moderately severe depression	58,772	27.42
20–27 Severe depression	55,727	26.00

Note. According to PHQ-9 scoring instructions (Kroenke et al., 2001). PHQ-9 = Patient Health Questionnaire-9.

The Stress Test, a feature added in February 2019, was completed 28,523 times. The goal conversion rate was 52.89% (28,523 submissions from 53,933 unique page views), indicating that just over half the visitors to the Stress Test page completed the Stress Test. The average time on page was 4 min 8 s, with a bounce rate of 72.40% and an exit rate of 48.88%. Since the launch of the Stress Test, it has received an average of 63 submissions per day. Figure 5 reports the five most frequently endorsed stressors by visitors who completed the Stress Test, revealing that lack of purpose or meaning in life and loneliness as the two stressors that were endorsed by more than half the Stress Test completers.

Figure 5.

Most Frequently Endorsed Stressors From Stress Test (n = 28,523).

The final goal considered for this study concerned sessions to the site that lasted more than 3 min. This goal was set in Google Tag Manager in October 2018; thus the time period considered for this goal was October 2018 to June 2020. There were 123,792 goal completions during this period, with a goal conversion rate of 11.53%. This means that approximately one in 10 visitors to the site had a session of at least 3 min.

Discussion

Principal Findings

There has been considerable development in the area of eHealth as a means to delivering tailored interventions for men. Yet, there has been little focus on developing eHealth programs specifically for men with depression, and by extension little is known about the uptake and usage patterns of potential end-users of such programs. The objective of the present study was to conduct an evaluation of web analytics, over a 5-year period, for HeadsUpGuys.org, an eHealth resource for men with depression. Through its first 5 years of operation, HeadsUpGuys had a total of 1,665,356 unique users, amounting to 1,948,481 sessions and 3,328,258 page views. Given the ubiquitousness of smartphones, perhaps it is not surprising that nearly three-quarters of the visitor sessions occurred on a mobile device. Being one of very few men’s eHealth resources focused specifically on depression and the first to report on 5-year user engagement data, it is difficult to position the findings in a relative perspective as the literature lacks direct comparators. Despite the lack of direct comparators, the reported user engagement findings compare favorably with other eHealth resources in some regards, for example, overall visitor traffic (King et al., 2019; Whiteside et al., 2019), return visitors (Jeong et al., 2019), and goal conversion (Murphy et al., 2018), but trailed in other metrics, for example, bounce rate (King et al., 2019) and average number of pages per session (Song et al., 2018). Such comparisons need to be considered with caution as significant discrepancies among the studies and eHealth resources exist (much like comparing apples to oranges). While most research on men’s mental health help-seeking has focused on “in person” clinical and community-based services (Seidler et al., 2016), the present study underscores the need to widen conceptualizations of gendered help-seeking to include men’s mental eHealth practices. Aided by the anonymity and absence of reliance of interpersonal relations (i.e., patient–physician), eHealth resources can assist men’s help-seeking. We suggest that key to the volume and engagement of visitors is HeadsUpGuys’ language and content. Some principles (and caveats) for men’s community-based health resources likely prevail in the eHealth space in this regard, including using plain language, norming the experience of depression among men, and being action-orientated, along with the value of men’s testimonials for norming help-seeking as a means to effectual self-management (Oliffe, Rossnagel, et al., 2020). Taking this a step further, the success of HeadsUpGuys rewrites (and perhaps overwrites) the long-standing tropes regarding men’s reticence for seeking mental health care in pointing to robust growth and strong goal conversion over 5 years. Retelling this narrative in and of itself norms men’s eHealth help-seeking to proactively reconsider where and how men engage mental health resources (Oliffe, Broom, et al., 2020).

Organic search traffic accounted for over half of all website sessions. The most obvious benefit of organic search traffic is a cost advantage, in that there are no direct acquisition costs for these visitors. The high organic search traffic also testifies to the growing credibility of HeadsUpGuys as a reliable mental health resource, especially considering that Google does not allow paid ads to target suicide. Suicidality featured prominently in the organic search traffic, representing four of the top 10 queries that brought users to the site, and correspondingly four of the top 10 pages that were accessed by those arriving by way of organic searches. The linkages between men’s depression and suicidality are well established, as are assertions that many men who experience suicidal ideation do not disclose those thoughts to others (Cleary, 2012). The predominance of suicidality search terms in the current study reveals many men’s eHealth help-seeking as candid, deliberate, and perhaps free of the constraints imposed by social- and self-stigmas for making such admissions directly to another person (Oliffe, Rossnagel, et al., 2020) or health care professional (Wide et al., 2011). Despite the need to get upstream of men’s suicidality as the lever for their mental health help-seeking, there are clearly some eHealth advantages that allow (and perhaps norm) men’s confidential suicidality search disclosures as a mechanism to formally self-evaluating and/or accessing self-management strategies.

The worldwide reach of HeadsUpGuys is evident in the array of countries represented. The predominance of American end-users might reflect barriers invoked by fee-for-service health care models, and the inevitable inequities flowing from that insurance and benefits based system. Indeed, the costs of, and fragmented pathways toward professional mental health care services amplify stigmas and gross social inequities (Livingston, 2020), heightening depression risk and barriers to care for vulnerable male subpopulations (Oliffe et al., 2019). The global reach of HeadsUpGuys also likely points to the limited availability of tailored eHealth resources that support men’s mental health. The high proportion of traffic that the website receives from organic searches may also speak to the relative absence of mental health resources for men; many individuals may end up at HeadsUpGuys because they perceive there to be few other credible eHealth options that speak directly to men’s mental health.

Outside of the home page, which serves as a landing and launch pad to other sections of the site, the most visited page was the Self Check page that contains the depression screening tool. Goal conversion associated with the Self Check was high relative to other reports of similar metrics (Song et al., 2018), with 6 in 10 visitors to the page completing the Self Check. Besides providing users of the Self Check with a score and prompts for action following completion, it was also used as an opportunity to inform visitors of the symptoms of depression and, in this way, help improve their mental health literacy around depression, further contributing to the usefulness of this particular website feature and underscoring the importance of the high conversion rate. Being one of the few pages with an interactive component, the findings are consistent with other reports noting interactivity being a preferred feature of eHealth resources (Smail-Crevier et al., 2019; Wang et al., 2016). Of the more than 200,000 Self Check completions, nearly 80% scored above the threshold for moderate depression, providing robust evidence that the site was drawing in those from its targeted market (i.e., men experiencing depression). In addition, more than half of the Self Check completers endorsed at least some suicidal ideation, which resonates with the findings of suicide-related pages being among the most visited on the site and with suicidality featuring prominently in the organic search traffic. These latter findings relating to suicidality point to the need for further resource development within HeadsUpGuys and beyond to support men who struggle with thoughts of ending their lives.

Although the Stress Test was a relatively new feature of the site, findings indicated good engagement by way of total number of completions (nearly 30,000) and high goal conversion (just over 50%). As with the Self Check, these findings may lend further evidence to the role of interactivity as a useful feature for stimulating visitor engagement. The Stress Test results are also revealing, with lack of purpose or meaning in life and loneliness emerging as significant stressors for more than half of the respondents. Considering the impact that meaning/purpose in life (Kealy et al., 2020; F. Li et al., 2016) and loneliness (Cox et al., 2020; Lee et al., 2021) each have on mental health, these findings point to targets for future content development for HeadsUpGuys and research to further understand their relationships to depression and suicidality among men.

Blog articles addressing specific issues of life with and recovery from depression were also among the top pages accessed. The high traffic to the HeadsUpGuys’ marijuana and depression web page, as one of these pages, is interesting to consider as it likely points to the legalization of cannabis in Canada and elsewhere mustering consumer explorations regarding the mental health values and risks of what has been traditionally positioned as a recreational drug. Trailing behind the legalization is evidence about the role of marijuana on men’s depression and anxiety. By opening up the conversation on HeadsUpGuys through blog entries, experiences and preliminary empirical insights can be shared to guide end-users about important considerations in deciding to opt in or out using marijuana for depression, anxiety, and other mental illness challenges.

In terms of the effectiveness of men’s eHealth interventions, it is fair to say that there has been a lack of empirical evidence to claim attribution. We suggest that the high volume of traffic affirms the perceived acceptability of the site to visitors, with the large proportion of organic search traffic indicating the site’s “fit” for visitor needs compared with other similar resources ranked by Google Search algorithms (with high page ranking for key terms related to “suicide”). Goal conversion rates point to important visitor engagements through the completion of site-defined meaningful actions (that aren’t tracked through default Google Analytics settings). There are no set “rules” when it comes to website conversions—conversions are unique to each website. Goal conversion is commonly used for commercial websites where a specific goal to convert a visit to a sale, for example, is easily and clearly defined. Defining goal conversions for an information-sharing website like HeadsUpGuys is more difficult. Of the three goals that were defined for the present study, two (Self Check completions, Stress Test completions) referred to specific visitor actions that reflected clear and important interactions with the website. The third goal, having a session that lasted more than 3 min, was less specific to a particular action on the site. In the absence of guiding standards, 3 min was specified for this goal because it is roughly the amount of time it takes to complete the Self Check, which was a major focus of visitor engagement. In retrospect, this goal was defined too broadly.

The age spread of visitors also reflects men’s depression and suicidality across the life course in confirming the wide reach and acceptability of the language, content, and interactivity of HeadsUpGuys. Acceptability of the site may also extend to female visitors, as just under half of the visitors who were logged into a Google-related service/account when visiting HeadsUpGuys were identified as “Female” in Google Analytics. Women (and those of other genders) may visit the site for a variety of reasons, including looking for information for a man in their lives. For example, 72.79% of visits to the “For Supporter -> Provide Support” page were identified as “Female” in Google Analytics. This is consistent with our expectation that female partners, mothers, and sisters of men would be the primary visitors to this section of the site. In addition, considering that females may also benefit from the content on the site, one can interpret the large portion of female visitors to the site as being reflective of the site’s acceptability to them.

We agree with emergent literature suggesting that the duration of men’s visits or bounce rates do not necessarily indicate relevant engagement (Yardley et al., 2016). It is hard to give an estimate of what a “successful” bounce rate is, as it will vary greatly depending on which page visitors enter a website. It may well be that men are able to quickly find what they want on HeadsUpGuys and move on or return for updates as needed. Furthermore, switching devices, logging in or out of accounts, clearing cookies, and viewing in private browser modes can result in “new visitors” when, in fact, it may be the same individual returning to a site. It is not possible to more accurately categorize these visitors through Google Analytics. A user-specific log-in would be needed to more accurately track user visits and get a clearer sense of the proportion that are true returning visitors. It has been suggested that metrics such as high bounce and return visit rates may be too conservative in evaluating the actual impact of eHealth interventions on users (Paschall et al., 2011). As Lo and colleagues (2020) indicate, there does not seem to be a standard as to what constitutes high or low engagement for digital mental health interventions, and there is currently no guidance on how to maximize the value of analytics data.

A significant blind spot in researching and understanding men’s mental health help-seeking resides in and around men’s preferences for self-management. While self-reliance has been reported to heighten suicide risk (Pirkis et al., 2017), eHealth strategies can be tailored to capitalize directly on men’s desire for independence and agency. Drawing on the transtheoretical model of change (Prochaska & DiClemente, 1983) contemplative men (as in contemplating seeking professional help) might access HeadsUpGuys’ content, such as “5 steps to overcoming suicidal thoughts,” as a means to building strategies for self-management, as well as understanding the potential limits of those efforts in bridging to professional help. This is especially important to consider in the current COVID-19 context with physical distancing restrictions changing the way men manage their health and engage with help services (Ogrodniczuk et al., 2021); Canadian virtual mental health consults, for example, have increased by 750% (Centre for Addiction and Mental Health, 2020). In this regard, HeadsUpGuys’ virtual platform, and the content messages about building “your” team of mental health resources norm and orientate men’s wider eHealth help-seeking, and in doing so, HeadsUpGuys may augment as well as be a gateway to men finding professional care.

Limitations

The findings of this study should be considered in the context of various limitations. As noted above, there are no guidelines for the use and interpretation of Google Analytics to demonstrate “success” of an eHealth platform. Google Analytics conforms to a marketing perspective of web-based behavior rather than to a comprehensive evaluation of impact on users’ health, behavior, or objectives (Clark et al., 2014). Thus, some analytic information may not be entirely relevant or valid for use in an eHealth context. In the absence of such guidelines, the various analytics may be used to observe trends in usage across different time periods where continual evaluation of the platform is encouraged. Another limitation is that the number of users may be inaccurate as a new client ID is given every time the user deletes the browser cookies, switches devices, or uses a different browser. Similarly, with regard to the calculation of average session duration, Google Analytics assigns a value of 0 s when a user visits a page but does not visit a second page or trigger an event. As such, the reported average session duration of 1 min 21 s is a very conservative estimate. An additional limitation is that the findings relating to age and sex were based only on a subset of visitors (i.e., those who were logged into their Google accounts when visiting the site). Finally, the study did not solicit users’ perspectives of the website regarding satisfaction with the site, whether it met their goals for visiting the site, or perceived impact of the site on their mental health literacy, self-stigma, or help-seeking behaviors. Future investigations will need to attend to such issues to provide a more comprehensive picture of HeadsUpGuys’ effect on its users.

Conclusion

Considering men’s low uptake of in-person mental health services and the increased risk of suicide among men with untreated depression, it is crucial to establish alternate avenues of engagement, especially for those men who might be isolated from other sources of support in their daily lives (Fogarty et al., 2017). The present study focused on a specific eHealth program, HeadsUpGuys, that was tailored to men experiencing depression, of which there are few examples in the literature. The 5-year review, the first of its kind for any men’s eHealth resource, revealed a high and rising volume of users, global reach, and good engagement (at least by some metrics, e.g., goal conversion). That there are no commercial interests underpinning HeadsUpGuys, rather an interest in sharing temporally fluid information and expertise, reveals the attraction of authentic conversations to catalyze men’s informed self-management. This is especially important in the context of men’s mental health in which self-reliance preferences can be satiated by encouraging men to consider, choose, and build upon a variety of health advancing strategies. Reflecting end-users’ patterns of engagement, the findings can help inform approaches to designing content and evaluating men’s mental eHealth resources. Taken together, the present study illustrates the potential of eHealth resources to support men’s mental health and provides some guidance to advancing the men’s eHealth field.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding received from Movember Canada (Grant No. LQMS, 11R18296)

Ethics Approval Statement

Ethics approval was provided by the University of British Columbia Behavioural Research Ethics Board (H13-02811; H17-01334).

Informed Consent Statement

Participants provided informed consent for the collection of their anonymous Self Check and Stress Test responses.

ORCID iD

John S. Ogrodniczuk

John L. Oliffe

References

Addis

M. E.

Mahalik

J. R.

(2003). Men, masculinity, and the contexts of help seeking. American Psychologist, 58(1), 5–14. https://doi.org/10.1037/0003-066X.58.1.5

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). American Psychiatric Publishing. https://doi.org/10.1176/appi.books.9780890425596

Beard

Hsu

K. J.

Rifkin

L. S.

Busch

A. B.

Björgvinsson

(2016). Validation of the PHQ-9 in a psychiatric sample. Journal of Affective Disorders, 193, 267–273. https://doi.org/10.1016/j.jad.2015.12.075

Bottorff

J. L.

Oliffe

J. L.

Sarbit

Sharp

Caperchione

C. M.

Currie

L. M.

Schmid

Mackay

M. H.

Stolp

(2016). Evaluation of QuitNow Men: An online, men-centered smoking cessation intervention. Journal of Medical Internet Research, 18(4), Article e83. https://doi.org/10.2196/jmir.5076

Centre for Addiction and Mental Health. (2020). CAMH enhances virtual capacity to respond to demand for mental health services. https://www.camh.ca/en/camh-news-and-stories/camh-enhances-virtual-capacity

Cheng

V. W. S.

Davenport

Johnson

Vella

Mitchell

Hickie

I. B.

(2020). Naturalistic evaluation of a sport-themed mental health and wellbeing app aimed at men (MindMax), that incorporates applied video games and gamification. Internet Interventions, 20, 100306. https://doi.org/10.1016/j.invent.2020.100306

Clark

D. J.

Nicholas

Jamali

H. R.

(2014). Evaluating information seeking and use in the changing virtual world: The emerging role of Google Analytics. Learned Publishing, 27(3), 185–194. https://doi.org/10.1087/20140304

Cleary

(2012). Suicidal action, emotional expression, and the performance of masculinities. Social Science and Medicine, 74(4), 498–505. https://doi.org/10.1016/j.socscimed.2011.08.002

Cox

D. W.

Ogrodniczuk

J. S.

Oliffe

J. L.

Kealy

Rice

S. M.

Kahn

J. H.

(2020). Distress concealment and depression symptoms in a national sample of Canadian men: Feeling understood and loneliness as sequential mediators. The Journal of Nervous and Mental Disease, 208(6), 510–513. https://doi.org/10.1097/NMD.0000000000001153

10.

Crump

Sundquist

Winkleby

M. A.

(2014). Sociodemographic, psychiatric and somatic risk factors for suicide: a Swedish national cohort study. Psychological Medicine, 44(2), 279–289. https://doi.org/10.1017/S0033291713000810

11.

Da Costa

Zelkowitz

Letourneau

Howlett

Dennis

C. L.

Russell

Grover

Lowensteyn

Chan

Khalifé

. (2017). HealthyDads.ca: What do men want in a website designed to promote emotional wellness and healthy behaviors during the transition to parenthood? Journal of Medical Internet Research, 19(10), Article e7415. https://doi.org/10.2196/jmir.7415

12.

Deady

Choi

Calvo

R. A.

Glozier

Christensen

Harvey

S. B.

(2017). eHealth interventions for the prevention of depression and anxiety in the general population: A systematic review and meta-analysis. BMC Psychiatry, 17(1), Article 310. https://doi.org/10.1186/s12888-017-1473-1

13.

Deady

Glozier

Calvo

Johnston

Mackinnon

Milne

Choi

Gayed

Peters

Bryant

Christensen

Harvey

S. B.

(2020). Preventing depression using a smartphone app: A randomized controlled trial. Psychological Medicine. Advance online publication. https://doi.org/10.1017/S0033291720002081

14.

Ellis

L. A.

Collin

Hurley

P. J.

Davenport

T. A.

Burns

J. M.

Hickie

I. B.

(2013). Young men’s attitudes and behaviour in relation to mental health and technology: Implications for the development of online mental health services. BMC Psychiatry, 13(1), Article 119. https://doi.org/10.1186/1471-244X-13-119

15.

Ferrari

A. J.

Somerville

A. J.

Baxter

A. J.

Norman

Patten

S. B.

Vos

Whiteford

H. A.

(2013). Global variation in the prevalence and incidence of major depressive disorder: A systematic review of the epidemiological literature. Psychological Medicine, 43(3), 471–481.

16.

Fogarty

A. S.

Proudfoot

Whittle

E. L.

Clarke

Player

M. J.

Christensen

Wilhelm

(2017). Preliminary evaluation of a brief web and mobile phone intervention for men with depression: Men’s positive coping strategies and associated depression, resilience, and work and social functioning. JMIR Mental Health, 4(3), Article e33. https://doi.org/10.1017/S0033291712001511

17.

Forbes

C. C.

Finlay

McIntosh

Siddiquee

Short

C. E.

(2019). A systematic review of the feasibility, acceptability, and efficacy of online supportive care interventions targeting men with a history of prostate cancer. Journal of Cancer Survivorship, 13(1), 75–96. https://doi.org/10.1007/s11764-018-0729-1

18.

Fridrici

Lohaus

(2009). Stress-prevention in secondary schools: Online- versus face-to-face-training. Health Education, 109(4), 299–313. https://doi.org/10.1108/09654280910970884

19.

Hirshfield

Downing

M. J.

Chiasson

M. A.

Yoon

I. S.

Houang

S. T.

Teran

R. A.

Grov

Sullivan

P. S.

Gordon

R. J.

Hoover

D. R.

Parsons

J. T.

(2019). Evaluation of sex positive! A video eHealth intervention for men living with HIV. AIDS and Behavior, 23(11), 3103–3118. https://doi.org/10.1007/s10461-019-02498-5

20.

Jeong

Cheng

St-Jean

Jalali

(2019). Evaluation of eMentalHealth.ca, a Canadian mental health website portal: Mixed methods assessment. JMIR Mental Health, 6(9), Article e13639. https://doi.org/10.2196/13639

21.

Johnson

J. L.

Oliffe

J. L.

Kelly

M. T.

Galdas

Ogrodniczuk

J. S.

(2012). Men’s discourses of help-seeking in the context of depression. Sociology of Health and Illness, 34(3), 345–361. https://doi.org/10.1111/j.1467-9566.2011.01372.x

22.

Kealy

Ben-David

Cox

D. W.

(2020). Early parental support and meaning in life among young adults: The mediating roles of optimism and identity. Current Psychology. Advance online publication. https://doi.org/10.1007/s12144-020-00907-w

23.

Keum

B. T.

Miller

M. J.

Inkelas

K. K.

(2018). Testing the factor structure and measurement invariance of the PHQ-9 across racially diverse US college students. Psychological Assessment, 30(8), 1096–1106. https://doi.org/10.1037/pas0000550

24.

King

Schlichthorst

Turnure

Phelps

Spittal

M. J.

Pirkis

(2019). Evaluating the effectiveness of a website about masculinity and suicide to prompt help-seeking. Health Promotion Journal of Australia, 30(3), 381–389. https://doi.org/10.1002/hpja.237

25.

Kocalevent

R. D.

Hinz

Brähler

(2013). Standardization of the depression screener patient health questionnaire (PHQ-9) in the general population. General Hospital Psychiatry, 35(5), 551–555. https://doi.org/10.1016/j.genhosppsych.2013.04.006

26.

Koneska

Appelbe

Williamson

P. R.

Dodd

(2020). Usage metrics of web-based interventions evaluated in randomized controlled trials: Systematic review. Journal of Medical Internet Research, 22(4), Article e15474. https://doi.org/10.2196/15474

27.

Kroenke

Spitzer

R. L.

Williams

J. B.

(2001). The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. https://doi.org/10.1046/j.1525-1497.2001.016009606.x

28.

Kuehner

(2017). Why is depression more common among women than among men? The Lancet Psychiatry, 4(2), 146–158. https://doi.org/10.1016/S2215-0366(16)30263-2

29.

Lee

S. L.

Pearce

Ajnakina

Johnson

Lewis

Mann

Pitman

Solmi

Sommerlad

Steptoe

Tymoszuk

Lewis

(2021). The association between loneliness and depressive symptoms among adults aged 50 years and older: A 12-year population-based cohort study. The Lancet Psychiatry, 8(1), 48–57. https://doi.org/10.1016/S2215-0366(20)30383-7

30.

Chen

Jing

Jiang

Sun

Luo

Kwan

Liu

(2016). The role of stress management in the relationship between purpose in life and self-rated health in teachers: A mediation analysis. International Journal of Environmental Research and Public Health, 13(7), 719. https://doi.org/10.3390/ijerph13070719

31.

Linardon

Cuijpers

Carlbring

Messer

Fuller-Tyszkiewicz

(2019). The efficacy of app-supported smartphone interventions for mental health problems: A meta-analysis of randomized controlled trials. World Psychiatry, 18(3), 325–336. https://doi.org/10.1002/wps.20673

32.

Livingston

J. D.

(2020). Structural stigma in health-care contexts for people with mental health and substance use issues: A literature review. Mental Health Commission of Canada.

33.

Shi

Hollenberg

Abi-Jaoudé

Johnson

Wiljer

(2020). Surveying the role of analytics in evaluating digital mental health interventions for transition-aged youth: Scoping review. JMIR Mental Health, 7(6), Article e15942. https://doi.org/10.2196/15942

34.

Luo

Sanger

Singhal

Pattrick

Shams

Shahid

Hoang

Schmidt

Lee

Haaber

Puckering

Buchanan

Lee

Sun

Kheyson

Cung

D. C.

Sanger

Thabane

Samaan

(2020). A comparison of electronically-delivered and face to face cognitive behavioural therapies in depressive disorders: A systematic review and meta-analysis. EClinicalMedicine, 24, 100442. https://doi.org/10.1016/j.eclinm.2020.100442

35.

Mackert

Guadagno

Lazard

Donovan

Rochlen

Garcia

Damásio

M. J.

Crook

(2018). Engaging men in prenatal health via eHealth: Findings from a national survey. JMIR Pediatrics and Parenting, 1(2), Article e7. https://doi.org/10.2196/pediatrics.9513

36.

Massoudi

Holvast

Bockting

C. L.

Burger

Blanker

M. H.

(2019). The effectiveness and cost-effectiveness of e-health interventions for depression and anxiety in primary care: A systematic review and meta-analysis. Journal of Affective Disorders, 245, 728–743. https://doi.org/10.1016/j.jad.2018.11.050

37.

Murphy

A. L.

Peltekian

Gardner

D. M.

(2018). Website analytics of a Google Ads campaign for a men’s mental health website: Comparative analysis. JMIR Mental Health, 5(4), Article e12428. https://doi.org/10.2196/12428

38.

Naghavi

(2019). Global, regional, and national burden of suicide mortality 1990 to 2016: Systematic analysis for the Global Burden of Disease Study 2016. British Medical Journal, 364, Article 194. https://doi.org/10.1016/S1474-4422(18)30322-3

39.

National Institute of Mental Health. (2019). Major depression. https://www.nimh.nih.gov/health/statistics/major-depression

40.

Ogrodniczuk

J. S.

Kealy

Laverdière

(2021). Who is coming through the door? A national survey of self-reported problems among post-secondary school students who have attended campus mental health services in Canada. Counselling and Psychotherapy Research, 21(4), 837–845. https://doi.org/10.1002/capr.12439

41.

Ogrodniczuk

J. S.

Oliffe

Beharry

(2018). HeadsUpGuys: Canadian online resource for men with depression. Canadian Family Physician, 64(2), 93–94. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5964376/pdf/0640093.pdf

42.

Oliffe

J. L.

Black

Yiu

Flannigan

R. K.

McCreary

D. R.

Goldenberg

S. L.

(2020). Mapping Canadian men’s recent and intended health behavior changes through the don’t change much electronic health program. Journal of Medical Internet Research, 22(5), Article e16174. https://doi.org/10.2196/16174

43.

Oliffe

J. L.

Broom

Rossnagel

Kelly

M. T.

Affleck

Rice

S. M.

(2020). Help-seeking prior to male suicide: Bereaved men perspectives. Social Science and Medicine, 261, 113173. https://doi.org/10.1016/j.socscimed.2020.113173

44.

Oliffe

J. L.

Ogrodniczuk

J. S.

Gordon

S. J.

Creighton

Kelly

M. T.

Black

Mackenzie

(2016). Stigma in male depression and suicide: A Canadian sex comparison study. Community Mental Health Journal, 52(3), 302–310. https://doi.org/10.1007/s10597-015-9986-x

45.

Oliffe

J. L.

Rossnagel

Bottorff

J. L.

Chambers

S. K.

Caperchione

Rice

S. M.

(2020). Community-based men’s health promotion programs: Eight lessons learnt and their caveats. Health Promotion International, 35(5), 1230–1240. https://doi.org/10.1093/heapro/daz101

46.

Oliffe

J. L.

Rossnagel

Seidler

Z. E.

Kealy

Ogrodniczuk

J. S.

Rice

S. M.

(2019). Men’s depression and suicide. Current Psychiatry Reports, 21(10), 1–6. https://doi.org/10.1007/s11920-019-1088-y

47.

Paschall

M. J.

Antin

Ringwalt

C. L.

Saltz

R. F.

(2011). Evaluation of an Internet-based alcohol misuse prevention course for college freshmen: Findings of a randomized multi-campus trial. American Journal of Preventive Medicine, 41(3), 300–308. https://doi.org/10.1016/j.amepre.2011.03.021

48.

Pirkis

Spittal

M. J.

Keogh

Mousaferiadis

Currier

(2017). Masculinity and suicidal thinking. Social Psychiatry and Psychiatric Epidemiology, 52(3), 319–327. https://doi.org/10.1007/s00127-016-1324-2

49.

Prochaska

J. O.

DiClemente

C. C.

(1983). Stages and processes of self-change of smoking: Toward an integrative model of change. Journal of Consulting and Clinical Psychology, 51(3), 390–395. https://doi.org/10.1037/0022-006X.51.3.390

50.

Rice

S. M.

Aucote

H. M.

Parker

A. G.

Alvarez-Jimenez

Filia

K. M.

Amminger

G. P.

(2017). Men’s perceived barriers to help seeking for depression: Longitudinal findings relative to symptom onset and duration. Journal of Health Psychology, 22(5), 529–536. https://doi.org/10.1177/1359105315605655

51.

Rice

S. M.

Oliffe

J. L.

Kealy

Seidler

Z. E.

Ogrodniczuk

J. S.

(2020). Men’s help-seeking for depression: Attitudinal and structural barriers in symptomatic men. Journal of Primary Care and Community Health. Advance online publication. https://doi.org/10.1177/2150132720921686

52.

Rice

S. M.

Purcell

McGorry

P. D.

(2018). Adolescent and young adult male mental health: Transforming system failures into proactive models of engagement. Journal of Adolescent Health, 62(3), S9–S17. https://doi.org/10.1016/j.jadohealth.2017.07.024

53.

Roche

A. M.

Pidd

Fischer

J. A.

Lee

Scarfe

Kostadinov

(2016). Men, work, and mental health: A systematic review of depression in male-dominated industries and occupations. Safety and Health at Work, 7(4), 268–283. https://doi.org/10.1016/j.shaw.2016.04.005

54.

Seidler

Z. E.

Dawes

A. J.

Rice

S. M.

Oliffe

J. L.

Dhillon

H. M.

(2016). The role of masculinity in men’s help-seeking for depression: A systematic review. Clinical Psychology Review, 49, 106–118. https://doi.org/10.1016/j.cpr.2016.09.002

55.

Seidler

Z. E.

Rice

S. M.

Ogrodniczuk

J. S.

Oliffe

J. L.

Dhillon

H. M.

(2018). Engaging men in psychological treatment: A scoping review. American Journal of Men’s Health, 12(6), 1882–1900. https://doi.org/10.1177/1557988318792157

56.

Smail-Crevier

Powers

Noel

Wang

(2019). Health-related internet usage and design feature preference for e-mental health programs among men and women. Journal of Medical Internet Research, 21(3), Article e11224. https://doi.org/10.2196/11224

57.

Smith

D. T.

Mouzon

D. M.

Elliott

(2018). Reviewing the assumptions about men’s mental health: An exploration of the gender binary. American Journal of Men’s Health, 12(1), 78–89. https://doi.org/10.1177/1557988316630953

58.

Song

M. J.

Ward

Choi

Nikoo

Frank

Shams

Tabi

Vigo

Krausz

(2018). A process evaluation of a web-based mental health portal (WalkAlong) using Google Analytics. JMIR Mental Health, 5(3), Article e50. https://doi.org/10.2196/mental.8594

59.

Strike

Rhodes

A. E.

Bergmans

Links

(2006). Fragmented pathways to care: The experiences of suicidal men. Crisis: The Journal of Crisis Intervention and Suicide Prevention, 27(1), 31–38. https://doi.org/10.1027/0227-5910.27.1.31

60.

U.S. Census Bureau. (2013). Annual estimates of the resident population for selected age groups by sex for the United States, states, counties, and Puerto Rico commonwealth and municipios: April 1, 2010 to July 1, 2012. https://www.census.gov/data/tables/time-series/demo/popest/2010s-national-detail.html

61.

Wang

Lam

R. W.

Attridge

Lashewicz

B. M.

Patten

S. B.

Marchand

Aiken

Schmitz

Gundu

Merali

(2016). Preferred features of e-mental health programs for prevention of major depression in male workers: Results from a Canadian national survey. Journal of Medical Internet Research, 18(6), Article e5685. https://doi.org/10.2196/jmir.5685

62.

White

McKee

Richardson

de Visser

Madsen

S. A.

de Sousa

B. C.

Hogston

Zatoński

Makara

(2011). Europe’s men need their own health strategy. British Medical Journal, 343, Article d7397. https://doi.org/10.1136/bmj.d7397

63.

Whiteside

Richards

Huh

Hidalgo

Nordhauser

Wong

A. J.

Zhang

Luxton

D. D.

Ellsworth

Lezine

(2019). Development and evaluation of a web-based resource for suicidal thoughts: NowMattersNow.org. Journal of Medical Internet Research, 21(5), Article e13183. https://doi.org/10.2196/13183

64.

Wide

Mok

McKenna

Ogrodniczuk

J. S.

(2011). Effect of gender socialization on the presentation of depression among men: A pilot study. Canadian Family Physician, 57(2), e74–e78. https://pubmed.ncbi.nlm.nih.gov/21642709/

65.

World Health Organization. (2017). Depression and other common mental disorders: Global health estimates. https://apps.who.int/iris/bitstream/handle/10665/254610/WHO-MSD-MER-2017.2-eng.pdf;jsessionid=2C7D54217E3B6147D1AA160C65AB4E4A?sequence=1

66.

Yardley

Spring

B. J.

Riper

Morrison

L. G.

Crane

D. H.

Curtis

Merchant

G. C.

Naughton

Blandford

(2016). Understanding and promoting effective engagement with digital behavior change interventions. American Journal of Preventive Medicine, 51(5), 833–842. https://doi.org/10.1016/j.amepre.2016.06.015

67.

Young

M. D.

Morgan

P. J.

(2018). Effect of a gender-tailored ehealth weight loss program on the depressive symptoms of overweight and obese men: Pre-post study. JMIR Mental Health, 5(1), Article e8920. https://doi.org/10.2196/mental.8920

68.

Yousaf

Grunfeld

E. A.

Hunter

M. S.

(2015). A systematic review of the factors associated with delays in medical and psychological help-seeking among men. Health Psychology Review, 9(2), 264–276. https://doi.org/10.1080/17437199.2013.840954

69.

Zanchetta

M. S.

Maheu

Kolisnyk

Mohamed

Guruge

Kinslikh

Christopher

J. J.

Stevenson

SanJose

Sizto

Byam

(2017). Canadian men’s self-management of chronic diseases: A literature analysis of strategies for dealing with risks and promoting wellness. American Journal of Men’s Health, 11(4), 1077–1095. https://doi.org/10.1177/1557988315577674