Abstract
Men’s e-health promotion programs can offer end-user anonymity and autonomy that provide avenues for supporting positive health behavior change. The twofold purpose of the current study was to use a benchmark cohort as a reference group to: (1) describe associations between men’s usage levels of the e-health program Don’t Change Much (DCM) and their recent and intended health behavior changes, and (2) report an exploratory analysis of the moderating effects of demographic variables on the associations between DCM users and their recent and intended health behavior changes. Based on self-report, DCM users were classified into limited (n = 613, 34.7%), low (n = 826, 46.8%), and high (n = 327, 18.5%) exposure groups. Compared with the benchmark cohort, DCM high-exposure respondents had significantly increased odds for eight of the nine recent behavior changes, with the largest effect size observed for “Made an effort to sit less and walk more” (odds ratio [OR] 2.996, 95% CI [2.347, 3.826]). Eight of the nine intended health behavior changes in the DCM high-exposure group had significantly increased odds compared to the benchmark cohort, with “Reduce stress level” (OR 3.428, 95% CI [2.643, 4.447]) having the largest effect size. Significantly greater total numbers of recent (F(12, 2850) = 29.32; p = .001; R2 = .086) and intended health behavior changes (F(12, 2850) = 34.59; p = .001; R2 = 0.100) were observed among high exposure respondents while adjusting for demographics. Younger age, being employed, and household income <$120,000 had an enhancing moderator effect on DCM users’ number of intended behavior changes.
Many health promotion programs have focused on reducing male risk factors such as unhealthy diets, substance use, smoking, and sedentary lifestyles (Lahoud & Franco, 2016). Investigations of the challenges for engaging men with self-health have highlighted the need for gender–sensitized health promotion programs (Oliffe et al., 2020a; Oliffe et al., 2020b; Soprovich et al., 2020). While the delivery of men’s health promotion programs is diverse, the e-health sector has grown significantly during the 2000s (Fogarty et al., 2017). By offering end-user anonymity and autonomy for selecting specific content, men’s e-health promotion programs can provide important avenues to augment and connect traditional services and aid men’s behavior changes and health promotion practices (Robinson & Robertson, 2010). Within the men’s e-health context, there is much debate about program effectiveness as well as broader concerns regarding the relative reach of interventions for specific male subgroups.
Background
Men’s e-health promotion programs have grown rapidly to address diverse contexts and deliver tailored information through an array of platforms (i.e., web, social media, email, etc.). While men’s health issues including prostate cancer (Bender et al., 2019), HIV prevention (Klein et al., 2017), weight loss (Young & Morgan, 2018), and health screening (Teo et al., 2019) have attracted e-health promotion programs, there is much debate about the effectiveness of these interventions. This relates, in part, to the predominance of needs analyses and acceptability studies wherein the feasibility and/or design of e-health programs are informed by potential male end-users. For example, a study of men with prostate cancer indicated 65% (n = 784) of respondents used the internet as a source of prostate cancer information, and that this subgroup was more likely to have unmet supportive care needs (Bender et al., 2019). These findings led the authors to recommend examining men’s e-health literacy as the next step to tailoring internet-based prostate cancer information (Bender et al., 2019). Differentiating these needs analyses, men’s e-health programs have also been pilot-tested with a view to adjusting content and delivery based on end-user and expert feedback. Examples include acceptability testing of a mobile application to improve men’s health screening uptake, which showed positive utility and usability amongst end-users (Teo et al., 2019). Some formal evaluations have also linked men’s behavior change to their use of specific e-health programs. A men’s e-health weight loss program was positively evaluated in a pre-post study design wherein short-term improvements in the mental health of overweight and obese men were reported (Young & Morgan, 2018). Man Central (a web and mobile phone intervention for men with depression) reported significant improvements in depression symptoms, depression risk, externalizing symptoms, and work and social functioning among end-users based on data collected through repeated measures from a single respondent group (Fogarty et al, 2017). A quasi-experimental, two-arm study by Klein et al. (2017) evaluated “Real Talk” (an e-health harm reduction intervention targeting Black men who have sex with men) and reported increased end-user HIV knowledge, though no significant differences for condom use or other risk reduction practices compared to the control group were found. These program reports and evaluations, and many other men’s e-health promotion studies, provide important insights; however, the findings and the men’s e-health field more broadly are limited by small sample sizes, cross-sectional study designs, and/or a lack of control groups (Forbes et al., 2019).
There has been research investigating the relative reach of men’s e-health promotion programs with a focus on distilling how social determinants of health influence men’s access, usage, and potential benefits (Francis, 2019; Nguyen et al., 2019). For example, low income (Dzinamarira et al., 2020) and homeless (Calvo et al., 2019) men have been highlighted as experiencing health inequities, which depending on the e-health content and platform, can be a barrier or avenue to promoting the health of such disadvantaged groups (Stone & Waldron, 2019). Older age has been associated with reduced e-health literacy in men, which can impose significant barriers for some males (Wills et al., 2020). In exploring the demographics of men who access e-health programs, and the relative benefits and barriers associated with specific end-user subgroups, important content and platform decisions and adjustments can be made (Oliffe, et al., 2020a; Soprovich et al., 2020).
In sum, evaluations of men’s e-health promotion programs, while encouraging, strongly support the need for additional evidence to map men’s health behavior changes and self-health promotion effects. The twofold purpose of the current study is to use a benchmark cohort as a reference group to: (1) describe associations between men’s usage levels of the e-health program Don’t Change Much (DCM) and their recent and intended health behavior changes, and (2) report an exploratory analysis of the moderating effects of demographic variables on the associations between DCM users and their recent and intended health behavior changes.
Methods
By way of background, the DCM e-health program was launched by the Canadian Men’s Health Foundation in 2014 to inspire and equip men and their families to lead healthier lives. Accessible research-informed content is provided through quick tips and lobbying men’s strength-based actions to incrementally do the work of self-health. Text, video, and audio information is purposefully delivered to improve diet, exercise, sleep, and stress management as well as reduce alcohol use and/or smoking. The DCM content and delivery recognizes end-users as having diverse backgrounds, needs, and alignments to self-health, and the materials are purpose built to work with men wherever they are at, to advance their health promotion practices.
With approval from the University of British Columbia behavioral ethics review board, written consent was confirmed ahead of respondents providing demographic and self-reported survey questionnaire data detailing recent and intended health behavior changes. Data were collected from two Canadian male cohorts: (1) benchmark (reference group who had not accessed DCM), and (2) DCM users. Benchmark data collection procedures, and cross-sectional (Oliffe et al., 2020c; Flannigan et al., 2019; McCreary et al., 2020) and comparative findings (Oliffe et al., 2020a) have been published elsewhere. In brief, the 15-min benchmark questionnaire was administered April 20–28, 2017, via an online panel provider, and 2000 men, stratified by age and location based on Canadian census data, completed the survey. DCM users completed the same survey questionnaire between April 1, 2018 and March 31, 2019, providing details of their usage (duration and frequency) for each of the three DCM formats (web, newsletter, and social media). DCM users were recruited through pop-up ads on the DCM website and e-newsletter invitations and incentivized by the option to enter a $500 prize draw. DCM users comprised 3597 respondents, which was reduced to 1766 (1611 incomplete, 123 straight lining and/or speeding, 70 were not male, 19 resided outside of Canada, eight were under 19 years of age).
Measures
Demographic data including age, education, employment, living arrangements (living with partner; children <19 years living at home), visible minority, sexual orientation, and household income (before taxes) were collected on the benchmark and DCM user cohorts. Respondents in both cohorts were also asked about recent and intended health behavior changes. To assess recent changes, respondents were asked “In the past 12 months, have you made any changes that would improve your health?” and invited to select all that applied from the following: (1) changed diet or improved eating habits, (2) made an effort to sit less and walk more, (3) increased exercise, sports, or physical activity, (4) drank less alcohol, (5) had a routine check-up or visit to doctor, (6) improved consistent sleep quality, (7) lost weight, (8) reduced stress level, (9) quit or reduced smoking. Intended changes data were collected through respondents selecting all that applied from the same nine items, with the stem question “In the next month (30 days), do you intend to make any changes that would improve your health?”
The work of Quinn and Chaudoir (2009) guided the conversion of categorical duration and frequency responses into continuous measures;
1) When did you FIRST use, or subscribe to, the following resources? Duration response options and continuous measure conversions: never before (0.0), in the past month (0.5), 1–6 months ago (3.5), 7–12 months ago (9.5), 13–24 months ago (18.5), more than 2 years ago (24.0), not sure (excluded).
2) How often do you use or access the following resources? Frequency response options and continuous measure conversions: several times a day (1095), once a day (365), several times a week (156), once a week (52), several times a month (36), once a month (12), several times a year (3), once a year (1), less often (.5), do not use (0), not sure (excluded).
The sum of the product of duration and frequency for each of the three DCM formats was used to calculate respondent exposure scores. For example, DCM users who responded “never before” (duration) and “do not use” (frequency) for all three DCM components were scored 0 and assigned to the limited exposure subgroup (n = 613, 34.7%). Respondents who used one DCM component “in the past month” (0.5 duration) “less often” (0.5 frequency) scored 0.25, and were classified low exposure (n = 826, 46.8%; range 0.25–680). Respondents who used two components, both “7–12 months ago” (9.5 duration) and “several times a month” (36 frequency) scored 684, and were classified high exposure (n = 327, 18.5%; range 684–30,570).
Data Analysis
Cohen’s d was used to assess the magnitude of difference in means between two or more independent groups. In cases where the means of three or more independent groups were compared, partial eta-squared was computed first and then converted to Cohen’s d using formulae in Cohen (1988). To calculate an effect size for the strength of association between two categorical variables, Cramer’s V was used to compute values ranging from 0 to 1 (inclusive) (McHugh, 2013).
Logistic regression analysis was used to model the association between the level of exposure to DCM (Independent Variable (IV)) and recent and intended health behavior changes (Dependent Variable (DV)). R2 values were used to measure the proportion of variance in a DV that can be explained by the IVs in the regression models (Cohen, 1988). To analyze recent changes (in the past 12 months), the DVs were user specified behavior changes. For intended changes, the DVs were predicted behavior changes (in the next month). Linear regression models were used to evaluate the association between the number of recent and intended changes and level of exposure to DCM. The models computed the number of recent changes (DV) and the number of intended changes (DV). Benchmark respondents were the reference group, and DCM users were classified as limited, low, or high exposure. All logistic and linear regression models controlled for the following covariates: age, employment, living arrangements (lives with partner; children <19 years living at home), education, visible minority, sexual orientation, and household income (before taxes). Odds ratios indicated the strength of association between predictor variables and dichotomous outcome variables (Chen et al., 2010). Variance inflation factors (VIF) were computed as a collinearity diagnostic check.
Simple exploratory moderation analyses to assess for the interactions of DCM end-user demographics on associations between DCM exposure level and recent and intended health changes were conducted with PROCESS (Hayes, 2017). Separate moderation models were computed for all of the demographic variables. All variables were entered in one step, with simple moderation models comprising: (1) the main effect of low or high composite DCM exposure, relative to the benchmark reference group, (2) the demographic main effect, and (3) the interaction between DCM exposure and the specified demographic.
Results
Benchmark respondents provided the reference group to compute the effects of DCM exposure on recent and intended health behavior changes. The magnitude of differences for demographic factors between benchmark respondents and DCM users indicated small to negligible effect sizes (Table 1). The majority of respondents in both cohorts were employed, lived with a partner, did not live with children under 19 years of age, had not graduated from university, and identified as heterosexual. Compared to benchmark respondents, a higher proportion of DCM users reported having a household income of $120,000 or more.
Benchmark vs. DCM User Sample Demographics.
DCM = Don’t Change Much.
As a ratio variable, the test performed was a t-test for this characteristic as indicated.
As a ratio variable, the effect size computed was a Cohen’s d for this characteristic as indicated.
Recent Health Behavior Changes
For the DCM group, all recent health behavior changes had significant associations with increased levels of exposure to DCM. Moderate effect sizes were observed in bivariate analyses between DCM exposure and “changed diet or improved eating habits” (χ23 = 222, p < .001, Cramer’s V = 0.243) and “made an effort to sit less and walk more” (χ23 = 160, p < .001, Cramer’s V = 0.206). Large effect sizes were observed in bivariate analyses between the total number of recent health changes and DCM high exposure level (Kruskal–Wallis H = 294, p < .001, Cramer’s V = 0.767). Compared to the benchmark cohort, high exposure respondents had significantly increased odds for all recent health behavior changes except “quit or reduced smoking” (OR = 0.920, 95% CI [0.606, 1.399]) while holding other predictor variables constant. Small effect sizes were observed for “made an effort to sit less and walk more” (OR = 2.996, 95% CI [2.347, 3.826]), “changed diet or improved eating habits” (OR = 2.879, 95% CI [2.252, 3.68]), and “improved consistent sleep” (OR = 2.655, 95% CI [2.05, 3.439]). Respondents with high exposure to DCM also had significantly greater total numbers of recent health changes while adjusting for covariates, with a small R2 effect size observed) (F12,3753 = 29.320, p < .001, R2 = 0.086) (Table 2).
Logistic Regressions Between Demographics and Recent Health Behavior Changes.
Note. DCM = Don’t Change Much.
p < .05; **p < .01; ***p < .001; separate multiple logistic regressions were conducted for each outcome variable with all predictor variables entered on the same step.
Intended Health Behavior Changes
For the DCM group, all intended health behavior changes had significant associations with increased levels of exposure to DCM. Moderate effect sizes were observed in bivariate analyses between DCM exposure and “improve consistent sleep quality” (χ23 = 187, p < .001, Cramer’s V = 0.223), “change diet or improve eating habits” (χ23 = 187, p < .001, Cramer’s V = 0.223), and “make an effort to sit less and walk more” (χ23 = 166, p < .001, Cramer’s V = 0.210). Large effect sizes were observed in bivariate analyses between the total number of intended health changes and DCM exposure level (Kruskal–Wallis H = 338, p < .001, Cramer’s V = 0.845). Compared to the benchmark cohort, high exposure respondents had significantly increased odds for all of the intended health behavior changes except “Quit or reduce amount smoked” (OR = 1.368, 95% CI [0.889, 2.105]) while holding other predictor variables constant. Small effect sizes were observed for “reduce stress level” (OR = 3.428, 95% CI [2.643, 4.447]), “improve consistent sleep quality” (OR = 3.239, 95% CI [2.522, 4.161]), and “change diet or improve eating habits” (OR = 2.915, 95% CI [2.28, 3.727]). Respondents with high exposure to DCM also had significantly greater total numbers of intended health changes while adjusting for covariates, with a moderate R2 effect size observed (F12,3753 = 34.591, p < .001, R2 = 0.100) (Table 3).
Logistic Regressions Between Demographics and Intended Health Behavior Changes.
Note. DCM = Don’t Change Much.
p < .05; **p < .01; ***p < .001; separate multiple logistic regressions were conducted for each outcome variable with all predictor variables entered on the same step.
Moderating Effects of Demographics
An exploratory moderation analysis indicated age, employment, living arrangements, education, and income had an interaction effect on associations between DCM exposure and four recent health behavior changes: “changed diet or improved eating habits,” “made an effort to sit less and walk more,” “had a routine check-up or visit to doctor,” and “reduced stress level.” Among these, higher education levels (graduated university) had an enhancing moderator effect on the association between DCM exposure and recent “changes to diet or improved eating habits” (OR = 1.3, 95% CI [1.027, 1.884]) and “made an effort to sit less and walk more” (OR = 1.434, 95% CI [1.056, 1.949]) (please see Table 4). Interaction effects of age, employment, education, and income were found for five intended health behavior changes. Models for the number of intended health behavior changes indicated younger age (R2 = .095, F(110.610), p = .001, b = −0.023), being employed (R2 = .092, F(106.80), p = .001, b = 0.442), and annual household income <$120,000 (R2 = .093, F(107.742), p = .001, b = −0.463) accounted for a significant proportion of the variance for intended health behavior changes (Table 5).
Analysis of Statistically Significant Moderating Effects Between Demographics and Recent Health Changes.
Note. DCM = Don’t Change Much.
Dichotomous variable, with 1 = low or high composite exposure, and 0 = benchmark.
p < .05; **p < .01; ***p < .001. Separate moderation models were computed for all nine recent health changes and all demographic variables (age, employment, lives with partner, lives with children, education, visible minority, sexual orientation, $59,999 or less, $120,000 or more). Models were also computed for the number of recent health changes and all demographics. Models with statistically insignificant moderator effects were omitted from display in the table.
Analysis of Statistically Significant Moderating Effects Between Demographics and Intended Health Changes.
Note. DCM = Don’t Change Much.
Dichotomous variable, with 1 = low or high composite exposure, and 0 = benchmark.
p < .05; **p < .01; ***p < .001. Separate moderation models were computed for all nine intended health changes and all demographic variables (age, employment, lives with partner, lives with children, education, visible minority, sexual orientation, $59,999 or less, $120,000 or more). Models were also computed for the number of intended health changes and all demographics. Models with statistically insignificant moderator effects were omitted from display in the table.
Discussion and Conclusion
The current study findings confirm and extend our previous work (Oliffe et al., 2020a), and by extension affirm the strong potential of men’s e-health promotion programs (Robinson & Robertson, 2010). Though not claiming attribution, the statistically significant positive associations between men’s DCM exposure levels and their recent and intended health behavior changes warrant discussion of three key issues as a means to scoping potential DCM adjustments, and offering recommendations to advance men’s e-health promotion work.
First, in describing specific recent and intended health behavior changes made available are opportunities to consider adjustments to DCM. Recent changes in “changed diet or improved eating habits” and “made an effort to sit less and walk more” reflect, at least in part, the appeal of DCM’s content and the lobby for men to make incremental and proportion-based adjustments (<sitting and >walking; balancing food groups). Within these contexts, the call to action is not wholesale change or abstinence; instead, DCM focuses on strategies for making adjustments (getting more steps and cooking [and eating] a variety of foods). That “reduce stress” and “improve consistent sleep quality” featured prominently as intended health behavior changes along with reports of recent “improved consistent sleep quality” underscores the importance building on the tailored DCM stress reduction and sleep-aiding resources. A review of interventions to promote sleep health in men (Soprovich et al., 2020) reported moderate level evidence for sleep health programs incorporating physical activity and stress management components. With this in mind, there may be traction for bundling some DCM content to engage men with interlocking strategies for concurrently reducing stress and increasing physical activity and sleep quality. Due diligence also includes attention to health behaviors drawing relatively little change. Specifically, the respondents’ low recent and intended changes for “quit or reduce smoking” likely reflect the small subpopulation of male smokers in Canada—and the fewer DCM end-users who smoke. In addition, because men’s e-health tobacco reduction and smoking cessation (TRSC) resources tend to be specialized (Bottorff et al., 2016), it is unlikely that DCM end-users who smoke arrived with the goal of TRSC.
Second, while e-health resources engage many men, the issues of reach and effectiveness continue to challenge the field. That the current study regression findings held when controlling across key demographics synonymous with social determinants of health (i.e., income, employment, education etc.) suggests DCM is accessible to, and engaging of, men from diverse backgrounds. That said, the results of the moderation analysis reveal enhancing interactions for younger, employed men, with an annual household income <$120,000. One possibility is that the DCM resources resonate most for men with those demographics. Of course, there is the possibility that these subgroups are also more motivated to make health changes. One challenge related to this finding is how best to sustain end-users drawing the biggest benefits while building content to grow those gains in other subgroups.
Third, while direct comparisons with previous DCM findings (Oliffe et al., 2020a) were not the aim of the current study, it is important to note that there were subtle changes to behavior change rankings and the strength of the associations with DCM exposure. For most end-users’ recent and intended health behavior changes, the odds ratios were lower (though significant) in the current study compared to the previous study results. One explanation for this is that the earlier study (Oliffe et al., 2020a) reported data collected January 1 through March 31 2018—a new-year period strongly linked to resolutions including health behavior changes. The current study data were collected over a 1-year period (April 1, 2018 through March 31, 2019), and this may have flattened the potential new-year peak reported in the previous study. Though speculation, this explanation should remind us that year-end through early new-year campaigns can be especially timely for engaging men with e-health promotion.
The high potential for familywise errors in conducting numerous separate regression analyses is a methodological limitation of the current study. Self-report biases for respondents’ DCM usage and recent and intended health behavior changes are limitations, as is the cross-sectional study design. These limitations can however be addressed by including qualitative interviews to further contextualize and augment statistical results. The current study design also underscores the need for longitudinal research to evaluate change more fully over time.
Within diverse platform delivery, fidelity, and focus, there is a pressing need to advance evaluative evidence in men’s e-health promotion. The tensions between reporting associations amid wanting to claim attribution might best be resolved with some concessions that men’s health behaviors shift over time and context in response to an array of influences. This is not to deny the need for some control trials to prove effect; rather, it is an acknowledgment that men’s e-health promotion programs can (and often do) assist men to advance their health, and by extension, the well-being of their families.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Canadian Men’s Health Foundation DCM eHealth program and DCM evaluation was supported with funding from the Public Health Agency of Canada. The DCM evaluation was subcontracted to and completed by Intensions Consulting (
); The American Urologic Association, Research Scholar Award supported the involvement of RF. JLO is supported by a Tier 1 Canada Research Chair in Men’s Health Promotion.