Sage Journals: Discover world-class research

Abstract

Magazines have traditionally been an effective medium for delivering health media messages to large populations or specific groups. In this retrospective cross-sectional study, we evaluated nine issues from 2016 publications of American men’s health-related magazines (Men’s Health and Men’s Fitness) to evaluate their recommendations and determine their validity by examining corresponding evidence found in the peer-reviewed scientific literature. We extracted health recommendations (n = 161) from both magazines and independently searched and evaluated evidence addressing the recommendations. We could find at least a case study or higher quality evidence addressing only 42% of the 161 recommendations (80 recommendations from Men’s Health and 81 recommendations from Men’s Fitness). For recommendations from Men’s Health, evidence supported approximately 23% of the 80 recommendations, while evidence was unclear, nonexistent, or contradictory for approximately 77% of the recommendations. For recommendations from Men’s Fitness, evidence supported approximately 25% of the 81 recommendations, while evidence was unclear, nonexistent, or contradictory for approximately 75% of the recommendations. The majority of recommendations made in men’s health-related magazines appear to lack credible peer-reviewed evidence; therefore, patients should discuss such recommendations with health-care providers before implementing.

Keywords

public health health-care issues evidence-based medicine men’s health programs health education general health and wellness

Historically, men have higher mortality rates and lower life expectancies compared to women (Baker et al., 2014; Crimmins et al., 2019). In a 2016 review of national-level surveys, Crimmins et al. identified that female life expectancy exceeded that of men in all 198 evaluated countries from 1960 to 2016 (Crimmins et al., 2019). Researchers suspect that the difference between male and female mortality rates is based upon various factors including sex differences in genetics and hormones, differences in socioeconomic structures, and differences in behavioral activities that depict stereotypes of masculinity or femininity (Baker et al., 2014; Crimmins et al., 2019). Women have historically expressed notions of femininity by seeking more health education or behavior than men (Rogers et al., 2010). Unfortunately, many men try to portray “notions of masculinity” by delaying health services, engaging in more harmful activities (e.g., excessive alcohol consumption or unprotected sexual activities) or with occupational hazards (e.g., physical or chemical exposures) (World Health Organization, Regional Office for Europe). These observed gender incongruities have prompted the World Health Organization (WHO) to endorse awareness programs specifically targeted to male populations. Programs recommended by WHO include implementing partnership projects with men (e.g., involving fathers in decisions about family health), implementing gender transformation interventions (e.g., educating males to reshape deconstructive male gender-related habits), and promoting mass media outreach initiatives (Dworkin et al., 2013; Hunt et al., 2014; Lindegren et al., 2012).

Mass media (e.g., television, radio, magazines, newspapers, and billboards) has historically been used for delivering health messages to the public (Bala et al., 2013; Marx et al., 2008). The primary benefit of mass media is the relatively inexpensive avenue to reach a large number of patients quickly. Magazines or periodicals have traditionally been an effective medium for delivering such health media messages to specific groups. Previous researchers have reported that patients prefer using periodicals as a means to access healthy lifestyle information (Cutilli, 2010; Maibach et al., 2006). However, research evaluating the quality of evidence regarding the advice and recommendations published in periodicals is sorely lacking. This concern is further prompted by evidence from a recent study on another form of mass media, that is, health television shows. This study suggested that many of the health-related recommendations made on health television shows were not supported by scientific, peer-reviewed evidence (Korownyk et al., 2014). Many in the medical and scientific community have voiced apprehension over the quality of recommendations made on health television shows. These concerns suggest that the shows have not consistently promoted interventions congruent with evidence-based medicine (Tilburt et al., 2017). While studies exist for health television shows, no studies to date have evaluated the validity of health recommendations from other forms of mass media, such as periodical publications (e.g., magazines) (Korownyk et al., 2014).

The primary objective of this study was to determine and compare the percentage of health recommendations in two popular men’s health-related magazines that are addressed by peer-reviewed scientific evidence.

Methods

This was a retrospective, cross-sectional study between two men’s health-related magazines aimed at characterizing and evaluating health-related recommendations made by them. The study mimicked a previous study done to evaluate health claims made in health television shows (Korownyk et al., 2014). The study was designed by the first and last author, and the protocol was approved by the Touro University California institutional review board (IRB Application # P-0117) in January 2017.

Men’s Health-Related Magazine Selection

A researcher (M.A.J.) conducted a search in the spring of 2017 on the Alliance for Audited Media website to identify the two most widely distributed men’s health-related magazines in the United States, as measured by total paid and verified circulation (using “The Alliance for Audited Media, 2016). The magazines with the highest likely exposure to men in the United States, Men’s Health (average 150,000+ circulation per issue) and Men’s Fitness (average 50,000 circulations per issue) were selected for the study (ABC, 2016).

The magazine issues were accessed electronically via the Texture digital magazine subscription service by two researchers (W.H. and A.H.) (Next Issue Media LLC, 2018). The study was limited to evaluating magazines from the United States as the authors were limited to electronic access within the United States via the Texture digital magazine subscription service (Next Issue Media LLC, 2018). Issues from each magazine with a publication date between January 2016 and December 2016 were eligible for study inclusion. Men’s health-related magazine issues published outside the 2016 time frame, magazines that primarily targeted women and magazines that primarily target men without a primary focus on health (e.g., GQ, Swagger, and Muscle Machines) were excluded from the study.

Data Collection and Outcomes Measures

The previously referenced study comparing recommendations from two health television shows was used to provide guidance in calculating the needed sample size (Korownyk et al., 2014). It was estimated that at least 54% of the recommendations would have evidence addressing them, then calculated and that 146 recommendations would give an 80% chance that the 95% confidence interval (CI) would have a precision within ±10%. To attain at least 146 recommendations, it was estimated researchers would need to review at least three to six issues from each magazine (Cutilli, 2010; Midwest Pharmacists Residents Conference, 2015). To ensure equal distribution throughout the year, author M.A.J. randomly selected one issue from each magazine from January to April, May to August, and September to December 2016 using the randomizer.org website. If additional issues for a magazine were required to obtain 80 recommendations, the originally chosen issues were retained and the selection process was repeated until the 80 recommendation threshold was achieved.

The primary objective of this study was to determine and compare percentages of health recommendations in two popular men’s health-related magazines that are addressed by peer-reviewed scientific evidence. For evidence found, the percentage of recommendations were further stratified into three outcomes: (a) evidence supports recommendation, (b) evidence is unclear (i.e., available evidence is inconclusive), or (c) evidence refutes recommendation. Secondary outcomes included characterization of each recommendation regarding topic area, and when found, the level of the medical evidence (e.g., systematic review/meta-analysis, randomized control trial, prospective or retrospective cohort study, and case-control or case series/case report) that addressed each recommendation.

Identification of Recommendations

Two researchers (W.H. and A.C.) independently surveyed the randomly selected magazine issues to identify and document the specific health recommendations made. To ensure independence, the researchers were encouraged to work in different locations as they submitted their extracted recommendations using an online data platform. The researchers documented the following from each recommendation: Magazine A or B classification, month of publication, page number of recommendation, exact quotation of recommendation, specific intervention, specific outcome, classification as a strong or weak recommendation, and recommendation topic area.

A health recommendation was considered for study inclusion only if it met the following selection criteria: (a) statement or sentence began with a verb addressing an action, (b) statement or sentence was “strongly worded,” and (c) statement contained a specific intervention associated with a specific health benefit or harm. Statements such as “To immediately lower your risk of cardiovascular disease, add 30 grams of nuts a day [to your diet],” or “. . .include a push, pull, lower-body multi-joint or core [interval to] each workout to prevent muscle pain development toward the end of the workout,” were both classified as recommendations for study purposes since they met all three criteria. A statement such as “Do the exercise no one else does, this will make you a champion” was not classified as a recommendation as no specific health benefit or harm was mentioned. A statement such as “Consider a flu vaccine this season as a way to possibly prevent workout fatigue” was excluded as it was not considered a “strong” recommendation due to the weak verbiage of the term “consider.” Similar to the methodology employed in the BMJ study, instances where the article statement author claimed they would personally undertake a specific intervention or recommendation were considered “strong” endorsements or recommendations (Korownyk et al., 2014). Recommendations were included if they were for or against an intervention as long a specific benefit or harm was associated with the intervention. For example, a recommendation discouraging the use of an intervention due to a perceived harm was included in this study as long as it met the three primary inclusion criteria.

It was observed early in data collection that more than one recommendation was often made within a single article, and sometimes within a single paragraph. For example, “Zinc boosts immune-response time by activating T cells to attack viruses and bacteria”. . .“Studies have shown that zinc is particularly effective at combating the rhinovirus and can help you get better faster”. . .“Don’t use the nasal sprays, which can affect your sense of smell and make you nauseated. Stick to lozenges (about 60 mg, three times a day).” In the above illustrative example, researchers created two separate recommendations and linked them with the identified benefit: (a) zinc use leads to faster resolution of cold/flu symptoms and (b) zinc lozenge administration is preferable to zinc nasal spray administration, as lozenges are less likely to affect the smell or cause nausea relative to nasal sprays. In the infrequent instances where the benefited outcome was dependent upon the combination of two or more specific interventions, the interventions were combined and analyzed as a single recommendation with a single benefit.

Researchers (M.A.J. and E.J.I.) refereed discrepancies in the appropriateness of inclusion of a recommendation between W.H. and A.C. Researchers (M.A.J. and E.J.I.) subsequently met on a single day to review all of the recommendations and confirm a final decision regarding recommendation inclusion for study evaluation. All of the included recommendations were further categorized regarding their specific topic area (e.g., exercise health, nutritional health, over-the-counter [OTC] medication use, consultation with a health care provider, and complementary/alternative medicine related or general health [e.g., immunizations, medical device use or other]) by a single researcher (M.A.J.). A flow diagram further detailing sample selection is provided in Figure 1.

Figure 1.

Flow chart of recommendation selection.

Systematic Search for Supportive/Conflicting/No Evidence

Recommendations, which were identified by researchers (W.H. and A.C.), were converted into a searchable clinical question by an evidence-based medicine expert on the research team (M.A.J.). Next, two researchers in the group (M.A.J. and N.A.) independently searched the literature for evidence addressing each clinical question while blinded to the originating magazine (Men’s Health or Men’s Fitness) of the recommendation. Available databases used for literature search included: PubMed (MEDLINE), Cochrane Database of Systematic Reviews, The Cochrane Central Register of Controlled Trials (CENTRAL), Natural Medicines, Trip Database, Google Scholar, and Google databases during May 2017 to September 2017. To standardize the search process, both researchers were instructed to use PubMed (MEDLINE), The Cochrane Central Register of Controlled Trials (CENTRAL), Trip Database, and Google Scholar for their searches regardless of the topic. Researchers were also advised to use other databases (e.g., Natural Medicines, CINAHL, and PyscINFO) based upon the overall topic of the created clinical question. As previous research has determined that adequate and efficient systematic review searches should include Embase, MEDLINE, Web of Science, and Google Scholar, the searches employed a strategy to ensure these databases would be included (Bramer et al., 2017). The two researchers (M.A.J. and N.A.) were generally allotted 30 min per question to conduct their systematic search for evidence. If 30 min had lapsed and no evidence had been found, researchers marked the question “no evidence.” Expert opinion articles were excluded from the literature search, as a majority of the recommendations from the men’s health-related magazines were by definition, being made by proclaimed experts. For this study, systematic review/meta-analyses were considered the highest level of available evidence and researchers were encouraged to search for this type of article first. Also, researchers were encouraged to use a provided evidence pyramid to guide their searches if they did not find any research in the highest level (Murad et al., 2016). For example, if the researcher could not find a systematic review/meta-analysis, they would then search for a randomized control trial. If a randomized control trial could not be found, then a search for a cohort study was conducted and this process was repeated until reaching the lowest level of evidence. Case series or case reports were considered the lowest level acceptable per the provided evidence pyramid (Murad et al., 2016).

If a researcher found a recent systematic review/meta-analysis specifically addressing the clinical question, the literature search was concluded. Researchers were encouraged to continue searching for evidence of the same or higher level (e.g., additional randomized control trials or other cohort studies) when a recent systematic review/meta-analysis was not readily identified. In searches where there was more than one level of study available, only studies from the highest level were retained (e.g., all randomized control trials). When it remained unclear whether evidence was available for a clinical question, an additional literature search was performed. When the researchers found different evidence regarding the same clinical question, the evidence from the highest level (e.g., randomized control trial) was selected to be retained and presented to the health experts. All relevant studies were subsequently summarized by two researchers (M.A.J. or N.A.) for review by the team of health experts. All study summaries included the following information in the following order: primary author last name, year of publication, study type, objective of the study, patient population, intervention, comparator, primary and secondary outcomes with statistic information, and reference link. Evidence grading systems (e.g., GRADE, SIGN, or USPSTF) were not included in the summaries to prevent any bias on how the health experts evaluated the literature summaries (Harbour et al., 2001; Harris et al., 2001; Guyatt et al., 2008).

Health Expert Review Consensus

The five health experts were chosen to represent a wide range of diverse clinical fields and included a Doctor of Osteopathic Medicine (DO), Doctor of Medicine (MD), Doctor of Pharmacy (PharmD), Doctor of Philosophy (PhD) of Nutrition, and an Exercise Scientist/Certified Strength and Conditioning Specialist (BA). It was determined a priori that any expert who reported a conflict of interest (e.g., financial relationship with a manufacturer or recommended product) would be excluded from the particular recommendation. No such relationships were identified in the course of the evaluations.

The five health experts (I.J.L., B.L., E.I., N.B., and M.J.T.) were presented with the 161 clinical questions. All five health experts independently reviewed the original magazine recommendation, the generated clinical question, and the summary of the found studies from the available literature. The health experts were given the universal resource locators (URLs) for all of the presented literature and encouraged to review the full-text articles before providing a final evaluation. The health experts were blinded to the magazine origin of the recommendation and independently evaluated the presented data to minimize bias. The health experts then independently voted if each recommendation was supported, refuted, or remained unclear (i.e., inconclusive) based on the available peer-reviewed scientific literature. The health experts submitted their votes using an online data platform to streamline data collection and verify their independence in evaluating the recommendations.

Statistical Analysis

This study utilized descriptive statistics to characterize the recommendations from the men’s health-related magazines. Rates reporting when evidence was provided in the literature for a given recommendation, along with the majority consensus of the health experts, was conveyed as frequency and percentage. It was decided a priori that in instances of a “tie,” for example, two health experts for “refute,” two health experts for “support,” and the fifth expert for “evidence unclear,” that a sixth expert rater would act as the tie-breaker between the top two choices. Rate of actual agreement among the expert reviewers was calculated using Krippendorff’s α, a relative conservative measure of inter-rater reliability which can be generalized across the nominal, ordinal, interval, and ratio variable types and is compatible with two or more raters per observation (Halgren et al., 2012). In addition, Krippendorff’s α is more flexible with missing observations than traditional measures, such as Cohen’s κ (Halgren et al., 2012; Hayes et al., 2007; Krippendorff et al., 2004). Comparison of rates for evidence found in the literature, as well as differences in the expert majority consensus, that is, support, refute, or evidence unclear, between the two magazines were made using a Chi-square (χ²) test. An α level of 0.05 was considered statistically significant. All results were analyzed using SAS version 9.4 (SAS Institute, Cary, NC).

Results

Overview

Baseline Characteristics of Recommendations

Three issues of Men’s Health magazine (Magazine A) and six issues of Men’s Fitness magazine (Magazine B) were selected to be reviewed for the study. The median age of readers from both magazines averaged 42.5 years of age and approximately a third of readers graduated from a college/university. The baseline characteristics of both magazines are shown in Table 1.

Table 1.

Baseline Characteristics of Men’s Health-Related Magazines (Men’s Health, 2016).

	Men’s Health Magazine	Men’s Fitness Magazine
Publication year	2016	2016
Number of issues published per year	10+	12+
Number of magazine issues evaluated in study	3	6
Specific publication months evaluated in study	April, May, November	January, March, August, September, November, December
Total audience (online and print)	13,392,000 (Men’s Health, 2017)	8,674,000 (Men’s Health, 2017)
Median age of readers (years)	43.6 (Men’s Health, 2017)	41.3 (Men’s Health, 2017)
Median household income of readers (HHI)	$84,637	$80,109
Marital status of readers: married	52.9%	50.3%
Readers who graduated college plus	34%	30.5%
Readers who own home	62.3%	58.2%
Editor(s) in chief during publication year	Matt Bean (Dool, 2016) Bill Philips (Steigrad, 2016)	Michael De Mederio (American Media Inc, 2012)
Publication location edition	United States edition	United States edition
Accessibility to researchers	Digital access	Digital access
Subscription access for readers/researchers	Paid subscription required	Paid subscription required
Number of raw recommendations extracted from magazine	168	119
Average number of extracted recommendations per issue	56	19.8

Recommendation Selection

One hundred sixty-eight recommendations were extracted from three issues of Men’s Health magazine (Magazine A), while 119 recommendations were extracted from six issues of Men’s Fitness magazine (Magazine B) (Figure 1). Eighty recommendations were retained from Men’s Health, and 81 recommendations were retained from Men’s Fitness after applying the inclusion and exclusion criteria.

Recommendation Categories

Specific categories of the recommendations are presented in Table 2. The most common category in both magazines was nutrition-related, namely 38.8% and 42.0%, for Men’s Health and Men’s Fitness, respectively. The second most common category for both magazines was exercise-related recommendations. In total, nutrition or exercise-related recommendations comprised a total of 63.8% and 80.2% of recommendations in Men’s Health and Men’s Fitness, respectively. Other frequent categories included OTC medication topics and healthcare provider consultation advice.

Table 2.

Categorization of Recommendations for Each Magazine.

Category of Recommendations	Men’s Health Magazine(n = 80)	Men’s Fitness Magazine (n = 81)
Exercise-related, N (%)	20 (25.0%)	31 (38.3%)
Nutrition-related, N (%)	31 (38.8%)	34 (42.0%)
Over-the-counter (OTC) medication related, N (%)	6 (7.5%)	3 (3.7%)
Consultation with a health-care provider related, N (%)	10 (12.5%)	0 (0.0%)
Complementary/alternative medicine related, N (%)	2 (2.5%)	7 (8.6%)
Other (General health, immunizations, medical devices, etc.) N (%)	11 (13.8%)	6 (7.4%)

Evidence Base for Recommendations

A summary of the peer-reviewed scientific evidence available for the 161 randomly selected recommendations found in both magazines is presented in Table 3. In total, peer-reviewed evidence was found addressing 69 (42.8%) of the 161 recommendations. After reviewing the literature for the 69 recommendations with available evidence, the panel of health experts rated 38 (23.6%) of the 161 as being supported, 20 (12.4%) as having unclear evidence and 11 (6.83%) as being refuted. In total, 76.4% (123/161) health-related recommendations in the two men’s health-related magazines lacked consistent peer-reviewed evidence. Consensus or agreement among the five expert reviewers, as measured by Krippendorff’s α, was above the threshold generally considered as an acceptable level of agreement (0.67, 95% CI [0.55, 0.77]) (Halgren et al., 2012; Hayes et al., 2007; Krippendorff et al., 2004).

Table 3.

Literature Evidence for Both Magazines Combined.

	Recommendations from Both Magazines (n = 161)
No evidence found, N (%)	92/161 (57.2%)
Literature found, N (%)	69/161 (42.8%)
Evidence supports recommendation*, N (%)	38/161 (23.6%)
Evidence refutes recommendation*, N (%)	11/161 (6.8%)
Evidence unclear*, N (%)	20/161 (12.4%)

Expert opinion consensus, Krippendorff’s α inter-rater reliability of 5 raters = 0.67 (95% CI [0.55, 0.77]).

A statistical analysis did not find any significant differences in whether the found evidence supported, refuted, or was unclear between the two magazines (Table 4). Only 22.5% and 24.7% of the recommendations found in Men’s Health and Men’s Fitness, respectively, had clear supportive evidence as determined by a panel of health experts reviewing the current literature (p = .743). However, significant differences were found in the percentage of recommendations supported by systematic reviews/meta-analyses between the two magazines. More specifically, 16.3% of the recommendations found in Men’s Health were supported by systematic reviews/meta-analyses, while 27.2% of the recommendations in Men’s Fitness were supported by such a review (p = .041) (Table 5).

Table 4.

Comparison of Literature Evidence by Magazine.

	Men’s Health Magazine (n = 80)	Men’s Fitness Magazine (n = 81)	p value
Literature found, N (%)	34 (42.5%)	35 (43.2%)	.945
Evidence supports Recommendations*, N (%)	18 (22.5%)	20 (24.7%)	.743
Evidence refutes Recommendations*, N (%)	4 (5.0%)	7 (8.6%)	.360
Evidence unclear*, N (%)	12 (15.0%)	8 (9.9%)	.251
Evidence unclear, refuted or no literature found, N (%)	62 (77.5%)	54 (75.3%)	.830

Expert opinion consensus, Krippendorff’s α inter-rater reliability of 5 raters = 0.67 (95% CI [0.55, 0.77]).

Table 5.

Secondary Outcomes.

Highest Type of Human-Based Medical Literature Found to Answer Clinical Questions	Men’s Health Magazine(n = 80)	Men’s Fitness Magazine (n = 81)	p value
Systematic review/meta-analysis, N (%)	13 (16.3%)	22** (27.2%)	0.041
Randomized control trial, N (%)	8 (10%)	6 (7.4%)	0.561
Open-label study, N (%)	5 (6.25%)	2 (2.5%)	0.256
Retrospective or prospective cohort study, N (%)	3 (3.75%)	5 (6.17%)	0.710
Cross-sectional study, N (%)	5 (6.25%)	0** (0%)	0.025

Expert opinion consensus, Krippendorff’s α inter-rater reliability of 5 raters = 0.67 (95% CI [0.55, 0.77]).

p < .05 between Men’s Health Magazine and Men’s Fitness Magazine.

Table 6.

Categorization of Recommendations for Men’s Health Magazine (N = 80).

Recommendation Category(N = # of Recommendations)	No Evidence Found	Evidence SupportsN (%)	Evidence RefutesN (%)	Evidence UnclearN (%)
All (N = 80)	46 (57.5 %)	18 (22.5%)	4 (5%)	12 (15%)
Nutrition related (N = 31)	13 (41.9%)	9 (29.0%)	3 (9.7%)	6 (19.4%)
Exercise related (N = 20)	13 (65.0%)	5 (25.0%)	—	2 (10.0%)
Over-the-counter (OTC) medication related (N = 6)	5 (83.3%)	—	—	1 (16.7%)
Consultation with a health-care provider related (N = 10)	8 (80.0%)	2 (20.0%)	—	—
Complementary/alternative medicine related (N = 2)	1 (50.0%)	—	—	1 (50.0%)
Other (general health, immunizations, medical devices, etc.) (N = 11)	6 (54.5%)	2 (18.2%)	1 (9.1%)	2 (18.2%)

Table 7.

Categorization of Recommendations for Men’s Fitness Magazine (N = 81).

Recommendation Category(N = # of Recommendations)	No Evidence Found	Evidence Supports (%)	Evidence Refutes (%)	Evidence Unclear (%)
All (N = 81)	46 (56.8%)	20 (24.7%)	7 (8.6%)	8 (9.9%)
Nutrition related (N = 34)	16 (47.1%)	10 (29.4%)	1 (2.9%)	7 (20.6%)
Exercise related (N = 31)	23 (74.2%)	5 (16.1%)	3 (9.7%)	—
Over-the-counter (OTC) medication related (N = 3)	—	3 (100%)	—	—
Consultation with a health care provider related (N = 0)	—	—	—	—
Complementary/alternative medicine related (N = 7)	1 (14.3%)	3 (42.8%)	2 (28.6%)	1 (14.3%)
Other (general health, immunizations, medical devices, etc.) (N = 6)	2 (33.3%)	2 (33.3%)	1 (16.7%)	1 (16.7%)

Discussion

Making strongly worded specific health-related recommendations occurs commonly in popular men’s health-related magazines. In this study, over 150 such health recommendations were found in just nine issues of two common magazines published within a single calendar year. The most common recommendations involved nutrition or exercise advice. Less than one-fourth (23.6%) of the recommendations were clearly supported by peer-reviewed scientific evidence. Further, when evaluating current literature, the panel of health experts decidedly refuted occasional recommendations made in both magazines. Thus, a majority of the frequent health-related recommendations made by popular men’s health-related magazines appear to lack credible peer-reviewed evidence.

Other researchers have reviewed the quality of health-related information disseminated to the public by television shows or evaluated the lifestyle impact of exposure to health-related magazines. Korownyk et al. (2014) compared the quality of recommendations between two syndicated health television shows, finding that only 32.5% to 52.5% of their recommendations had at least some level of evidence when reviewing peer-reviewed literature. Their rates were higher than what was discovered in this study of men’s health-related magazine recommendations. While the current amount of any professional medical interventions/recommendations with supportive, credible evidence is purported to be below 50%, this study found that men’s health-related magazine-based recommendations appear to be much lower (Institute of Medicine [US], 2008; Masic et al., 2008). Individuals who read magazine-like articles focusing on diet and exercise often try to follow the recommendations being promoted (Ricciardelli et al., 2000; Utter et al., 2003). Utter et al. evaluated the impact of reading weight loss magazine articles on health behaviors, finding that increasing the frequency of magazine reading increased the likelihood of engaging in healthy weight control behaviors among adolescents (Utter et al., 2003). Since many individuals follow recommendations promoted in magazine articles, it is even more imperative that these recommendations are supported by evidence-based medicine.

Strengths and Limitations of the Study

Strengths of this study include the utilization of objective criteria to identify a recommendation, detailed data extraction, incorporation of thorough literature searches and summaries, and involvement of health experts from various practice and clinical settings. This was the first study to actively describe and compare the quality of recommendations in men’s health-related magazines. Multiple limitations to the study should be noted. First, the number of appropriate extractable recommendations per issue of magazine was difficult to accurately gauge prestudy and differed substantially between the two magazines. The random sample of issues from 2016 may not be reflective of the current content of the magazines due to frequent changes in article types, editors, writers, editorial advisory board members, and so forth. For example, Men’s Fitness was combined with another magazine, Men’s Journal during the 2017 to 2018 publication year (Men’s Journal Magazine, 2018).

Developing clinically searchable questions was the most challenging aspect of the study, often due to the generalizability or ambiguity regarding the stated recommendations or potential benefits. At times, clinical judgment on the part of the researcher (M.A.J.) was required to extrapolate an appropriate clinical question from the text. Other researchers trying to replicate this study may not generate identical clinical literature search questions even if utilizing the same articles. The search strategy was limited to two researchers (M.A.J. and N.A.) who, at baseline had varying degrees of medical literature search training. However, N.A. received comprehensive training from M.A.J. prior to performing the literature searches to ensure comparable search quality. Ideally, M.A.J should not have developed the clinical questions and been involved in the medical literature extractions. Some recommendations did not have any readily searchable evidence due to the recommendation being considered “standard medical practice.” For example, a recommendation suggesting patients drink water to remain hydrated during workouts would likely not have randomized controlled trial addressing whether water is effective for staying healthy or hydrated compared to placebo (i.e., no water). The five health experts had varied levels of medical training, which may have affected the inter-rater agreement and reliability. The computed rate for Krippendorff’s α was 0.667, just at the lower limit to be considered accepted agreement. A subgroup analysis identified that the Krippendorff’s α values increased when only the votes of the physicians (MD or DO), or the practicing clinicians (MD, DO, or PharmD) were considered, suggesting that an expert panel excluding nonphysicians or nonclinicians may have yielded greater agreement (results not reported). Future researchers may use such information to guide research protocols for developing their panel of health experts.

Future Research and Insight

As the data were derived from 2016 issues, researchers may consider performing a follow-up study on more recent issues to compare the quality of recommendations within a more contemporary time frame. Researchers may also consider implementing a similar study with other consumer health-related magazines (Health, The Good Life, and Prevention) that target other specific patient demographics. Licciardone et al. (2001) reported that newspapers or magazines were recently rated as the most likely publication types for accessing health information within the United States, despite the perceived ease of use with the internet. In addition, health-related magazines are likely to be more accessible at health food stores, gyms, health facilities, or traditional book stores, where patients may be intrinsically motivated to engage in healthy activities. Future researchers could identify and evaluate the most commonly distributed health-related magazines at the popular health food store, gyms, health facilities, and so forth.

Men’s health-related magazines are popular and generally receptive in nature to feedback. It is hoped they wish to ensure that the majority of their health recommendations have credible evidence supporting them. To execute this intervention, magazines may consider collaborating with academic centers or health news review-like websites before their recommendations are communicated to the public (Schwitzer, 2007). Magazines could also collaborate with academic centers to ensure their journalists undergo training or utilize checklists to improve their health reporting skills (Ashoorkhani et al., 2012). Wilson et al. (2010) reported that the quality of health stories was significantly higher among journalists who were trained in health reporting versus journalists who were not. The utilization of health news review-like websites may also serve as an effective medium for training journalists on how to improve the quality of their health articles (Schwitzer, 2007). In a 2013 study, Schwitzer and his health news review team reviewed 1889 health stories from newspapers, magazines, and television studies to assess their quality. After looking at 7 years of data, researchers reported that the majority of stories were considered unsatisfactory since most did not clearly communicate costs, benefits, harms, quality of the evidence, or comparisons with alternative interventions (Schwitzer, 2013). Magazines may consider adopting the criteria used by such watchdog-like organizations to evaluate health articles prior to publication of related recommendations (Schwitzer, 2007, 2013).

Media-related regulatory organizations (e.g., Federal Communications Commission and Centers for Medicare and Medicaid Services) may consider requiring that all consumer publications report any financial relationships or conflict of interests by all of the editorial team, editorial advisory board members, and authors within each issue. Such transparency could help ensure readers that magazine-based health recommendations are less likely to be influenced by commercial conflicts of interest (Bero et al., 2005).

Magazine editors may also consider making clear that patients should confirm implementing any intervention with their health-care provider by utilizing specific symbols or notification images. Consistent graphic notifications may instinctively remind men to start engaging with their health-care provider. Men are known to have an underutilization of primary care services, which has been a key factor linked to lower life expectancies when compared to women (Baker et al., 2014). Consistent notifications may prompt primary care visit follow-ups for men to allow them to verify the effectiveness of implementing any magazine-based health recommendations.

Finally, health-care providers should consider investigating whether patients have adopted any new magazine-based health recommendations during routine visits and clarify whether there is any evidence addressing the new lifestyle. Health-care providers should be open to verifying recommendations that are suggested by patients after reading health-related magazines. To validate such recommendations, clinicians can use the approach of converting the recommendations into a clinical question, searching the medical literature, and evaluating the found information. Historically, patients are more likely to utilize an intervention if their health-care provider utilizes the intervention for their own personal health or personally recommends it (Pearson et al., 2000; Shapiro, 2008; Thom et al., 2004).

Conclusion

Male consumers should be cognizant regarding recommendations provided in men’s health-related magazines. In this study of more than 160 recommendations made by two popular men’s health-related journals, only 38 (23.6%) were supported when evaluated by a panel of health experts reviewing summaries of available medical literature findings. Male consumers should engage with their health-care providers before implementing any medical interventions obtained from a magazine. Health-care providers should be vigilant in verifying recommendations that are presented from health magazines when engaging their patients.

Footnotes

Author Contributions

M.A.J. (Mohamed Alieu Jalloh) and E.J.I. (Eric J Ip) conceived and designed the study. A.C. (Alex Cullum) and W.A. (Walid Aljayosi, PharmD) extracted the magazine recommendations. N.A. (Nicholas Alonzo) helped to conduct a medical literature search for the magazine recommendations. E.J.I., B.L. (Bonnie Lau), N.B. (Nathalie Bergeron), I.L. (Ingrid Lopes), and M.J.T. (Michael J. Tenerowicz) served as the five health experts who reviewed the recommendations and provided their professional opinions. M.A.J., M.J.B., and E.J.I. acquired, analyzed, or interpreted the data. M.J.B. conducted the statistical analysis. M.A.J., M.J.B., and E.J.I. drafted the manuscript and revised the manuscript prior to submission.

Disclaimers

The views expressed in the submitted article are from the listed authors and not an official position of their institution or any funder.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: All authors have completed the ICMJE uniform disclosure form at and declare: no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years; and no other relationships or activities that could appear to have influenced the submitted work.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Mohamed A Jalloh

References

ABC. (2016, October). https://www.abc.org.uk/

Alliance For Audited Media (2016, October). The Top 10 U.S. Magazines with Circulation Gains. https://auditedmedia.com/

American Media Inc. Cision PR Newswire. (2012, January). Michael De Medeiros Named Editor-in-Chief of Men’s Fitness. https://www.prnewswire.com/news-releases/michael-de-medeiros-named-editor-in-chief-of-mens-fitness-138133553.html

Ashoorkhani

Majdzadeh

(2012). Improving the quality of health news. International Journal of Preventative Medicine, 3(7), 440–443.

Baker

Dworkin

S. L.

Tong

Banks

Shand

Yamey

(2014). The men’s health gap: men must be included in the global health equity agenda. Bulletin of The World Health Organization, 92(8), 618–620.

Bala

M. M.

Strzeszynski

Topor-Madry

Cahill

(2013). Mass media interventions for smoking cessation in adults. Cochrane Database of Systematic Reviews, 6(6), CD004704.

Bero

L. A.

Glantz

Hong

M. K.

(2005). The limits of competing interest disclosures. Tobacco Control, 14(2), 118–126.

Bramer

W. M.

Rethlefsen

M. L.

Kleijnen

Franco

O. H.

(2017). Optimal database combinations for literature searches in systematic reviews: A prospective exploratory study. Systematic Reviews, 6(1), 245.

Crimmins

E. M.

Shim

Zhang

Y. S.

Kim

J. K.

(2019). Differences between men and women in mortality and the health dimensions of the morbidity process. Clinical Chemistry, 65(1), 135–145.

10.

Cutilli

C. C.

(2010). Seeking health information: what sources do your patients use? Orthopaedic Nursing, 29(3), 214–219.

11.

Dool

(2016, October 18). Big data and entrepreneurship: Matt Bean’s vision for men’s Health. Folio Mag. https://www.foliomag.com/big-data-entrepreneurship-matt-beans-vision-mens-health/

12.

Dworkin

S. L.

Treves-Kagan

Lippman

S. A.

(2013). Gender-transformative interventions to reduce HIV risks and violence with heterosexually active men: A review of the global evidence. AIDS and Behavior, 17(9), 2845–2863.

13.

Guyatt

G. H.

Oxman

A. D.

Vist

G. E.

Kunz

Falck-Ytter

Alonso-Coello

Schünemann

H. J.

, & GRADE working Group. (2008). GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. British Medical Journal, 336(7650), 924–926.

14.

Halgren

K. A.

(2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23–34.

15.

Harbour

Miller

(2001). A new system for grading recommendations in evidence based guidelines. British Medical Journal, 323(7308), 334–336.

16.

Harris

Atkins

Berg

A. O.

Best

Eden

K. B.

Feightner

J. W.

US preventative services task force. (2001). US preventive services task force procedure manual. Agency for Healthcare Research and Quality

17.

Hayes

A. F.

Krippendorff

(2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1(1), 77–89.

18.

Hunt

Wyke

Gray

C. M.

Anderson

A. S.

Brady

Bunn

Donnan

P. T.

Fenwick

Grieve

Leishman

Miller

Mutrie

Rauchhaus

White

Treweek

(2014). A gender-sensitised weight loss and healthy living programme for overweight and obese men delivered by Scottish Premier League football clubs (FFIT): A pragmatic randomised controlled trial. Lancet, 383(9924), 1211–1221.

19.

Institute of Medicine (US). (2008). Evidence-based medicine and the changing nature of healthcare: 2007 IOM annual meeting summary. National Academies Press (US).

20.

Korownyk

Kolber

M. R.

McCormack

Lam

Overbo

Cotton

Finley

Turgeon

R. D.

Garrison

Lindblad

A. J.

Banh

H. L.

Campbell-Scherer

Vandermeer

Allan

G. M.

(2014). Televised medical talk shows—What they recommend and the evidence to support their recommendations: A prospective observational study. British Medical Journal, 349, g7346.

21.

Krippendorff

(2004). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30(3):411–433.

22.

Licciardone

J. C.

Smith-Barbaro

Coleridge

S. T.

(2001). Use of the internet as a resource for consumer health information: Results of the second osteopathic survey of health care in America (OSTEOSURV-II). Journal of Medical Internet Research, 3(4): E31.

23.

Lindegren

M. L.

Kennedy

C. E.

Bain-Brickley

Azman

Creanga

A. A.

Butler

L. M.

Spaulding

A. B.

Horvath

Kennedy

G. E.

(2012). Integration of HIV/AIDS services with maternal, neonatal and child health, nutrition, and family planning services. Cochrane Database of Systematic Reviews, 9(9), CD010119.

24.

Maibach

E.W.

Weber

Massett

Hancock

G. R.

Price

(2006). Understanding consumers’ health information preferences: Development and validation of a brief screening instrument. Journal of Health Communication, 11(8): 717–736.

25.

Marx

J. J

Nedelmann

Haertle

Dieterich

Eicke

B. M.

(2008). An educational multimedia campaign has differential effects on public stroke knowledge and care-seeking behavior. Journal of Neurology, 255(3): 378–384.

26.

Masic

Miokovic

Muhamedagic

(2008). Evidence based medicine - new approaches and challenges. Acta Informatica Medica, 16(4), 219–225.

27.

Men’s Health. (2017, April). Media Kit. Men’s Health Website. https://www.menshealthmediakit.com/wp-content/uploads/2017/04/18-001_MK_MEDIA-KIT.pdf

28.

Men’s Journal Magazine Editors. (2018). Meet the New Dynamic Duo of Print Publishing: ‘Men’s Fitness’ and ‘Men’s Journal.’ Men’s Journal Website. https://www.mensjournal.com/health-fitness/meet-new-dynamic-duo-print-publishing-mens-fitness-and-mens-journal/

29.

Midwest Pharmacists Residents Conference. (2015, May). Recommendations in men’s health-related magazines evidence-based? A retrospective analysis. http://www.mprconference.com/wp-content/uploads/2009/10/2015PresentationSchedule_Website.pdf

30.

Murad

M. H.

Asi

Alsawas

Alahdab

(2016). New evidence pyramid. British Medical Journal Evidence-Based Medicine, 21(4), 125-127.

31.

Next Issue Media LLC. (2017). Texture. https://www.texture.com/

32.

Pearson

S. D.

Raeke

L. H.

(2000). Patients’ trust in physicians: many theories, few measures, and little data. Journal of General Internal Medicine 15(7):509–513.

33.

Ricciardelli

L. A

McCabe

M. P.

Banfield

(2000). Body image and body change methods in adolescent boys. Role of parents, friends and the media. The Journal of Psychosomatic Research, 49(3), 189–197.

34.

Schwitzer

(2007) HealthNewsReview.org: Criteria for excellence in health and medical journalism. Virtual Mentor 9(3), 225–228.

35.

Schwitzer

(2013). Addressing tensions when popular media and evidence-based care collide. BMC Medical Informatics and Decision, 13(Suppl 3), S3.

36.

Shapiro

(2008). Walking a mile in their patients’ shoes: empathy and othering in medical students’ education. Philosophy, Ethics, and Humanities in Medicine, 3, 10.

37.

Steigrad

(2016, June 10). Men’s Health Editor in Chief Bill Phillips Out Amid Turmoil at Rodale. WWD Website. https://wwd.com/business-news/media/mens-health-editor-in-chief-bill-phillips-out-michael-lafavore-letter-maria-rodale-10451037/

38.

The Alliance For Audited Media. (2016). The Top 10 U.S. Magazines with Circulation Gains. https://auditedmedia.com/

39.

Thom

D. H

Hall

M. A.

Pawlson

L. G.

(2004). Measuring patients’ trust in physicians when assessing quality of care. Health Affairs, 23(4), 124–132.

40.

Tilburt

J. C

Allyse

Hafferty

F. W.

(2017). The Case of Dr. Oz: ethics, evidence, and does professional self-regulation work? American Medical Association Journal of Ethics, 19(2), 199–206.

41.

Utter

Neumark-Sztainer

Wall

Story

(2003). Reading magazine articles about dieting and associated weight control behaviors among adolescents. Journal of Adolescent Health, 32(1), 78–82.

42.

Wilson

Robertson

McElduff

Jones

Henry

(2010) Does it matter who writes medical news stories? Public Library of Science (PLoS) Medicine, 7(9), e1000323.

43.

World Health Organization, Regional Office for Europe. (2013). Review of social determinants and the health divide in the WHO European Region: final report http://www.instituteofhealthequity.org/projects/who-european-review

Men’s Health-Related Magazines: A Retrospective Study of What They Recommend and the Evidence Addressing Their Recommendations

Abstract

Keywords

Methods

Men’s Health-Related Magazine Selection

Data Collection and Outcomes Measures

Identification of Recommendations

Systematic Search for Supportive/Conflicting/No Evidence

Health Expert Review Consensus

Statistical Analysis

Results

Overview

Baseline Characteristics of Recommendations

Recommendation Selection

Recommendation Categories

Evidence Base for Recommendations

Discussion

Strengths and Limitations of the Study

Future Research and Insight

Conclusion

Footnotes

Author Contributions

Disclaimers

Declaration of Conflicting Interests

Funding

ORCID iD

References