Content and Face Validation of the Lifestyle Medicine Assessment

Abstract

Introduction: There are no validated global lifestyle medicine brief screening tools that measure health behaviors in all six lifestyle domains. The Lifestyle Medicine Assessment (LMA) tool was initially developed and revised based on feedback elicited from colleagues, experts, and patients. During the developmental process, every item underwent language changes. Three of the original 24 items were removed. However, there have not been any formal validation efforts. This study aims to formally evaluate the face and content validity of the LMA. Methods: A survey was emailed to 12 board-certified lifestyle medicine experts asking them to rank items in the LMA on a 1-4 scale for content relevance and clarity. Content and face validity were quantitatively determined using the item-level content validity index (I-CVI), scale-level content validity index (S-CVI), and item-level face validity index (I-FVI), scale-level face validity index (S-FVI), respectively. Literature accepted thresholds of I-CVI/I-FVI ≥.79 and S-CVI/S-FVI average ≥ .80 were used. Results: Eleven experts returned evaluations of the 21-item LMA. All 21 items had I-CVI for relevance ≥.91 and I-FVI ≥.81 with excellent kappa values. The S-CVI/I-FVI average for relevance and clarity were .99 and .95, respectively. Conclusion: The 21-item LMA is a brief global lifestyle medicine tool that has demonstrated excellent content and face validity.

“The LMA was developed as a global screening tool that quickly assesses the most impactful health behaviors based on current evidence.”

Introduction

It is estimated that behavioral patterns are responsible for ∼40% of premature death^1,2 and may be responsible for 78% of chronic diseases. ³ Major drivers of morbidity and mortality include poor diet, physical inactivity, and substance use.^3,4 The impact of sleep, stress management, and connectedness on overall health and well-being has been increasingly recognized as well.^5–7 This recognition has given rise to the field of lifestyle medicine, which focuses on improving health behaviors in these domains to prevent, treat, and potentially reverse chronic disease. ⁸

Despite the tremendous interest and growth in the field, there are not universally recognized brief validated global lifestyle medicine specific screening tools that cover the six pillars of lifestyle medicine. The Health-Promoting Lifestyle Profile II instrument is a validated option of 52 items that cover physical activity, nutrition, spiritual growth, interpersonal relations, and stress management. It also includes health responsibility, though it does not address substance use. ⁹ Surveys such as the RAND-36 focus on quality-of-life measures as opposed to monitoring lifestyle behaviors. ¹⁰ Other tools like the American Heart Association’s Life’s Simple 7 combine biometrics and behaviors to estimate heart disease risk. ¹¹ While it includes physical activity, tobacco use, and diet, the Life’s Simple 7 lacks other behaviors that have been shown to significantly impact morbidity and mortality.

Targeted domain specific validated tools are available for each lifestyle medicine domain;^12–17 however, it is impractical and often unnecessary to use 5 or 6 different survey instruments for patients to investigate each domain. A rapid measure that assesses all these components would improve the ease of evaluating and monitoring lifestyle behaviors. The Lifestyle Medicine Assessment (LMA) was designed to address this need. ¹⁸

The original LMA items were chosen to reflect the 6 domains (diet, physical activity, social connection, sleep, stress management, and substance use) of the American College of Lifestyle Medicine (ACLM). ⁸ Domain specific validated instruments were used to identify relevant questions and were adapted, when possible. Other impactful health behaviors were then added to their respective domains to succinctly cover the most influential health behaviors in each domain.

The LMA items focus exclusively on modifiable health behaviors that are all mapped to research that has demonstrated their impact on morbidity or mortality. The tool itself is organized into 5 domain areas: connectedness, movement, nutrition, recovery (combining sleep and stress), and substance use. Zero to 10 points can be earned in each area, with a potential total lifestyle score of 50 points. Each item is weighted in an attempt to reflect the magnitude of its effect on morbidity and mortality (e.g., not using tobacco is worth 6 points whereas resistance training is worth 2 points). Higher scores reflect healthier behaviors, and presumably better health outcomes.

As part of an initial quality improvement project, the original LMA was sent electronically to approximately 52 lifestyle medicine professionals and researchers for feedback. It was also given to 10 patients who were asked to provide verbal feedback regarding the tool, domains, items, and wording. Revisions were made to the survey based on suggestions provided. The updated survey was then re-circulated to eight of the original reviewers.

During this process, all 24 items underwent some type of revision. For most of the items, this consisted of minor wording changes to make the items easier to understand by participants. However, 3 of the 24 items were ultimately removed based on feedback from colleagues and experts. The 3 removed items evaluated stretching duration each week, ingestion of illegal substances, and number of missed doses of prescription medications.

This original LMA, along with its rationale and design, were published in 2021 in the Lifestyle Medicine Handbook 2nd Edition. ¹⁸ Subsequent to that, the 21-item version has undergone minor language revisions to reach its current form. It has been translated into a free printable PDF and an Excel version through the American Academy of Family Physicians (AAFP) (https://www.aafp.org/family-physician/patient-care/prevention-wellness/healthy-lifestyle.html). ¹⁹ An online version has also been made freely available by Fundación Mapfre in English and Spanish (https://survey.fundacionmapfre.org/index.php/211124?lang=en). ²⁰ Both the Excel file and online versions automatically calculate the domain specific and lifestyle scores. The PDF includes manual scoring instructions. Figure 1 includes the current LMA items. Figure 2 is an example of the Excel file results page. Figure 3 is an example of the online version results page.

Figure 1.

Lifestyle medicine assessment form. Note: A PDF version with item weighing and scoring instructions is freely available at https://www.aafp.org/family-physician/patient-care/prevention-wellness/healthy-lifestyle.html under ‘Lifestyle Medicine Assessment (LMA) Tool: Survey Format.

Figure 2.

Electronic LMA example results page. Excel file can be found at https://www.aafp.org/family-physician/patient-care/prevention-wellness/healthy-lifestyle.html.

Figure 3.

Online LMA results example. Courtesy of Fundación Mapfre: https://survey.fundacionmapfre.org/index.php/211124.

The LMA can be completed by individuals in a few minutes and can be repeated as often as weekly, or at any lesser frequency desired. Importantly, the behavioral nature of the items communicates the need to practice these habits consistently over time to reap their health rewards.

The LMA is being widely used by clinicians around the world. However, no formal validation has been done. This study examines the face and content validity of the current LMA instrument.

Methods

Formal face and content validation generally require a minimum of 5-10 experts to have sufficient control over chance agreement. ²¹ To achieve this range, twelve lifestyle medicine experts were invited to participate. They were identified based on their role nationally within leading national lifestyle medicine organizations including the American Board of Lifestyle Medicine and the American College of Lifestyle Medicine. Participation was voluntary and no financial incentives were provided. Electronic invitations were sent requesting their opinions about the relevancy and clarity of each of the current 21-item LMA version. Experts were asked to rank each item on a 1-4 ordinal scale for relevancy and clarity, with 1 being not relevant/clear and 4 being very relevant/clear. Table 1 describes the scoring guide provided to the experts. ²¹

Table 1.

Expert Item Scoring Guide.

Relevancy	Clarity
1 [not relevant]	1 [not clear]
2 [item needs some revision]	2 [ item needs some revision]
3 [relevant but needs some minor revision]	3 [clear but needs minor revision]
4 [very relevant]	4 [very clear]

The content validity index (CVI) and face validity index (FVI) were calculated for all individual items (I-CVI/I-FVI) and for the overall scale-level content/face validity index (S-CVI/S-FVI). The I-CVI/I-FVI is expressed as a fraction between 0 and 1 (higher numbers being better) and was calculated as the number of experts rating the item a 3 or 4 for relevancy or clarity, divided by the total number of experts. The S-CVI/S-FVI was calculated for relevancy by averaging all the I-CVI/I-FVI relevancy scores (S-CVI/Ave/S-CVI/Ave) and using the universal agreement approach (S-CVI/UA). Previous research has proposed that 80% agreement or higher among judges be used as an accetpable threshold for new instruments.^21–23 Judgment on each item was made based on the I-CVI/I-FVI as follows: items with I-CVI/I-FVIs >79% are considered appropriate, 70%–79% require revision, and <70% are eliminated.^21,24 Kappa (K) was calculated for all items using the I-CVI/I-FVI to evaluate the possibility of chance agreement using the following formulas:

• P_c = [N!/A! (N-A)!]*0.5^N

• K = (I-CVI-P_c)/(1-P_c)

• K = (I-FVI-P_c)/(1-P_c)

P_c: probability of chance agreement

N: number of experts

A: number of experts ranking an item with a 3 or 4

Kappa values of .74 or higher are considered excellent, .60-.74 are considered good, and .40-.59 are considered fair.^21,25

Microsoft Excel was used for data entry and tabulation of I-CVI/I-FVI, S-CVI/S-FVI, and kappa. This study was determined to be exempt by the Stanford University IRB (eProtocol #: 72 027).

Results

Eleven of the 12 (92%) board-certified lifestyle medicine experts invited to participate returned their surveys and their responses were included in the analysis. Additional demographic data were not collected. Item domain, relevance, and clarity scores, along with kappa calculations are included in Table 2. All 21-items had content relevance I-CVI scores ≥.91 with associated kappa scores ≥.91. All 21-items had clarity I-FVI scores ≥.82 with associated kappa scores ≥.81. The whole instrument S-CVI/Ave and S-FVI/Ave scores were .99 and .95, respectively. The S-CVI/UA was .90.

Table 2.

LMA Item Categorization and Calculated Item/Scale Clarity and Face Validity Indices.

Item #	Domain	Relevance		Clarity
		I-CVI	Kappa	I-FVI	Kappa
1	Connectedness	1	1	1	1
2	Nutrition	.91	.91	.91	.91
3	Connectedness	1	1	1	1
4	Recovery (sleep and stress)	1	1	1	1
5	Connectedness	1	1	1	1
6	Substance use	1	1	.91	.91
7	Connectedness	1	1	1	1
8	Recovery (sleep and stress)	1	1	1	1
9	Connectedness	1	1	1	1
10	Recovery (sleep and stress)	.91	.91	.82	.81
11	Nutrition	1	1	.82	.81
12	Movement	1	1	.91	.91
13	Nutrition	1	1	.82	.81
14	Substance use	1	1	1	1
15	Nutrition	1	1	.91	.91
16	Recovery (sleep and stress)	1	1	.91	.91
17	Nutrition	1	1	1	1
18	Movement	1	1	1	1
19	Substance use	1	1	.91	.91
20	Nutrition	1	1	1	1
21	Movement	1	1	1	1
S-CVI/Ave .99 S-FVI/Ave .95

Legend: I-CVI (item-content validity index), I-FVI (item-face validity index), S-CVI/Ave (scale-content validity index average), S-FVI/Ave (scale-face validity index average).

Discussion

The LMA was developed as a global screening tool that quickly assesses the most impactful health behaviors based on current evidence. International adoption and widespread use have demonstrated its clinical utility. Initial face validity was sought by using the ACLM’s lifestyle medicine domains to create the initial questions, with item revisions occurring based on feedback from patients and providers. This study used a quantitative validation process with board-certified lifestyle medicine experts to test the content and face validity of the LMA.

The literature suggests that I-CVI/I-FVI of new instruments should be ≥ .80 to be considered appropriate.^21–23 All 21-items of the LMA demonstrated a relevance I-CVI of .91 or higher, suggesting that they are all appropriate (19 items had a I-CVI of 1 and 2 items had a I-CVI of .91). The likelihood of chance agreement was very low as demonstrated by the kappa values for all items being >.90 (values ≥.74 are considered excellent).^21,25 The complete instrument performed extremely well as evidenced by the S-CVI/Ave of .99 (the maximum score is 1). Using the more stringent universal agreement approach, the S-CVI/UA was still .90. Similarly, the I-FVI for all 21-items was at least .82 with a kappa ≥.82. The S-FVI/Ave of .95 indicates that the overall face validity is very good too.

Collectively, these quantitative content and face validation processes strongly support the current version of the LMA. The overall performance of the tool in this study is likely a product of the multiple iterations the instrument received prior to this formal evaluation. Conceptually, the tool has remained largely unchanged since its inception, though it has benefited substantially from the feedback of lay and content expert audiences. The initial informal face validation processes helped refine the language and design of the instrument, making it both clinically relevant and patient friendly. The formal quantitative validation process among board-certified lifestyle medicine experts builds upon this work to demonstrate quantitative face and content validity.

These initial validation processes are a critical first step in having a standardized assessment tool to quickly screen for healthy behaviors. The relative brevity of this tool makes it easy to use during any preventive or chronic disease management visit. It can quickly identify lifestyle domains that are worthy of additional investigation. It can prompt specific follow up questions or suggest strategic use of domain specific questionnaires. This saves patient and clinician time. It also helps direct the conversation towards areas of a patient’s health that may warrant attention and that could have the largest impact on their overall health and well-being.

Limitations

Establishing a new survey instrument without similar comparators is challenging. It is resource prohibitive to compare the LMA to gold standard measures in all lifestyle medicine domains, so fundamental survey validation techniques must be used. The board-certified lifestyle medicine expert participants in this study represent a select sample who were chosen specifically for their national leadership roles within established lifestyle medicine organizations. This may have skewed the feedback in ways that could make the tool less generalizable. Although all surveys were done individually, similar experiences, biases, and perceptions may be held by this cohort of experts. While this study described the face and content validation of the tool, the predictive capacity of the tool has not been demonstrated. It is conceivable that every item and point in the LMA is meaningful, though larger studies in patient populations with associated health outcomes will be required to demonstrate this. As new research emerges, it is possible that some items may need to be added, removed, or edited to reflect current evidence.

Future Directions

Future studies should address other aspects of validation. They should also explore additional applications of the LMA tool in specific patient populations. Of particular interest will be the potential predictive validity and ability to discern meaningful health outcomes based on LMA scores. It will also be valuable to investigate how tools like the LMA can be combined with more domain specific in-depth questionnaires. For example, the LMA provides a quick snapshot of basic diet principles. However, complementing the LMA with a diet-specific tool could prove valuable.

Conclusion

This study demonstrated the face and content validity of the 21-item Lifestyle Medicine Assessment (LMA) tool. Items in the tool were shown to have high levels of relevance and clarity among board-certified lifestyle medicine experts. This initial validation work of the 21-item global LMA should lay the foundation for future lifestyle medicine assessment research.