The Fully Understanding Eating and Lifestyle Behaviors (FUEL) trial: Protocol for a cohort study harnessing digital health tools to phenotype dietary non-adherence behaviors during lifestyle intervention

Abstract

Objective

Lifestyle intervention can produce clinically significant weight loss and reduced disease risk/severity for many individuals with overweight/obesity. Dietary lapses, instances of non-adherence to the recommended dietary goal(s) in lifestyle intervention, are associated with less weight loss and higher energy intake. There are distinct “types” of dietary lapse (e.g., eating an off-plan food, eating a larger portion), and behavioral, psychosocial, and contextual mechanisms may differ across dietary lapse types. Some lapse types also appear to impact weight more than others. Elucidating clear lapse types thus has potential for understanding and improving adherence to lifestyle intervention.

Methods

This 18-month observational cohort study will use real-time digital assessment tools within a multi-level factor analysis framework to uncover “lapse phenotypes” and understand their impact on clinical outcomes. Adults with overweight/obesity (n = 150) will participate in a 12-month online lifestyle intervention and 6-month weight loss maintenance period. Participants will complete 14-day lapse phenotyping assessment periods at baseline, 3, 6, 12, and 18 months in which smartphone surveys, wearable devices, and geolocation will assess dietary lapses and relevant phenotyping characteristics. Energy intake (via 24-h dietary recall) and weight will be collected at each assessment period.

Results

This trial is ongoing; data collection began on 31 October 2022 and is scheduled to complete by February 2027.

Conclusion

Results will inform novel precision tools to improve dietary adherence in lifestyle intervention, and support updated theoretical models of adherence behavior. Additionally, these phenotyping methods can likely be leveraged to better understand non-adherence to other health behavior interventions.

Trial Registration

This study was prospectively registered https://clinicaltrials.gov/study/NCT05562427

Keywords

Dietary lapse adherence phenotyping lifestyle intervention overweight/obesity dietary assessment passive sensing ecological momentary assessment

Background

Overweight and obesity remain a significant public health problem, as these conditions contribute to increased risk for a multitude of chronic diseases as well as increased healthcare expenditure. Lifestyle interventions, the frontline non-surgical treatment for obesity and other weight-related chronic diseases, have been shown to produce clinically meaningful weight loss and reduced disease risk and severity.¹ These evidence-based interventions focus on helping individuals make gradual and sustainable changes to their exercise (e.g., increased aerobic activity) and eating (e.g., decreased caloric intake, improved diet quality), with the goal of achieving modest weight loss (5–10%) that typically confers important health benefits.² Poor adherence in these programs, particularly to the recommended dietary guidelines, has been shown to stymie weight loss and contribute to weight regain, thereby negating the potential health benefits.^3,4 Prior research has highlighted that dietary lapses (i.e., self-reported instances of non-adherence to the recommended dietary goal(s) in lifestyle interventions) occur frequently and are associated with less weight loss.^5–9 The impact of dietary lapses is likely, at least in part, a function of increased energy intake; a recent study demonstrated that individuals consumed an estimated average of 131.3 calories above their stated calorie goal on days when they reported having lapses compared to non-lapse days.¹⁰ Despite the known potential for dietary lapses to influence weight and overall intake, the mechanisms underlying this complex adherence behavior are still poorly understood.

Unlike most health behavior lapses (e.g., medication adherence, smoking), dietary lapses can manifest themselves through many different behaviors resulting in excess energy intake.^11,12 Data from studies utilizing ecological momentary assessment (EMA; repeated daily surveys of behavior and experiences administered via smartphone) indicate that individuals in lifestyle interventions tend to identify distinct behaviors associated with a dietary lapse (e.g., eating an off-plan food, planning a lapse, eating too large a portion of food).^5,13,14 These data have supported the concept of “dietary lapse types” (i.e., specific eating behavior(s) and contextual factors underlying a dietary lapse), and research thus far has shown that there is notable variability in the potential mechanisms and clinical impact of these different “lapse types.” For example, two studies of individuals with overweight/obesity enrolled in lifestyle interventions found that the different dietary lapse types experienced by participants shared some contributing behavioral, psychosocial, and contextual factors (e.g., sleep, having a prior lapse) but were distinct in others (e.g., race, cravings, planning meals).^13,15 Few participants in these studies (∼5%) reported engaging in only one lapse type throughout a weight loss attempt, highlighting that lapse behavior could vary within individuals (i.e., there is unlikely a type of lapser, rather people engage in multiple lapse types).^13,15 Importantly, dietary lapse types conferred different associations with weight change, indicating that some lapse types could be more detrimental than others for weight control.¹³ Taken together, these data point to substantial variability in lapse behaviors (i.e., a “heterogeneity problem”) that is likely obfuscating the mechanisms and consequences of lapses.^16,17

Having established that dietary lapse is a common and complex contributor to lifestyle intervention outcomes, an important next step of this research is to clearly differentiate dietary lapse types and their associated mechanisms, and understand their impact on clinical course and outcomes. The prior research showing that the mechanisms contributing to lapse may differ across dietary lapse types, and that these relationships may vary within individuals and over time, offers evidence for emerging “lapse phenotypes.” An example of how heterogeneity in lapse behaviors and mechanisms may result in potential lapse phenotypes can be seen in Figure 1, which depicts potential hypothetical phenotypes. In Figure 1, Person 1 experiences two kinds of lapses (eating larger portions, eating at unplanned times) that have differing phenotypes over the course of treatment. Person 2 experiences a unique lapse type (eating avoided foods) and also shares a lapse type with Person 1 (eating at unplanned times), but the phenotypic expression of that shared lapse type looks very different. For Persons 1 and 2, lapse phenotypes also differentially change over time.

Figure 1.

Depiction of hypothetical lapse phenotypes with mechanisms that vary within individuals and over time.

Behavioral phenotyping methods have strong promise to advance the study of dietary non-adherence by offering ways to understand heterogeneity in lapse behavior; behavioral phenotyping is a precision medicine framework that improves upon traditional “one-size-fits all” approaches by revealing underlying patterns that elucidate individual susceptibility to treatment failure, which can then inform tailored intervention.^18–21 As an example, the phenotype patterns from Person 1 in Figure 1 indicate that successful interventions to prevent lapsing via eating larger portions of food may directly target underlying mechanisms such as sleep (e.g., keeping a sleep diary, carefully planning sleep/wake times, reducing screentime before bed), location (e.g., just-in-time alerts when the person is near certain locations), and/or eating pace (e.g., instructions for mindful/intuitive eating, wearable devices to give feedback on slowing eating). Importantly, those intervention tools could dynamically shift as lapse phenotypes change over the course of treatment (e.g., when attempting to maintain their lost weight later in treatment, Person 1 may require interventions targeting mood and support during social situations).

This paper describes the design of a behavioral phenotyping trial that seeks to establish lapse phenotypes that can be directly targeted in future interventions to promote dietary adherence. Despite the promise of phenotyping methods, phenotyping studies for eating behavior have been limited to only a few phenotypes (e.g., food responsivity, loss of control eating) that have not been validated to show they increase susceptibility to reduced weight loss in lifestyle interventions.^22–25 Studies also typically cluster individuals via one-time assessment of unique characteristics, which fails to capture the time-varying nature of eating and its many contextual influences.^26,27 In contrast, this trial applies behavioral phenotyping to repeated, technology-mediated assessment data in order to understand a specific behavior within individuals, such as lapse. Such methods will allow for nuanced phenotyping that accounts for multiple sources of heterogeneity (e.g., heterogeneity between and within individuals, heterogeneity over time). The outcomes of this project will be a set of lapse phenotypes, and knowledge of their underlying mechanisms, that can inform personalized (likely digital) interventions to promote dietary adherence and thus improve weight loss outcomes in lifestyle interventions.

Methods

Objectives

This trial is sponsored by the National Institutes of Health/National Institute of Diabetes and Digestive and Kidney Diseases (NIH/NIDDK), and aims to conduct an 18-month observational cohort study focused on phenotyping dietary lapses from lifestyle intervention among individuals with overweight/obesity. This study protocol is based on an established behavioral phenotyping framework, which involves using rigorous assessment to establish underlying phenotypes of a behavior and then validating those emerging phenotypes by evaluating their associations with clinical outcomes.^28,29

Participants are enrolled into a well-established online lifestyle intervention for 12 months, with a 6-month maintenance period, for a total study participation of 18 months. The online intervention provides dietary recommendations with calorie goals for weight loss and maintenance; we are studying lapses from these dietary guidelines. Dietary lapses, phenotyping characteristics (i.e., behavioral, psychosocial, and contextual factors), and clinical outcomes (i.e., weight change and energy intake) are assessed during 14-day periods at baseline, 3, 6, 12, and 18 months. We include a comprehensive battery of phenotyping characteristics primarily measured via smartphone (EMA surveys and geolocation) and two wrist-worn sensors, and propose to analyze these data via multi-level factor analysis to derive meaningful lapse phenotypes from self-reported dietary lapses. We also assess weight change and energy intake during each assessment period, which are critical for understanding the impact of lapse phenotypes on clinical outcomes both concurrently (during the 14-day assessment) and prospectively (at subsequent assessment points). This study has three aims:

Aim 1: Establish dietary lapse phenotypes by identifying clusters of behavioral, psychosocial, contextual, and individual-level factors that differentiate lapse behaviors during weight loss and maintenance.

Aim 2: Test the association of dietary lapse phenotypes with energy intake and weight change during weight loss and maintenance to determine which lapse phenotypes have the greatest impact on lifestyle intervention outcomes.

Aim 3: Evaluate individual variability in the occurrence of dietary lapse phenotypes during weight loss and maintenance to determine the generalizability of lapse phenotypes across individuals.

Study population

We are currently recruiting men and women with body mass index (BMI) 25–50 kg/m², aged 18–70. Exclusion criteria are as follows: currently pregnant or breastfeeding, has been pregnant within 6 months prior to enrolling, plans to become pregnant within 6 months of enrolling, currently or recently (<6 months) enrolled in a commercial weight loss program, weight loss of ≥5% of initial body weight in the 6 months prior to enrolling, reports chest pain or loss of consciousness in the 12 months prior to enrolling, reports any medical condition that would affect the safety of participating in unsupervised physical activity or contraindicates weight loss, currently taking weight loss medication, reports surgical procedure for weight loss, reports any condition that would result in inability to follow the study protocol (e.g., terminal illness, substance abuse, or untreated major psychiatric illness), or history of a clinically diagnosed eating disorder excluding binge eating disorder.

Online lifestyle intervention: Rx Weight Loss

All participants in this trial receive a 12-month online lifestyle intervention, called Rx Weight Loss (RxWL), which is followed by a 6-month no-treatment weight loss maintenance period. The RxWL program has already been refined and validated in multiple prior NIH-funded trials, and has consistently produced average weight losses of 4–5% when tested with primary care patients, in worksites, and in community settings.^30–33 RxWL is compatible with most devices with access to the Internet, such as desktop computers, laptops, tablets, and smartphones. Because RxWL is standardized across participants and based on gold-standard practices set forth by the Diabetes Prevention Program (DPP) and Look AHEAD trials, it represents a rigorous and controlled setting in which to study dietary lapse.^34,35 RxWL consists of: (a) 45 min introductory session in which research staff introduces the program eating and activity goals, how to use RxWL, and provides brief instruction on self-monitoring; (b) 12 weekly 10–15 min interactive multimedia lessons for training in behavioral skills based on the intervention approach used in the DPP and Look AHEAD trials, followed by 9 monthly lessons for continued weight loss and/or maintenance; (c) online self-monitoring of weight, diet, and physical activity with the option of using the FitBit platform for recording these data; (d) 12 weeks of weekly automated feedback on the aforementioned self-monitoring record, followed by automated feedback once monthly for 9 months; and (e) “drop-in” hours held via videoconference by the study principal investigator (a licensed clinical psychologist) on a weekly (followed by monthly) basis. Consistent with DPP and Look AHEAD, there is one lesson that teaches general skills for coping with lapses and relapses, but dietary lapses are not directly targeted in this program.

Participants are given a goal of losing 1–2 lbs (0.45–0.91 kg) per week to achieve a total weight loss of 5–10% of initial body weight. To achieve this weight loss goal, participants are given a daily calorie goal and dietary recommendations (described further below). Participants are also given a physical activity goal starting in week 3, in which they are encouraged to do aerobic activities of their choosing (e.g., brisk walking, biking, swimming). There is a weekly minutes goal, tailored to initial activity level (e.g., 10 min/day over 5 days per week) and increasing each week to a maximum 200 min per week (e.g., 40 min/day over 5 days per week).³⁶ Participants are instructed to follow these diet and physical activity guidelines, as well as self-monitor weight, diet, and physical activity for the 12-month intervention period; they are encouraged to continue using these strategies but will not have access to the RxWL platform during the 6-month maintenance period. Only lapses from the dietary guidelines will be studied in this trial.

Consistent with prior work, participants in RxWL are given a calorie goal of 1200–1800 kcal/day tailored on initial weight (starting weight <250 lbs (113.4 kg) = 1200–1500 kcal/day, starting weight >250 lbs (113.4 kg) = 1500–1800 kcal/day) and are also provided guidelines to follow their choice of a low-fat or Mediterranean diet.^37–39 Lapses from these dietary guidelines are defined as “eating or drinking in which you exceeded your allotted calorie target for a meal or snack.”^6,8 At the outset of the program, participants are given instructions (by staff and via the RxWL online platform) to segment their daily calorie goal into specific calorie targets for each meal and snack. Setting calorie targets for meals and snacks is consistent with strategies employed in most lifestyle interventions to ensure that individuals are not consistently exceeding their daily calorie goal (e.g., meal planning tools).² Importantly for this trial, setting calorie targets for each meal and snack also creates an objective definition of “dietary lapse” for each participant to report on during the 14-day assessment periods. While participants are encouraged to self-define their meal and snack targets to meet their specific needs and eating patterns, they have the option to begin with a suggested set of targets that has been tested in our prior research: ∼15% of daily calories for breakfast, ∼25% for lunch, ∼40% for dinner, and two snacks of ∼10% each.^6,7 Participants are regularly reminded to review their meal and snack targets in the RxWL platform, are encouraged to change them at any time (e.g., if the previous targets felt frustrating or unrealistic), and given guidance in how to make those decisions for themselves. These reminders occur on a weekly basis for the first 12 weeks and monthly during the remainder of the study. If a participant sets their calorie targets too low (e.g., meal and snack targets are only accounting for 1000 kcal per day instead of the recommended 1500 kcal), participants are prompted via RxWL to choose more appropriate targets. Lastly, project staff monitor the targets and contact a participant directly if they persist in attempting to make inappropriate selections.

Procedures

Setting

This single-site trial is being conducted at an academic medical center in Rhode Island, USA. This research is approved by The Miriam Hospital Institutional Review Board (IRB). Trial conduct is reviewed quarterly by an independent safety monitor, and reviewed by the IRB and study sponsor annually. Protocol modifications are reported to the IRB as they arise and to the study sponsor annually.

Recruitment and enrollment

Participant recruitment began in September 2022 and is anticipated to close in August 2025 in order to complete data collection by February 2027. Recruitment is occurring via advertisements in local media and targeted online advertising; flyers and advertisements posted in waiting rooms and exam rooms in primary care offices; community events; informational materials made available to employees in the study site's health system; and direct mailings. Interested individuals are given a brief study description and screened via a REDCap survey to determine eligibility. Those who appear eligible are invited to an online orientation (held via video call), where the informed consent is thoroughly reviewed and the first study visit is scheduled. Participants are given the option to provide written informed consent at the first study visit or sign electronically (via REDCap) prior to study initiation, in accordance with procedures approved by The Miriam Hospital IRB.

During the first in-person study visit, participants have height and weight measured by trained research staff and receive training in how to use the study equipment. Participants complete a 1-week run-in in which they must complete >70% of the prompted EMA surveys on their smartphone and wear both wrist devices for >10 h/day to proceed with the study. These data are discarded, but ensure that participants are willing to adhere to study procedures prior to beginning treatment. Participants return for a “kick-off” visit, in which they are trained to use the online lifestyle intervention. Immediately following this visit, participants begin the intervention as well as their baseline 14-day phenotyping assessment period.

Assessment protocol

Participants complete 14-day phenotyping assessment periods at baseline (first 2 weeks of intervention), 3, 6, 12, and 18 months (last 2 weeks of weight loss maintenance period). Baseline and 18-month assessments occur during active treatment because characterizing dietary lapses necessitates actively following a dietary prescription. Each assessment period involves two visits with trained research staff, one at the beginning and one at the end of the 14-day period. At the first visit, participants: have their height and weight measured, complete questionnaires, and undergo (re)training in using study equipment. Participants are instructed to wear the wrist sensor devices and complete the smartphone-based EMA surveys daily for 14 days, and complete three non-consecutive telephone-based 24-h dietary recalls to assess total energy intake. At the end of 14 days, participants will return for their second visit in which they have their weight measured, complete additional questionnaires, return study equipment, and receive payment. Each participant can earn up to $110 for each assessment ($550 total for the trial). Participants can earn up to $42 for wearing both wrist devices for >10 h per day ($1.50/d for each device), up to $28 ($2/d) for responding to >75% of the daily EMA surveys, and $30 for attending the study visits ($15/visit). Participants can earn a bonus $10 per assessment period for consistent overall adherence to EMA surveys and device wear. Participants receive text messages to remind them to charge and wear the wrist-worn devices. Participants without a personal smartphone are lent one during the study assessment periods to complete EMA. Staff regularly review EMA survey completion rates and, if non-adherence persists, the participant is offered (re)training and help with problem-solving technical barriers.⁴⁰ Individuals who drop-out of the study will no longer be followed (as measuring dietary lapses is dependent on attempting to follow a dietary prescription), but will be included in the proposed analyses if their data are adequate for modeling.

Participant data confidentiality is protected through a multi-tiered approach that includes using a unique identification (ID) number that will be used on all documents and data sources with no references to personal information and using encryption of stored and transmitted data to protect data collected via electronic means. A certificate of confidentiality has been issued by the National Institutes of Health for this research to prevent risk of forced third party disclosure, which protects the participant from other organizations and governmental bodies from accessing their de-identified study data and potentially using the highly granular location and behavioral pattern data to re-identify the individual. Adverse events are collected and reported using a standardized medical event form from the beginning of study-related procedures to the end of the study follow-up period for an individual participant.

Measures overview

Each 14-day phenotyping assessment period includes measurement of: phenotyping characteristics, dietary lapses, and clinical outcomes. Phenotyping characteristics and dietary lapse will be used to establish lapse phenotypes by identifying clusters of characteristics that differentiate lapse behaviors. The clinical outcomes will be used to evaluate associations of lapse phenotypes with energy intake and weight loss. To reduce bias, objective measures of behavior and environment are used where possible and subjective measures are assessed via EMA. EMA involves collecting data on behaviors, psychosocial experiences, and environmental conditions in near real-time over multiple occasions during an individual's daily life.⁴¹ Having multiple reports of a behavior or experience that are reported by an individual in close proximity to the event reduces the impact of self-report bias and thus supports more reliable and valid assessment.⁴²

Measures of phenotyping characteristics

The selected lapse phenotyping characteristics are based on the reciprocal interactions framework (RIF), rooted in Social Cognitive Theory,^43,44 which is a simple and comprehensive model that is a commonly used framework for studying adherence behaviors.^4,45,46 RIF offers three broad interacting factors that influence health behavior: personal factors (i.e., cognitive, affective, and physiological experiences), environmental factors (i.e., factors in someone's micro- or macro-level environment), and behavioral factors (i.e., routines, cues to action). Table 1 provides an overview of each phenotyping characteristic measure and its corresponding RIF construct, state of the evidence for its inclusion (i.e., published or unpublished evidence regarding role in adherence behavior more generally or dietary lapse, specifically), and methods of assessment.

Table 1.

Overview of lapse phenotyping characteristics.

RIF construct	Measure	State of evidence linking construct to dietary lapse types	Assessment method
Personal-affective	Affect	Published evidence (general adherence)^5,47–49	EMA
	Affect	Published evidence (lapse types)^13,15	EMA
	Energy	Published evidence (lapse types)^13,15	EMA
	Stress	Published evidence (general adherence)^5,48,50	EMA
	Stress	Published evidence (lapse types)¹³	EMA
Personal-cognitive	Dietary self-efficacy	Published evidence (general adherence)^51,52	EMA
	Dietary self-efficacy	Published evidence (lapse types)¹⁵	EMA
	Motivation for change	Published evidence (general adherence)^48,53–55	EMA
	Motivation for change	Published evidence (lapse types)¹⁵	EMA
	Reactions to lapse	Published evidence (general adherence)^21,56,57	EMA
	Executive function (EF)	Published evidence (general adherence)^58–60	EMA
	Executive function (EF)	Published evidence (lapse types)¹⁵	EMA
Personal-physiological	Homeostatic and hedonic hunger	Published evidence (general adherence)^4,5,61,62	EMA
	Homeostatic and hedonic hunger	Published evidence (lapse types)^13,15	EMA
	Satiety	Published evidence (general adherence)^4,63	EMA
	Food cravings	Published evidence (general adherence)^49,51	EMA
	Food cravings	Published evidence (lapse types)¹⁵	EMA
	Dietary composition	Unpublished evidence (general adherence)	EMA
	Dietary composition	Unpublished evidence (lapse types)	EMA
	Basal metabolic rate	Published evidence (general adherence)⁶⁴	Weight, height, age
Behavioral	Sleep	Published evidence (lapse types)¹⁵	ActiGraph xGT3X-BT
	Physical activity	Published evidence (general adherence)^49,65	ActiGraph xGT3X-BT
	Sedentary behavior	Published evidence (general adherence)^66,67	ActiGraph xGT3X-BT
	Objective eating characteristics	Unpublished evidence (lapse types)	ActiGraph GT9X
	Alcohol intake	Published evidence (general adherence)^68,69	EMA
	Alcohol intake	Published evidence (lapse types)¹⁵	EMA
	Adherence to treatment and self-monitoring	Published evidence (general adherence)^70,71	Logged via Rx WL platform
Environmental	Location	Published evidence (general adherence)⁴⁹	GPS/GIS
	Location	Published evidence (lapse types)¹³	GPS/GIS
	Social context and support	Published evidence (general adherence)^48,51,72	EMA
	Social context and support	Published evidence (lapse types)¹⁵	EMA
	Time of day	Published evidence (general adherence)⁵¹	Self-reported at lapse
	Time of day	Published evidence (lapse types)^13,15	Self-reported at lapse
	Television watching	Published evidence (lapse types)¹⁵	EMA
	Immediate food availability	Published evidence (general adherence)^5,51	EMA
	Immediate food availability	Published evidence (lapse types)^13,15	EMA
	Neighborhood socio-economics	Published evidence (general adherence)^73–75	Census tracts (Home Address)
	Food accessibility	Published evidence (general adherence)^76–78	GPS/GIS
–	Demographic information	Published evidence (general adherence)^73,74	Questionnaire
–	Demographic information	Published evidence (lapse types)¹⁵	Questionnaire

Ecological momentary assessment

This experience sampling method supports examination of the state-level factors (moment-to-moment changes) associated with dietary lapse types, as well as obtain a measure of trait-level factors (differences between individuals) contributing to lapse. EMA during the 14-day phenotyping assessment periods is facilitated by Catalyst (MetricWire Inc.), which is a web-based platform that allows researchers to specify EMA survey questions and determine the schedule by which participants are prompted to complete EMA surveys on their smartphone. The platform has a corresponding smartphone “app” that is downloaded onto the participants’ smartphones. The app prompts participants via vibration and audible tone to complete six self-report surveys at semi-random intervals throughout the day (every 2–3 h), beginning 1-h after the participant's stated wake time. Participants are given 90 min to respond to a survey before it expires, with reminders at 30, 60, and 75 min. Surveys have approximately 40–45 questions (dependent on number of meals/snacks and lapses reported). While this number of questions is high compared to previous work, the brief nature of each question (e.g., “Did you eat a meal or snack…?,” “Rate the extent to which you feel happy right now…”) with consistent and simple response rating systems (e.g., Yes/No vs. rating 1–100) allows the surveys to be answered within approximately 5 min; this timeframe is roughly consistent with recommendations made in prior literature.^79,80 EMA surveys are only administered during the 14-day phenotyping assessment periods and include the following measures (see Supplementary File 1 in the online supplemental materials for a complete list of questions):

Affect and energy levels. Selected items from the Positive and Negative Affect Scale (PANAS)⁸¹ are used to assess momentary affect and energy levels. The PANAS has been validated for use via EMA.^82,83

Stress. Stress is being measured by three items of Cohen's Perceived Stress Scale⁸⁴ adapted for EMA.⁸⁵

Motivation for change and dietary self-efficacy. Consistent with research using EMA to study self-efficacy and motivation, participants are asked to rate their commitment to their weight loss goals⁸⁶ and their confidence in their ability to follow their diet.^51,56,87

Reactions to lapse. After reporting a lapse, participants respond to selected items from the Dichotomous Thinking in Eating Disorder Scale⁸⁸ and the Self-Compassion Scale-Short-Form,⁸⁹ which have been adapted for EMA and used in prior studies of dietary lapses.⁵⁷

Executive function. Availability of cognitive resources is assessed via two items from Quality of Life in Neurological Disorders (Neuro-QoL) item bank,⁹⁰ which has been adapted for EMA in previous studies.⁹¹

Hunger. Participants rate their homeostatic hunger (i.e., drive to eat due to physical hunger) and hedonic hunger (i.e., drive to eat in absence of energy need)⁹² using items from the Power of Food Scale that have been adapted for EMA.⁹³

Satiety. Consistent with prior research, participants rate current levels of desire to eat and how “satisfied” they feel with the amount of food eaten.⁹³

Food cravings. Consistent with our prior work and other studies of cravings,^6,94,95 participants are asked to rate the presence and intensity of food cravings.

Dietary composition. After reporting any eating event, participants choose the foods consumed from a checklist (e.g., starchy foods, cheese) based on the Dietary Targets Monitor, which has been adapted for EMA in prior work.^96,97

Alcohol intake. Participants are asked if they consumed alcohol since the prior EMA survey and, if yes, they are asked how many drinks were consumed.⁹⁸

Social context and support. Consistent with prior EMA studies evaluating social context and eating behaviors, participants are asked about whether other people were eating around them and to rate the level of approval and encouragement of others during the eating episode.⁹⁹

Television watching. Consistent with prior work, participants are asked to report if they have watched television (or online streaming equivalent) since the last survey.^6,100

Immediate food availability. Participants are asked to report if tempting, high-calorie foods or healthy foods have been visible and accessible,¹⁴ and then prompted to note any factors that made the food/drink easier to eat (regardless of whether or not they ate the food/drink), e.g., food in close proximity, low cost, being offered by someone

Objective measurement of eating

Participants are asked to wear the ActiGraph GT9X Link (ActiGraph, LLC, Pensacola, FL, USA) on their dominant wrist daily during waking hours for each 14-day phenotyping assessment period. The ActiGraph uses an inertial measurement unit (IMU) to detect wrist-roll motion as food is being brought to the mouth; at present, all-day use of IMU requires overnight charge and so participants are instructed to charge the device nightly. Members of the study team have developed classifiers that use the IMU data to estimate the number of bites taken during eating with 86% sensitivity¹⁰¹ and infer the duration of eating (i.e., start/stop times of eating) with up to 89% accuracy,^102,103 both of which can be used to calculate rate of eating (i.e., seconds per bite).¹⁰⁴ These eating characteristics will be calculated for lapse eating (reported via EMA, see below) and summarized for pre- and post-lapse (e.g., three eating episodes occurred before reported lapse, average duration of eating episodes prior to lapse was 5 min). Individuals often eat while engaging in other activities (e.g., walking, working, socializing) and accounting for these “secondary” activities can improve performance of eating detection and characterization algorithms.¹⁰⁵ Participants are therefore asked to self-report via EMA if they were engaging in other activities while eating (e.g., working at a desk, walking, chores) (see Supplementary File 1 in the online supplemental materials).

Objective measurement of physical activity (PA), sedentary behavior (SB), and sleep

While the ActiGraph GT9X Link typically assesses PA, SB, and sleep, the overnight charging requirement for enabling IMU precludes using it to monitor sleep and the dominant wrist-wear requirement for enabling eating detection makes it less ideal for valid PA and SB monitoring.^106,107 Participants therefore wear the xGT3X-BT ActiGraph (ActiGraph, LLC, Pensacola, FL, USA) on their non-dominant wrist for 24 h/day during each 14-day phenotyping assessment period. The xGT3X-BT accurately and reliably measures PA, SB, and sleep in free-living adults.^108–111 Variables of interest are total PA min/day, moderate-to-vigorous intensity PA (MVPA) min/day, and SB min/day. Validated cut-points will be used to categorize and timestamp bouts of PA, MVPA and SB for use in analysis.¹¹² The three-axis accelerometer and ambient light sensors produce validated estimates of total sleep time and sleep efficiency (i.e., time spent asleep vs. time spent in bed). Participants also self-report sleep/wake times as well as fatigue during EMA (see Supplementary File 1 in the online supplemental materials), which will be used to validate objective sleep data. Participants also complete the Paffenbarger Physical Activity Questionnaire¹¹³ at the end of each assessment period to recall PA amount and type over the previous 14 days, which will supplement objective data in the event of device failure and/or low wear-time.

Objective measurement of location and food accessibility

EMA will be integrated with Global Positioning System (GPS) tracking data, a method referred to as geographically explicit EMA, which allows for spatial analyses of relationships between participant self-report (e.g., lapses), location types (e.g., home), and objective environment data (e.g., food accessibility).¹¹⁴ The Catalyst app that is being used to facilitate EMA during the 14-day phenotyping assessment periods also automatically captures participants’ geographic location (i.e., latitude and longitude coordinates) via smartphone GPS receivers continuously throughout the day (every 2–3 min, based automatically on participant movement patterns) and at each completed EMA survey. Previous research has shown that smartphone GPS systems have high degree of positional accuracy for this proposed use.^115,116 By using Geographic Information Systems (GIS) to input and analyze GPS data, we will overlay participant location (e.g., home, work) and the food accessibility at that location. ArcGIS Pro software,¹¹⁷ along with a network database created by the study team from sources such as participant self-report (to identify home, work, school, friend/family residences) and known food-related business locations derived from public datasets (e.g., Data Axle of Wharton Research Data Services, health.ri.gov) will be used to: (1) geocode location from GPS coordinates, and (2) determine food accessibility by calculating the least-cost driving proximity to food source¹¹⁸ and density of food sources within a 0.5-mile radius (e.g., supermarkets, convenience stores, fast food) from GPS coordinates. EMA questions, such as the labeled location of each eating episode (e.g., home, work) and reports of being inside food-related establishments (vs. in a parking lot or store nearby), are being used to further contextualize results from GIS analysis (see Supplementary File 1 in the online supplemental materials).

Adherence to online lifestyle intervention

RxWL automatically records frequency of lesson access and days on which diet, physical activity, and weight were self-monitored each week.

Trait-level information

Trait-level variables are being assessed at baseline and updated at each assessment period. Demographic information includes factors such as: age, sex, gender identity, race, ethnicity, nativity, marital/relationship status, occupational status, level of education, and household income. The Harris–Benedict equation will be used to estimate basal metabolic rate.¹¹⁹ Census tracts will be used to extract median household income and proportion of the population whose income to poverty ratio is 100% below the federal poverty line.¹²⁰

Measurement of dietary lapses

Consistent with prior work, a lapse is defined for the participant as “eating or drinking in which you exceeded your allotted calorie target for a meal or snack.”^6,8 As described above, each participant receives extensive guidance and support in segmenting their daily calorie goal to create calorie targets for daily meals and snacks. Participants are also frequently reminded and encouraged to change these targets as necessary for themselves and these changes are tracked by the RxWL system. During the 14-day phenotyping assessment periods, participants are asked at each EMA survey to report if they have experienced a “lapse” since the last survey. Participants are also allowed to report a lapse at any time in the Catalyst app.⁴² Participants are asked to record the time of day that the lapse occurred. To obtain self-report descriptions of dietary lapse types, participants are asked “how would you describe the lapse?” and can select all that apply of the following options (“I ate a larger portion,” “I ate when I hadn’t intended,” “I ate a type of food I wanted to avoid,” “I planned to lapse,” “I did not know the calories in the food,” and “None of these descriptions fit my lapse”).¹⁵ Participants are (re)trained on reporting dietary lapses at each assessment period. Any dietary lapse that is reported via EMA with a valid timestamp will be considered a verified lapse; objective measurement of eating will not be used to verify lapses, as these methods for dietary lapse assessment (i.e., combining self-report with objective measures) have not been rigorously tested.

Measurement of clinical outcomes

The two clinical outcomes in this trial are weight change and energy intake, as both outcomes are impacted by lapse and are conduits to achieving the clinically meaningful distal health benefits of weight loss.^121–123

Weight and height

Body weight is measured in lab at the beginning and end of each assessment period. Weight is measured to the nearest 0.1 kg using a calibrated digital scale. Height is measured in centimeters using a wall-mounted stadiometer at the beginning of each assessment period. Measurements are made in light indoor clothing without shoes. Weight loss will be expressed as total percent weight loss from baseline to each assessment, percent weight loss between assessments (from day 14 of assessment #1 to day 1 of assessment #2), and percent weight loss during each 14-day assessment (from day 1 to 14).

Assessment of dietary intake

Energy intake is measured during each assessment period via three non-consecutive 24-h dietary recalls. Recalled intake is being collected over 2 weekdays and 1 weekend day, and weekend/weekday status will be controlled for in analyses. The Nutrition Data System for Research (NDSR), a computer-based software application developed at the University of Minnesota Nutrition Coordinating Center (NCC), facilitates the collection of recalls in a standardized fashion via a multiple pass interview approach.¹²⁴ The 24-h dietary recall has been shown to be highly valid and is therefore considered a gold-standard in dietary assessment.¹²⁵ NDSR will generate daily estimates of energy intake that will be used to characterize usual intake during each assessment period.

Statistical analysis plan

We propose to identify lapse phenotypes via clusters of behavioral, psychosocial, and contextual characteristics (aim 1), validate emerging lapse phenotypes by testing their associations with clinical outcomes of interest (aim 2), and evaluate individual variability in the occurrence of lapse phenotypes throughout lifestyle intervention (aim 3). Statistical analysis and reporting will follow best practices for EMA studies by following CREMAS (Checklist for Reporting EMA Studies) and STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) guidelines,^126,127 including a study flow diagram from initial eligibility assessment to study completion. All data will be checked for errors and cleaned prior to analysis. Data from EMA, geolocation, and wrist-worn sensors must also be processed and reduced. Preliminary analyses will include descriptive statistics and exploratory graphing for all variables of interest measured at all assessment points. Initial exploratory data analysis will be used to identify outliers such as measurement and recording errors, logical inconsistencies in data, and values extreme in the marginal distributions of the variables in question. Key baseline variables (e.g., baseline BMI, age, sex) will be considered for use as covariates in the below analyses. Under the assumptions that data are missing at random and distributions are normal, we will use maximum likelihood and Bayesian parameter estimation to obtain unbiased parameter estimates in the presence of missing data.¹²⁸ Other types of parameter estimation will be explored for non-normally distributed data. To allow for the possibility that the missing at random assumption may not hold, we will also use a third approach, pattern mixture models, in which the distribution of the outcome is assumed to follow a mixture of two distributions: one for those who complete follow up and another for those who do not.¹²⁹

We will use structural equation modeling approaches for the multilevel data generated by this intensive longitudinal study, which accommodates the clustered design, imbalance across persons over time, and non-independence of observations.¹³⁰ Aim 1 will use multilevel factor analyses within an exploratory structural equations modeling framework.^130,131 Multilevel factor analysis involves estimating within (repeated assessments within people) and between (person level) covariance matrices, and separately factor analyzing the covariance structure at these two levels. In the context of this study, where the multilevel structure derives from individuals repeatedly observed over time, the “within” level refers to specific lapse events and the “between” level refers to individual person level effects that carry across repeated assessment occasions. Strength of factor loadings will be summarized with the omega coefficient,¹³² and model fit will be evaluated using multiple metrics (e.g., root meal square residual, confirmatory fit index).^133,134 Results of these exploratory models will be evaluated for reproducibility and will be scrutinized using 10-fold internal cross-validation.¹³⁵ Where applicable, parameter estimates will include 95% confidence intervals. Multilevel factor analysis will allow us to identify latent structures underlying dietary lapse behaviors and interpret them to infer lapse phenotypes.

Aim 2 analyses will be conducted using multilevel location-scale models and estimated within a dynamic structural equation models (SEM) approach.¹³⁶ Dynamic SEM methods were developed to accommodate intensive longitudinal data and the location-scale model is notable for incorporating means and variances of personal characteristics into the model. We will initially consider the effects of each lapse phenotype on clinical outcomes in separate models and, if indicated, will explore more complex models (e.g., three or more phenotypes). Due to the potential multiplicity in hypothesis testing, we will use the Benjamini–Yekutieli false discovery rate limiting procedure to control for false discovery.¹³⁷ Aim 3 will use descriptive methods to understand variability of lapse phenotypes within participants over each assessment period, at the person level, and over time.

Sample size and power estimates

We will collect data from N = 150 participants across five measurement occasions (baseline, and months 3, 6, 12, and 18). Per prior research, we anticipate an average of eight lapses per participant during each assessment period. Accounting for 15% attrition, this leads to an expected total of 5541 lapses for analysis for Aim 1 among the 150 participants (between-level sample size), or 37 lapses per participant (within sample size). For Aim 1, these sample size estimates are consistent with recommendations for multilevel factor analysis that suggest a between-level sample size of 100 will have a 94.6% probability that within-level population values for factor loadings are contained within the sample-estimated 95% confidence intervals.¹³⁸ We confirmed these estimates using a simulation of a multilevel model confirmatory factor analysis. For Aim 2, we used the following parameters to determine minimum detectable associations between lapse phenotypes and clinical outcomes: with N = 150, accounting for 15% attrition, alpha = .05, power = .80, and estimated intraclass correlation coefficients of 0.31–0.50 for energy intake and weight change respectively based on our preliminary studies, we will be able to detect correlations as low as 0.16–0.19 (small effects in Cohen's taxonomy).¹³⁹ This is equivalent to 174 additional calories and .01 kg less weight loss per assessment period (14 days) per lapse. Importantly, we are powering on this minimum clinically significant effect, but expect much larger effects to provide compelling evidence of the clinical predictive validity of lapse phenotypes. For Aim 3, we determined that differences in our sample relative to expected populations values will likely be small (for N = 150, there is approximately 80% probability of returning a mean estimate that is within 0.11 SD units of the population mean).

Trial status

As of the date on this manuscript submission, the trial is ongoing and this manuscript represents protocol version 3 (revised twice following receipt of funding). Data collection began on 31 October 2022 and the last participant is scheduled to complete the study by February 2027. Because this research involves no more than minimal risk, there will be no interim analysis, and data and safety monitoring will occur in accordance with NIH and the IRB of record. Ethical approval was granted by The Miriam Hospital IRB and the trial was pre-registered in ClinicalTrials.Gov on 26 September 2022 prior to participant enrollment (NCT05562427).

Discussion

The above-described trial represents a novel application of behavioral phenotyping, a well-established strategy for supporting precision medicine, to facilitate a deeper understanding of the mechanisms underlying dietary non-adherence as well as the roles of specific non-adherence behaviors in lifestyle intervention outcomes. A major limitation of the current research on dietary non-adherence is that no single study has comprehensively assessed behavioral, psychosocial, and contextual characteristics of lapse. This trial therefore improves upon prior work by having an inclusive assessment battery based in an a-priori theoretical model, combining EMA with other rigorous measures (e.g., passive sensing), and adopting a full appreciation of momentary factors before, during, and after lapse. While we include a comprehensive battery of phenotyping characteristics, our proposed multi-level factor analyses will use these data to derive meaningful lapse phenotypes from known dietary lapse episodes so that future applications can then use focused measurement of the most salient variables that are relevant to lapse types of interest. Ultimately, results from this project are expected to inform future tailored intervention for dietary adherence as well as contribute to the science of adherence more broadly.

Existing interventions to improve dietary adherence in lifestyle interventions are typically based on the assumption that all dietary lapses contribute equally to clinical outcomes and share the same underlying mechanisms.^2,13,140 However, studies of lapses within both in-person and mobile lifestyle interventions have shown that not all lapse behaviors are associated with poor weight loss outcomes.^15,141 Attempting to intervene on lapse behaviors that are not associated with distal health outcomes may lead to administering unnecessary interventions and also reinforces an unrealistic goal that all lapses can be prevented, both of which can cause frustration and disengagement with intervention.^47,142–145 In contrast, the proposed work answers the question of which lapse types are most important to intervene on and identifies specific behavioral, psychosocial, and contextual mechanisms (i.e., drivers) of lapse that can be targeted by future theory-driven interventions.¹⁴⁶ In order to translate the findings from the current study into personalized intervention, the steps following this research could focus on: (1) developing and/or adapting approaches that effectively target the mechanisms underlying emerging lapse phenotypes, including understanding how to best target multiple potential co-occurring mechanisms of action (e.g., sleep, location, and, mood) for each phenotype; (2) optimizing the delivery modality and timing for such interventions (e.g., “just-in-time” adaptive interventions); and (3) devising digital health tools and methods that are sustainable for repeatedly assessing changing lapse phenotypes throughout treatment.

This project is also expected to advance the science of adherence through generalizable knowledge. Because lapse phenotypes are being studied in the context of a standardized online lifestyle intervention that is based on gold-standard practices, it is expected that the phenotypes identified during this trial will generalize to other similar evidence-based lifestyle intervention programs.¹⁴⁷ Future studies could evaluate whether results also generalize to other chronic conditions for which dietary adherence is critical (e.g., diabetes, heart disease, renal disease, gastrointestinal diseases). Additionally, the method of phenotyping specific lapse behaviors is a unique strength and innovation of this trial that could be leveraged for future research. While phenotyping has been used to inform precision approaches to many health behaviors and disease states, it is underutilized in the field of obesity. Especially with regards to eating behavior, phenotyping has been limited to using paper-and-pencil questionnaires to uncover static individual traits. However, these methods significantly underestimate the complexity and time-varying nature of adherence behaviors and their contributing factors.^4,48,148 Alternatively, our approach to phenotyping specific lapse behaviors via digital health tools instead of characterizing an individual as one type of lapser has the potential to offer a deeper understanding of behavior and more precise interventions. If successful, this methodology can ultimately be applied to study other important putative targets in lifestyle intervention, such as physical activity adherence.¹⁸ For example, while not as well-documented as dietary lapses, potential heterogeneity in types of physical activity “lapses” (e.g., missing an exercise day, exercising for fewer minutes than planned) is one indicator that this non-adherence behavior could benefit from the phenotyping approach applied in the current study.

This study has several limitations that may impact the proposed procedures and interpretation of resulting data. First, participants are required to wear two separate wrist-worn devices, one of which needs to be charged each night. While pilot data has indicated that participants will be willing to wear both devices for a short assessment period (14 days), it impacts the scalability of this protocol as well as compliance and data quality. By the end of the trial (2027), it is likely that passive sensing technology will have advanced enough so that all measurement functions can be accomplished via one device. If so, this limitation will have less of an impact on future research and practice. Second, the assessment of dietary lapse is subjective and dependent on self-report. Future research may consider validating methods for passively capturing lapse behaviors. Third, the list of phenotyping characteristics to be evaluated in this study is comprehensive, yet not exhaustive. The research team attempted to balance EMA survey length (as to not overburden participants) with obtaining a full understanding of the numerous factors influencing eating behavior (as to identify as many emerging lapse phenotypes as possible). We therefore included measures based on best available theory and prior evidence, and we favored state-level assessments (e.g., EMA questions, wearable sensors) over trait-level assessments given the known complexities of lapse behaviors. As a result, it is possible that participant burden of survey length may negatively impact EMA compliance and yet there may still be important state- and trait-level factors not measured in this study that will need to be investigated in future research (e.g., food reward sensitivity, delay discounting).

Conclusions

In sum, the proposed study is based on a strong foundation of preliminary data and is being conducted by a multidisciplinary team of researchers, with goals to advance the science of adherence behavior and clinical impacts of lifestyle interventions. This research is significant because it addresses the current limitations of lifestyle intervention by focusing on a primary driver of weight change and health benefit, dietary adherence. The project is highly innovative in its aim to phenotype dietary lapses and their mechanisms, rather than phenotyping individuals, to inform more nuanced intervention targets in the context of an already established and standardized online lifestyle intervention.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241271783 - Supplemental material for The Fully Understanding Eating and Lifestyle Behaviors (FUEL) trial: Protocol for a cohort study harnessing digital health tools to phenotype dietary non-adherence behaviors during lifestyle intervention

Supplemental material, sj-docx-1-dhj-10.1177_20552076241271783 for The Fully Understanding Eating and Lifestyle Behaviors (FUEL) trial: Protocol for a cohort study harnessing digital health tools to phenotype dietary non-adherence behaviors during lifestyle intervention by Stephanie P. Goldstein, Kevin M. Mwenda, Adam W. Hoover, Olivia Shenkle, Richard N. Jones and John Graham Thomas in DIGITAL HEALTH

Supplemental Material

sj-doc-2-dhj-10.1177_20552076241271783 - Supplemental material for The Fully Understanding Eating and Lifestyle Behaviors (FUEL) trial: Protocol for a cohort study harnessing digital health tools to phenotype dietary non-adherence behaviors during lifestyle intervention

Supplemental material, sj-doc-2-dhj-10.1177_20552076241271783 for The Fully Understanding Eating and Lifestyle Behaviors (FUEL) trial: Protocol for a cohort study harnessing digital health tools to phenotype dietary non-adherence behaviors during lifestyle intervention by Stephanie P. Goldstein, Kevin M. Mwenda, Adam W. Hoover, Olivia Shenkle, Richard N. Jones and John Graham Thomas in DIGITAL HEALTH

Footnotes

Acknowledgments

We would like to thank our project staff, as well as the current and future trial participants, for their efforts. KMM acknowledges support from the Population Studies and Training Center at Brown University, which receives funding from the NIH (P2C HD041020).

Contributorship

SPG is the Principal Investigator; she conceived the study, led the proposal and protocol development, and was the lead writer on this manuscript. KMM contributed to the study design and development of the proposal; he oversees all project components pertaining to geolocation (e.g., collection of global positioning system data, geographic information systems analysis). AWH contributed to the study design and development of the proposal; he oversees project components pertaining to wearable devices, with particular emphasis on processing, reducing, and aggregating data. OS contributed to study protocol development and is taking the lead on data acquisition for this project. RNJ contributed to the study design and the development of the proposal; he serves as the lead analyst. JGT assisted in study conception and contributed to the study design and development of the proposal; he oversees administration of the online lifestyle intervention. All authors read and approved the final manuscript.

Declaration of conflicting interests

SPG, KMM, AWH, OS, and RNJ declare no conflict of interest with respect to the research, authorship, and/or publication of this article. JGT participates in a scientific advisory board and serves as a paid consultant for Lumme Health, Inc. and Medifast, Inc.

Ethics approval and consent to participate

Study procedures have been approved by the Miriam Hospital Institutional Review Board (Board Reference #200622, Protocol ID: 1836936).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article. This work was supported by the National Institute of Diabetes and Digestive and Kidney Diseases [grant number: R01 DK132210; Principal Investigator: SPG] and the National Institute of General Medical Sciences [P20 GM139743] of the National Institutes of Health.

Guarantor

SPG.

ORCID iD

Stephanie P. Goldstein

Supplemental material

Supplemental material for this article is available online.

References

Poirier

Giles

Bray

, et al. Obesity and cardiovascular disease: pathophysiology, evaluation, and effect of weight loss: an update of the 1997 American Heart Association Scientific Statement on Obesity and Heart Disease from the Obesity Committee of the Council on Nutrition, Physical Activity, and Metabolism. Circulation 2006; 113: 898–918.

Wadden

Foster

. Behavioral treatment of obesity. Med Clin N Am 2000; 84: 441–461.

Alhassan

Kim

Bersamin

, et al. Dietary adherence and weight loss success among overweight women: results from the A TO Z weight loss study. Int J Obes 2008; 32: 985–991.

Middleton

Anton

Perri

. Long-term adherence to health behavior change. Am J Lifestyle Med 2013; 7: 395–404.

Forman

Schumacher

Crosby

, et al. Ecological momentary assessment of dietary lapses across behavioral weight loss treatment: characteristics, predictors, and relationships with weight change. Ann Behav Med 2017; 51: 741–753.

Goldstein

Zhang

Thomas

, et al. Application of machine learning to predict dietary lapses during weight loss. J Diabetes Sci Technol 2018; 12: 1045–1052.

Forman

Goldstein

Zhang

, et al. Ontrack: development and feasibility of a smartphone app designed to predict and prevent dietary lapses. Transl Behav Med 2019; 9: 236–245.

Goldstein

Thomas

Foster

, et al.

Refining an algorithm-powered just-in-time adaptive weight control intervention: a randomized controlled trial evaluating model performance and behavioral outcome. Health Informatics J 2020; 26: 2315–2331.

Goldstein

Brick

Thomas

, et al. Examination of the relationship between lapses and weight loss in a smartphone-based just-in time adaptive intervention. Transl Behav Med 2020; 11: 993–1005.

10.

Goldstein

Hoover

Evans

, et al. Combining ecological momentary assessment, wrist-based eating detection, and dietary assessment to characterize dietary lapse: a multi-method study protocol. Digital Health 2021; 7: 2055207620988212.

11.

Grilo

Shiffman

Wing

. Relapse crises and coping among dieters. J Consult Clin Psychol 1989; 57: 488.

12.

Johnson

Wildman

O’Brien

. The assessment of program adherence: The Achilles' heel of behavioral weight reduction? Behav Assess 1980; 2: 297–301.

13.

Forman

Goldstein

Zhang

, et al. Ontrack: development and feasibility of a smartphone app designed to predict and prevent dietary lapses. Transl Behav Med 2018; 9: 236–245.

14.

Thomas

Doshi

Crosby

, et al. Ecological momentary assessment of obesogenic eating behavior: combining person-specific and environmental predictors. Obesity 2011; 19: 1574–1579.

15.

Goldstein

Thomas

Brick

, et al. Identifying behavioral types of dietary lapse from a mobile weight loss program: preliminary investigation from a secondary data analysis. Appetite 2021; 166: 105440.

16.

Feczko

Miranda-Dominguez

Marr

, et al. The heterogeneity problem: approaches to identify psychiatric subtypes. Trends Cogn Sci 2019; 23: 584–601.

17.

Feczko

Fair

. Methods and challenges for assessing heterogeneity. Biol Psychiatry 2020; 88: 9–17.

18.

Bryan

Jakicic

Hunter

, et al. Behavioral and psychological phenotyping of physical activity and sedentary behavior: implications for weight management. Obesity (Silver Spring) Oct 2017; 25: 1653–1659.

19.

Yanovski

. Toward precision approaches for the prevention and treatment of obesity. Jama 2018; 319: 223–224.

20.

MacLean

Rothman

Nicastro

, et al. The accumulating data to optimally predict obesity treatment (ADOPT) core measures project: rationale and approach. Obesity (Silver Spring) 2018; 26: S6–s15.

21.

Kangovi

Asch

. Behavioral phenotyping in health promotion: embracing or avoiding failure. JAMA 2018; 319: 2075–2076.

22.

Asta

Miller

Retzloff

, et al. Eating in the absence of hunger and weight gain in low-income toddlers. Pediatrics 2016; 137: e20153786.

23.

Fisher

Birch

. Eating in the absence of hunger and overweight in girls from 5 to 7 y of age. Am J Clin Nutr Jul 2002; 76: 226–231.

24.

Boutelle

Peterson

Crosby

, et al. Overeating phenotypes in overweight and obese children. Appetite 2014; 76: 95–100.

25.

Tanofsky-Kraff

Cohen

Yanovski

, et al. A prospective study of psychological predictors of body fat gain among children at high risk for adult obesity. Pediatrics 2006; 117: 1203–1209.

26.

Brandl

Skurk

Rennekamp

, et al. A phenotyping platform to characterize healthy individuals across four stages of life – the enable study. Front Nutr 2020; 7: 582387–582387.

27.

Gibbons

Finlayson

Dalton

, et al. Metabolic phenotyping guidelines: studying eating behaviour in humans. J Endocrinol 2014; 222: G1–G12.

28.

Bouhlal

McBride

Trivedi

, et al. Identifying eating behavior phenotypes and their correlates: a novel direction toward improving weight management interventions. Appetite 2017; 111: 142–150.

29.

Chen

Changolkar

Navathe

, et al. Association between behavioral phenotypes and response to a physical activity intervention using gamification and social incentives: secondary analysis of the STEP UP randomized clinical trial. PLoS ONE 2020; 15: e0239288.

30.

Thomas

Leahey

Wing

. An automated internet behavioral weight-loss program by physician referral: a randomized controlled trial. Diabetes Care 2015; 38: 9–15.

31.

Ross

Wing

. Implementation of an internet weight loss program in a worksite setting. J Obes 2016; 1: 9372515.

32.

Leahey

Thomas

Fava

, et al. Adding evidence-based behavioral weight loss strategies to a statewide wellness campaign: a randomized clinical trial. Am J Public Health 2014; 104: 1300–1306.

33.

Thomas

Panza

Espel-Huynh

, et al. Pragmatic implementation of a fully automated online obesity treatment in primary care. Obesity 2022; 30: 1621–1628.

34.

Diabetes Prevention Program Research Group. The Diabetes Prevention Program (DPP): description of lifestyle intervention. Diabetes Care 2002; 25: 2165–2171.

35.

Look AHEAD Research Group, Wadden

West

, et al. The Look AHEAD study: a description of the lifestyle intervention and the evidence supporting it. Obesity (Silver Spring) 2006; 14: 737–752.

36.

Piercy

Troiano

Ballard

, et al. The physical activity guidelines for Americans. Jama 2018; 320: 2020–2028.

37.

Esposito

Kastorini

Panagiotakos

, et al. Mediterranean diet and weight loss: meta-analysis of randomized controlled trials. Metab Syndr Relat Disord 2011; 9: 1–12.

38.

Estruch

Ros

Salas-Salvadó

, et al. Primary prevention of cardiovascular disease with a Mediterranean diet. N Engl J Med 2013; 368: 1279–1290.

39.

Shai

Schwarzfuchs

Henkin

, et al. Weight loss with a low-carbohydrate, Mediterranean, or low-fat diet. N Engl J Med 2008; 359: 229–241.

40.

Ziesemer

König

Boushey

, et al. Occurrence of and reasons for ‘missing events’ in mobile dietary assessments: results from three event-based ecological momentary assessment studies. JMIR Mhealth Uhealth 2020; 8: e15430.

41.

Stone

Shiffman

. Ecological momentary assessment (EMA) in behavioral medicine. Ann Behav Med 1994; 16: 199–202.

42.

Shiffman

Stone

Hufford

. Ecological momentary assessment. Annu Rev Clin Psychol 2008; 4: 1–32.

43.

Schunk

Dibenedetto

. Motivation and social cognitive theory. Contemp Educ Psychol 2019; 60: 101832.

44.

Bandura

. Social foundations of thought and action: a social cognitive theory. Englewood Cliffs, NJ, USA: Prentice-Hall, 1986.

45.

Baglione

Gong

Boukhechba

, et al. Leveraging mobile sensing to understand and develop intervention strategies to improve medication adherence. IEEE Pervasive Comput 2020; 19: 24–36.

46.

DiClemente

Salazar

Crosby

. Health behavior theory for public health: principles, foundations, and applications. Burlington, MA, USA: Jones & Bartlett Learning, 2013.

47.

Carels

Douglass

Cacciapaglia

, et al. An ecological momentary assessment of relapse crises in dieting. J Consult Clin Psychol 2004; 72: 341.

48.

Burgess

Hassmén

Pumpa

. Determinants of adherence to lifestyle intervention in adults with obesity: a systematic review. Clin Obes 2017; 7: 123–135.

49.

Carels

Hoffman

Collins

, et al. Ecological momentary assessment of temptation and lapse in dieting. Eat Behav 2001; 2: 307–321.

50.

Ahola

Forsblom

Harjutsalo

, et al. Perceived stress and adherence to the dietary recommendations and blood glucose levels in type 1 diabetes. J Diabetes Res 2020; 2020: 3548520.

51.

McKee

Ntoumanis

Taylor

. An ecological momentary assessment of lapse occurrences in dieters. Ann Behav Med 2014; 48: 300–310.

52.

Warziski

Sereika

Styn

, et al. Changes in self-efficacy and dietary adherence: the impact on weight loss in the PREFER study. J Behav Med 2008; 31: 81–92.

53.

Cruwys

Norwood

Chachay

, et al. “An important part of who I am”: the predictors of dietary adherence among weight-loss, vegetarian, vegan, paleo, and gluten-free dietary groups. Nutrients 2020; 12: 970.

54.

VanWormer

Boucher

. Motivational interviewing and diet modification: a review of the evidence. Diabetes Educ 2004;30:404–406, 408–410, 414–416 passim.

55.

West

DiLillo

Bursac

, et al. Motivational interviewing improves weight loss in women with type 2 diabetes. Diabetes Care May 2007; 30: 1081–1087.

56.

Schumacher

Martin

Goldstein

, et al. Ecological momentary assessment of self-attitudes in response to dietary lapses. Health Psychol 2018; 37: 148–152.

57.

Thøgersen-Ntoumani

Dodos

Stenling

, et al.

Does self-compassion help to deal with dietary lapses among overweight and obese adults who pursue weight-loss goals?

Br J Health Psychol 2021; 26: 767–788.

58.

Alosco

Spitznagel

van Dulmen

, et al. Cognitive function and treatment adherence in older adults with heart failure. Psychosom Med 2012; 74: 965–973.

59.

Walø-Syversen

Kvalem

Kristinsson

, et al. Executive function, eating behavior, and preoperative weight loss in bariatric surgery candidates: an observational study. Obes Facts 2019; 12: 489–501.

60.

Manasse

Crochiere

Dallal

, et al. A multimodal investigation of impulsivity as a moderator of the relation between momentary elevations in negative internal states and subsequent dietary lapses. Appetite 2018; 127: 52–58.

61.

Åberg

Edman

Rössner

. Perceived hunger, palatability, and adherence: a comparison of high-and low-fat diets. Obes Res Clin Pract 2008; 2: 101–110.

62.

O'Neil

Theim

Boeka

, et al. Changes in weight control behaviors and hedonic hunger during a 12-week commercial weight loss program. Eat Behav 2012; 13: 354–360.

63.

Arguin

Tremblay

Blundell

, et al. Impact of a non-restrictive satiating diet on anthropometrics, satiety responsiveness and eating behaviour traits in obese men displaying a high or a low satiety phenotype. Br J Nutr 2017; 118: 750–760.

64.

Henry

Ponnalagu

, et al.

Does basal metabolic rate drive eating rate?

Physiol Behav 2018; 189: 74–77.

65.

Crochiere

Kerrigan

Lampe

, et al.

Is physical activity a risk or protective factor for subsequent dietary lapses among behavioral weight loss participants?

Health Psychol 2020; 39: 240.

66.

Epstein

Roemmich

Paluch

, et al. Influence of changes in sedentary behavior on energy and macronutrient intake in youth. Am J Clin Nutr 2005; 81: 361–366.

67.

Epstein

Paluch

Consalvi

, et al. Effects of manipulating sedentary behavior on physical activity and food intake. J Pediatr 2002; 140: 334–339.

68.

Hofmann

Förster

Stroebe

, et al. The great disinhibitor: alcohol, food cues, and eating behavior. Handbook of behavior, food and nutrition. New York, NY, USA: Springer; 2011: 2977–2991.

69.

Kase

Piers

Schaumberg

, et al. The relationship of alcohol use to weight loss in the context of behavioral weight loss treatment. Appetite 2016; 99: 105–111.

70.

Acharya

Elci

Sereika

, et al. Adherence to a behavioral weight loss treatment program enhances weight loss and improvements in biomarkers. Patient Prefer Adherence 2009; 3: 151–160.

71.

Lemstra

Bird

Nwankwo

, et al. Weight loss intervention adherence and factors promoting adherence: a meta-analysis. Patient Prefer Adherence 2016; 10: 1547–1559.

72.

Patel

Schlundt

. Impact of moods and social context on eating behavior. Appetite 2001; 36: 111–118.

73.

Bodnar

Simhan

Parker

, et al. Racial or ethnic and socioeconomic inequalities in adherence to national dietary guidance in a large cohort of US pregnant women. J Acad Nutr Diet 2017; 117: 867–877.e3.

74.

Kirkpatrick

Dodd

Reedy

, et al. Income and race/ethnicity are associated with adherence to food-based dietary guidance among US adults and children. J Acad Nutr Diet 2012; 112: 624–635.e6.

75.

McInerney

Csizmadi

Friedenreich

, et al. Associations between the neighbourhood food environment, neighbourhood socioeconomic status, and diet quality: an observational study. BMC Public Health 2016; 16: 984.

76.

Stroebe

van Koningsbruggen

Papies

, et al. Why most dieters fail but some succeed: a goal conflict model of eating behavior. Psychol Rev 2013; 120: 110–138.

77.

Gorin

Raynor

Niemeier

, et al. Home grocery delivery improves the household food environments of behavioral weight loss participants: results of an 8-week pilot study. Int J Behav Nutr Phys Act 2007; 4: 58.

78.

Smalls

Gregory

Zoller

, et al. Assessing the relationship between neighborhood factors and diabetes related health outcomes and self-care behaviors. BMC Health Serv Res 2015; 15: 445.

79.

Stone

Shiffman

Schwartz

, et al. Patient compliance with paper and electronic diaries. Control Clin Trials 2003; 24: 182–199.

80.

Timmer

Hickson

Launer

. The use of ecological momentary assessment in hearing research and future clinical applications. Hear Res 2018; 369: 24–28.

81.

Watson

Clark

Tellegen

. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol 1988; 54: 1063.

82.

Engel

Wonderlich

Crosby

, et al. The role of affect in the maintenance of anorexia nervosa: evidence from a naturalistic assessment of momentary behaviors and emotion. J Abnorm Psychol 2013; 122: 709.

83.

Wonderlich

Lavender

Wonderlich

, et al. Examining convergence of retrospective and ecological momentary assessment measures of negative affect and eating disorder behaviors. Int J Eating Disord 2015; 48: 305–311.

84.

Cohen

Kamarck

Mermelstein

. A global measure of perceived stress. J Health Soc Behav 1983; 24: 385–396.

85.

Reichenberger

Pannicke

Arend

A-K

, et al. Does stress eat away at you or make you eat? EMA measures of stress predict day to day food craving and perceived food intake as a function of trait stress-eating. Psychol Health 2021; 36: 129–147.

86.

Morgenstern

Kuerbis

Houser

, et al. Within-person associations between daily motivation and self-efficacy and drinking among problem drinkers in treatment. Psychol Addict Behav 2016; 30: 630–638.

87.

Zenk

Horoi

McDonald

, et al. Ecological momentary assessment of environmental and personal factors and snack food intake in African American women. Appetite 2014; 83: 333–341.

88.

Byrne

Allen

Dove

, et al. The reliability and validity of the dichotomous thinking in eating disorders scale. Eat Behav 2008; 9: 154–162.

89.

Raes

Pommier

Neff

, et al. Construction and factorial validation of a short form of the Self-Compassion Scale. Clin Psychol Psychother 2011; 18: 250–255.

90.

Cella

Lai

Nowinski

, et al. Neuro-QOL: brief measures of health-related quality of life for clinical research in neurology. Neurology 2012; 78: 1860–1867.

91.

Kratz

Murphy

Braley

. Ecological momentary assessment of pain, fatigue, depressive, and cognitive symptoms reveals significant daily variability in multiple sclerosis. Arch Phys Med Rehabil 2017; 98: 2142–2150.

92.

Lowe

Butryn

. Hedonic hunger: a new dimension of appetite? Physiol Behav 2007; 91: 432–439.

93.

Goldstein

Thomas

Vithiananthan

, et al. Multi-sensor ecological momentary assessment of behavioral and psychosocial predictors of weight loss following bariatric surgery: study protocol for a multicenter prospective longitudinal evaluation. BMC Obes 2018; 5: 27.

94.

Richard

Meule

Reichenberger

, et al. Food cravings in everyday life: an EMA study on snack-related thoughts, cravings, and consumption. Appetite 2017; 113: 215–223.

95.

Roefs

Boh

Spanakis

, et al. Food craving in daily life: comparison of overweight and normal-weight participants with ecological momentary assessment. J Hum Nutr Diet 2019; 32: 765–774.

96.

Elliston

Ferguson

Schüz

, et al. Situational cues and momentary food environment predict everyday eating behavior in adults with overweight and obesity. Health Psychol 2016; 36: 337–345.

97.

Lean

MEJ

Anderson

Morrison

, et al. Evaluation of a dietary targets monitor. Eur J Clin Nutr 2003; 57: 667–673.

98.

Dvorak

Pearson

Day

. Ecological momentary assessment of acute alcohol use disorder symptoms: associations with mood, motives, and use on planned drinking days. Exp Clin Psychopharmacol 2014; 22: 285–297.

99.

Schüz

Papadakis

Ferguson

. Situation-specific social norms as mediators of social influence on snacking. Health Psychol 2017; 37: 153–159.

100.

Gemming

Doherty

Utter

, et al. The use of a wearable camera to capture and categorise the environmental and social context of self-identified eating episodes. Appetite 2015; 92: 118–125.

101.

Dong

Hoover

Scisco

, et al. A new method for measuring meal intake in humans via automated wrist motion tracking. Appl Psychophysiol Biofeedback 2012; 37: 205–215.

102.

Sharma

Hoover

. Top-down detection of eating episodes by analyzing large windows of wrist motion using a convolutional neural network. Bioengineering 2022; 9: 70.

103.

Tang

Patyk

Jolly

, et al. Detecting eating episodes from wrist motion using daily pattern analysis. IEEE J Biomed Health Inform 2024; 28: 1054–1065.

104.

Dong

Hoover

Muth

. A device for detecting and counting bites of food taken by a person during eating. 2009 IEEE Int'l Conf on Bioinformatics and Biomedicine 2009; 1: 265–268.

105.

Sharma

Jasper

Muth

, et al. The impact of walking and resting on wrist motion for automated detection of meals. ACM Trans Comput Healthc 2020; 1: 1–19.

106.

Full

Kerr

Grandner

, et al. Validation of a physical activity accelerometer device worn on the hip and wrist against polysomnography. Sleep Health 2018; 4: 209–216.

107.

Buchan

Boddy

McLellan

. Comparison of free-living and laboratory activity outcomes from ActiGraph accelerometers worn on the dominant and non-dominant wrists. Meas Phys Educ Exerc Sci 2020; 24: 247–257.

108.

Quante

Kaplan

Cailler

, et al. Actigraphy-based sleep estimation in adolescents and adults: a comparison with polysomnography using two scoring algorithms. Nat Sci Sleep 2018; 10: 13–20.

109.

Leinonen

A-M

Ahola

Kulmala

, et al. Measuring physical activity in free-living conditions – comparison of three accelerometry-based methods. Original research. Front Physiol 2017; 7: 681.

110.

Vanhelst

Mikulovic

Bui-Xuan

, et al. Comparison of two ActiGraph accelerometer generations in the assessment of physical activity in free living conditions. BMC Res Notes 2012; 5: 187.

111.

Kim

Barry

Kang

. Validation of the ActiGraph GT3X and activPAL accelerometers for the assessment of sedentary behavior. Meas Phys Educ Exerc Sci 2015; 19: 125–137.

112.

Kamada

Shiroma

Harris

, et al. Comparison of physical activity assessed using hip-and wrist-worn accelerometers. Gait Posture 2016; 44: 23–28.

113.

Paffenbarger

Wing

Hyde

. Paffenbarger physical activity questionnaire. Am J Epidemiol 1978; 108: 161–175.

114.

McQuoid

Thrul

Ling

. A geographically explicit ecological momentary assessment (GEMA) mixed method for understanding substance use. Soc Sci Med 2018; 202: 89–98.

115.

Zandbergen

Barbeau

. Positional accuracy of assisted GPS data from high-sensitivity GPS-enabled mobile phones. J Navig. 2011; 64: 381.

116.

Barber

. GPS Distance Accuracy Test: Smartphone Apps vs. Dedicated GPS, singletracks. https://www.singletracks.com/mtb-gear/gps-distance-accuracy-test-smartphone-apps-vs-dedicated-gps/

117.

Esri Inc. ArcGIS Pro (Version 2.7). Redlands, CA, USA: Esri Inc, 2020.

118.

Guevara

Smego

Vivier

. A spatial analysis of the food environment and overweight and obesity among Rhode Island youth. R I Med J 2020; 103: 56–61.

119.

Harris

Benedict

. A biometric study of human basal metabolism. Proc Natl Acad Sci USA 1918; 4: 370.

120.

Krieger

Williams

Moss

. Measuring social class in US public health research: concepts, methodologies, and guidelines. Annu Rev Public Health 1997; 18: 341–378.

121.

Kant

. Consumption of energy-dense, nutrient-poor foods by adult Americans: nutritional and health implications. The third National Health and Nutrition Examination Survey, 1988–1994. Am J Clin Nutr 2000; 72: 929–936.

122.

Goldstein

. Beneficial health effects of modest weight loss. Int J Obes Relat Metab Disord 1992; 16: 397–415.

123.

Wing

Lang

Wadden

, et al. Benefits of modest weight loss in improving cardiovascular risk factors in overweight and obese individuals with type 2 diabetes. Diabetes Care 2011; 34: 1481–1486.

124.

Feskanich

Sielaff

Chong

, et al. Computerized collection and analysis of dietary intake information. Comput Methods Programs Biomed 1989; 30: 47–57.

125.

Blanton

Moshfegh

Baer

, et al. The USDA Automated Multiple-Pass Method accurately estimates group total energy and nutrient intake. J Nutr 2006; 136: 2594–2599.

126.

Liao

Skelton

Dunton

, et al. A systematic review of methods and procedures used in ecological momentary assessments of diet and physical activity research in youth: an adapted STROBE checklist for reporting EMA studies (CREMAS). J Med Internet Res 2016; 18: e151.

127.

Vandenbroucke

Altman

, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. Ann Intern Med 2007; 147: W-163–W-194.

128.

Enders

. Applied missing data analysis. New York, NY, USA: Guilford press, 2010.

129.

Shen

Weissfeld

. Application of pattern-mixture models to outcomes that are potentially missing not at random using pseudo maximum likelihood estimation. Biostatistics 2005; 6: 333–347.

130.

Asparouhov

Muthén

. Exploratory structural equation modeling. Struct Equ Model Multidiscip J 2009; 16: 397–438.

131.

Reise

Ventura

Nuechterlein

, et al. An illustration of multilevel factor analysis. J Pers Assess 2005; 84: 126–136.

132.

Peters

G-J

. The alpha and the omega of scale reliability and validity: why and how to abandon Cronbach's alpha and the route towards more comprehensive assessment of scale quality. Eur Health Psychol 2014; 16: 56–59.

133.

Asparouhov

Muthén

. SRMR in Mplus. http://www statmodel com/download/SRMR2 pdf. 2018;

134.

Bentler

. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model Multidiscip J 1999; 6: 1–55.

135.

Knafl

Grey

. Factor analysis model evaluation through likelihood cross-validation. Stat Methods Med Res 2007; 16: 77–102.

136.

McNeish

Hamaker

. A primer on two-level dynamic structural equation models for intensive longitudinal data in Mplus. Psychol Methods 2020; 25: 610.

137.

Benjamini

Yekutieli

. The control of the false discovery rate in multiple testing under dependency. Ann Stat 2001; 29: 1165–1188.

138.

Meuleman

Billiet

. A Monte Carlo sample size study: how many countries are needed for accurate multilevel SEM? Surv Res Methods 2009; 3: 45–58.

139.

Cohen

. Statistical power analysis for the behavioral sciences. New York, NY, USA: Academic press, 2013.

140.

Grilo

Shiffman

Wing

. Coping with dietary relapse crises and their aftermath. Addict Behav 1993; 18: 89–102.

141.

Goldstein

Dochat

Schumacher

, et al. Using ecological momentary assessment to better understand dietary lapse types. Appetite 2018; 129: 198–206.

142.

Nahum-Shani

Smith

Spring

, et al. Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2018; 52: 446–462.

143.

Riley

Rivera

Atienza

, et al.

Health behavior models in the age of mobile interventions: are our theories up to the task?

Transl Behav Med 2011; 1: 53–71.

144.

Mooney

Burling

Hartman

, et al. The abstinence violation effect and very low calorie diet success. Addict Behav 1992; 17: 319–324.

145.

Parks

Marlatt

. Relapse prevention therapy: a cognitive-behavioral approach. Natl Psychol 2000; 9: 3.

146.

Wang

Miller

. Just-in-the-moment adaptive interventions (JITAI): a meta-analytical review. Health Commun 2020; 35: 1531–1544.

147.

Van der Staay

Steckler

. The fallacy of behavioral phenotyping without standardisation. Genes Brain Behav 2002; 1: 9–13.

148.

McBride

Bryan

Bray

, et al.

Health behavior change: can genomics improve behavioral adherence?

Am J Public Health 2012; 102: 401–405.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB

0.03 MB