Noncompletion of ecological momentary assessment (EMA) surveys is a common issue and may yield bias in results if not properly handled. Using data observed at scheduled times as well as data retrieved later to fill missing responses, this study aims to investigate predictors of EMA completion, including demographic characteristics, time-related factors, and momentary experiences/behaviors. Data were from a 30-day EMA study including 145 women currently experiencing intimate partner violence (IPV) and using substances. The average rate of EMA completion was initially 51.4% at the scheduled times and increased to 72.6% after incorporating data from later-retrieved surveys. Participants who were younger, had more children, or had lower mean levels of negative affect dysregulation showed lower completion rates. At the momentary survey level, more days into the study and afternoon/evening reports (vs. morning reports) were associated with lower completion; lower levels of negative affect dysregulation, less smoking or alcohol use, and experiencing IPV were linked to lower momentary completion. Implications of the results for handling missing data in EMA are discussed and have important ramifications for future research, practice, and theory.
AndersenA. M.DoganM. V.BeachS. R.PhilibertR. A. (2015). Current and future prospects for epigenetic biomarkers of substance use disorders. Genes, 6(4), 991–1022. https://doi.org/10.3390/genes6040991
2.
ArmourC.TsaiJ.DurhamT. A.CharakR.BiehnT. L.ElhaiJ. D.PietrzakR. H. (2015). Dimensional structure of DSM-5 posttraumatic stress symptoms: Support for a hybrid Anhedonia and Externalizing Behaviors model. Journal of Psychiatric Research, 61, 106–113. https://doi.org/10.1016/j.jpsychires.2014.10.012
3.
ArndtS. (2009). Stereotyping and the treatment of missing data for drug and alcohol clinical trials. Substance Abuse Treatment, Prevention, and Policy, 4(1), 2. https://doi.org/10.1186/1747-597X-4-2
4.
BattistaS. R.MacKinnonS. P.SherryS. B.BarrettS. P.MacNevinP. D.StewartS. H. (2015). Does alcohol reduce social anxiety in daily life? A 22-day experience sampling study. Journal of Social and Clinical Psychology, 34(6), 508–528.
5.
BellI. H.LimM. H.RossellS. L.ThomasN. (2017). Ecological momentary assessment and intervention in the treatment of psychotic disorders: A systematic review. Psychiatric Services, 68(11), 1172–1181. https://doi.org/10.1176/appi.ps.201600523
6.
BoukhechbaM.CaiL.ChowP. I.FuaK.GerberM. S.TeachmanB. A.BarnesL. E. (2018). Contextual analysis to understand compliance with smartphone-based ecological momentary assessment. In Proceedings of the 12th EAI international conference on pervasive computing technologies for healthcare. https://dl.acm.org/doi/10.1145/3240925.3240967
7.
BurkeL. E.ShiffmanS.MusicE.StynM. A.KriskaA.SmailagicA.SiewiorekD.EwingL. J.ChasensE.FrenchB. (2017). Ecological momentary assessment in behavioral research: Addressing technological and human participant challenges. Journal of Medical Internet Research, 19(3), 37–53.
8.
ColomboD.Fernández-ÁlvarezJ.Suso-RiberaC.CipressoP.ValevH.LeufkensT.SasC.Garcia-PalaciosA.RivaG.BotellaC. (2020). The need for change: Understanding emotion regulation antecedents and consequences using ecological momentary assessment. Emotion, 20(1), 30–36.
9.
ColonnaA.LombardoL.SicilianiG.BracciA.Guarda-NardiniL.DjukicG.ManfrediniD. (2020). Smartphone-based application for EMA assessment of awake bruxism: Compliance evaluation in a sample of healthy young adults. Clinical Oral Investigations, 24(4), 1395–1400.
10.
CorkreyR.ParkinsonL. (2002). A comparison of four computer-based telephone interviewing methods: Getting answers to sensitive questions. Behavior Research Methods, Instruments, & Computers, 34(3), 354–363. https://doi.org/10.3758/BF03195463
11.
CourvoisierD. S.EidM.LischetzkeT. (2012). Compliance to a cell phone-based ecological momentary assessment study: The effect of time and personality characteristics. Psychological Assessment, 24(3), 713–720. https://doi.org/10.1037/a0026733
12.
DockrayS.GrantN.StoneA. A.KahnemanD.WardleJ.SteptoeA. (2010). A comparison of affect ratings obtained with ecological momentary assessment and the day reconstruction method. Social Indicators Research, 99(2), 269–283.
13.
DziuraJ. D.PostL. A.ZhaoQ.FuZ.PeduzziP. (2013). Strategies for dealing with missing data in clinical trials: From design to analysis. The Yale Journal of Biology and Medicine, 86(3), 343–358.
14.
GlynnR. J.LairdN. M.RubinD. B. (1993). Multiple imputation in mixture models for nonignorable nonresponse with follow-ups. Journal of the American Statistical Association, 88(423), 984–993. https://doi.org/10.2307/2290790
15.
GomilaR.ClarkC. S. (2022). Missing data in experiments: Challenges and solutions. Psychological Methods, 27(2), 143–155. https://doi.org/10.1037/met0000361
16.
GratzK. L.RoemerL. (2004). Multidimensional assessment of emotion regulation and dysregulation: Development, factor structure, and initial validation of the difficulties in emotion regulation scale. Journal of Psychopathology and Behavioral Assessment, 26(1), 41–54. https://doi.org/10.1023/B:JOBA.0000007455.08539.94
17.
HarveyP. D.MillerM. L.MooreR. C.DeppC. A.ParrishE. M.PinkhamA. E. (2021). Capturing clinical symptoms with ecological momentary assessment: Convergence of momentary reports of psychotic and mood symptoms with diagnoses and standard clinical assessments. Innovations in Clinical Neuroscience, 18(1–3), 24–30.
18.
HedekerD.GibbonsR. D. ( 2006). Longitudinal data analysis. Hoboken, New Jersey: John Wiley & Sons, Inc.
19.
HedekerD.MermelsteinR. J. (1996). Application of random-effects regression models in relapse research. Addiction, 91(12s1), 211–229.
20.
HeronK. E.EverhartR. S.McHaleS. M.SmythJ. M. (2017). Using mobile-technology-based ecological momentary assessment (EMA) methods with youth: A systematic review and recommendations. Journal of Pediatric Psychology, 42(10), 1087–1107.
21.
HeronK. E.SmythJ. M. (2010). Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments. British Journal of Health Psychology, 15, 1–39. https://doi.org/10.1348/135910709x466063
22.
HuffordM. R. (2007). Special methodological challenges and opportunities in ecological momentary assessment. In StoneA. A.ShiffmanS.AtienzaA. A.NebelingL. (Eds.), The science of real-time data capture: Self-reports in health research (pp. 54–75). Oxford University Press.
23.
JonesA.RemmerswaalD.VerveerI.RobinsonE.FrankenI. H.WenC. K. F.FieldM. (2019). Compliance with ecological momentary assessment protocols in substance users: A meta-analysis. Addiction, 114(4), 609–619.
24.
JunghaenelD. U.StoneA. A. (2020). Ecological momentary assessment for the psychosocial study of health. In SweenyK.RobbinsM. L.CohenL. M. (Eds.), The Wiley encyclopedia of health psychology (pp. 105–112). Wiley. https://doi.org/10.1002/9781119057840.ch56
25.
LiaoY.SkeltonK.DuntonG.BrueningM. (2016). A systematic review of methods and procedures used in ecological momentary assessments of diet and physical activity research in youth: An adapted STROBE checklist for reporting EMA studies (CREMAS). Journal of Medical Internet Research, 18(6), e151.
26.
LittM. D.CooneyN. L.MorseP. (1998). Ecological momentary assessment (EMA) with treated alcoholics: Methodological problems and potential solutions. Health Psychology, 17(1), 48–52. https://doi.org/10.1037//0278-6133.17.1.48
27.
LittleR. J.D’AgostinoR.CohenM. L.DickersinK.EmersonS. S.FarrarJ. T.FrangakisC.HoganJ. W.MolenberghsG.MurphyS. A.NeatonJ. D.RotnitzkyA.ScharfsteinD.ShihW. J.SiegelJ. P.SternH. (2012). The prevention and treatment of missing data in clinical trials. New England Journal of Medicine, 367(14), 1355–1360. https://doi.org/10.1056/NEJMsr1203730
28.
MaherJ. P.RebarA. L.DuntonG. F. (2018). Ecological momentary assessment is a feasible and valid methodological tool to measure older adults’ physical activity and sedentary behavior. Frontiers in Psychology, 9, 1485. https://doi.org/10.3389/fpsyg.2018.01485
29.
MartinezG. J.MattinglyS. M.Robles-GrandaP.SahaK.SirigiriA.YoungJ.ChawlaN.De ChoudhuryM.D’MelloS.MarkG. (2021). Predicting participant compliance with fitness tracker wearing and ecological momentary assessment protocols in information workers: Observational study. JMIR Mhealth and Uhealth, 9(11), e22218.
30.
MofsenA. M.RodebaughT. L.NicolG. E.DeppC. A.MillerJ. P.LenzeE. J. (2019). When all else fails, listen to the patient: A viewpoint on the use of ecological momentary assessment in clinical trials. JMIR Mental Health, 6(5), e11845. https://doi.org/10.2196/11845
31.
MurrayA. L.BrownR.ZhuX.SpeyerL. G.YangY.XiaoZ.RibeaudD.EisnerM. (2022). Momentary predictors of compliance in an ecological momentary assessment study of young adults’ mental health. Center for Open Science. https://doi.org/10.31234%2Fosf.io%2F5r9js
32.
MuthénL. K.MuthénB. O. (1998-2017). Mplus User’s Guide (8th ed.).
33.
PalermoT. M.ValenzuelaD.StorkP. P. (2004). A randomized trial of electronic versus paper pain diaries in children: Impact on compliance, accuracy, and acceptability. Pain, 107(3), 213–219.
34.
PiaseckiT. M. (2019). Assessment of alcohol use in the natural environment. Alcoholism, Clinical and Experimental Research, 43(4), 564–577. https://doi.org/10.1111/acer.13975
35.
PiaseckiT. M.AlleyK. J.SlutskeW. S.WoodP. K.SherK. J.ShiffmanS.HeathA. C. (2012). Low sensitivity to alcohol: Relations with hangover occurrence and susceptibility in an ecological momentary assessment investigation. Journal of Studies on Alcohol and Drugs, 73(6), 925–932. https://doi.org/10.15288/jsad.2012.73.925
36.
RintalaA.WampersM.Myin-GermeysI.ViechtbauerW. (2020). Momentary predictors of compliance in studies using the experience sampling method. Psychiatry Research, 286, 112896.
37.
RussellM. A.GajosJ. M. (2020). Annual research review: Ecological momentary assessment studies in child psychology and psychiatry. Journal of Child Psychology and Psychiatry, 61(3), 376–394.
38.
SAS Institute Inc. (2002-2014). SAS® 9.4 help and documentation.
39.
SchattenH. T.AllenK. J. D.ArmeyM. F. (2020). Assessment of emotion dysregulation using ecological momentary assessment. In BeauchaineT. P.CrowellS. E. (Eds.), The Oxford handbook of emotion dysregulation (pp. 411–426). Oxford University Press.
40.
SchüzN.WaltersJ. A.FrandsenM.BowerJ.FergusonS. G. (2013). Compliance with an EMA monitoring protocol and its relationship with participant and smoking characteristics. Nicotine & Tobacco Research, 16(Suppl. 2), S88–S92.
41.
SeidmanA. J.GeorgeC. J.KovacsM. (2022). Ecological momentary assessment of affect in depression-prone and control samples: Survey compliance and affective yield. Journal of Affective Disorders, 311, 63–68.
42.
SerreF.FatseasM.SwendsenJ.AuriacombeM. (2015). Ecological momentary assessment in the investigation of craving and substance use in daily life: A systematic review. Drug and Alcohol Dependence, 148, 1–20.
43.
ShiffmanS. (2009). Ecological momentary assessment (EMA) in studies of substance use. Psychological Assessment, 21(4), 486–497. https://doi.org/10.1037/a0017074
44.
ShiffmanS.StoneA. A. (1998). Introduction to the special section: Ecological momentary assessment in health psychology. Health Psychology, 17(1), 3–5.
SilviaP. J.KwapilT. R.EddingtonK. M.BrownL. H. (2013). Missed beeps and missing data: Dispositional and situational predictors of nonresponse in experience sampling research. Social Science Computer Review, 31(4), 471–481.
47.
SmythJ. M.HeronK. E. (2014). Ecological momentary assessment (EMA) in family research. In McHaleS. M.AmatoP.BoothA. (Eds), Emerging methods in family research (pp. 145–161). Springer.
48.
SokolovskyA. W.MermelsteinR. J.HedekerD. (2014). Factors predicting compliance to ecological momentary assessment among adolescent smokers. Nicotine & Tobacco Research, 16(3), 351–358.
49.
StoneA. A.BroderickJ. E.SchwartzJ. E.ShiffmanS.Litcher-KellyL.CalvaneseP. (2003). Intensive momentary reporting of pain with an electronic diary: Reactivity, compliance, and patient satisfaction. Pain, 104(1–2), 343–351.
50.
StoneA. A.ShiffmanS. (1994). Ecological momentary assessment (EMA) in behavorial medicine. Annals of Behavioral Medicine, 16(3), 199–202.
51.
StoneA. A.ShiffmanS. (2002). Capturing momentary, self-report data: A proposal for reporting guidelines. Annals of Behavioral Medicine, 24(3), 236–243. https://doi.org/10.1207/s15324796abm2403_09
52.
SullivanT. P.KhondkaryanE.Dos SantosN. P.PetersE. N. (2011). Applying experience sampling methods to partner violence research: Safety and feasibility in a 90-day study of community women. Violence Against Women, 17(2), 251–266.
53.
SullivanT. P.WeissN. H.FlanaganJ. C.WillieT. C.ArmeliS.TennenH. (2016). PTSD and daily co-occurrence of drug and alcohol use among women experiencing intimate partner violence. Journal of Dual Diagnosis, 12(1), 36–42. https://doi.org/10.1080/15504263.2016.1146516
54.
TargumS. D.SauderC.EvansM.SaberJ. N.HarveyP. D. (2021). Ecological momentary assessment as a measurement tool in depression trials. Journal of Psychiatric Research, 136, 256–264. https://doi.org/10.1016/j.jpsychires.2021.02.012
55.
WangL. P.MaxwellS. E. (2015). On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychological Methods, 20(1), 63–83. https://doi.org/10.1037/met0000030
56.
WangY.Jean JacquesJ.LiZ.SibilleK. T.CookR. L. (2021). Health outcomes among adults initiating medical cannabis for chronic pain: A 3-month prospective study incorporating ecological momentary assessment (EMA). Cannabis, 4(2), 69–83. https://doi.org/10.26828/cannabis/2021.02.006
57.
WeathersF. W.LitzB. T.KeaneT. M.PalmieriP. A.MarxB. P.SchnurrP. P. (2013). The PTSD Checklist for DSM-5 (PCL-5). National Center for PTSD. www.ptsd.va.gov
58.
WeissN. H.BoldK. W.SullivanT. P.ArmeliS.TennenH. (2017). Testing bidirectional associations among emotion regulation strategies and substance use: A daily diary study. Addiction, 112(4), 695–704.
59.
WeissN. H.DaroshA. G.ContractorA. A.SchickM. M.Dixon-GordonK. L. (2019). Confirmatory validation of the factor structure and psychometric properties of the Difficulties in Emotion Regulation Scale-Positive. Journal of Clinical Psychology, 75(7), 1267–1287. https://doi.org/10.1002/jclp.22768
60.
WeissN. H.Dixon-GordonK. L.BrickL. A.GoldsteinS. C.SchickM. R.KieferR.ContractorA. A.SullivanT. P. (in press). Measuring emotion dysregulation in daily life: An experience sampling study.
61.
WeissN. H.GratzK. L.LavenderJ. M. (2015). Factor structure and initial validation of a multidimensional measure of difficulties in the regulation of positive emotions: The DERS-Positive. Behavior Modification, 39(3), 431–453. https://doi.org/10.1177/0145445514566504
62.
WenC. K. F.SchneiderS.StoneA. A.Spruijt-MetzD. (2017). Compliance with mobile ecological momentary assessment protocols in children and adolescents: A systematic review and meta-analysis. Journal of Medical Internet Research, 19(4), e6641.
63.
Williams-KerverG. A.SchaeferL. M.HazzardV. M.CaoL.EngelS. G.PetersonC. B.WonderlichS. A.CrosbyR. D. (2021). Baseline and momentary predictors of ecological momentary assessment adherence in a sample of adults with binge-eating disorder. Eating Behaviors, 41, 101509.
64.
YangC.LinasB.KirkG.BollingerR.ChangL.ChanderG.SiconolfiD.BraxtonS.RudolphA.LatkinC. (2015). Feasibility and acceptability of smartphone-based ecological momentary assessment of alcohol use among African American men who have sex with men in Baltimore. JMIR Mhealth and Uhealth, 3(2), e67. https://doi.org/10.2196/mhealth.4344
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
