Missing values are nearly inescapable within social science research. The problem of missing data is especially troubling in longitudinal and intensive longitudinal studies when participants miss an entire collection cycle compared to item nonresponse. The present study examines some of the contextual influences of missed study participation using unobtrusive measures of experience in conjunction with the experience sampling method (ESM). A sample of 66 participants generated 2,940 observations across a 7-day study, yielding a missing response rate of 34%. Multilevel binary logistic regression was used to estimate the probability of missing the study participation signal based upon study time and physical activity states. Results indicate that the probability of missing collection signals increases throughout the duration of the study and with high levels of physical activity. The strongest predictor of missed participation occurred when participants’ activity monitors were set to “asleep” mode. Implications of these findings and recommendations for future ESM studies are discussed.
AcockA. C. (2005). Working with missing values. Journal of Marriage and Family, 67, 1012–1028.
2.
AsakawaK. (2004). Flow experience and autotelic personality in Japanese college students: How do they experience challenges in daily life?Journal of Happiness Studies, 5, 123–154.
BealD. J. (2015). ESM 2.0: State of the art and future potential of experience sampling methods in organizational research. Annual Review of Organization Psychology and Organizational Behavior, 2, 383–407.
5.
BolgerN.LaurenceauJ.-P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research. New York, NY: Guilford Press.
6.
BuhiE. R.GoodsonP.NeilandsT. B. (2008). Out of sight, not out of mind: Strategies for handling missing data. American Journal of Health Behavior, 32, 83–92.
7.
ConnerT. S.MehlM. R. (2015). Ambulatory assessment—Methods for studying everyday life. In ScottR.KosslynS.PinkertonN. (Eds.), Emerging trends in the social and behavioral sciences: An interdisciplinary, searchable, and linkable resource (pp. 1–13). Hoboken, NJ: Wiley.
8.
CourvoisierD. S.EidM.LischetzkeT. (2012). Compliance to a cell phone-based ecological momentary assessment study: The effect of time and personality characteristics. Psychological Assessment, 24, 713–720. doi:10.1037/a0026733
9.
CranoW. D.BrewerM. B.LacA. (2015). Principles and methods of social research (3rd ed.). New York, NY: Routledge.
10.
CsikszentmihalyiM.LarsonR. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Diseases, 175, 526–536.
11.
CsikszentmihalyiM.LarsonR.PrescottS. (1977). The ecology of adolescent activity and experience. Journal of Youth and Adolescence, 6, 281–294.
12.
DohertyS. T.LemieuxC. J.CanallyC. (2014). Tracking human activity and well-being in natural environments using wearable sensors and experience sampling. Social Science and Medicine, 106, 83–92. doi:http://doi.org/10.1016/j.socscimed.2014.01.048
13.
DuntonG. F.HuhJ.LeventhalA. M.RiggsN.Spruijt-MetzD.PentzM. A. (2014). Momentary assessment of affect, physical feeling states, and physical activity in children. Health Psychology, 33, 255–263.
14.
FranklinM. S.SmallwoodJ.ZedeliusC. M.BroadwayJ. M.SchoolerJ. W. (2016). Unaware yet reliant on attention: Experience sampling reveals that mind-wandering impedes implicit learning. Psychonomic Bulletin & Review, 23, 223–229.
15.
GarberC. E.BlissmerB.DeschenesM. R.FranklinB. A.LamonteM. J.LeeI. M.…SwainD. P. (2011). Quantity and quality of exercise for developing and maintaining cardiorespiratory, musculoskeletal, and neuromotor fitness in apparently healthy adults: Guidance for prescribing exercise. Medicine and Science in Sports and Exercise, 43, 1334–1359. doi:10.1249/MSS.0b013e318213fefb
16.
GelmanA.HillJ. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge, MA: Cambridge University Press.
17.
GoetzT.BiegM.HallN. C. (2016). Assessing academic emotions via the experience sampling method. In ZembylasM.SchutzP. A. (Eds.), Methodological advances in research on emotion and education (pp. 245–258). Switzerland: Springer International Publishing.
18.
GrahamJ. W.DonaldsonS. I. (1993). Evaluating interventions with differential attrition: The importance of nonresponse mechanisms and use of follow-up data. The Journal of Applied Psychology, 78, 119–128. doi:10.1037/0021-9010.78.1.119
19.
HedekerD.GibbonsR. D. (1997). Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods, 2, 64–78. doi:10.1037/1082-989X.2.1.64
20.
HektnerJ. M.SchmidtJ. A.CsikszentmihalyiM. (2007). Experience sampling method: Measure the quality of life. Thousand Oaks, CA: Sage.
21.
LittleR. J. A. (1995). Modeling the drop-out mechanism in repeated-measures studies. Journal of the American Statistical Association. doi:10.1080/01621459.1995.10476615
22.
LopezR. B.MilyavskayaM.HofmannW.HeathertonT. F. (2016). Motivational and neural correlates of self-control of eating: A combined neuroimaging and experience sampling study in dieting female college students. Appetite, 103, 192–199.
23.
MarshallS. J.LevyS. S.Tudor-LockeC. E.KolkhorstF. W.WootenK. M.JiM.…AinsworthB. E. (2009). Translating physical activity recommendations into a pedometer-Based Step Goal. American Journal of Preventive Medicine, 36, 410–415. doi:10.1016/j.amepre.2009.01.021
24.
MehlM. R. (2007). Eavesdropping on health: A naturalistic observation approach for social health research. Social and Personality Psychology Compass, 1, 359–380.
25.
MehlM. R.PennebakerJ. W.CrowM.DabbsJ.PriceJ. (2001). The Electronically Activated Recorder (EAR): A device for sampling naturalistic daily activities and conversations. Behavior Research Methods, Instruments, and Computers, 33, 517–523.
26.
ReedJ.OnesD. S. (2006). The effect of acute aerobic exercise on positive activated affect: A meta-analysis. Psychology of Sport and Exercise, 7, 477–514. doi:http://doi.org/10.1016/j.psychsport.2005.11.003
27.
RothP. L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47, 537–560. doi:10.1111/j.1744-6570.1994.tb01736.x
28.
SilviaP. J.KwapilT. R.EddingtonK. M.BrownL. H. (2013). Missed beeps and missing data: Dispositional and situational predictors of nonresponse in experience sampling research. Social Science Computer Review, 31, 471–481. doi:10.1177/0894439313479902
29.
WrzusC.WagnerG. G.RiedigerM. (2016). Personality-situation transactions from adolescence to old age. Journal of Personality and Social Psychology, 110, 782–799. doi:10.1037/pspp0000054
