Nonparticipation Bias in Accelerometer-Based Studies and the Use of Propensity Scores

Abstract

Relatively little attention has been paid to the effects of nonparticipation on data quality in population-based studies that use accelerometers to measure physical activity. We examine these issues using data from the 2013 Longitudinal Internet Studies for the Social Sciences (LISS) panel and 2013–2014 National Health and Nutrition Examination Survey (NHANES) accelerometer studies, both of which collected survey data in advance and therefore permit comparisons of self-reported physical activity between participants and nonparticipants to the accelerometer studies. While individuals with high levels of self-reported physical activity are overrepresented in the participant samples, the differences are modest in both studies. However, in the LISS panel this difference led to overestimates of physical activity that are not fully corrected by propensity score weighting adjustments (i.e., non-ignorable selection bias). This finding underscores the importance of assessing the potential influence of nonparticipation on accelerometer-derived estimates of physical activity.

Keywords

sensors physical activity monitors selection bias weighting adjustments national health and nutrition examination survey longitudinal internet studies for the social sciences

Get full access to this article

View all access options for this article.

References

Bauman

A. E.

Reis

R. S.

Sallis

J. F.

Wells

J. C.

Loos

R. J.

Martin

B. W.

Lancet Physical Activity Series Working Group . (2012). Correlates of physical activity: Why are some people physically active and others not? Lancet (London, England), 380(9838), 258–271. https://doi.org/10.1016/S0140-6736(12)60735-1

Bethlehem

(2009). Applied survey methods: A statistical perspective. John Wiley & Sons.

Centers for Disease Control and Prevention (CDC). National Center for Health Statistics (NCHS) . (2022). National health and nutrition examination survey, 2013-2014: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2013

Chaudhury

Esliger

Craig

(2008). Accelerometry in adults. In Craig

Mindell

Hirani

(Eds.), Health Survey for England 2008: Physical Activity and Fitness (pp. 59–88). The Health and Social Care Information Centre.

Cohen

(1988). Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates.

Doherty

Jackson

Hammerla

Plötz

Olivier

Granat

M. H.

White

van Hees

V. T.

Trenell

M. I.

Owen

C. G.

Preece

S. J.

Gillions

Sheard

Peakman

Brage

Wareham

N. J.

Wareham

N. J.

(2017). Large scale population assessment of physical activity using wrist worn accelerometers: The UK biobank study. PLoS One, 12(2), Article e0169649. https://doi.org/10.1371/journal.pone.0169649

Eckman

Unangst

Dever

J. A.

Antoun

(2023). The precision of estimates of nonresponse bias in means. Journal of Survey Statistics and Methodology, 11(4), 758–783. https://doi.org/10.1093/jssam/smac019

Farina

Sherlock

Thomas

Lowry

R. G.

Banerjee

(2019). Acceptability and feasibility of wearing activity monitors in community‐dwelling older adults with dementia. International Journal of Geriatric Psychiatry, 34(4), 617–624. https://doi.org/10.1002/gps.5064

Groves

R. M.

(2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646–675. https://doi.org/10.1093/poq/nfl033

10.

Hassani

Kivimaki

Elbaz

Shipley

Singh-Manoux

Sabia

(2014). Non-consent to a wrist-worn accelerometer in older adults: The role of socio-demographic, behavioural and health factors. PLoS One, 9(10), Article e110816. https://doi.org/10.1371/journal.pone.0110816

11.

Kapteyn

Banks

Hamer

Smith

J. P.

Steptoe

Van Soest

Koster

Htay Wah

Wah

S. H.

(2018). What they say and what they do: Comparing physical activity across the USA, England and The Netherlands. Journal of Epidemiology & Community Health, 72(6), 471–476. https://doi.org/10.1136/jech-2017-209703

12.

Kern

Wang

(2021). Boosted kernel weighting – using statistical learning to improve inference from nonprobability samples. Journal of Survey Statistics and Methodology, 9(5), 1088–1113. https://doi.org/10.1093/jssam/smaa028

13.

Keusch

Bähr

Haas

G. C.

Kreuter

Trappmann

(2023). Coverage error in data collection combining mobile surveys with passive measurement using apps: Data from a German national survey. Sociological Methods & Research, 52(2), 841–878. https://doi.org/10.1177/0049124120914924

14.

Keusch

Struminskaya

Antoun

Couper

M. P.

Kreuter

(2019). Willingness to participate in passive mobile data collection. Public Opinion Quarterly, 83(Suppl 1), 210–235. https://doi.org/10.1093/poq/nfz007

15.

Lathia

Sandstrom

G. M.

Mascolo

Rentfrow

P. J.

(2017). Happier people live more active lives: Using smartphones to link happiness and physical activity. PLoS One, 12(1), Article e0160589. https://doi.org/10.1371/journal.pone.0160589

16.

Lee

P. H.

Macfarlane

D. J.

Lam

T. H.

(2013). Factors associated with participant compliance in studies using accelerometers. Gait & Posture, 38(4), 912–917. https://doi.org/10.1016/j.gaitpost.2013.04.018

17.

Little

R. J.

Vartivarian

(2005). Does weighting for nonresponse increase the variance of survey means? Survey Methodology, 31(2), 161.

18.

Matthews

C. E.

Hagströmer

Pober

D. M.

Bowles

H. R.

(2012). Best practices for using physical activity monitors in population-based research. Medicine & Science in Sports & Exercise, 44(1 Suppl 1), S68–S76. https://doi.org/10.1249/MSS.0b013e3182399e5b

19.

Plasqui

Bonomi

A. G.

Westerterp

K. R.

(2013). Daily physical activity assessment with accelerometers: New insights and validation studies. Obesity Reviews: An Official Journal of the International Association for the Study of Obesity, 14(6), 451–462. https://doi.org/10.1111/obr.12021

20.

Revilla

Couper

M. P.

Ochoa

(2019). Willingness of online panelists to perform additional tasks. Methods, Data, Analyses, 13(2), 223–252. https://doi.org/10.12758/mda.2018.01

21.

Rosenbaum

P. R.

Rubin

D. B.

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.2307/2335942

22.

Roth

M. A.

Mindell

J. S.

(2013). Who provides accelerometry data? Correlates of adherence to wearing an accelerometry motion sensor: The 2008 health survey for England. Journal of Physical Activity and Health, 10(1), 70–78. https://doi.org/10.1123/jpah.10.1.70

23.

Rubin

D. B.

(1976). Inference and missing data. Biometrika, 63(3), 581–592. https://doi.org/10.1093/biomet/63.3.581

24.

Scherpenzeel

A. C.

Das

(2010). “True” longitudinal and probability-based Internet panels: Evidence from The Netherlands. In Das

Ester

Kaczmirek

(Eds.), Social and behavioral research and the Internet: Advances in applied methods and research strategies (pp. 77–104). Taylor & Francis.

25.

Struminskaya

Lugtig

Keusch

Höhne

J. K.

(2020). Augmenting surveys with data from sensors and apps: Opportunities and challenges. Social Science Computer Review. https://doi.org/10.1177/0894439320979951

26.

Struminskaya

Toepoel

Lugtig

Haan

Luiten

Schouten

(2020). Understanding willingness to share smartphone-sensor data. Public Opinion Quarterly, 84(3), 725–759. https://doi.org/10.1093/poq/nfaa044

27.

Toepoel

Luiten

Zandvliet

(2021). Response, willingness, and data donation in a study on accelerometer possession in the general population. Survey Practice, 14(1), 1–19. https://doi.org/10.29115/sp-2021-0005

28.

Troiano

R. P.

Berrigan

Dodd

K. W.

Masse

L. C.

Tilert

McDowell

(2008). Physical activity in the United States measured by accelerometer. Medicine & Science in Sports & Exercise, 40(1), 181–188. https://doi.org/10.1249/mss.0b013e31815a51b3

29.

Troiano

R. P.

McClain

J. J.

Brychta

R. J.

Chen

K. Y.

(2014). Evolution of accelerometer methods for physical activity research. British Journal of Sports Medicine, 48(13), 1019–1023. https://doi.org/10.1136/bjsports-2014-093546

30.

Trost

S. G.

Owen

Bauman

A. E.

Sallis

J. F.

Brown

(2002). Correlates of adults’ participation in physical activity: Review and update. Medicine & Science in Sports & Exercise, 34(12), 1996–2001. https://doi.org/10.1097/00005768-200212000-00020

31.

Van Hees

V. T.

Gorzelniak

Dean León

E. C.

Eder

Pias

Taherian

Ekelund

Renström

Franks

P. W.

Horsch

Brage

(2013). Separating movement and gravity components in an acceleration signal and implications for the assessment of human daily physical activity. PLoS One, 8(4), Article e61691. https://doi.org/10.1371/journal.pone.0061691

32.

Wang

Graubard

B. I.

Katki

H. A.

(2020). Improving external validity of epidemiologic cohort analyses: A kernel weighting approach. Journal of the Royal Statistical Society: Series A, 183(3), 1293–1311. https://doi.org/10.1111/rssa.12564

33.

Wenz

Jäckle

Couper

M. P.

(2019). Willingness to use mobile technologies for data collection in a probability household panel. Survey Research Methods, 13(1), 1–22.

34.

World Health Organization . (2010). Global recommendations on physical activity for health. World Health Organization. https://www.who.int/publications/i/item/9789241599979

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.88 MB