Development and Interrater Reliability of an Observational School Environment Checklist: A Practical,Comprehensive Tool to Assess Healthy Eating and Physical Activity Opportunities in Schools

Abstract

Introduction

Comprehensive, objective assessment of schools’ eating and physical activity environments is critical to developing and evaluating policies and interventions to reduce pediatric obesity inequities; however, few tools exist that describe the entire school comprehensively and are feasible with restricted resources. This study describes development and reliability of the observational school environment checklist (OSEC), a comprehensive observational audit tool.

Method

We developed the OSEC through iterative adaptations of existing instruments and pilot testing. The tool assesses four focus areas: cafeteria, lobby/hallway, gym, and outdoor areas. For reliability testing, two trained auditors independently completed the OSEC and met to resolve disagreements. For items with poor agreement, a third independent coder coded photographs taken during auditing. Percent agreement and Cohen’s kappa were calculated for all items and across four evidence-based constructs: atmosphere, accessibility, attractiveness, and advertising.

Results

After iterative development, the 88-item OSEC was tested for reliability in 18 schools. Items with poor (<80%) agreement or redundancy were discarded or reworded (n = 16 items). All four constructs had acceptable agreement, ranging by focus area: 72.3% (attractiveness), 86.3% to 97.1% (atmosphere), 82.9% to 100% (accessibility), and 92.9% (advertising). Cohen’s kappa ranges were acceptable: 0.66–0.91 (atmosphere), 0.60–1.00 (accessibility), 0.46 (attractiveness), and 0.77 (advertising). After adding similar items across domains (n = 49) to improve comprehensiveness, the final tool contained 121 binary items.

Implications

The OSEC reliably and comprehensively captures the school environment. It requires few resources or expertise to administer, has acceptable reliability, and can assess atmosphere, accessibility, attractiveness, and advertising in school areas where students engage in eating and physical activity.

Keywords

child/adolescent health school health health promotion obesity

Get full access to this article

View all access options for this article.

References

California Department of Education. (2020). Smarter lunchrooms movement. https://www.cde.ca.gov/ls/nu/he/smarterlunchrooms.asp

Centers for Disease Control and Prevention. (2019). CDC healthy schools: School health index. https://www.cdc.gov/healthyschools/shi/index.htm

Centers for Disease Control and Prevention. (2020). CDC healthy schools: Whole School, Whole Community, Whole child (WSCC). https://www.cdc.gov/healthyschools/wscc/index.htm

Frerichs

Brittin

Sorensen

Trowbridge

M. J.

Yaroch

A. L.

Siahpush

Tibbits

Huang

T. T.

(2015). Influence of school architecture and design on healthy eating: A review of the evidence. American Journal of Public Health, 105(4), e46–e57. https://doi.org/10.2105/ajph.2014.302453

Glanz

Sallis

J. F.

Saelens

B. E.

(2015). Advances in physical activity and nutrition environment assessment tools and applications: Recommendations. American Journal of Preventive Medicine, 48(5), 615–619. https://doi.org/10.1016/j.amepre.2015.01.023

Glasgow

R. E.

Riley

W. T.

(2013). Pragmatic measures: What they are and why we need them. American Journal of Preventive Medicine, 45(2), 237–243. https://doi.org/10.1016/j.amepre.2013.03.010

Hanks

A. S.

Just

D. R.

Wansink

(2013). Smarter lunchrooms can address new school lunchroom guidelines and childhood obesity. Journal of Pediatrics, 162(4), 867–869. https://doi.org/10.1016/j.jpeds.2012.12.031

Jones

N. R.

Jones

van Sluijs

E. M.

Panter

Harrison

Griffin

S. J.

(2010). School environments and physical activity: The development and testing of an audit tool. Health Place, 16(5), 776–783. https://doi.org/10.1016/j.healthplace.2010.04.002

Kenney

E. L.

Gortmaker

S. L.

Cohen

J. F.

Rimm

E. B.

Cradock

A. L.

(2016). Limited school drinking water access for youth. Journal of Adolescent Health, 59(1), 24–29. https://doi.org/10.1016/j.jadohealth.2016.03.010

10.

Kumanyika

S. K.

(2019). A framework for increasing equity impact in obesity prevention. American Journal of Public Health, 109(10), 1350–1357. https://doi.org/10.2105/AJPH.2019.305221

11.

Lane

H. G.

Calvert

Deitch

Hager

Turner

Babatunde

Harris

Pitts

(2020). Usability of existing observational tools to measure schools’ health-promoting environment: A systematic review. Health & Place, 66, Article 102388. https://doi.org/10.1016/j.healthplace.2020.102388

12.

Lane

H. G.

Deitch

Wang

Black

M. M.

Dunton

G. F.

Aldoory

Turner

Parker

E. A.

Henley

S. C.

Saksvig

Song

H. J.

Hager

E. R.

(2018). “Wellness champions for change”: A multi-level intervention to improve school-level implementation of local wellness policies: Study protocol for a cluster randomized trial. Contemporary Clinical Trials, 75, 29–39. https://doi.org/10.1016/j.cct.2018.10.008

13.

Local School Wellness Policy Implementation Under the Healthy, Hunger-Free Kids Act of 2010, 7 C.F.R. 210 and 220 (2016). https://www.federalregister.gov/documents/2016/07/29/2016-17230/local-school-wellness-policy-implementation-under-the-healthy-hunger-free-kids-act-of-2010

14.

McHugh

M. L.

(2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/BM.2012.031

15.

Micha

Karageorgou

Bakogianni

Trichia

Whitsel

L. P.

Story

Peñalvo

J. L.

Mozaffarian

(2018). Effectiveness of school food environment policies on children’s dietary behaviors: A systematic review and meta-analysis. PLOS ONE, 13(3), e0194555. https://doi.org/10.1371/journal.pone.0194555

16.

Morton

K. L.

Atkin

A. J.

Corder

Suhrcke

van Sluijs

E. M.

(2016). The school environment and adolescent physical activity and sedentary behaviour: A mixed-studies systematic review. Obesity Review, 17(2), 142–158. https://doi.org/10.1111/obr.12352

17.

National Center for Education Statistics. (2019). Digest of Education Statistics 2018: Chapter 1. https://nces.ed.gov/programs/digest/d18/index.asp

18.

National Center for Education Statistics. (2020). Common core of data. https://nces.ed.gov/ccd/pubschuniv.asp

19.

O’Halloran

Eksteen

Gebremariam

Alston

(2020). Measurement methods used to assess the school food environment: A systematic review. International Journal of Environmental Research Public Health, 17(5), 1623. https://doi.org/10.3390/ijerph17051623

20.

Patel

A. I.

Podrabsky

Hecht

A. A.

Morris

Yovanovich

Walkinshaw

L. P.

Ritchie

Hecht

(2020). Development and validation of a photo-evidence tool to examine characteristics of effective drinking water access in schools. Journal of School Health, 90(4), 271–277. https://doi.org/10.1111/josh.12873.

21.

R Core Team. (2020). R: A language and environment for statistical computing. https://www.R-project.org/

22.

Sallis

J. F.

Conway

T. L.

Prochaska

J. J.

McKenzie

T. L.

Marshall

S. J.

Brown

(2001). The association of school environments with youth physical activity. American Journal of Public Health, 91(4), 618–620. https://doi.org/10.2105/ajph.91.4.618

23.

Sallis

J. F.

Glanz

(2009). Physical activity and food environments: Solutions to the obesity epidemic. Milbank Quarterly, 87(1), 123–154. https://doi.org/10.1111/j.1468-0009.2009.00550.x

24.

Saluja

Rawal

Bassi

Bhaumik

Singh

Park

M. H.

Kinra

Arora

(2018). School environment assessment tools to address behavioural risk factors of non-communicable diseases: A scoping review. Preventive Medicine Report, 10, 1–8. https://doi.org/10.1016/j.pmedr.2018.01.014

25.

Story

Nanney

M. S.

Schwartz

M. B.

(2009). Schools and obesity prevention: Creating school environments and policies to promote healthy eating and physical activity. Milbank Quarterly, 87(1), 71–100. https://doi.org/10.1111/j.1468-0009.2009.00548.x

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.16 MB