Restricted accessResearch articleFirst published online 2025-08
Enhancing Machine Learning Explainability of Disaster Preparedness Models from the FEMA National Household Survey to Inform Tailored Population Health Interventions
Devastating mortality, morbidity, economic, and quality of life impacts have resulted from disasters in the United States. This study aimed to validate a preexisting machine learning (ML) model of household disaster preparedness. Data from 2021 to 23 Federal Emergency Management Agency’s National Household Surveys (n = 21,294) were harmonized. Importance features from the preexisting random forest ML model were transferred and tested in multiple linear and logistic regression models with updated datasets. Multiple regression models explained 42%–53% of the variance in household disaster preparedness. Features that improved the odds of overall disaster preparedness included detailed evacuation plans (odds ratios [OR] = 3.5–5.5), detailed shelter plans (OR = 4.3–11.0), having flood insurance (OR = 1.5–2.0), and higher educational attainment (OR = 1.1). Having no specified source of disaster information lowered preparedness odds (OR = 0.11–0.53). When stratified further by older adults with Black racial identities (n = 350), television as a main source of disaster-related information demonstrated associations with increased preparedness odds (OR = 2.2). These results validate the importance of detailed evacuation and shelter planning and the need to consider flood insurance subsidies in population health management to prepare for disasters. Tailored preparedness education for older adults with low educational attainment and targeted television media for subpopulation disaster-related information are indicated. By demonstrating a feasible use case to import ML model findings for regression testing in new datasets, this process promises to enhance population management health equity for those in sites that do not yet utilize local ML.
Get full access to this article
View all access options for this article.
References
1.
National Centers for Environmental Information. Assessing the Global Climate in January 2025. National Oceanic and Atmospheric Administration. February 12, 2025. Available from: https://www.ncei.noaa.gov/news/global-climate-202501 [Accessed February 16, 2025].
2.
The Intergovernmental Panel on Climate Change. Reports. The Intergovernmental Panel on Climate Change. 2025. Available from: https://www.ipcc.ch/reports/ [Accessed February 16, 2025].
EgedeLE, WalkerRJ, CampbellJA, LindeS, HawksLC, BurgessKM. Modern day consequences of historic redlining: Finding a path forward. J Gen Intern Med, 2023; 38(6):1534–1537; doi: 10.1007/s11606-023-08051-4
5.
MarshallJ, SawyerC, KingH, DeSalvoB, PeacockW. More than half a million north carolinians under disaster declaration after hurricane helene were at high social vulnerability to disasters. October 7, 2024. Available from: https://www.census.gov/library/stories/2024/10/hurricane-helene.html [Accessed November 28, 2024].
6.
KatzL. A Racist Past, a Flooded Future: Formerly Redlined Areas Have $107 Billion Worth of Homes Facing High Flood Risk—25% More Than Non-Redlined Areas. March 14, 2021. Available from: https://www.redfin.com/news/redlining-flood-risk/ [Accessed November 28, 2024].
7.
EstienCO, WilkinsonCE, Morello-FroschR, SchellCJ. Historical redlining is associated with disparities in environmental quality across California. Environ Sci Technol Lett, 2024; 11(2):54–59; doi: 10.1021/acs.estlett.3c00870
8.
HowellJ, ElliottJR. Damages done: The longitudinal impacts of natural hazards on wealth inequality in the United States. Soc Probl, 2019; 66(3):448–467; doi: 10.1093/socpro/spy016
9.
AmbersonT, HeageleT, Wyte-LakeT, et al.Social support, educational and behavioral modification interventions for improving household disaster preparedness in the general community-dwelling population: A systematic review and meta-analysis. Front Public Health. Published Online, 2023; doi: 10.3389/fpubh.2023.1257714
10.
MonsenKA, PetersonJJ, MathiasonMA, KimE, VotavaB, PieczkiewiczDS. Discovering public health nurse–specific family home visiting intervention patterns using visualization techniques. West J Nurs Res, 2017; 39(1):127–146; doi: 10.1177/0193945916679663
11.
ShuklaM, AmbersonT, HeageleT, et al.Tailoring household disaster preparedness interventions to reduce health disparities: Nursing implications from machine learning importance features from the 2018–2020 FEMA national household survey. Int J Environ Res Public Health, 2024; 21(5):521; doi: 10.3390/ijerph21050521
12.
Adler-MilsteinJ, AggarwalN, AhmedM, et al.Meeting the moment: Addressing barriers and facilitating clinical adoption of artificial intelligence in medical diagnosis. NAM Perspect, 2022; 2022(9); doi: 10.3147/8/202209c
13.
National Academies of Sciences E and M. Examining the History, Consequences, and Effects of Race-Based Clinical Algorithms on Health Equity. (AlperJ, TaffeR, MartinezRM., eds.). National Academies Press; 2023. doi:10.1722/6/27301
MolnarC. Interpretable Machine Learning: A Guide For Making Black Box Models Explainable. 2nd ed.. Leanpub; 2022.
16.
VandenbrouckeJP, von ElmE, AltmanDG, et al.; STROBE Initiative. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): Explanation and elaboration. PLoS Med, 2007; 4(10):e297; doi: 10.1371/journal.pmed.0040297
PfaffER, GirvinAT, CrosskeyM, et al.; N3C and RECOVER Consortia. De-black-boxing health AI: Demonstrating reproducible machine learning computable phenotypes using the N3C-RECOVER Long COVID model in the All of Us data repository. J Am Med Inform Assoc, 2023; 30(7):1305–1312; doi: 10.1093/jamia/ocad077
U.S. Department of Homeland Security. Emergency Alerts. Ready.gov. October 29, 2024. Available from: https://www.ready.gov/alerts [Accessed November 28, 2024].
24.
KakoM, SteenkampM, RyanB, ArbonP, TakadaY. Best practice for evacuation centres accommodating vulnerable populations: A literature review. Inter J Disaster Risk Reduction, 2020; 46:101497; doi: 10.1016/j.ijdrr.2020.101497
25.
HeageleTN. A qualitative survey of household emergency preparedness for the elderly and the medically frail living in coastal urban communities. J Emergen Manag, 2021; 19(2):143–163; doi: 10.5055/jem.0493
26.
WilsonB, TateE, EmrichCT. Flood recovery outcomes and disaster assistance barriers for vulnerable populations. Front Water, 2021; 3; doi: 10.3389/frwa.2021.752307
27.
FloresAB, SullivanJA, YuY, FriedrichHK. Health disparities in the aftermath of flood events: A review of physical and mental health outcomes with methodological considerations in the USA. Curr Environ Health Rep, 2024; 11(2):238–254; doi: 10.1007/s40572-024-00446-7
28.
BellSA, SingerD, SolwayE, KirchM, KullgrenJ, MalaniP. Predictors of Emergency Preparedness Among Older Adults in the United States. Disaster Med Public Health Prep, 2021; 15(5):624–630; doi: 10.1017/dmp.2020.80
PaekHJ, HilyardK, FreimuthV, BargeJK, MindlinM. Theory-Based Approaches to understanding public emergency preparedness: Implications for effective health and risk communication. J Health Commun, 2010; 15(4):428–444; doi: 10.1080/10810731003753083
QiaoM, HaraguchiM, LallU. Enhancing urban resilience to extreme weather: The roles of human transition paths among multiple transportation modes. Inter J Geographical Inform Sci, 2024; 28:1–20; doi: 10.1080/13658816.2024.2418040
33.
LiJ, SpidalieriK. Home is where the safer ground is: The need to promote affordable housing laws and policies in receiving communities. J Environ Stud Sci, 2021; 11(4):682–695; doi: 10.1007/s13412-021-00702-4
34.
PlattS, MahdavianF, CarpenterO, WiensM, SchultmannF. Were the floods in the UK 2007 and Germany 2013 game-changers? Philos Trans A Math Phys Eng Sci, 2020; 378(2168):20190372; doi: 10.1098/rsta.2019.0372
35.
MulchandaniR, SmithM, ArmstrongB, BeckCR, OliverI, English National Study of Flooding and Health Study Group. Effect of insurance-related factors on the association between flooding and mental health outcomes. Int J Environ Res Public Health, 2019; 16(7):1174; doi: 10.3390/ijerph16071174
36.
McKenzieJW, LongmanJM, BailieR, et al.Insurance issues as secondary stressors following flooding in Rural Australia—A mixed methods study. Int J Environ Res Public Health, 2022; 19(11):6383; doi: 10.3390/ijerph19116383
37.
FriedmanS, FussellE, NakatsukaM, YucelR. Hispanic disaster preparedness in the United States, 2017: Examining the association with residential characteristics. Cityscape, 2021; 23(3):205–239.
38.
KouskyC. Disasters as learning experiences or disasters as policy opportunities? examining flood insurance purchases after hurricanes. Risk Anal, 2017; 37(3):517–530; doi: 10.1111/risa.12646
Al-RousanTM, RubensteinLM, WallaceRB. Preparedness for natural disasters among older us adults: A nationwide survey. Am J Public Health, 2015; 105(Suppl 4):S621–S626; doi: 10.2105/AJPH.2013.301559r
41.
ChaeMM, ChoudhuryMS, Franco-CastanoBJ, AdepojuPMOE. Self-perceived disaster preparedness in minority older adults: A cross-sectional study. Am J Disaster Med, 2022; 17(2):117–125; doi: 10.5055/ajdm.2022.0425
42.
BeckerJS, PatonD, JohnstonDM, RonanKR, McClureJ. The role of prior experience in informing and motivating earthquake preparedness. Inter J Disaster Risk Reduction, 2017; 22:179–193; doi: 10.1016/j.ijdrr.2017.03.006
43.
LawrenceJ, QuadeD, BeckerJ. Integrating the effects of flood experience on risk perception with responses to changing climate risk. Nat Hazards, 2014; 74(3):1773–1794; doi: 10.1007/s11069-014-1288-z
AdepojuO, HerreraL, ChaeM, HanD. Optimizing disaster preparedness planning for minority older adults: One size does not fit all. Int J Environ Res Public Health, 2022; 20(1):401; doi: 10.3390/ijerph20010401
46.
AmbersonT, NdayishimiyeO, CloudQY, CastnerJ. Cognitive interview validation of a novel household hazard vulnerability assessment instrument. West J Nurs Res, 2024; 46(2):90–103; doi: 10.1177/01939459231217935
Mendes-Da-SilvaW, LucasEC, CarvalhoJVdF. Flood insurance: The propensity and attitudes of informed people with disabilities towards risk. J Environ Manage, 2021; 294:113032; doi: 10.1016/j.jenvman.2021.113032
49.
ObermeyerZ, EmanuelEJ. Predicting the future—Big data, machine learning, and clinical medicine. N Engl J Med, 2016; 375(13):1216–1219; doi: 10.1056/NEJMp1606181
50.
ChenJH, AschSM. Machine learning and prediction in medicine—Beyond the peak of inflated expectations. N Engl J Med, 2017; 376(26):2507–2509; doi: 10.1056/NEJMp1702071
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
