Abstract
Objective:
We describe the methodologic approach to rapidly launching a population-based surveillance study in spring 2020 to examine the lasting physical, mental, and economic effect of COVID-19 among adults in Michigan.
Materials and Methods:
We established a partnership between the University of Michigan School of Public Health and the Michigan Department of Health and Human Services to conduct this study. Using a sequential stratified sampling strategy, we randomly selected adults with polymerase chain reaction–confirmed SARS-CoV-2 infection in the Michigan Disease Surveillance System. From 2020 through 2022, respondents completed detailed surveys on the lasting effect of their COVID-19 illness online in English or by telephone in English, Spanish, or Arabic, reflecting the diverse population in Michigan. We created and used sampling weights to reduce survey nonresponse bias and tested the performance of the weights with a nonresponse bias analysis.
Results:
Of all sampled people (n = 17 584), 5521 completed our baseline survey a median of 4.5 months after their COVID-19 onset, for a response rate of 32.1%. Most respondents completed the survey online in every region except Detroit, where 67.1% completed the survey by telephone, highlighting the importance of multimode surveys to increase accessibility and generalizability. Our findings suggest minimal nonresponse bias in the weighted baseline sample.
Practice Implications:
This unique academic–state partnership resulted in timely and actionable findings related to the lasting effect of COVID-19 that were unavailable elsewhere. While this effort was successful, it was built out of necessity given the limited resources available to local and state health departments to conduct surveillance during the COVID-19 pandemic.
Although the acute stage of the COVID-19 pandemic has passed, the lasting physical, mental, and economic effects of COVID-19 continue to unfold. An estimated 10% of individuals infected with SARS-CoV-2 have not recovered from their acute illness, a condition known as post–COVID-19 condition, or colloquially as “long COVID.” 1 Additionally, many people have experienced social and economic effects of COVID-19 illness and recovery, including stigma and financial strain.2-4 It is critical to understand the toll of COVID-19 on society to support people who are experiencing long-term consequences of COVID-19 illness and other pandemic-related hardships and develop appropriate safety nets for future public health emergencies. However, surveillance efforts to document the lasting effect of COVID-19 are limited.
Because of historic disinvestment in public health, 5 sufficient resources were unavailable to health departments tasked with tracking local SARS-CoV-2 outbreaks and implementing containment measures. 6 The network of public health departments in the United States was often unable to adequately support essential COVID-19 surveillance efforts, including case investigation and contact tracing.7-9 Although several health departments attempted to conduct follow-up interviews to better understand the recovery process, 10 most were unable to track anything beyond the acute phase.
Many early studies relied on clinical samples to estimate population metrics related to the prolonged health effect of COVID-19 illness. However, clinical samples have limited generalizability, because they include patient populations with more severe illness or better access to care than the general population. 11 Furthermore, clinical cohorts often lack pertinent information on the social and economic effects of COVID-19 illness and recovery. Although convenience samples, such as online surveys conducted by the Patient-Led Research Collaborative, 12 have been informative in characterizing post–COVID-19 condition, they too lack the generalizability needed to estimate the prevalence of post–COVID-19 condition and do not capture other effects of COVID-19, such as job loss, for those without post–COVID-19 condition.
We describe a unique academic–state partnership established between the University of Michigan School of Public Health and the Michigan Department of Health and Human Services (MDHHS) to rapidly deploy a statewide surveillance study of people at least 30 days after their COVID-19 onset. Before the long-term effects of COVID-19 were known, epidemiologists at both institutions recognized the importance of monitoring the lasting effect of COVID-19 illness and the pandemic writ large. The Michigan COVID-19 Recovery Surveillance Study (MI CReSS) aimed to (1) conduct public health surveillance to learn about COVID-19 illness and recovery, including physical, mental, and economic effects, using a representative sample of adults with polymerase chain reaction (PCR)–confirmed SARS-CoV-2 and (2) document sociodemographic differences in COVID-19 treatment and the differential effect of COVID-19 illness and recovery on communities throughout the state. We describe the procedures, data collection metrics, and lessons learned from this surveillance study.
Materials and Methods
Setting
Our study was conducted in Michigan, a racially and ethnically diverse Midwestern state with more than 10 million residents. Fourteen percent of Michigan residents identify as Black or African American and nearly 6% identify as Hispanic or Latino. 13 Michigan is also home to the second-largest Arabic-speaking population in the United States 14 ; 1.8% of residents aged ≥5 years speak Arabic. 15 Michigan was a hot spot for COVID-19 at several points during the pandemic. In March 2020, Michigan had the third-highest number of COVID-19 deaths of all US states. 16 In 2021, Michigan had the highest national COVID-19 case rate per capita, 17 prompting the US government to send clinical support teams. These factors make Michigan a useful setting in which to conduct COVID-19 surveillance.
Academic–State Partnership
In April 2020, epidemiologists from MDHHS approached faculty at the University of Michigan School of Public Health to explore the joint development of a surveillance study to monitor the effect of COVID-19 illness and recovery on the population in Michigan. MDHHS, which was leading Michigan’s COVID-19 pandemic response, did not have the capacity to conduct this study, and no funding was earmarked for this type of surveillance at this time. A small group of faculty members, postdoctoral fellows, students, and staff at the University of Michigan volunteered to develop a survey tool for what was presumed to be a short-term surveillance study. Small-scale internal funding from various programs at the University of Michigan supported initial data collection by a team of Department of Epidemiology graduate students who conducted interviews in English, Spanish, and Arabic. Although MDHHS was not able to provide funding initially, MDHHS epidemiologists served as active collaborators, meeting weekly with the MI CReSS principal investigator and other faculty members to strategically guide the project. However, beginning in March 2020, Congress signed several COVID-19 relief packages into law, allocating funding for the Centers for Disease Control and Prevention (CDC) to bolster local COVID-19 surveillance efforts. Our existing academic–state partnership facilitated the use of CDC funds, administered via MDHHS, to transition MI CReSS into a sustained surveillance effort with both student and professional interviewers, including the addition of a cohort component.
Data Source and Sampling Procedure
As part of the academic–state partnership, the University of Michigan established a data use agreement with MDHHS to allow access to records of people with PCR-confirmed SARS-CoV-2 in the Michigan Disease Surveillance System (MDSS). 18 Michigan’s Public Health Code 19 requires that certain communicable diseases, including SARS-CoV-2, are reported to local health authorities by testing facilities. 20 These reports are aggregated in MDSS, a web-based platform that facilitates statewide tracking of communicable diseases and coordination between local and state jurisdictions. Records should reflect all people with PCR-confirmed SARS-CoV-2 in Michigan.
We restricted the MI CReSS sampling frame to noninstitutionalized adults aged ≥18 years who were alive at the time of the sample draw, per their MDSS record, and had zip code and/or county information and a telephone number in MDSS. At the time of contact, we further restricted the sample to those who were able to complete the survey in English, Spanish, or Arabic by themselves or with assistance from a proxy.
From this sampling frame, we drew stratified probability samples using 13 geographic regions as strata so the resulting data could be summarized at both the state and regional levels. The strata included Detroit, 6 counties in southeast Michigan (Macomb, Monroe, Oakland, St. Clair, and Washtenaw counties and Wayne County, excluding Detroit), and 6 public health preparedness regions. 21 We sampled at the city and county level in southeast Michigan because this was the epicenter of the Michigan outbreak when we were developing our sampling approach. 22
We drew sequential random samples throughout data collection, which typically covered COVID-19 onset dates during a 1.5- to 2-month period. For each sample, we drew a base number of 50 to 70 people from each geographic stratum to ensure adequate sample size for reporting regional results. We drew the remainder of the sample proportionate to the number of cases in each stratum during the time frame to reach a total sample size of 2000 (sample 1) or 1000 (subsequent samples). The University of Michigan (HUM00181068) and MDHHS institutional review boards deemed this study public health surveillance. All participants provided informed consent verbally (telephone survey) or electronically (online survey).
Data Collection Tool
We drew from validated surveys and forms to develop the data collection tool, including the Behavioral Risk Factor Surveillance System, 23 the California Health Interview Survey, 24 the PhenX toolkit, 25 and the MDHHS COVID-19 case investigation form. After initial review by the University of Michigan Department of Epidemiology faculty, we went through several iterations with MDHHS to finalize the tool. A team of MI CReSS student interns developed a Qualtrics survey to collect data for both the telephone and online survey. The survey tool was updated multiple times throughout data collection to reflect emerging aspects of the pandemic (eg, post–COVID-19 condition, vaccination).
Interviewer Training
We developed a training protocol for interviewers, including (1) a 2-hour virtual onboarding training and (2) asynchronous training materials, including an interviewer manual, videos on interviewing tips and handling difficult conversations related to COVID-19 stress and grief, and excerpts from the University of Illinois at Chicago Survey Research Laboratory Telephone Survey Training Manual. All interviewers completed the Program for the Education and Evaluation of Responsible Research and Scholarship training on Responsible Conduct of Research and Scholarship and Human Subjects Research Protections. Throughout baseline data collection, we met with interviewers weekly to discuss progress and troubleshoot recruitment issues.
Recruitment
We sent an introductory recruitment letter and consent document to all sampled people by mail. The introductory letter and mailed reminder letters included our study website and a unique code for the selected individual to access the survey online in English. Several weeks after sending the introductory letter, interviewers began calling sampled individuals who had not completed the survey online to offer a telephone interview in English, Spanish, or Arabic. To streamline efforts, we identified Hispanic and Arab surnames and assigned these individuals to Spanish- and Arabic-speaking interviewers, respectively. If available, proxies were allowed to assist respondents who did not speak English, Spanish, or Arabic; were hearing impaired; or did not have the mental capacity to independently complete the survey.
Interviewers used InContact, an MDHHS telephone banking system, to place calls. Interviewers were required to make a minimum of 5 call attempts to sampled individuals who had not yet completed the survey or had not refused to participate. Respondents in early samples received a $10 gift card for compensation. After more substantial funding was received from MDHHS, respondents received a $25 gift card.
Community Advisory Committee
Because of the disproportionate effect of the pandemic on racial and ethnic minority populations and rural communities, we established a MI CReSS Community Advisory Committee in December 2020 with individuals from these groups to guide our efforts. We recruited advisory members through multiple channels, including University of Michigan resources to link academic researchers to community members and connections made via MDHHS and local health departments. We also asked recruited individuals for recommendations on additional individuals to contact. Throughout the study, our Community Advisory Committee had 8 to 10 members representing Black, Latino, Indigenous, Arab American, and rural White communities from across Michigan, comprising 7 of the 8 public health preparedness regions. Committee members received $150 per 2-hour meeting, with meetings held bimonthly or quarterly. The Community Advisory Committee has been invaluable to study efforts, providing input on the appropriateness and relevance of survey questions for their respective communities, context for interpreting survey findings, and plans for data dissemination.
Weighting and Nonresponse Bias Analysis
We created sampling weights so that the weighted distribution of the survey data matched the age and sex distribution of the MDSS sampling frame overall and within geographic strata by using generalized regression estimation. 26 This approach is equivalent to nonresponse weighting with calibration because no additional variables were available for sampled individuals beyond these sampling frame measures.
To test the performance of the weights in accounting for nonresponse, we conducted a nonresponse bias analysis in which we multiply imputed missing values for the entire sampled dataset (respondents and nonrespondents) by using sequential regression under the missing-at-random assumption. 27 To impute values for nonrespondents, we used a sample indicator, selection weights, and available information on age, sex, race and ethnicity, and geographic stratum in MDSS. We also geocoded individuals in our sample with matched address information in the Census Geocoder tool 28 (16 721 of 17 584; 95.1%) to account for the following census tract–level variables from the 2020 American Community Survey 5-year estimates 29 : proportion who are unemployed, who are living below the federal poverty level, who are aged ≥25 years, who have ≥bachelor’s degree, who are non-Hispanic White, and who have home internet access; proportion of owner-occupied housing units; and median annual household income. We then compared prevalence estimates from the imputed and nonimputed datasets for 12 survey questions representing 4 broad categories: access to care, COVID-19 severity and recovery, social stigma and stressors, and employment. We imputed 20 datasets separately for the 4 broad categories, except for 2 items related to fear of disclosing COVID-19 diagnosis, which were imputed separately because the survey questions were added midway through sample 1 data collection. We obtained 95% CIs for weighted estimators by using Taylor Series approximations in Stata version 18 (StataCorp).
Results
Nearly all sampled people were eligible to participate (17 211 of 17 584; 97.9%). Of all eligible sampled people, including those we never successfully reached, 5521 (32.1%) with COVID-19 onset between February 29, 2020, and May 31, 2022, completed our baseline survey (eTable 1 in Supplemental Material). Of those we successfully reached, 51.1% completed our baseline survey. Respondents completed baseline surveys a median (IQR) of 4.4 (3.4-5.7) months after their COVID-19 onset. Most respondents completed the survey online in every region except Detroit, where 67.1% completed the survey by telephone (eTable 2 in Supplemental Material). Our weighted baseline sample is similar to the MDSS case population with respect to age, sex, and geographic stratum but less comparable with respect to known race and ethnicity (Table 1). Given the large proportion of people missing data on race or ethnicity in MDSS (31.1%), we were unable to account for race or ethnicity in our sampling weights.
Demographic and geographic comparison of baseline respondents, sampled individuals, and the sampling frame, Michigan COVID-19 Recovery Surveillance Study, 2020-2022
Weighted with the final weight, accounting for the probability of selection and calibration to the sampling frame (Michigan Disease Surveillance System).
Weighted with the selection weight, accounting for the probability of selection.
Sex missingness did not exclude individuals from being sampled but did exclude them from contributing to the calibration weights based on the sampling frame.
Excluding Detroit.
Results from the nonresponse bias analysis suggest minimal bias in our weighted baseline sample (Table 2). Point estimates were generally similar across the weighted baseline sample (respondents only) and imputed baseline sample with all sampled individuals (respondents and nonrespondents). However, the imputed sample had slightly higher estimates than the weighted baseline sample (1 to 2 percentage points) for COVID-19 severity and recovery measures, suggesting our weighted baseline sample may be slightly biased toward people who were less severely affected by COVID-19.
Nonresponse bias analysis comparing estimates from the baseline sample with all sampled individuals, Michigan COVID-19 Recovery Surveillance Study, 2020-2022
Experienced by respondent or respondent’s family member.
Employed respondents whose physical presence was required at work.
Discussion
Because population-based samples were limited for most of the pandemic, the COVID-19 recovery literature is largely based on clinical populations or convenience samples, although the field is evolving with the widespread recognition of post–COVID-19 condition. For example, the National Institutes of Health Researching COVID to Enhance Recovery (RECOVER) initiative, the largest coordinated effort to understand post–COVID-19 condition, uses electronic health records30-33 and opt-in observational cohorts34,35 to examine biological and clinical aspects of post–COVID-19 condition. Additionally, convenience samples, often driven by patient advocates, have helped define and characterize post–COVID-19 condition.12,36,37 However, clinical and convenience samples lack the generalizability needed to estimate the prevalence of post–COVID-19 condition and contain limited, if any, data on other aspects of recovery, including social and economic effects. Increasingly, national population-based surveys are incorporating questions on post–COVID-19 condition. The largest national study with data on post–COVID-19 condition is the Household Pulse Survey, which is limited by a repeated cross-sectional design, online-only format, and response rates of 4% to 7%. 1 Other national surveillance systems (eg, Behavioral Risk Factor Surveillance System) and cohort studies (eg, Panel Study of Income Dynamics) have recently added questions on post–COVID-19 condition but lack a detailed measurement of post–COVID-19 condition or other aspects of COVID-19 recovery. MI CReSS is unique in that it facilitates an in-depth understanding of the physical, mental, and economic effects of COVID-19 illness and recovery by using a population-based sample, making a valuable contribution to the scientific community’s understanding of the effect of COVID-19.
Lessons Learned
Quickly deploying a statewide surveillance study during the pandemic resulted in many lessons learned. First, it was wise to heavily invest in the study early on, even though we had minimal funding and initially envisioned MI CReSS as a short-term effort. To quote Hyder et al, “Planning and deploying proactive and innovative public health practices during a global pandemic is not an easy task, but it is possible with the right set of collaborators and conditions. One such condition is the willingness of collaborators to take informed risks and set aside organizational or professional goals and objectives for the sake of the greater and common good.” 38 In our case, the successful launch and sustained COVID-19 recovery surveillance were enabled by a strong collaboration between MDHHS and the University of Michigan. Although we could not predict the duration of the pandemic, or extended need for MI CReSS, the risk paid off in terms of having actionable information on COVID-19 recovery early in the pandemic. Relatedly, our sequential sampling strategy proved extremely adaptable, allowing us to modify the survey tool in real time to capture changing circumstances, such as the emergence of post–COVID-19 condition and vaccine availability. Additionally, once MDHHS received COVID-19 relief funding, we had the financial support to extend baseline data collection and add a cohort component.
Consistent with other large probability surveys,39,40 we experienced challenges with response rates. The creation of sampling weights was helpful in minimizing nonresponse bias in our sample. Although imputation was a useful tool to estimate bias because of survey nonresponse, we do not recommend imputation for all sampled individuals in lieu of weighting respondent data to account for nonresponse, given that we had to impute data for 68.5% of sampled individuals to obtain imputed estimates. Additionally, we found that the use of a multimode survey (telephone and online) was critical to increasing accessibility and generalizability. For example, 67.1% of respondents in Detroit completed the survey by telephone, compared with 30% to 40% of respondents in other Michigan regions. This difference is likely because Detroit has a lower median annual household income ($37 761) than the state of Michigan overall ($68 505) and a lower proportion of households with a broadband internet subscription (79.4%) than the state of Michigan (87.8%).13,41
Finally, although SARS-CoV-2 is a reportable disease, many sampled individuals were surprised, and sometimes upset, that we were aware of their SARS-CoV-2 infection. This reaction may have occurred because of COVID-19 stigma, 2 particularly early in the pandemic, and pandemic politicization. 42 We worked closely with interviewers throughout data collection to understand the feedback they were receiving from sampled individuals and kept a living Frequently Asked Questions document with suggested responses for interviewers to use in challenging situations.
Dissemination and Next Steps
We have packaged MI CReSS findings for academic, professional, and lay audiences. In addition to publishing articles in peer-reviewed journals, we produced 10 reports for MDHHS to aid in understanding the effect of COVID-19. Reports included information on access to COVID-19–related health care, exposure, illness severity and recovery, social stigma and stressors, and COVID-19–related employment findings, documenting disparities across race and ethnicity, rurality, and disability status. MDHHS shared this information widely with governmental decision makers, local health departments, clinicians and health systems, the media, and the Michigan Coronavirus Task Force on Racial Disparities. 43 We also created public-facing versions of the reports suitable for the general public. 44 Additionally, we worked with our Community Advisory Committee and the University of Michigan’s Institute for Healthcare Policy and Innovation 45 to develop regional infographics 46 with key findings for each region, county, or city in our sampling strata. Additional manuscripts, reports, and infographics are underway. 44
Given our baseline results reflecting the lasting effect of COVID-19, we pursued a follow-up study with the MI CReSS baseline cohort. With funding from MDHHS, we enlisted the University of Michigan Survey Research Operations 47 to coordinate and conduct the cohort component. The initial follow-up survey was completed 1 to 2 years after respondents’ baseline interview, with a second follow-up survey currently in progress.
Practice Implications
This unique academic–state partnership resulted in timely and actionable findings related to COVID-19 illness and recovery that were unavailable elsewhere. Our success was due, in large part, to the willingness of team members at both the University of Michigan and MDHHS to devote time and resources to establishing this surveillance effort with minimal funding early in the pandemic. The engagement of a Community Advisory Committee was also instrumental in focusing analyses and dissemination on topics that were especially relevant to populations disproportionately affected by COVID-19. While this effort was successful, this partnership was built out of necessity given the limited resources available to local and state health departments to conduct surveillance during the pandemic. Academic–state partnerships can serve as an important avenue to move critical public health work forward, particularly during times of resource scarcity.
Supplemental Material
sj-docx-1-phr-10.1177_00333549251323859 – Supplemental material for Methodology for a COVID-19 Recovery Surveillance Study Conducted Through an Academic–State Partnership
Supplemental material, sj-docx-1-phr-10.1177_00333549251323859 for Methodology for a COVID-19 Recovery Surveillance Study Conducted Through an Academic–State Partnership by Jana L. Hirschtick, Yanmei Xie, Blair Whittington, Akash Patel, Michael R. Elliott, Kristi Allgood, Joseph Coyle and Nancy L. Fleischer in Public Health Reports
Footnotes
Acknowledgements
The authors thank the Michigan COVID-19 Recovery Surveillance Study participants and interviewers for making this study possible, as well as the study’s Community Advisory Committee, including Ghada Aziz; Rev. Sarah Bailey, PhD, MA; Ashley Bauer, BSN; Vicki Dobbins, BA; Carlton Evans, BS; Adnan Hammad, PhD; Chuqui King, BS; Tyler Migizii Migwan LaPlaunt, MS; Marta Larson, MA; Roquesha O’Neal; LaKila Shea Salter, BA; Joseph Sowmick; Christina Vasquez; and Filiberto Villa-Gomez.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Michigan Department of Health and Human Services, Michigan Public Health Institute, University of Michigan (UM) Institute for Data Science, UM Rogel Cancer Center, UM Epidemiology Department, and the Centers for Disease Control and Prevention (CDC) of the US Department of Health and Human Services (HHS) (grant no. 6NU50CK000510-02-04). The contents of this article are those of the authors and do not necessarily represent the official views of, nor an endorsement by, CDC/HHS or the US government.
Supplemental Material
Supplemental material for this article is available online. The authors have provided these supplemental materials to give readers additional information about their work. These materials have not been edited or formatted by Public Health Reports’s scientific editors and, thus, may not conform to the guidelines of the AMA Manual of Style, 11th Edition.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
