Abstract
Our objective was to understand the rates of missed follow-up appointments following discharge from a tertiary neonatal intensive care unit (NICU) over the course of 1 year. We abstracted demographic and clinical information from the medical record into a database, including appointment referrals and outcomes of those appointments. Follow-up compliance data were reviewed for one calendar year following NICU discharge for each patient. Using our definition of “lost-to-follow-up” (LTFU), of 154 neonates reviewed, 111 (72.1%) were not LTFU, while 43 (27.9%) were LTFU for at least one sub-specialty. Adjusted for co-variates, children who were LTFU had more follow-up appointments (odds ratio [OR]: 1.53 for one appointment increase, 95% confidence interval [CI]: [1.20, 1.99]), were more likely to be discharged with a medical device (OR: 1.00, 95% CI: [0.36, 2.77]) and were more likely to have Medicaid or no health insurance at admission (OR: 3.45, 95% CI: [1.43, 9.19]). Our study suggests that financial stability in relation to insurance type may warrant further attention for missing medically necessary follow-up services. Additionally, increased number of follow-up appointments recommended or scheduled at the time of NICU discharge puts those infants at higher risk for becoming LTFU. Overall, maximizing the use of multidisciplinary follow-up clinics to reduce the total number of clinic appointments required may increase NICU follow-up attendance.
Plain language summary
Infants admitted to the neonatal intensive care unit (NICU) continue to require care following discharge from hospitalization. However, little is known about what influences attendance required for follow-up appointments. Using electronic medical health records, we extracted care appointment and social characteristics of 599 infants discharged from the NICU with a HIPAA-compliant database to ensure privacy. We used regression modeling to visualize the connections between multiple variables and their appointment follow-ups. Our study found that many determinants of adequate follow-up, such as distance from medical services and length of NICU admission, were not associated with a greater loss to follow-up. However, children with more specialty appointments were more likely to become lost-to-follow-up, and children with Medicaid or no insurance were more likely to miss appointments than those with private insurance. The data show that an increased number of follow-up appointments, among other determinants, place post-NICU infants at a higher risk of not attending and receiving the appropriate care. These relationships can be used to streamline hospital admittance processes. Though this data can optimize the coordinated care of post-NICU infants, there are limitations to consider. Because of our limited focus on data from the electronic medical record, it can be reasonably assumed that there may have been missing or excluded data from outside of our hospital system. However, this study is one of the first to identify factors that may place infants at risk for missing medically necessary services. It is important to continually protect the health of post-NICU children.
Introduction
In the United States, 10% to 15% of infants will require an admission to a neonatal intensive care unit (NICU), which is approximately 500,000 neonates annually (Goodstein et al., 2021). Babies admitted to the NICU experience both short- and long-term medical and neurodevelopmental risks (Eichenwald & Stark, 2008; Lemons et al., 2001; Mikkola et al., 2005; Robertson et al., 2009), requiring coordinated, multidisciplinary follow-up after discharge from the NICU (Goldstein & Malcolm, 2019). As admission rates have increased and survival rates have improved in the last decade, appropriate follow-up enables these children to thrive, by receiving the NICU follow-up support they require. Coordinated outpatient follow-up services are essential for enhancing the health and developmental outcomes of this particularly vulnerable population (Hill et al., 2003; McCormick et al., 2006; Spittle & Treyvaud, 2016). When a child misses their recommended follow-up appointments, crucial opportunities for early referral and intervention programs are missed.
While post-NICU follow-up care was previously well-documented in the literature (Campbell et al., 1993; Catlett et al., 1993; Slater et al., 1987; Tin et al., 1998; Tyson et al., 1988; Wariyar & Richmond, 1989), fewer studies have been performed in recent years, limiting our understanding of current neonatal practices, societal factors and mobility, or medical accessibility. A recent Canadian study identified single-parent mothers who self-reported worry about substance use or who lived further from the clinic as negative predictive factors on NICU follow-up attendance rates (Ballantyne et al., 2014). Conversely, mothers who experienced greater stressors at the time of NICU hospitalization were more likely to attend the follow-up clinic. Similarly, a study based in the U.S. supported these findings, noting maternal drug use, multiple gestation pregnancy, male sex and greater distance from the hospital as being associated with lower appointment compliance (Harmon et al., 2013).
To date, no published reviews exist of follow-up compliance across multiple specialties. It is important to understand the rates of lost-to-follow-up (LTFU) to identify areas for improvement and targeted interventions. This retrospective chart review reports the subspecialty-specific rates of missed NICU follow-up appointments following discharge from a large tertiary-care NICU over the course of 1 year and describes patient-specific risk factors for being LTFU. The results from this study may inform future planning and practice related to NICU follow-up in similar tertiary referral centers.
Methods
Study Population
A retrospective chart review was performed for neonates discharged alive from a Midwestern children’s hospital, from January 1, 2016, to December 31, 2016. This hospital has one of the state’s only comprehensive level IV NICU nurseries, with a 60-bed NICU and a High-Risk Infant Follow-Up Care Program. Typically, just over 600 neonates are admitted each year, and numerous medical subspecialty clinical programs are provided by the health system, serving the entire Midwestern state. Over the study period, 50% of discharged patients were selected for inclusion in the study using a random number generator. Individuals were excluded from review after sampling if they had died or were discharged to another hospital, hospice care, or a long-term care facility.
Data Collection
Three reviewers abstracted information from the medical record into a REDCap database (Harris et al., 2009), including medical history, demographic and social characteristics, and appointment referrals and outcomes. Follow-up compliance data were reviewed for one calendar year following NICU discharge for each patient.
Of note, two specialties, primary care and audiology, were excluded from the definition of LTFU due to lack of available data. Because of the characteristics of the service area represented by the population, most neonates discharged from the NICU received primary care services outside of the hospital system and its electronic health record system, which made tracking compliance difficult. Audiology follow-up at 9 to 12 months of age is recommended by the state health department for neonates’ primary care providers to arrange, and so appointments are not scheduled prior to NICU discharge. As a result, many patients receive audiology follow-up outside of the hospital system. This was confirmed when we found only 20% of patients received audiology follow-up within our hospital system in the recommended window. Thus, patients receiving only primary care and/or audiology follow-up were excluded from these analyses.
For inter-rater reliability, a random sample of approximately 25% of all included records (n = 44) were reviewed by two of the three total reviewers to ensure data quality. Disagreements between raters were addressed continuously to improve future reviews. Inter-rater reliability was calculated using Cohen’s kappa, which accounts for the possibility of agreement occurring by chance. For selected records, κ = 0.64, indicating good agreement. In total, only six children had discordant classifications.
Definitions
For the primary outcome of interest, LTFU was defined as failing to attend at least one NICU follow-up within 90 days of the recommended or originally scheduled appointment date. Distance from the hospital, where most, but not all, follow-up appointments occur, was calculated using the R package zipcodeR, which calculated the patient’s distance to travel using the hospital’s address and the U.S. Department of Housing and Urban Development-defined longitude and latitude values for the patient’s ZIP code (Rozzi, 2021).
Statistical Methods
All analyses were performed using R version 4.2.3 (R Core Team, 2020). Statistical significance was defined as p < .05. Patients who were LTFU were compared to those who were not with respect to medical and socioeconomic characteristics. Categorical variables were assessed using the chi-squared test, while continuous variables were evaluated with the Wilcoxon rank sum test. Logistic regression was used to examine the relationship between patient characteristics and LTFU. Variables showing associations in univariate analysis were included in multivariate logistic regression models. Given the sample size, linearity between continuous variables and the log odds of LTFU was not checked. Based on the final model, residual plots demonstrated independence of observations (residuals within ±2 and random pattern), and variance inflation factors showed no multicollinearity (all variance inflation factors less than 2). Cook’s distance was employed to identify influential observations and sensitivity analysis showed consistent results after removing the first three influential observations. Additionally, the attendance of each subspecialty was summarized.
Ethics
This study was approved by the accompanying Institutional Review Board as an exempt study (secondary research for which consent is not required). As a retrospective chart review involving no direct contact with human subjects and using only previously collected clinical data, the study posed minimal risk to participants. All identifiable information was removed or encrypted, and data were handled using HIPAA-compliant methods to ensure privacy and confidentiality. No consent was obtained from families due to the study design.
The study design minimized risk of harm by relying solely on existing medical records, without introducing any interventions or requiring interaction with patients or families. No procedures or data collection activities altered patient care or imposed additional burden on the participants. The potential benefits of this research—namely, identifying factors associated with being LTFU among high-risk neonates and informing future strategies to improve continuity of care—were deemed to outweigh any potential risks. The findings have the potential to improve health outcomes and care coordination for vulnerable neonatal populations on a broader scale.
Results
Out of 599 discharged neonates, 253 (42.2%) were randomly selected for this analysis. Of these 253, 29 were excluded due to being deceased or transferred to an outside hospital, hospice, or long-term care. After removing those without follow-up appointments, missing information on service type or status, or only having primary or audiology services at follow-up, 154 neonates (580 visits) remained. Following the exclusion of duplicate visits with the same service on the same date and of visits for primary care and Audiology, the final analysis included 154 neonates and 429 visits. Of those, 111 (72.1%) were not LTFU, while 43 (27.9%) were LTFU for at least one sub-specialty. Figure 1 provides a flow diagram of recruitment and analysis.
Flow diagram for patient inclusion and analysis.
Children who were LTFU were of lower gestational age, weighed less at birth, and had longer hospitalizations than children who were not LTFU; they had more follow-up appointments and were more likely to be discharged with a medical device (Table 1). Mothers of children who were LTFU were less likely to carry private insurance.
Characteristics of the Sample Subsets.
In univariate analysis, there were lower odds of LTFU with increasing of gestation age (odds ratio [OR]: 0.93 for 1 week increase, 95% confidence interval [CI]: [0.86, 1.00]). Additionally, those with more appointments (OR: 1.56, 95% CI: [1.28, 1.94]), having a medical device (OR: 2.96, 95% CI: [1.39, 6.69]), and having Medicaid or no health insurance at time of hospital discharge (OR: 3.59, 95% CI: [1.59, 8.95]) were more likely to be LTFU. In multivariable analysis, more appointments (OR: 1.53, 95% CI: [1.20, 1.99]) and having Medicaid or no health insurance (OR: 3.45, 95% CI: [1.43, 9.19]) were associated with LTFU (Table 2).
Univariable and Multivariable OR for LTFU Factors.
Of those that were not LTFU, 44 (39.6%) had only one appointment, while 6 (5.4%) had six or more appointments. Conversely, of those that were LTFU, 6 (14.0%) had only one appointment, while 8 (18.6%) had six or more appointments. Other than the NICU follow-up clinic, which had no appointments kept among our cohort, the proportion of appointments attended differed substantially across subspecialties, ranging from 64.7% (Developmental Behavioral Pediatrics/Complex Care) to 96.3% (Urology) (Table 3).
Frequency of Follow-up Appointments Attended.
Discussion
This retrospective chart review provides an updated analysis of factors associated with appointment attendance after discharge from a large Level IV NICU. When adjusted for other possible risk factors, our study found that only insurance type and the number of recommended/scheduled follow-up appointments influenced the rates of LTFU. No other clinically relevant factors were identified to have influenced LTFU rates, supporting the importance of medical coordination upon discharge to ensure appropriate follow-up.
Our finding that follow-up rates vary with insurance type has been reported previously (Harmon et al., 2013). The authors of that study hypothesized that families with private insurance may have greater financial stability to return for follow-up appointments. That is a reasonable hypothesis that could explain our findings when looking across multiple subspecialty services. Another study focusing on healthy newborn follow-up rates found that the only factor influencing appointment lateness in both univariate and multivariate analysis was Medicaid status, matching our findings (Feinberg & Hicks, 2003). These trends exist outside perinatology as well, as demonstrated by a study of hypertensive African American patients who missed their follow-up appointments, which identified no insurance as a significant risk factor (Nwabuo et al., 2014). Those with private insurance provided by their employer may be supported by company policies that allow time off for medical appointments. Furthermore, those with private insurance may have greater financial stability and resources by which to coordinate transportation to clinics, overcome unforeseen logistical obstacles, access home health nursing support, etc. The confluence of economic disadvantage may contribute to the greater likelihood for adverse birth outcomes among infants who were LTFU, such as lower gestational age and birth weight and longer periods of hospitalization.
Overall, the rates of follow-up across subspecialties were satisfactory, excluding the NICU follow-up clinic. Endocrine, Infectious Diseases, Home Ventilator Clinic, and Urology had either a perfect or near-perfect attendance rate for their follow-up, though differences in follow-up rates are likely attributable to reasons for follow-up with each subspecialty, such as the need for surgical intervention or home ventilator, as well as the subspecialty’s strategies and processes for engaging patients after hospital discharge. For example, some departments provide phone call reminders just prior to scheduled appointments. Our follow-up rates across all subspecialties are greater than those published for one regional high-risk NICU follow-up database, where they found that just 52.0% (516/993) of eligible infants in a 3-year study period attended follow-up appointments in the 18- to 24-month corrected age window (Litt et al., 2020). If we had only examined our NICU follow up clinic’s attendance rates, we would have drastically overestimated the degree of LTFU within this hospital population. It is probable that parents perceived the growth and development monitoring occurring within the NICU follow-up clinic as duplicative to what is performed by their primary care physicians, which would explain the consistently low rates of follow-up seen within these clinics.
Other works have focused narrowly on attendance rates at multidisciplinary NICU follow-up clinics. These studies will then dichotomize appointment attendance as either “compliant” or “non-compliant,” based on whether patients attended these specific visits (Harmon et al., 2013; Nehra et al., 2009). A strength of our study is that we defined LTFU as a patient who was “lost” to any of our subspecialty clinics. Analyzing the LTFU data across subspecialty clinics provides a unique view into our population and the potential impact of clinic-specific factors on non-compliance in a way that has not been studied previously. Future efforts can include comparing the procedures and resources of clinics with relatively high LTFU rates and those with low rates to identify areas for targeted intervention to improve follow-up rates. Similar efforts should be made at other institutions to understand the impact that each service line has on the LTFU rates of NICU graduates. Additionally, there is merit in considering a qualitative, observation study in the future to address gaps in patient and clinician perspectives that may have impacted LTFU, such as perceptions of the quality of clinician interactions within each subspecialty.
One limitation of our study was the absence of data from healthcare providers outside of our related hospital system and electronic health record. It can be reasonably assumed that the majority of our NICU patients remained within the system for outpatient NICU follow-up care, as there is only one alternative referral center in our state with limited subspecialties offered; however, we cannot exclude the possibility that some patients scheduled NICU follow-up appointments elsewhere. A related limitation was the exclusion of follow-up data for audiology and primary care appointments. Because there are many regional options for audiology testing and follow-up across the state, it was difficult to obtain data on which infants attended their recommended screening. As such, there remains a potential for selection bias which should be considered when interpreting the results of this study, given the exclusion of audiology and primary care visits due to limited data. Though steps were taken to ensure concordance in classification of each neonate, selection bias remains a challenge.
In summary, LTFU rates are useful for understanding and identifying which patients within an institution may be at highest risk for missing medically necessary follow-up services after discharge from the NICU. Interpreting the potential role of insurance type’s association with LTFU is complex, as it may be a secondary indicator related to financial stability and parent work infrastructure. However, it is also clear that increased number of follow-up appointments recommended or scheduled at the time of NICU discharge puts those infants at higher risk for becoming LTFU. Multidisciplinary follow-up clinics that include multiple subspecialty services may be beneficial to reducing the distinct number of appointments required after NICU discharge, which may in turn increase follow-up rates.
Footnotes
Acknowledgements
This study was conducted when the first three authors were pediatric residents at Indiana University, Riley Hospital for Children, Indianapolis, IN. We would like to acknowledge Melissa Thomas for her careful editing and review of this manuscript.
Ethical Considerations
This study was approved by the accompanying Institutional Review Board as an exempt study. No consent was obtained from families due to the study design. All identifiable information was encrypted and handled with HIPAA-compliant methods.
Author Contributions
EAR contributed to data acquisition and analysis and drafted the manuscript. SJ, JK, and RS contributed to data acquisition. AR contributed to data analysis and interpretation. MM and AH contributed to conception and design of the study. All authors critically revised the manuscript, gave final approval, and agree to be accountable for all aspects of work ensuring integrity and accuracy.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.