Abstract
PURPOSE:
The goal of this study was to evaluate the performance of a pediatric stratification tool that incorporates health and non-medical determinants to identify children and youth with special health care needs (CYSHCN) patients according to increasing levels of complexity and compare this method to existing tools for pediatric populations.
METHODS:
This retrospective cohort study examined pediatric patients aged 0 to 21 years who received care at our institution between 2012 and 2015. We used the St. Luke’s Children’s Acuity Tool (SLCAT) to evaluate mean differences in dollars billed, number of encounters, and number of problems on the problem list and compared the SLCAT to the Pediatric Chronic Conditions Classification System version2 (CCCv2).
RESULTS:
Results indicate that the SLCAT assigned pediatric patients into levels reflective of resource utilization and found that children with highly complex chronic conditions had significantly higher utilization than those with mild and/or moderate complex conditions. The SLCAT found 515 patients not identified by the CCCv2. Nearly half of those patients had a mental/behavioral health diagnosis.
CONCLUSIONS:
The findings of this study provide evidence that a tiered classification model that incorporates all aspects of a child’s care may result in more accurate identification of CYSHCN. This would allow for primary care provider and care coordination teams to match patients and families with the appropriate amount and type of care coordination services.
Introduction
Children and youth with special health care needs (CYSHCN) comprise approximately 18–20% of all pediatric patients [1, 2] and account for a large portion of child health expenditures [1]. CYSHCN have a wide range of conditions from mild chronic (e.g. asthma or eczema) to complex chronic conditions with the most complex cases characterized by high system utilization, including primary care and specialist encounters, emergency department (ED) visits, and inpatient admissions [2–4]. Care coordination (CC) was developed as part of the medical home model to address the complexity of care and relieve family/caregiver burden while improving cost-efficiency for health systems and payers [5].
Many private and state Medicaid programs have developed algorithmic risk-based methods to better identify children who would benefit from additional services by assigning them to levels, or tiers, based on utilization to predict future use and determine risk classification [6, 7]. These methods classify medical complexity using claims, electronic health records, or administrative data. However, these algorithmic-based methods have had inconsistent success [7, 8]. These mixed results are due in part to the heterogeneous nature of childhood conditions compared to adults where many health problems can be narrowed down to a select few (e.g., cancer, diabetes, cardiovascular disease). Overall health is impacted by several variables outside of the healthcare system like mental health, family factors, and social determinants of health which are not available in claims data or the electronic health record (EHR) [9–12]. This information is best collected during a personal encounter, face to face, or over the phone [13]. Given the limitations of identifying and risk-tiering based on billing data, there is need for simpler and more inclusive tools to identify and categorize CYSHCN. To address the need for such a tool, the St. Luke’s Children’s Acuity Tool (SLCAT) was developed and implemented in 2010 to identify children who would most benefit from CC services based on the severity of their diagnoses, as well as social determinants of health not currently factored into other readily available stratification tools.
Development of the SLCAT
The SLCAT was initially created and implemented in an outpatient pediatric clinic to improve identification of children who may benefit from care coordination services and provide a common nomenclature for providers and families. As of 2015, the tool has identified and stratified over 4,000 children across all providers within the health system using the guidelines for data collection and tiering. The SLCAT is currently implemented in several pediatric clinics, subspecialty practices, and inpatient settings.
To create the tool, a working group that included primary care pediatricians, a developmental pediatrician, and care coordinators met to identify critical areas of patient assessment not currently covered in risk-tiering methods. The group invited and received feedback from patients, families, and community service providers on what information and circumstances were important to assess the needs of CYSHCN. Four specific areas were identified: (1) mental/behavioral needs, (2) family factors and relationships, (3) other social determinants, and (4) community services need (e.g. education, transportation). A matrix of guidelines to stratify children using the SLCAT was then created with five evaluative dimensions (medical health, family status, developmental/behavioral, community services, and anticipated intensity of CC services) and five levels of increasing complexity or utilization (see Fig. 1). Once consensus had been reached, the SLCAT was then implemented in a pediatric clinic that utilized a full-time care coordination program and team.
St. Luke’s Children’s Acuity Tool.
Pediatric patients are first evaluated by their primary care provider (PCP) who assigns a SLCAT level based on a clinical/medical evaluation along with information gained relative to the other domains. Notably, the SLCAT does not require separate time or labor-intensive questioning/examination during the evaluation as this is provided within the scope of the evaluation as information about the patient, their diagnosis, and social determinants are discussed or provided during their first appointment. The first two levels of the SLCAT include typical healthy children from birth to maturation. Recognizing the increased utilization and work of the clinical team during the first three years of life, a distinction was made between this population and that of older healthy children. Level 1 is assigned to healthy children 3 years and older while Level 2 is used to designate healthy children under 3 years of age.
Levels 3 through 5 are considered CYSHCN with differing and increasing levels of complexity in the assessment domains. These children have conditions which are expected to last for a period of at least one year or would significantly increase their need for coordinated care by involving one or more specialists. Once familiar with the SLCAT, the provider can quickly determine whether a child is best categorized as Level 3 (mild complex), Level 4 (moderate complex), or Level 5 (highly complex). This designation is placed into the EHR by the provider.
If a child is assigned by their PCP to level three or above, then they and their family may be referred for further evaluation by an in-clinic care coordinator. If referred, the care coordinator conducts a more thorough assessment of the child and family needs for each SLCAT domain. After completing the CC evaluation and scoring each of the SLCAT domains, the care coordinator assigns a level based on their needs. The recommendation of the care coordinator and the full medical and CC assessment are reviewed by the PCP, who may adjust their original level assignment based on the comprehensive CC assessment. Thus, the acuity level reflects not only the severity of the medical condition, but the actual clinic, school, and community support needed. For example, a child with well-controlled moderate asthma would normally be categorized as Level 3. However, the same child in a refugee family, new to the country, might be designated as Level 4 or 5, depending on the family situation and the extent of coordination support needed to meet the needs of the child. It is important to note that during a child’s ongoing care, the medical and other SLCAT categories are regularly assessed. Consequently the SLCAT level may be updated based on changes to the status or needs of the child and their family.
To evaluate the SLCAT’s stratification outcomes, this study used a retrospective cohort analysis to analyze how the SLCAT predicted patient healthcare utilization across a three-year period (January 1, 2012 to December 31, 2015). Additionally, the ability of the SLCAT to predict if a child had a score of 1 or higher using the Pediatric Chronic Conditions Classification System version2 (CCCv2) was evaluated. The CCCv2 is a computer-based algorithm that uses ICD-9 and ICD-10 codes to identify patients with chronic conditions, as well as those with transplants or technology dependence [15]. Originally, the CCC was developed to examine mortality patterns and palliative care in pediatric patient populations. It has subsequently been used in studies relating to risk adjustment, predicting populations with high health care utilization, and predicting health outcomes [15–19]. Although the CCCv2 does not take behavioral health or social determinants into consideration, it is a validated standardized tool used to identify children with high system utilization [20] and is openly distributed for public access and use.
The aim of this study was to examine the SLCAT’s ability to stratify CYSHCN into actionable categories of system utilization based on complexity, and to compare the SLCAT to an established pediatric chronic condition classification system so that future studies could examine whether SLCAT stratification resulted in improved clinical outcomes in patients assigned care coordination services based on their acuity level.
Methods
All patients from birth through 21 years of age followed by a single PCP at a primary care pediatric clinic in Boise, Idaho were eligible for inclusion. The panel was composed of a suburban/rural population and was somewhat weighted towards complex care, but included a full spectrum of age and complexity. Each patient was assigned a risk tier for the SLCAT by the PCP and CC team.
All outcome information was extracted from Epic [16], an EHR database maintained by St. Luke’s Health System (SLHS) in Boise, ID. Data was extracted at the patient level beginning three years prior to their last visit in 2015. Demographic data included age, sex, and insurance type (i.e. private, Medicaid, uninsured). Diagnosis (International Classification of Disease Ninth and Tenth Revisions [ICD 9 & 10]) [17] and Current Procedural Terminology codes were collected for each patient encounter including the total dollars billed (TDB) for the specified period. This study was approved by our Institutional Review Boards.
Statistical analysis
Descriptive statistics were calculated for system utilization variables, including TDB, number of encounters, and number of problems at each SLCAT complexity level. To assess how the SLCAT stratified children, a series of independent-samples Kruskal-Wallis tests were performed to evaluate if the SLCAT categorized children into statistically significant different groups for number of encounters and number of problems. A one-way ANOVA was conducted to evaluate if the SLCAT categorized children into statistically significantly different groups for TDB. In order to compare the SLCAT to the CCCv2, crosstabulation was used to describe the agreement between a patient’s SLCAT level and conditions identified according to the CCCv2. Statistical analyses were completed using SPSS version 25.0.
Results
A total of 1303 patients from the pediatrician’s panel were examined during the study. Of the initial 1303, 107 were removed because they had seen by the PCP fewer than three times. The final analysis included 1196 patients. A detailed breakdown of the patient panel by SLCAT level by gender for payment source, age, length of care with the PCP, and age can be seen in Table 1. The age and length of care for children classified as Level 2 differs from that of Level 1 and Level 3–5 children. Given that Level 2 children are pediatric patients under the age of 3 years, they were described by age and length of time seen by the PCP in the 36-month study period which is shown in Table 1.
Patient Panel Demographic Characteristics and SLCAT Level
Patient Panel Demographic Characteristics and SLCAT Level
*Average Time Seen by PCP during the 36-month study period.
A Kruskal-Wallis test revealed a significant main effect for total encounters (χ2(4) = 268.15, p < 0.001; Fig. 2A). Pairwise comparisons, adjusted by the Bonferroni correction for multiple tests, revealed that children categorized as Level 5 had significantly higher total encounters than children categorized as Level 4 (p < 0.05), Level 3 (p < 0.001), Level 2 (p < 0.001), and Level 1 (p < 0.001). Similarly, children categorized as Level 4 had significantly higher total encounters than children categorized as Level 3 (p < 0.001), Level 2 (p < 0.01), and Level 1 (p < 0.001). Children categorized as Level 3 had significantly higher total encounters than children categorized as Level 1 (p < 0.001). Lastly, children categorized as Level 2 had significantly higher total encounters than children categorized as Level 1 (p < 0.001) (see Fig. 2).
Healthcare Resource Utilization by SLCAT Acuity Level.
An independent-samples Kruskal-Wallis test for number of problems revealed a significant main effect (χ2(4) = 451.88, p < 0.001, Fig. 2B). Pairwise comparisons, adjusted by the Bonferroni correction for multiple tests, found that children categorized as Level 5 had a significantly higher number of problems than children categorized as Level 3 (p < 0.001), Level 2 (p < 0.001), and Level 1 (p < 0.001), but not a significantly higher number of problems than children categorized as Level 4. Children categorized as Level 4 had a significantly higher number of problems than children categorized as Level 3 (p < 0.001), Level 2 (p < 0.001), and Level 1 (p < 0.001). Children categorized as Level 3 had a significantly higher number of problems than children categorized as Level 2 (p < 0.001) and Level 1 (p < 0.001). Children categorized as Level 2 did not have significantly more problems than children categorized as Level 1 (see Fig. 2).
Lastly, a one-way ANOVA revealed a significant main effect for TDB (F(4, 1195) = 37.29, p < 0.001, Fig. 2C). Post-hoc analyses revealed that children categorized as Level 5 had significantly higher TDB than children categorized as Level 4 (p < 0.001, 95% CI [8324.54, 18385.86]), Level 3 (p < 0.001, 95% CI [12514.51, 21458.55]), Level 2 (p < 0.001, 95% CI [10778.51, 21436.60]), and Level 1 (p < 0.001, 95% CI [14387.78, 23172.12]). Children categorized as Level 4 had significantly higher TDB than children categorized as Level 3 (p < 0.05, 95% CI [474.46, 6788.20]) and Level 1 (p < 0.001, 95% CI [2382.06, 8467.45]). To assess the impact of outliers on the data, 135 extreme outliers were removed from the data set and a one-way ANOVA was conducted, revealing a significant main effect for TDB. Given that the main effect and post hoc analyses were not impacted by the outliers, the ANOVA findings reported included the extreme outliers (see Fig. 2).
A comparison between the SLCAT and CCCv2 was conducted using crosstabulation to describe the overlap in SLCAT levels and flagged conditions on the CCCv2. There were 631 patients that were classified as a Level 1 or 2 on the SLCAT that did not have any chronic conditions on the CCCv2. However, there were 11 children identified as having chronic conditions on the CCCv2 that were not identified as a Level 3–5 on the SLCAT. Further analysis identified these children as having 4 cases of “other cardiac arrhythmia” (ICD10 = I498), 2 hyperbilirubinemia (ICD9 = 2774), 2 scoliosis (ICD10 = 7373), 1 irregular heart beat (ICD9 = 4279), 1 gastrostomy in place (ICD9 = v441), and 1 chest wall asymmetry (ICD9 = 7563). Upon further review by the research team and PCP who issued the diagnosis and SLCAT level, 9 of the 11 cases were flagged as healthy since the conditions had resolved or required no further services other than routine PCP visits.
Interestingly, the SLCAT identified 515 CYSCHCN that the CCCv2 did not flag with a chronic diagnosis or procedure code. Of those cases, 254 had mental/behavioral health diagnoses. Further examination revealed that the remaining patients had varying diagnoses of less severe chronic conditions and additional factors contributing to their higher SLCAT level (see Table 2).
Patient Chronic Diagnoses Identified by the SLCAT but not the CCCv2
Patient Chronic Diagnoses Identified by the SLCAT but not the CCCv2
To our knowledge, this is the first classification developed and evaluated with the specific factors listed in the SLCAT. The unique contribution of this study to pediatric risk stratification for CC is the use of information outside typical EHR data, and the identification of three potentially distinct groups within CYSHCN contrasted with binary classification of either medically complex or not [18]. Using the SLCAT, we were able to show significant differences in utilization and complexity between children with mild, moderate, and highly complex chronic conditions. In fact, there was an approximate doubling of TDB, encounters, and number of problems on the problem list between children designated as Level 3, Level 4, and Level 5 when social determinants and other factors are considered.
We were also able to demonstrate the increased clinical utilization and support needed for healthy infants and toddlers versus healthy older children. Some may feel this distinction is not clinically useful and that all healthy children should be grouped into a single category. This is certainly open to discussion. However, from the vantage point of population health, the distinct categories of Level 1 through Level 5 aids significantly in understanding both the complexity of a given pediatrician’s panel, and in predicting the system support needs, such as care coordination.
In comparing the SLCAT with the CCCv2, we found that the SLCAT did very well in identifying children with chronic conditions. Very few were identified by the CCCv2 as having a chronic condition not recognized by the SLCAT. Upon review, these children were primarily those whose chronic conditions had resolved with time or treatment. In contrast, there were over 500 children in this population who were identified by the SLCAT as having chronic health conditions (Level 3, 4, 5) who were not identified by the CCCv2. Nearly half of the discrepancy can be accounted for by mental and behavioral health diagnoses, which are not identified by the CCCv2. At the time of this study, there were no algorithm-based acuity tools for children which took mental health codes into account. As can be seen in Table 2, there were many children with significant chronic medical conditions who were not identified by the CCCv2. These include conditions such as diabetes, chronic renal failure, premature puberty, and seizures. Many mild chronic conditions affecting the child’s overall function were also included such as amblyopia, scoliosis, severe eczema, and chronic abdominal pain. Therefore, we conclude that in an ambulatory primary care population, the SLCAT was better than the CCCv2 in identifying and categorizing the acuity level of CYSHCN.
Strengths of this study include a sample composed of medically diverse children who received care supervised by a single PCP and CC team within an integrated health care system. The medical diversity of the sample provides a contrast to single disease studies and reflects a panel more typical of a pediatric practice setting. In addition, utilizing three years of data provided results which could consider significant fluctuations in utilization over time. The lack of total cost data is a limitation potentially biasing the sample and affecting the predictive power of the models. However, these differences may be small since the children in this panel receive most of their care within SLHS. It is expected that the inclusion of cost data for care provided outside SLHS (primarily mental health therapies, OT/PT) would magnify the differences observed as these services are provided more often to higher complexity children. An additional limitation of the study is that the SLCAT was compared solely to the CCCv2 and not other measures that might be more likely to be used in pediatric clinics, such as the Pediatric Medical Complexity Algorithm (PMCA) [19] or the Clinical Risk Groups [20]. However, the SLCAT tool development began prior to the development and dissemination of the PMCA and the intent was to focus on freely available and publicly distributed measures to stratify pediatric children with medical complexity. Future studies should incorporate measures, such as the PMCA, that were developed for a more similar purpose for comparison. Additionally, although the purpose of this study was to validate the SLCAT as a means of stratifying complex pediatric patients, the current study cannot speak to the reliability of classifications across providers as this was done within a single provider’s patient base. However, future studies will be able to examine intra- and interrater reliability.
The SLCAT approach addresses a common weakness of algorithms based on billing and other readily available EHR data. It improves estimation because a human systematically collects data from the patient and family regardless of plan coverage or frequency of patient visits. If recorded at all, these data are generally located in the notes portion of the record and present problems for automated processing. This improvement is important especially for clinicians or clinics that are not a part of a larger health system and do not have access to complete claims records. However, even large health systems lack the entire picture and may benefit from using the guidelines provided by the SLCAT.
Conclusion
The SLCAT is a novel tool used to identify and stratify all children by level of complexity, including CYSHCN. It considers several domains affecting a child’s health which are not currently available other than by human interaction. Also, it addresses a common weakness shared by algorithm-based acuity models for children which rely on billing or other data accessible within the EHR. By applying the SLCAT to a population of CYSHCN, it was possible to stratify mild chronic, moderate chronic, and complex chronic populations, each showing clinically and statistically different patterns of utilization and service needs. Though the traditional focus on children with the highest medical complexity is important, this study demonstrates that tools such as the SLCAT offer the ability to identify and stratify all children, especially CYSHCN, thus allowing more effective application of care coordination and other population health tools to improve the health and function of children more accurately.
Footnotes
Acknowledgments
The authors have no acknowledgements and received no financial support for the research, authorship, and/or publication of this article.
Conflict of interest
The authors have no conflicts of interest to report.