Sage Journals: Discover world-class research

Abstract

Children’s mental health care plays a vital role in many social, health care, and education systems, but there is evidence that appropriate targeting strategies are needed to allocate limited mental health care resources effectively. The aim of this study was to develop and validate a methodology for identifying children who require access to more intense facility-based or community resources. Ontario data based on the interRAI Child and Youth Mental Health instruments were analysed to identify predictors of service complexity in children’s mental health. The Resource Intensity for Children and Youth (RIChY) algorithm was a good predictor of service complexity in the derivation sample. The algorithm was validated with additional data from 61 agencies. The RIChY algorithm provides a psychometrically sound decision-support tool that may be used to inform the choices related to allocation of children’s mental health resources and prioritisation of clients needing community- and facility-based resources.

Keywords

Children’s mental health interRAI resource allocation

Introduction

Mental health issues are one of the leading health problems encountered by children and youth in Canada. In fact, 14% of children between 4 and 18 years of age experience mental health issues that are serious enough to cause distress and impairment in multiple contexts, such as school, home, and in the community.¹ Of these children, over 75% do not receive appropriate treatment. The presence of mental health issues in childhood, as well as their continuance into adolescence and adulthood, results in substantial costs for Canadian society.²

In Canada, children’s mental health services play an increasingly prominent role in our service system, including acute hospitalisation and residential placement.² Although children’s mental health accounts for only a small fraction of the total expenditures in the Canadian health care and social system, children’s mental health issues continue to increase in prevalence. For instance, the World Health Organization (WHO) estimated that by the year 2020, childhood neuropsychiatric disorders will increase by over 50% internationally.³ Although mental health issues in children and youth are becoming increasingly common, a small percentage of children and youth – approximately 10% of those referred for mental health services – present with increased complexity.^4,5 The expense for caring for these children accounts for a disproportionate amount of health care costs, as these children require ‘episodic, chronic, and ongoing care’ from multiple service sectors.^6–9 Unfortunately, there is little Canadian evidence regarding the costs related to various service options for children’s mental health.^10,11 However, recent data from the United States suggest that one-third of children’s mental health costs are accounted for by inpatient mental health services, which is five times greater than the costs per outpatient.^12,13 In light of the high costs associated with specialised mental health care and the limited availability of such resources, it is imperative that mental health service providers allocate their resources efficiently.

Given that children’s mental health problems are costly from both social and economic standpoints,^14,15 a thorough understanding of the factors that influence mental health service usage is needed to allocate resources appropriately. Currently, within the children’s mental health system, service providers are required to make decisions about the allocation of community and institutional resources for children, youth, and their families in the community. These decisions are commonly made on the basis of comprehensive assessments; however, in many jurisdictions, these assessments are not standardised and have not been tested for reliability and validity.^7,16,17 Moreover, most assessment systems are not accompanied by decision-support algorithms that aid in the interpretation of the assessment findings.¹⁸ Consequently, the decisions of service providers are often subjectively based and inconsistent due to idiosyncratic differences among clinicians, reducing the likelihood that resources are allocated effectively.^2,18 The cost of mental health care can also be exacerbated by poor communication among health professionals as well as unnecessary and avoidable admissions to residential and inpatient services.¹⁸

To address this gap in the literature, this study sought to develop an algorithm to identify factors associated with high service use within the children’s mental health system within the province of Ontario, Canada. Since no existing system was found within the Ontario context, an effort was launched to develop a new decision-support algorithm for allocating resources. The Resource Intensity for Children and Youth (RIChY) was created to assist service providers in determining resource allocation that should be attached to a child or youth based on need. The aim of this article is to describe the development and validation efforts undertaken as part of the development of the RIChY algorithm.

Methods

Sample

All data used came from the implementation of assessments in Ontario, Canada. The sample for the derivation of the RIChY algorithm was 685 records obtained from the interRAI Child and Youth Mental Health (ChYMH⁷) and the Child and Youth Mental Health and Developmental Disability (ChYMH-DD¹⁹) assessment instruments completed as part of an initial pilot project and subsequently used as part of standard clinical practice. The participating agencies in the pilot project included selected agencies providing representation across the service spectrum including primary, secondary, and tertiary care services within the Province of Ontario, thereby providing a range of light-care to heavy-care users. This allowed for variability in resources rather than with respect to representativeness. The 685 children and youth had a mean age of 10.8 years (69% were males).

Depending on the situation of the child or youth being assessed, either the ChYMH or the ChYMH-DD was used. Each version has many assessment items in common and these were used for the derivation analysis. Assessed children and youth ranging from 4 to 18 years of age were included, and the sample represented the first assessment completed for unique individuals between October 2012 and July 2014 when the analytic project began. Many of these children were receiving services within the agencies or have received services from the current or previous agency prior to their initial assessment. Eight mental health organisations were included in the original development efforts of the RIChY algorithm. Subsequent secondary data analyses were completed to validate the algorithm across 61 organisations in total using 12 172 case records from August 2014 to September 2017. These facilities represented the majority of the mental health facilities across various catchment areas within the Province of Ontario providing similar services to that of the derivation sample. Access to services was also sufficiently similar across catchment areas within the province. Data included all available assessments completed as part of the Ontario implementation, excluding those used in the derivation.

All children and youth were referred to mental health facilities through their family physician, paediatricians, school personnel, parents, or other allied professionals. Trained child/youth mental health professionals (ie social workers, psychologists, child and youth workers, occupational therapists, and speech and language pathologists) who completed the assessments had a diploma or degree in the mental health field, at least 2 years of clinical experience with children and youth, and had completed a 2.5-day training programme for the administration of the interRAI ChYMH and ChYMH-DD. All possible sources of information were reviewed before coding of the assessment was completed such as interviews with the child/youth and family, medical records, and reports from other service providers.

Assessment information was recorded using a secure online tool that required responses to conform to acceptable values and for all required responses to be entered before the assessment could be signed as complete, thus promoting data quality. Personal identifiers were removed for the analytic dataset. Ethics clearance for this research was provided by Western University (REB#: 106741).

Measures

The ChYMH⁷ and ChYMH-DD¹⁹ are new assessment instruments from interRAI, a not-for-profit collaboration of researchers and clinicians from more than 30 countries. Assessment items are designed to be equivalent or compatible with other interRAI instruments, such as the interRAI Mental Health²⁰ and interRAI Community Mental Health²¹ used with adult populations. A detailed manual accompanies the instrument that outlines the definition, intent, and coding rules for each item. The result is a valid, reliable, and comprehensive assessment that outlines individual strengths and needs with an emphasis on functioning while providing data and valuable information to examine resource intensity.

Both assessments are based on a semi-structured interview format to address a broad range of mental health problems to assess key domains of functioning, physical health, social support, and service usage. These instruments take approximately 1 hour to complete when conducted by trained clinical staff using all sources of information available to identify the child’s or youth’s strengths, needs, preferences, and areas of risk to inform care planning.

The ChYMH and ChYMH-DD contain many items specific to clinical assessment and care planning for children and youth, and contains over 350 items spanning 20 clinical domain areas. In addition to individual items, computed scales (eg anxiety) use multiple items that are available along with care planning triggers designed to flag the need for attention in specific areas. Scale scores are calculated from items with varied response options and use algorithms based on pathways with different risk levels to obtain measures of status or functioning.²² Higher scores reflect increased severity of symptoms or frequency of problems related to functioning.^23,24

Rigorous studies have confirmed the reliability and validity of interRAI instruments.^25–30 Several studies have found scales embedded in the interRAI suite of instruments to have strong psychometric properties for children and youth.^{23,24,31–33} interRAI instruments for adults have been used to develop systems to support resource allocation in long-term care, home care, developmental services, and inpatient mental health settings.^34–39

Analysis

With the goal of the algorithm to predict those who require the most complex service needs, an important consideration was the specification of the dependent variable. Based on a facilitated session where a group of clinical experts was asked to identify services received among these most complex cases, a list of these represented in the ChYMH and ChYMH-DD instruments was identified (see Table 1). In the absence of a gold standard indicator of service complexity, a broad list of services is consistent with the clinical diversity in this population, with more complex individuals receiving more of these services.

Table 1.

Dependent variable items.

1. Any prior lifetime inpatient mental health admission

2. Psychiatrist in the last 7 days

3. Social worker in the last 7 days

4. Psychologist in the last 7 days

5. Child protection in the last 7 days

6. Currently receiving case management

7. Scheduled in the next 30 days or received in the past 30 days three or more of the following interventions:
Life skills Crisis intervention
Social skills Family preservation
Family functioning Family support
Anger management Medication management
Behavioural management

8. Two or more total of acute hospital admissions, emergency department visits, or physician visits, in the last 90 days

With respect to the developed efforts of the algorithm, a variety of components were used to develop the outcome (dependent) variable reflective of intensive service usage. Service complexity was constructed by using three different types of services reflective of more intensive service needs: (1) formal care provided to the patient which included services from a psychiatrist, psychologist, psychometrist, social worker, child protection worker, and case management; (2) the number of admissions to mental health facilities across the lifespan; and (3) interventions provided within the last 30 days including life skills, training, social skills, crisis management, family functioning, anger management, family preservation, behaviour management, family support, and medication management. Eight individual items were summed based on the items, resulting in a count of services, with each additional endorsement resulting in increased service complexity. These services ranged from a 7-day look back to 90-day timeframe, depending on the type of service used. For example, often children and youth are seen on a weekly basis for formal care (eg social work visit), while other services (eg emergency department or physician visits) are more episodic in nature. Although some error may be attached to service availability and timing of the services, these variations are likely randomly distributed and not likely to impact the utility of the algorithm.

The eight service domains used were formulated without any specific weighting in mind, only that higher counts represented higher resource use in a very broad way. Assessed clients represent a range of individual cases who were receiving services prior to assessment, with the cross-sectional assessment timing representing an opportunity to summarise a variety of client presentations and mix of services. Based on the distribution of the total (0-8), a cut-point of 3 or more was established to represent a proportion of cases with the highest counts, and this dichotomous variable became the dependent variable (see Appendix 1 for further information regarding the outcome variable, including their prevalence). Modelling of a dichotomous measure is appropriate here, since modelling the mean of the sum would be improperly affected by its non-normal distribution. The cut-point of 3 was chosen to strike a reasonable balance regarding a high enough count such that one or two services would not dominate the dependent variable, and a proportion (here just over one quarter) that was sufficiently prevalent such that associations between the dependent and potential independent variables would be robust (Table 2).

Table 2.

Distribution of raw dependent variable in derivation sample prior to dichotomising sum of ‘Service’ items/constructs.

Count	N	%	Cumulative %	%
0	204	29.8	29.8	73.0
1	182	26.6	56.4
2	114	16.6	73.0
3	92	13.4	86.4	27.0
4	55	8.0	94.5
5	26	3.8	98.2
6	11	1.6	99.9
7	1	0.1	100.0
Total	685

Independent variables were identified based on the same session with clinical experts who informed the specification of the dependent variable. During the session, experts (eg child psychiatrist, psychologist, or expert allied professionals) were asked to identify items or constructs that were considered related to complex service needs and represented characteristics of the client or the family. Items describing the care provider or under direct control of the care provider were avoided, as were those with a risk of misrepresentation by family members or clinicians, to preferentially bias access to services. The ability to explain variation in the dependent variable as well as confidence in psychometric properties was also considered.

Modelling was done using an interactive decision tree approach. SAS Enterprise Miner supports a classification modelling tool where the operator chooses branches of the tree using information on the power of all candidate independent variables, allowing many alternative constructs to be explored. The strength of decision trees, as opposed to conventional regression models, is that the interactions among predictors can be more naturally discovered and implemented. The tree modelling tool presents for a given node, which represents those observations that follow the logic up to that point, those splitting options that provide the greatest statistical differentiation in the dependent variable. A subsequent branch may not yield satisfactory additional branches based on clinical and statistical judgements, and therefore, the original split is exchanged for another. The initial splits of the tree structure are especially important. After exploring various options for initial splits, three age groups were selected based on both clinical judgement and the finding that items and structures suggested within each of the age groups tended to be distinct. In the course of the modelling, insight was achieved into natural groupings in the data; since the modelling technology does not support the regrouping of different branches (so that they can be considered together in subsequent splits), it became apparent that combining two items (bullying peers and conflict/repeated criticism of close friends) was useful to combine into a single group. Overall, a high number of options were explored, especially regarding final splits where available assessment counts become small. Both clinical judgement and statistical explanatory power were considered in selecting the final tree model. This is a common approach used in case mix research, and the advantage of decision tree analyses provides the opportunity to test complex interaction terms more appropriately than traditional statistical models and allows one to identify rare and expensive sub-groups.³⁴

There were 18 terminal nodes which were subsequently grouped as six levels of increasing proportions of the dependent variable. Logistic regression was subsequently completed using the six groups of the algorithm, applying the Firth penalised likelihood method due to the near-zero result in the lowest group. SAS 9.4 and SAS Enterprise Miner 13.1 were used for the analysis.

Results

Figure 1 provides a schematic representation of the final RIChY algorithm. Measures differentiating level of service complexity varied by age range. The RIChY algorithm categorises children and youth into levels of risk that suggest the need for high intensity services based on criteria as identified from the ChYMH and ChYMH-DD. Children and youth assessed using either instrument were included given that they have comorbid conditions and do not exhibit purely distinct characteristics as they both have mental health issues and related symptomatology. For children aged 7 years and below, the scores range from 0 to 3; for children and youth aged 8+ years, the scores range from 0 to 5 with higher scores indicating a higher priority for intense service needs. The child or youth may fall into a given level via a number of pathways that represent different combinations of the criteria/risk factors. High service use was concentrated in a small minority of children, youth, and their families (approximately 10% scored very high on the RIChY).

Figure 1.

Resource Intensity for Children and Youth Algorithm (RIChY) logic diagram.

For children below 7 years of age, intimidation of others, threatening violence, and destructive behaviour towards property predicted high service usage on the RIChY. For children between the ages of 8 and 11 years, lack of close friends, bullying/victimisation, dysfunctional family functioning, victim of abuse, medication side effects/medical issues, poor parenting, and sleep problems were related to higher scores on the algorithm. For those above 12 years, bullying peers, interpersonal conflict, anxiety, suicide or self-harm, or risk of harming others along with dysfunctional family functioning were related to high ratings on the RIChY.

The proportions in each of the six levels and the proportion of complex service are shown in Table 3, with odds ratio representing the increments in odds with higher RIChY levels. The C-statistic represents the area under the receiver operator characteristic (ROC) curve, and was 0.821 for the derivation dataset. The count of services that defined the dependent variable prior to it being dichotomised was found to have a correlation coefficient of 0.575 with the RIChY score value. After adjusting for RIChY level, further analyses showed that gender and developmental disability were not significantly associated with the dependent variable. Assessment as an inpatient was associated with a significant odds ratio of 3.17 for complex resource use. Further collapsing the RIChY score into dichotomous groups (0, 1, 2 as low vs 3, 4, 5 as high) results indicated a sensitivity of 74.6% and specificity of 73.6% for identifying high intensity cases.

Table 3.

Derivation results of Resource Intensity for Child and Youth (RIChY) algorithm.

RIChY level	N (proportion of sample), %	Mean ‘Service Item’ count	High resource use (%)	Odds ratio, 95% confidence interval
0	198 (28.9)	0.56	1.5	0.12 (0.03-0.44)
1	97 (14.2)	0.90	11.3	1.00 (reference)
2	120 (17.5)	1.60	27.5	2.97 (1.41-6.24)
3	130 (19.0)	2.21	38.5	4.89 (2.38-10.4)
4	73 (10.7)	2.64	58.9	11.21 (5.13-24.49)
5	67 (9.8)	3.13	67.2	15.99 (7.12-35.90)
All	685		27.0

To validate the RIChY algorithm, data from 12 172 new records were used. The validation results are presented in Table 4. The C-statistic for the validation sample was 0.691 with a high level of consistency across multiple years. Although the percentage of high service users within categories is lower over time, there is a consistent, substantial increase in service complexity for every increment in RIChY. Table 5 provides the Ontario results for the algorithm derivation by year comparing RIChY priority levels against the dependent variable.

Table 4.

Validation results of Resource Intensity for Children and Youth (RIChY) algorithm.

RIChY level	N (proportion of sample), %	High resource use (%)	Odds ratio, 95% confidence interval
0	26.4	11.9	0.59 (0.50-0.70)
1	12.3	18.9	1.00 (reference)
2	22.3	28.0	1.69 (1.45-1.97)
3	13.8	36.9	2.54 (2.16-2.99)
4	14.9	43.4	3.33 (2.84-3.90)
5	10.2	54.1	5.11 (4.31-6.06)
All	12 172	28.8

Table 5.

High service usage for validation results of Resource Intensity for Children and Youth (RIChY) algorithm by year.

RIChY level	2014 (N = 913), %	2015 (N = 2001), %	2016 (N = 4249), %	2017 (N = 5009), %
0	18.5	13.0	12.3	10.1
1	25.4	24.2	16.7	16.9
2	50.8	32.3	28.7	22.0
3	49.7	36.2	39.7	31.3
4	62.9	45.8	46.2	38.2
5	62.3	63.5	50.8	50.0
High service usage (%)	42.4	32.0	29.4	24.6

As can be seen in Table 6, older children scored higher on the RIChY algorithm indicating that they had higher service needs than younger children. Specifically, for children below 7 years of age, only 12.9% were classified as high on the RIChY algorithm compared with higher scores for children aged 8-11 years (24.4%) and those above 12 years (35.8%). In addition, as can be seen from Table 7, inpatients scored significantly higher on the RIChY algorithm than outpatients (74.2% vs 22.1%), reflecting more complex, intensive service needs for the former group compared with the latter one. With respect to diagnoses related to higher service use, mood disorder, disruptive behaviours, learning/communication disorders, anxiety, attention deficit hyperactivity, and autism were most related to service complexity (see Table 8).

Table 6.

High service usage for validation results of Resource Intensity for Children and Youth (RIChY) algorithm by age and gender.

RIChY level	Age (years)			Gender
RIChY level	7 and below (N = 1902), %	8-11 (N = 4186), %	12+ (N = 6769), %	Male (N = 7414), %	Female (N = 4740), %
0	9.0	12.4	12.7	12.9	10.1
1	12.1	19.5		19.5	17.0
2		30.7	27.2	30.4	24.3
3	23.8	36.0	50.2	35.9	39.1
4		39.6	44.5	45.0	41.9
5		47.1	55.8	57.1	50.0
High service usage (%)	12.9	24.4	35.8	29.0	28.4

Table 7.

High service usage for validation results of Resource Intensity for Children and Youth (RIChY) algorithm by instrument and patient type.

RIChY level	Instrument		Patient type
RIChY level	ChYMH (N = 11 777), %	DD (N = 1080), %	Inpatient (N = 1625), %	Outpatient (N = 11 232), %
0	10.9	15.9	56.8	8.9
1	17.9	22.8	63.6	14.6
2	25.5	43.9	69.1	22.1
3	35.6	48.6	76.6	29.2
4	43.7	47.6	76.0	38.1
5	54.6	59.6	85.6	41.0
High service usage (%)	28.0	36.9	74.2	22.1

Table 8.

High service usage for validation results of Resource Intensity for Children and Youth (RIChY) algorithm by DSM diagnosis.

RIChY level	ADHD (N = 3302), %	Anxiety (N = 2781), %	Disruptive behaviour (N = 1417), %	Autism spectrum (N = 1163), %	Mood (N = 1148), %	Learning/communication (N = 1605), %
0	15.1	12.3	15.8	17.3	24.8	15.2
1	24.0	19.4	20.3	18.4	43.8	25.4
2	32.3	29.0	40.4	36.3	37.5	34.0
3	34.9	42.1	43.3	39.0	56.1	42.9
4	46.4	47.1	58.6	45.8	48.1	55.1
5	54.7	58.2	68.5	57.1	67.9	59.4
High service usage (%)	31.1	32.1	43.5	30.4	47.2	36.2

With respect to the derivation and validation of the algorithm, the C-statistic dropped (0.81 - 0.69) between the derivation analysis and the validation analysis. While it is typical for the C-statistic to be lower on validation data than the initial derivation of the algorithm, it should also be noted that the original derivation included children and youth with more intensive needs to provide a better basis for differentiation. Therefore, it is possible that temporal instability is related to a change in sites as there appears to have been fewer tertiary cases in the validation sample. Specifically, the derivation sample represented more children and youth requiring more intensive mental health needs, and consequently, the validation sample had lighter care clients and families than the derivation sample. Despite this, what the analysis exhibited is that the clinical variables within the algorithm are variables known to drive services and that would typically be expected to drive cost. Consequently, the resulting risk grouping generated should be effective across Ontario given that our validation sample provides a strong mix of agencies and organisations that provide mental health services for children and youth across the province.

To give some context about the Ontario service delivery system, approximately 45% of children and youth receive services from more than one service sector within Ontario (either concurrently or sequentially), with those with serious emotional disturbances exhibiting disproportionately higher use of different services across multiple sectors.⁴⁰ Consequently, Ontario has been known for its fragmented mental health infrastructure and ‘siloed’ nature of service delivery^41,42 with repeat clients using a significant proportion of child psychiatric and mental health services.⁴³ Consequently, our outcome variable was designed to incorporate services reflective of higher service use across multiple service sectors.

Discussion

High service intensity was predicted by a variety of different contributors depending on the age range of the child or youth. Children who are 7 years and below who intimidate others, threaten violence, and exhibit destructive behaviour towards property received the highest scores on the RIChY algorithm. Consistent with previous research, children referred to mental health services due to acting-out, destructive, and violent behaviours tend to receive more immediate and intensive services.^44–46 Furthermore, lifelong aggressive behaviour has been found to be positively associated with specialised health care use.³⁸

For children between 8 and 11 years of age, many of the contributing factors related to high service usage were external circumstances such as a lack of close friends, family dysfunction, and parenting problems, along with traumatic life events. These factors have been linked to socio-emotional problems and psychological distress⁴⁷ and may reflect a limited social support network, an important psychosocial buffer when faced with various risk factors.^48,49

A lack of social support has also been found to be linked to signs and symptoms of internalising problems, including anxiety.⁵⁰ For example, previous studies have found that children and youth experiencing symptoms indicative of anxiety are more likely to use specialised mental health services (eg inpatient as opposed to outpatient services), especially when additional risk factors are present. Children using specialised health care receive significantly less social support, experience higher levels of family dysfunction and conflict, and greater levels of caregiver distress than those using primary care only.^51,52 Previous research has also found that parental depression is associated with increased use of emergency departments as well as inpatient and outpatient services for children’s mental health issues.⁵³ These difficulties, coupled with child-related risk factors and trauma, further exacerbate the need for increased mental health resources.^54,55

The experience of a traumatic life event and subsequent trauma-related symptoms are associated with greater service usage as are obsessive-compulsive disorder.⁵⁶ These associations are accounted for by the RIChY algorithm because it contains items that are associated with increased service usage such as obsessive thoughts, compulsive behaviours, nightmares, hypervigilance, intrusive thoughts, and flashbacks. In addition, the strong association between traumatic life experiences and increased service usage has been well documented in the literature.^57–60 Consistent with the findings herein, several studies have found that active forms of maltreatment (eg physical abuse and sexual abuse) predict higher use of mental health services, whereas children who experience neglect are much less likely to receive services.^57–60 In addition, multiple exposures to stressors may have a cumulative effect which causes alterations in stress responses that negatively impact physical health leading to medical problems and increased intensity and chronicity of service use.^3,61,62 These medical issues, along with medical side effects, are associated with higher scores on the RIChY algorithm. This is consistent with literature which indicates that children with chronic physical or medical problems have higher rates of service usage, including case management services, school-related mental health service, and emergency department use for emotional or behaviour concerns.^63–65

For children above 12 years of age, the combination of contributors to high resource use included bullying peers, high rates of interpersonal conflict, suicide or self-harm, risk of harming others, poor family functioning, and differential rates of anxiety (depending on the combination of contributing factors).

Previous research indicates that multiple forms of self-injury and suicide are associated with more emergent and intensive services.⁶⁶ The influence of self-injury for this age group is consistent with studies that indicate the age of onset of these behaviours to be around the age of 13 or 14 years.⁶⁷ The studies also suggest that the majority of youth with self-harming behaviours and/or suicidal ideation seek support from those within their social system^68–72 and greater service need is required for those who report fewer positive and supportive connections to family members.^73,74

In contrast to the 8-11 age group, youth with low rates of anxiety, coupled with high rates of bullying and interpersonal conflict, tend to be at highest need on the RIChY algorithm for children above 12 years of age. Higher rates of behavioural inhibition and anxiety have been known to be a protective factor against youth delinquency.⁷⁵ Youth with low levels of anxiety and significant conflict with peers and authority figures (eg parents and school staff) tend to be in conflict with the law and are more likely to be directed towards higher intensity services (eg secure custody and residential treatment), especially when family support or functioning is low.^76–78

As evidenced by both the results of the current study and previous literature, several factors predict service usage as children age, and variable combinations become more complex. More intensive services may also be required as children move into adolescence and behaviours such as aggression and conflict with others become potentially threatening and increasingly difficult for families to manage.⁷⁹ Suicidal ideation and self-harm are also associated with emergent, intensive services due to the life-threatening nature of the behaviours.^62,80–82 Consequently, these behaviours often result in placement outside the home which is more common for children older than 11 years of age.⁸³

Use and utility of the RIChY

Based on the findings, RIChY is an empirically based decision-support tool that may be used to inform intensity of scope and nature of service needs for children and youth needing community- or facility-based services. It is a good predictor of high intensity needs among children and youth which will allow service providers to make more systematic evaluations of the resource needs.

Service providers who have completed the interRAI ChYMH and ChYMH-DD assessment can obtain the RIChY results automatically from the software in which the algorithm is embedded, and these results then provide a context against which service needs can be identified for the child/youth and family. It should be noted, however, that the intent is not to use RIChY as an automated decision-making system, absent of clinical judgement. The RIChY score, along with other information obtained during the assessment process, should be used to assist the clinical team in determining the need and urgency for complex and intense services. It is the responsibility of the clinical team to use professional judgement to decide whether the score accurately reflects the child’s or youth’s need for complex, intense service, given all available information. If a score is in the upper range, it is recommended that the clinical team consider the child/youth to be at high need for intense services. If the score is in the lower range, it is recommended that further discussion occur to determine whether the level of need is appropriate. In all situations, the child/youth, or his or her family, if necessary, should be involved in the decision-making process and consider their needs, strengths, and preferences.¹⁸ For example, some children who score low on the RIChY algorithm may be showing signs of depression or may have had frequent visits to the emergency room or hospital. In such a case, it would be inappropriate to treat the child with psycho-education only. Conversely, a child with a very high RIChY score may not require inpatient services because they have family members who are both willing and able to address their current needs.

In addition to being used for individualised resource allocation decisions, the RIChY algorithm can also provide comprehensive, standardised data across large catchment areas, allowing for the identification of needs across the system, and providing justification for expenditures.¹⁸ Similar to other interRAI algorithms, populations can be stratified according to the RIChY levels and then be used to compare the performance of mental health agencies with respect to outcomes of care within the RIChY levels.

Consequently, practice patterns can be evaluated at the regional, organisational, national, and international levels.²⁹ Such a benchmarking system may be used to identify jurisdictions where RIChY-adjusted residential admissions, for example, are higher than expected based on the experience of other regions.³¹ In addition, RIChY levels at intake can be used to examine variations across regions with respect to how services are used by level of need. The main benefit of implementing the RIChY would be that children, youth, and their families with higher levels of need would be directed towards more extensive resources and services than those with lower-level needs. This does not, however, preclude the possibility of children and youth at the lowest level receiving appropriate services.

The findings of this study need to be considered within the context of its limitations. First, this study is cross-sectional rather than a longitudinal. As such, older children have had a longer opportunity to be involved in more complex services, and hence, it is not surprising that these children would score higher with respect to service complexity than younger children. In the future, longitudinal data are needed to examine service complexity as a child grows and develops. Second, staff time measurement and actual cost data were not available. Consequently, we were unable to identify and differentiate cases that were least and most costly to directly determine how cost relates to the RIChY algorithm. Future research will link actual cost data to scores obtained on the RIChY algorithm as part of further validation efforts.

Conclusion

The RIChY algorithm provides an empirically based decision-support tool that may be used to inform choices related to the allocation of resources and prioritisation of children and youth needing community- or facility-based services. RIChY may be used at the individual level to support clinical decision-making and can be used with aggregated data to inform policy development and planning.

Footnotes

Appendix

Appendix 1

% with any	RIChY							Logistic regression
Service items	0	1	2	3	4	5	All	Odds ratio	C-statistic
Lifetime admissions	4.1%	6.9%	26.2%	29.0%	59.7%	51.3%	24.7%	1.68 (1.53-1.85)	0.781
Psychiatrist	2.6%	2.3%	4.0%	9.7%	19.5%	15.4%	7.4%	1.46 (1.27-1.68)	0.720
Social worker	1.6%	6.9%	11.1%	33.1%	37.7%	46.2%	18.8%	1.69 (1.52-1.87)	0.796
Psychologist/psychometrist	1.0%	1.1%	3.2%	5.6%	5.2%	5.1%	3.2%	1.28 (1.05-1.57)	0.665
Child protection	1.0%	3.4%	4.0%	8.9%	7.8%	15.4%	5.7%	1.44 (1.23-1.68)	0.721
Case management	19.7%	35.6%	63.5%	69.4%	61.0%	83.3%	50.7%	1.57 (1.43-1.71)	0.743
Any of: life skills, social skills, family functioning, behavioural management, crisis intervention, family preservation, family support, medication management	13.5%	11.5%	28.6%	41.9%	33.8%	57.7%	28.5%	1.39 (1.28-1.52)	0.695
Two or more visits: acute hospital, ED, physician visits	13.0%	21.8%	19.0%	23.4%	39.0%	38.5%	22.9%	1.27 (1.17-1.39)	0.637

Acknowledgements

The authors would like to thank the children, youth, and families, as well as the service providers within the agencies for their participation in the research. This manuscript is for a special issue focused on interRAI instruments and applications. This is a unique contribution to the journal as it relates to resource allocation in children and youth with mental health issues.

Declaration of conflicting interests:

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding:

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We would like to thank the London Community Foundation and the Public Health of Canada. We would also like to thank CPRI for their contribution and support with training across the Province of Ontario.

Author Contributions

SLS, JWP, and JPH conceived and designed the research project; JWP analysed the data; SLS and JWP wrote the first draft of the manuscript; SLS, JWP, JPH, and ET contributed to the writing of the manuscript, agreed with manuscript results and conclusions, jointly developed the structure and arguments for the paper, and made critical revisions and approved final version. All authors reviewed and approved the final manuscript.

Disclosures and Ethics

As a requirement of publication, author(s) have provided to the publisher signed confirmation of compliance with legal and ethical obligations including but not limited to the following: authorship and contributorship, conflicts of interest, privacy and confidentiality, and (where applicable) protection of human and animal research subjects. The authors have read and confirmed their agreement with the ICMJE authorship and conflict of interest criteria. The authors have also confirmed that this article is unique and not under consideration or published in any other publication, and that they have permission from rights holders to reproduce any copyrighted material. Any disclosures are made in this section. The external blind peer reviewers report no conflicts of interest.

References

Waddell

Offord

Shepherd

Hua

McEwan

Child psychiatric epidemiology and Canadian public policy-making: the state of the science and the art of the possible. Can J Psychiatry. 2002;47:825–832.

Waddell

McEwan

Shepherd

Offord

Hua

JM.

A public health strategy to improve the mental health of Canadian children. Can J Psychiatry. 2005;50:226–233.

World Health Organization. World Health Statistics. Geneva, Switzerland: World Health Organization; 2010.

Epstein

Kutash

Duchnowski

. Outcomes for Children and Youth With Emotional and Behavioral Disorders and Their Families: Programs and Evaluation Best Practices. 2nd ed. Austin, TX: Pro-Ed; 2004:656.

Reid

Cunningham

Tobon

et al . Help-seeking for children with mental health problems: parents’ efforts and experiences. Adm Policy Ment Health. 2011;38:384–397.

Tobon

Reid

Brown

JB.

Continuity of care in children’s mental health: parent, youth and provider perspectives. Community Ment Health J. 2015;51:921–930.

Stewart

Hirdes

Curtin-Telegdi

et al . interRAI Child and Youth Mental Health (ChYMH) Assessment Form and User’s Manual: For Use With In-Patient and Community-Based Assessments. Washington, DC: interRAI; 2015.

Reid

Brown

JB.

Money, case complexity, and wait lists: perspectives on problems and solutions at children’s mental health centers in Ontario. J Behav Health Serv Res. 2008;35:334–346.

Reid

Stewart

Zaric

et al . Defining episodes of care in children’s mental health using administrative data. Adm Policy Ment Health. 2015;42:737–747.

10.

Gandhi

Chiu

Lam

Cairney

Guttman

Kurdyak

Mental health service use among children and youth in Ontario: population trends over time. Can J Psychiatry. 2016;61:119–124.

11.

Beecham

Annual research review: child and adolescent mental health interventions: a review of progress in economic studies across different disorders. J Child Psychol Psychiatry. 2014;55:714–732.

12.

Ringel

Sturm

National estimates of mental health utilization and expenditures for children in 1998. J Behav Health Serv Res. 2001;28:319–333.

13.

Martin

Leslie

Psychiatric inpatient, outpatient, and medication utilization and costs among privately insured youths, 1997-2000. Am J Psychiatry. 2003;160:757–764.

14.

U.S. Department of Health and Human Services. Report of the Surgeon General’s Conference on Children’s Mental Health: A National Action Agenda. Washington (DC): US Department of Health and Human Services.

15.

Kieling

Baker-Henningham

Belfer

et al . Child and adolescent mental health worldwide: evidence for action. Lancet. 2011;378:1515–1525.

16.

Hirdes

Smith

Rabinowitz

et al . The Resident Assessment Instrument-Mental Health (RAI-MH): inter-rater reliability and convergent validity. J Behav Health Serv Res. 2002;29:419–432.

17.

Stewart

Currie

Arbeau

Leschied

Kerry

. Assessment and Planning for Community and Custodial Services: The Application of interRAI Assessment in the Youth Justice System. In: Corrado RR, Lussier P, Leschied AD, eds. Serious and Violent Young Offenders and Youth Criminal Justice: A Canadian Perspective. Simon Fraser University Publications. 2015:355–397.

18.

Hirdes

Poss

Curtin-Telegdi

The Method for Assigning Priority Levels (MAPLe): a new decision-support system for allocating home care resources. BMC Med. 2008;6:9.

19.

Stewart

LaRose

Gleason

et al . interRAI Child and Youth Mental Health – Developmental Disabilities (ChYMH-DD) Assessment Form and User’s Manual. Version 1. Washington, DC: interRAI; 2015.

20.

Hirdes

Marhaba

Smith

et al . Development of the resident assessment instrument – mental health (RAI-MH). Hosp Q. 2000;4:44–51.

21.

Hirdes

Curtin-Telegdi

Rabinowitz

et al . interRAI Community Mental Health (CMH) Assessment Form and User’s Manual. Version 9.2. Washington, DC: interRAI; 2010.

22.

Martin

Hirdes

Fries

Smith

TF.

Development and psychometric properties of an assessment for persons with intellectual disability – the interRAI ID. J Policy Pract Intel Disabil. 2007;4:23–29.

23.

Stewart

Hamza

CA.

The Child and Youth Mental Health Assessment (ChYMH): an examination of the psychometric properties of an integrated assessment developed for clinically referred children and youth. BMC Health Serv Res. 2017;17:82.

24.

Lau

Stewart

Saklofske

Tremblay

Hirdes

Psychometric evaluation of the interRAI Child and Youth Mental Health Disruptive/Aggression Behaviour Scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiatry Hum Dev. 2018;49:279–289.

25.

Carpenter

GI.

Accuracy, validity and reliability in assessment and in evaluation of services for older people: the role of the interRAI MDS assessment system. Age Ageing. 2006;35:327–329.

26.

Gray

Berg

Fries

et al . Sharing clinical information across care settings: the birth of an integrated assessment system. BMC Health Serv Res. 2009;9:71.

27.

Hirdes

Smith

Rabinowitz

et al . The Resident Assessment Instrument-Mental Health (RAI-MH): inter-rater reliability and convergent validity. J Behav Health Serv Res. 2002;29:419–432.

28.

Hirdes

Ljunggren

Morris

et al . Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system. BMC Health Serv Res. 2008;8:277.

29.

Morris

Jones

Fries

Hirdes

JP.

Convergent validity of minimum data set-based performance quality indicators in postacute care settings. Am J Med Qual. 2004;19:242–247.

30.

Poss

Jutan

Hirdes

et al . A review of evidence on the reliability and validity of minimum data set data. Healthc Manage Forum. 2008;21:133–139.

31.

Phillips

Hawes

The interRAI pediatric home care (PEDS HC) assessment: evaluating the long-term community-based service and support needs of children facing special healthcare challenges. Health Serv Insights. 2015;8:17–24.

32.

Phillips

Patnaik

Moudouni

et al . Summarizing activity limitations in children with chronic illnesses living in the community: a measurement study of scales using supplemented interRAI items. BMC Health Serv Res. 2012;12:19.

33.

Stewart

Tohver

Klassen

The validation of the interRAI ChYMH internalizing subscale in Canadian children and youth. Personal Ind Differ. 2016;101:518.

34.

Fries

Schneider

Foley

Gavazzi

Burke

Cornelius

Refining a case-mix measure for nursing homes: Resource Utilization Groups (RUG-III). Med Care. 1994;32:668–685.

35.

Björkgren

Fries

Shugarman

LR.

A RUG-III case-mix system for home care. Can J Aging/La Rev Can Vieil. 2000;19:106–125.

36.

Poss

Hirdes

Fries

McKillop

Chase

Validation of Resource Utilization Groups version III for Home Care (RUG-III/HC): evidence from a Canadian home care jurisdiction. Med Care. 2008;46:380–387.

37.

Martin

Fries

Hirdes

James

Using the RUG-III classification system for understanding the resource intensity of persons with intellectual disability residing in nursing homes. J Intellect Disabil. 2011;15:131–141.

38.

Minoletti

Alegría

Barrioneuvo

et al . Translating Psychiatric Diagnosis and Classification into Public Health Usage. In: Saxena S, ed. Public Health Aspects of Diagnosis and Classification of Mental and Behavioral Disorders: Refining the Research Agenda for DSM-5 and ICD-11. American Psychiatric Pub. 2012:201.

39.

Canadian Institute for Health Information (CIHI). Ontario Mental Health Reporting System: Case Mix System for Classification of In-patient Psychiatry and SCIPP Weighted Patient Days. Ottawa, ON, Canada: CIHI; 2012. http://www.cihi.ca/CIHI-ext-portal/pdf/internet/OMHRS_SCIPP_SWPD_FAQ2012_EN.

40.

Farmer

Burns

Phillips

Angold

Costello

EJ.

Pathways into and through mental health services for children and adolescents. Psychiatr Serv. 2003;54:60-66.

41.

Canadian Institutes for Health Research. Access and wait time in child and youth mental health: a background paper, 2010. http://www.excellenceforchildandyouth.ca/sites/default/files/policy_access_and_wait_times.pdf.

42.

Kirby

Keon

WJ.

Out of the Shadows at Last: Transforming Mental Health, Mental Illness, and Addiction Services in Canada. Ottawa, ON, Canada: Standing Senate Committee on Social Affairs, Science and Technology; 2006.

43.

Christodulu

Lichenstein

Weist

Schager

Simore

Psychiatric emergencies in children. Pediatr Emerg Care. 2002;118:268–270.

44.

Hoven

Bird

et al . Depressive and disruptive disorders and mental health service utilization in children and adolescents. J Am Acad Child Adolesc Psychiatry. 1999;38:1081–1090.

45.

Totsika

Toogood

Hastings

Lewis

Persistence of challenging behaviours in adults with intellectual disability over a period of 11 years. J Intellect Disabil Res. 2008;52:446–457.

46.

Minshawi

Hurwitz

Fodstad

Biebl

Morriss

McDougle

CJ.

The association between self-injurious behaviors and autism spectrum disorders. Psychol Res Behav Manag. 2014;7:125–136.

47.

Kawachi

Berkman

LF.

Social ties and mental health. J Urban Health. 2001;78:458–467.

48.

Birmaher

Ryan

Williamson

et al . Childhood and adolescent depression: a review of the past 10 years. Part I. J Am Acad Child Adolesc Psychiatry. 1996;35:1427–1439.

49.

Greening

Stoppelbein

Religiosity, attributional style, and social support as psychosocial buffers for African American and White adolescents’ perceived risk for suicide. Suicide Life Threat Behav. 2002;32:404–417.

50.

de Graaf

Bijl

Ten Have

Beekman

Vollebergh

WA.

Pathways to comorbidity: the transition of pure mood, anxiety and substance use disorders into comorbid conditions in a longitudinal population-based study. J Affect Disord. 2004;82:461–467.

51.

Albert

Becker

Mccrone

Thornicroft

Social networks and mental health service utilisation – a literature review. Int J Soc Psychiatry. 1998;44:248–266.

52.

Guzzetta

Miglio

Santone

et al . First-ever admitted psychiatric inpatients in Italy: clinical characteristics and reasons contributing to admission: findings from a national survey. Psychiatry Res. 2010;176:62–68.

53.

Dreyer

Williamson

Hargreaves

Rosen

Deeny

SR.

Associations between parental mental health and other family factors and healthcare utilisation among children and young people: a retrospective, cross-sectional study of linked healthcare data. BMJ Pediatr Open. 2018;2:e000266.

54.

Fryers

Brugha

Childhood determinants of adult psychiatric disorder. Clin Pract Epidemiol Ment Health. 2013;9:1–50.

55.

Shonkoff

Garner

Siegel

et al . The lifelong effects of early childhood adversity and toxic stress. Pediatrics. 2012;129:e232–e246.

56.

de Graaf

Bijl

Smit

Vollebergh

Spijker

Risk factors for 12-month comorbidity of mood, anxiety, and substance use disorders: findings from the Netherlands Mental Health Survey and Incidence Study. Am J Psychiatry. 2002;159:620–629.

57.

Bellamy

Gopalan

Traube

DE.

A national study of the impact of outpatient mental health services for children in long-term foster care. Clin Child Psychol Psychiatry. 2010;15:467–479.

58.

Garland

Landsverk

Hough

Ellis-MacLeod

Type of maltreatment as a predictor of mental health service use for children in foster care. Child Abuse Negl. 1996;20:675–688.

59.

Leslie

Landsverk

Ezzet-Lofstrom

Tschann

Slymen

Garland

AF.

Children in foster care: factors influencing outpatient mental health service use. Child Abuse Negl. 2000;24:465–476.

60.

Tarren-Sweeney

Concordance of mental health impairment and service utilization among children in care. Clin Child Psychol Psychiatry. 2010;15:481–495.

61.

Way

Taylor

SE.

The serotonin transporter promoter polymorphism is associated with cortisol response to psychosocial stress. Biol Psychiatry. 2010;67:487–492.

62.

Shonkoff

Boyce

McEwen

BS.

Neuroscience, molecular biology, and the childhood roots of health disparities: building a new framework for health promotion and disease prevention. JAMA. 2009;301:2252–2259.

63.

Wilcox

Rains

Belcher

et al . Behavioral Problems and Service Utilization in Children with Chronic Illnesses Referred for Trauma-Related Mental Health Services. J Dev Behav Pediatr. 2015;37:62–70.

64.

Sareen

Cox

Stein

Afifi

Fleet

Asmundson

GJ.

Physical and mental comorbidity, disability, and suicidal behavior associated with posttraumatic stress disorder in a large community sample. Psychosom Med. 2007;69:242–248.

65.

Urbanoski

Rush

Wild

Bassani

Castel

Use of mental health care services by Canadians with co-occurring substance dependence and mental disorders. Psychiatr Serv. 2007;58:962–969.

66.

Merikangas

Mehta

Molnar

et al . Comorbidity of substance use disorders with mood and anxiety disorders: results of the international consortium in psychiatric epidemiology. Addict Behav. 1998;23:893–907.

67.

Nock

Joiner

Gordon

Lloyd-Richardson

Prinstein

MJ.

Non-suicidal self-injury among adolescents: diagnostic correlates and relation to suicide attempts. Psychiatry Res. 2006;144:65–72.

68.

Freedenthal

Stiffman

AR.

‘They might think I was crazy’: young American Indians’ reasons for not seeking help when suicidal. J Adolesc Res. 2007;22: 58–77.

69.

Nada-Raja

Morrison

Skegg

A population-based study of help-seeking for self-harm in young adults. Aust N Z J Psychiatry. 2003;37:600–605.

70.

O’Donnell

Stueve

Wardlaw

O’Donnell

Adolescent suicidality and adult support: the reach for health study of urban youth. Am J Health Behav. 2003;27:633–644.

71.

Whitlock

Eckenrode

Silverman

Self-injurious behaviors in a college population. Pediatrics. 2006;117:1939–1948.

72.

Ystgaard

Arensman

Hawton

et al . Deliberate self-harm in adolescents: comparison between those who receive help following self-harm and those who do not. J Adolesc. 2009;32:875–891.

73.

Muehlenkamp

Gutierrez

PM.

An investigation of differences between self-injurious behavior and suicide attempts in a sample of adolescents. Suicide Life Threat Behav. 2004;34:12–23.

74.

Muehlenkamp

Gutierrez

PM.

Risk for suicide attempts among adolescents who engage in non-suicidal self-injury. Arch Suicide Res. 2007;11:69–82.

75.

Kerr

Tremblay

Pagani

. Boys’ behavioral inhibition and the risk of later delinquency. In: Ballenger

Frances

Jensen

et al . (eds) Year Book of Psychiatry and Applied Mental Health. Maryland Heights, MO: Mosby Year Book; 1999:1–2.

76.

Rosenblatt

Biggs

EE.

Criminal behavior and emotional disorder: comparing youth served by the mental health and juvenile justice systems. J Behav Health Serv Res. 2000;27:227–237.

77.

Garland

Hough

McCabe

Yeh

Wood

Aarons

GA.

Prevalence of psychiatric disorders in youths across five sectors of care. J Am Acad Child Adolesc Psychiatry. 2001;40:409–418.

78.

Lyons

Royce Baerger

Quigley

Erlich

Griffin

Mental health service needs of juvenile offenders: a comparison of detention, incarceration, and treatment settings. Child Serv. 2001;4:69–85.

79.

Stewart

Falah Hassani

Poss

Hirdes

The determinants of service complexity in children with intellectual disabilities. J Intellect Disabil Res. 2017;61:1055–1068.

80.

Hawton

Saunders

O’Connor

RC.

Self-harm and suicide in adolescents. Lancet. 2012;379:2373–2382.

81.

Olfson

Gameroff

Marcus

Greenberg

Shaffer

Emergency treatment of young people following deliberate self-harm. Arch Gen Psychiatry. 2005;62:1122–1128.

82.

Mann

Currier

DM.

Stress, genetics and epigenetic effects on the neurobiology of suicidal behavior and depression. Eur Psychiatry. 2010;25:268–271.

83.

Barth

Lloyd

Green

James

Leslie

Landsverk

Predictors of placement moves among children with and without emotional and behavioral disorders. J Emot Behav Disord. 2007;15:46–55.

Resource Intensity for Children and Youth: The Development of an Algorithm to Identify High Service Users in Children’s Mental Health

Abstract

Keywords

Introduction

Methods

Sample

Measures

Analysis

Results

Discussion

Use and utility of the RIChY

Conclusion

Footnotes

Appendix

Acknowledgements

Declaration of conflicting interests:

Funding:

Author Contributions

Disclosures and Ethics

References