Sage Journals: Discover world-class research

Abstract

French

Objective

To examine the usage of a standardized screening tool bundle in guiding care recommendations within a regional coordinated access service (1Call1Click.ca) for children, youth and their families seeking mental health, addictions, and substance use health (MHASUH) care. To explore how the screening tools align with each other and with Level of Need, an assigned designation used to indicate acuity, needs, and guide service recommendations.

Method

Between May 31, 2021, and December 31, 2023, 8,280 clients aged 6–21 were administered the CASH screening tool bundle (CASH is an acronym for the included screening tools, the CRAFFT, Ask Suicide Questions {ASQ}, HEADS-ED (Over 6 and Under 6) and assigned a Level of Need (used to describe acuity and guide referrals). We describe the most common needs, correlations between Level of Need and each screening tool, and agreement between the HEADS-ED Domains of Suicidality and Drugs and alcohol with the ASQ and CRAFFT screening tools.

Results

We observe a significant moderate correlation (r(8,280) = .507, p < .001) between the total HEADS-ED score and the assigned Level of Need and significant differences in HEADS-ED total score between each Level of Need (F(4, 8,275) = 766.810, p < .001). Additionally, ASQ and CRAFFT results align with the Suicidality and Drugs and alcohol domains of the HEADS-ED. Referrals were accepted by services at a rate of 94.4%.

Conclusion

The CASH screening tool bundle is effective for screening child and youth MHASUH concerns. The HEADS-ED can give a non-specialist healthcare provider an overview of the nature and acuity of child and youth MHASUH and guide the application of more specialized tools included in the bundle. Used as described in this manuscript, the standardized screening bundle is a useful means for assessing needs and guiding referrals to specialized care, including in-depth specialized assessments.

Plain Language Summary Title

The CASH Bundle: a set of 3 screening tools used together at 1Call1Click.ca, a program designed to connect children and youth to the mental health, addictions, and substance use health care they need

Plain Language Summary

Screening tools are questionnaires that can help identify individuals at heightened risk for mental health, addictions, and substance use (MHASUH) challenges so they can be prioritized for more in-depth assessment and treatment. We examined the use of a set of screening tools (The CASH Bundle) in combination with each other. The CASH Bundle includes screening tools for substance use (the CRAFFT), suicide risk (the Ask Suicide Questions (ASQ)), and a general assessment screening tool for children and youth (the HEADS-ED (Over 6 and Under 6)).

The CASH bundle is used at 1Call1Click.ca, a program for infants, children, and youth struggling with MHASUH concerns to find the service that matches their needs. When someone contacts 1Call1Click.ca, they speak to a healthcare provider who uses the CASH Bundle to determine their Level of Need (a measure of severity that is linked to treatment types). The provider uses this information to connect the client directly to a local MHASUH treatment resource. This study included 8280 1Call1Click.ca clients aged 6 to 21 who were screened with the CASH bundle and assigned a Level of Need.

Our results showed that each of the tools in the CASH Bundle supported the healthcare providers' assessment of the clients' Level of Need.

The CASH Bundle can help healthcare providers quickly gain an overview of the client's MHASUH needs and determine which type and level of care those clients will benefit from the most. This is important because care resources are both limited and in high demand. Assigning care resources quickly, efficiently, and appropriately is critical to making the best use of available resources and making sure that people receive the care they need as quickly as possible.

Keywords

adolescence mental health services suicide substance use access to care addictions screening

Problem

Identifying and addressing the mental health, addictions, and substance use health (MHASUH) needs of children and youth is critical. In Canada, 38% of people with MHASUH challenges say that the symptoms started before the age of 15.^1,2 Millions of youth with mental health needs across North America are without care from a mental health professional.^3,4 More than half of youth who try to access MHASUH services report that it is not easy to do so,⁵ and many (particularly those also using substances) find themselves in the Emergency Department before connecting with any form of outpatient care.⁶

Research suggests that these challenges are both serious and growing, and recent work has emphasized the importance of early intervention in youth MHASUH.^7,8 Several studies have found an increasing incidence of anxiety and other internalizing problems.^3,9,10 Suicide is a leading cause of death in youth worldwide, second only to accidental death in Canadian youth.^11,12 Meanwhile, Emergency Department visits for cannabis use increased from 3.8 in 2003 to 17.9 of 10,000 in 2017 in Ontario.¹³ Nearly half of North American youth (44%) have indicated drinking within the past year, 57% of whom engage in high-risk drinking behaviour,¹⁴ while between 10% and 24% of adolescents engage in daily vaping.^15,16 The substantial overlap between suicidality, substance use, and youth mental health challenges^14,17–19 demands immediate action towards prevention and intervention. While the need to facilitate access to youth MHASUH treatment and services is clear, it is also critical to use currently available resources and services effectively.

In this manuscript, we present the CASH Bundle, which includes the CRAFFT, ASQ, and HEADS-ED (original and Under 6), as an example of how standardized screening tools can be used to identify risk and match children and youth to MHASUH services that meet their needs and acuity. Each of these screening tools is widely used, free, valid, and reliable.^20–28 The HEADS-ED and HEADS-ED Under 6 are age-dependant communimetric screening tools that can guide healthcare providers through a psychosocial interview to quickly assess MHASUH needs. The CRAFFT and ASQ are specialized screening tools for assessing drug and alcohol use or suicidality, respectively.

1Call1Click.ca, a regional coordinated access and service navigation system designed to connect infants, children, and youth (0–21 years) to MHASUH care, uses the CASH bundle in their intake. The program uses a stepped care model, through which intake workers use information gathered with the CASH bundle to assign clients a Level of Need.²⁹ Embedded within each Level of Need is a guide used in directly referring clients to the local community agency or hospital-based services most appropriate to their needs.

Our objectives are to: (1) examine the effectiveness of the CASH screening tool bundle within the context of a coordinated access and service navigation program to inform Level of Need determination and guide referrals, (2) explore screening tool scores and agreement between the tools, and (3) examine differences in screening tool findings across intake workers assigned Level of Need.

Methods

Service Access and Intake

1Call1Click.ca serves a five-county region in Southeastern Ontario, home to over 1.5 million people. A child, youth or caregiver in Eastern Ontario can make a self-scheduled appointment through the 1Call1Click.ca website, by phone, or through referral from a primary care provider. The client speaks to an intake worker who inquires about the client's needs and goals, presenting problems, and demographics before administering the CASH bundle (Figure 1). All tools in the CASH Bundle are administered verbally by the intake worker. The HEADS-ED is used for all clients 6 years old and over.^21,27,28 The CRAFFT Part A is used for clients 12 and over, and the CRAFFT Part B is administered for clients endorsing substance use in Part A.³⁰ The ASQ is used for any clients 6 or older who endorse any suicidality as captured on the HEADS-ED (safety procedures are engaged for clients determined to be at high risk for suicide).^22,31 The HEADS-ED Under 6 is used to screen clients under 6 years of age, and outcomes from that tool in relation to Level of Need at 1Call1Click.ca are reported elsewhere.²³ The use of each of these screening tools is additionally dependent on the intake worker's judgement of each tool's appropriateness and utility on a case-by-case basis.

Figure 1.

A depiction of the process through which clients can connect with 1Call1Click.ca, go through an intake and be connected to an MHASUH service. The arrows lead the reader through how clients engage with 1Call1Click.ca, how the intake worker screens the client using the CASH Bundle in a way that is flexible to the client's characteristics and needs, and how that information is combined by the intake worker to refer the client to the services best suited to their needs.

This information is stored in EPIC, an electronic health record system, where it can be accessed by the referral destination and be retrieved for research and quality improvement purposes.

Screening Tools, Level of Need, and Service Recommendations

ASQ. The Ask Suicide Questions (ASQ) includes 4 yes-no questions about the client's history of suicidal ideation and suicidal gestures.^22,31 A client who responds “no” to each question is a negative screen (low risk). If the client answers “yes” to any questions, a fifth question on imminent suicidal ideation is asked. A response of “yes” on the fifth question indicates an acute positive screen (high risk), where a “no” indicates a non-acute positive screen (moderate risk).

CRAFFT 2.1 + N. The CRAFFT 2.1 + N is a 3-part tool with strong concurrent validity with substance use diagnostic criteria.²⁰ The CRAFFT Part A consists of 5 questions regarding the youth's frequency of alcohol, marijuana, drug, and nicotine use within the past year and whether the client has driven or ridden in a vehicle operated by someone using substances (the “Car” question). Clients who have not used substances in the past year and answer “No” to the “Car” question are scored as Low Risk, otherwise they are asked the 5 questions in part B. The questions in the CRAFFT 2.1 + N Part B are related to the substance use-related behaviours, motivations, and consequences. Clients assessed as Medium Risk either have no substance use history and answered “yes” to the “Car” question or have used in the past year and answered “yes” to one or fewer of the CRAFFT Part B questions. Clients with use in the past year who answer “yes” to two or more questions are designated High Risk. The third part of the CRAFFT 2.1 + N (the Hooked on Nicotine Checklist)³² does not contribute to CRAFFT risk assessment and was not included in our analyses.

HEADS-ED. HEADS-ED is a communimetric MHASUH screening tool for children and youth aged 6 or older that has good concurrent validity with the Child Development Inventory (https://www.heads-ed.com/).^21,27,28,33 The HEADS-ED guides a healthcare professional through an interview, allowing them to quickly assess the needs of the child or youth across the domains of child and youth MHASUH for which HEADS-ED is a mnemonic (Home, Education and employment, Activities and peers, Drugs and alcohol, Suicidality, Emotions, behaviours and thought disturbances, and Discharge or current resources). The healthcare provider or intake worker asks questions related to each domain and assigns a score of 0 (No Action Needed), 1 (Needs Action but not Immediate/Moderate Functional Impairment) or 2 (Needs Immediate action/Severe impairment) to each. At the end, each domain score is added together, with higher total scores indicating greater acuity; a total score of greater than or equal to 8 or a Suicidality score of 2 indicates a need for a specialized intake.

The CASH Bundle embeds the CRAFFT and ASQ into the HEADS-ED interview. The HEADS-ED domains Drugs and alcohol and Suicidality overlap with the subject matter of the CRAFFT and ASQ and center on subjects that may be difficult or painful for clients to discuss. For example, the ASQ, when used, is administered during the HEADS-ED prompted discussion of suicidality.^34,35 Embedding these tools into the HEADS-ED interview maintains conversation flow and reduces the need to return to these topics later.

Level of Need. The intake worker uses the gathered information obtained during intake to assign the client a Level of Need. This designation is based on the Ontario Government's Roadmap to Wellness²⁹ and includes five different levels. These levels range from General Population (the lowest level, including health promotion and prevention supports) to Severe or Complex Needs (requiring the most intense and highly specialized services). Level of Need is used both as an indicator of acuity and in service selection, as each level is linked to types of services available to be recommended to clients assessed at that level.

Inclusion Criteria

Between May 31, 2021 (service launch) and December 31, 2023, 9,694 intakes were completed at 1Call1Click.ca. For the following analyses, we included only clients who met the following criteria: (1) were between the ages of 6 and 21 at the time of intake, (2) were assessed with the HEADS-ED, and (3) were assigned a Level of Need by the intake worker. 85.4% (8,280/9,694) of intakes met these criteria and were included. Children under 6 comprised 7.5% (724/9,694) of the total clientele, while 7.1% (690/9,694) of clients were 6 or older but not assigned a Level of Need.

Data Analysis

Data were analyzed using JASP 0.19.³⁶ Descriptive statistics were used for frequencies and to examine the characteristics of the data. Univariate Analysis of Variance (ANOVA) with a Bonferroni post-hoc correction was used to compare HEADS-ED total score across Level of Need. Spearman Rank-Order Correlations were used to examine the relationships between Level of Need, number of presenting problems, and HEADS-ED total scores. Spearman Rank-Order correlation was used to examine relationships between the Suicidality domain and ASQ results, and the Drugs and alcohol domain and the CRAFFT. Effect sizes of ANOVA (η²) and Spearman's rho correlation (r) coefficients are reported.^37,38

Results

Client Demographics

Overall, 8,280 clients (Mean=12 years, 6 months; Mode=15 years; Table 1) met the inclusion criteria. Over half of the clients identified as female (Table 1). Most clients were designated Moderate Need, followed by the Low Need and Moderate to Severe Needs designations (Table 1). Clients presented with an average of 2.6 (Median=2, IQR = 1.00) presenting problems, most commonly anxiety, depression, and suicidal ideation (Table 1).

Table 1.

Frequencies of Demographics and Presenting Problems among 1Call1Click.ca Clientele.

		Number of Clients (n = 8,280)	Percentage of Clients
Age (inclusive)	15 to 17 years	2,820	34.1%
	12 to 14 years	2,590	31.3%
	9 to 11 years	1,574	19.0%
	6 to 8 years	1,096	13.2%
	18 to 21 years	200	2.4%
Gender Identity	Female	4,631	55.9%
	Male	3,152	38.1%
	Prefer not to answer/ No response	219	2.6%
	Non-Binary	124	1.5%
	Trans Male	85	1.0%
	Other	47	0.7%
	Trans Female	14	0.2%
	Agender	<10
	Two Spirit	<10
	Agender	<10
Racial Background	White	5,967	72.1%
	Multiracial	625	7.5%
	Black	318	3.8%
	Prefer not to answer/ No response	399	4.8%
	Arab	254	3.1%
	Indigenous	160	1.9%
	South Asian	134	1.6%
	Other	92	1.1%
	Southeast Asian	86	1.0%
	Latin American	84	1.0%
	Chinese	81	1.0%
	West Asian	44	0.5%
	Korean	16	0.2%
	Filipino	15	0.2%
	Japanese	<10
Presenting Problem	Anxiety	5,418	65.4%
	Depression	3,000	36.2%
	Suicidal Ideation	1,791	21.6%
	Aggression	1,780	21.5%
	Inattention and Impulsivity	1,727	20.9%
	Parent-Child Relationships	1,611	19.5%
	Stress and Trauma	1,472	17.8%
	Other	1,019	12.3%
	Sleep Problems	809	9.8%
	Eating Problems	795	9.6%
	Non-Suicidal Self-Injury	689	8.3%
	Learning Disabilities	616	7.4%
	Thought disruption	586	7.1%
	Obsessions and Compulsions	376	4.5%
	Substance Use	293	3.5%
	Somatization	128	1.5%

The demographics and presenting problems of clients with a completed intake at1Call1Click.ca, ordered from the most to least frequently endorsed responses for each grouping. “Multiracial” is a calculated variable including all individuals who reported 2 or more racial backgrounds. Each percentage is from the total number of clients included, and specific numbers of clients endorsed by fewer than 10 clients are obscured to protect client privacy. Presenting problems do not indicate clinical diagnoses, but rather the challenges the client is currently struggling with. As clients can indicate multiple presenting problems (M = 2.6, Mode = 2), the numbers presented here add to a sum greater than the number of clients. The category “Other” in presenting problems includes a range of challenges, including struggles with specific life challenges or transitions, such as grief, medical challenges or pain, and other challenges not captured by the list including but not limited to school attendance, peer relationships or bullying, or body dysmorphia.

Tool Completion

The HEADS-ED was used in all intakes (n = 8,280), with intake calls lasting an average of 48 min (Figure 1). 42.2% of clients were flagged for suicidality using the HEADS-ED Suicidality score of 1 or 2 and screened using an ASQ (3,536/8,280). Guidelines recommended that intake workers use the ASQ with clients scoring a 1 or higher on the HEADS-ED suicidality domain, and these guidelines were followed for 95.1% of clients (7,785/8,183). Specifically, 94.7% of clients with scores of 1 or 2 on the Suicidality domain were screened (3,221/3,401), while 95.4% (4,564/4,782) were not screened with the ASQ following a score of 0 on the Suicidality domain. Intake workers, using their clinical judgement, used the ASQ with 218 clients (4.6%) with a Suicidality rating of 0, and did not use the ASQ with 180 clients (5.3%) rated at a 1 (n = 170) or 2 (n = 10) in the Suicidality domain. 2.7% (97/3,536) of these included only partial ASQ data from which an ASQ risk level could not be calculated; these clients were excluded from analyses, including the ASQ.

Guidelines indicated that the CRAFFT should be used for clients aged 12 and over (n = 5,610). Of those clients, 70.6% (3,959/5,610) were assessed with the CRAFFT, and intake workers screened an additional 6 clients under the age of 12. Of the eligible clients who were not screened with the CRAFFT, 91.6% (1,513/1,651) were rated as a 0 on the Drug and alcohol domain of the HEADS-ED.

Of those screened with the CRAFFT, 75.5% (2,995/3,965) were screened with only Part A of the CRAFFT, though 11 of these were missing data and could not be scored. A large proportion (70.6%) were screened only with Part A and were not asked the “Car” question (2,116/2,995). Of these 2,116 clients, 87.6% (n = 1,853) indicated 0 substance use on questions 1–4 in Part A, and 85.6% (1,812/2,116) were 16 or younger. For research purposes, a missing response to the “Car” question was treated as a “no.” The remaining 24.5% (970/3,965) were screened with both Part A and Part B. Part B includes 5 follow-up questions regarding behaviours and motivations related to substance use. Of 970 clients with a total CRAFFT score, 28.9% (n = 280) were scored as Medium Risk and 71.1% (n = 690) were assigned a High Risk level.

Level of Need

A significant difference was observed between the HEADS-ED total score and each assigned Level of Need (F(4, 8,275) = 766.810, p < .001, η²=.270), a small to medium effect (Table 2). Pairwise comparisons reveal a significant difference between each Level of Need. The lowest Level of Need (General Population) has the lowest mean total scores, with each rising level rising congruently to its intended level of acuity. Spearman rank-order correlations demonstrated a significant moderate positive correlation between HEADS-ED total score and the assigned Level of Need (Table 3).

Table 2.

Level of Need Descriptions and HEADS-ED Outcomes.

Level of Need	Style of Care	Core Service Recommendations	Number of Clients (n = 8,280)	Average Minutes Per Intake Call	Mean HEADS-ED Total Score	Standard Error	Level of Need HEADS-ED Total Differences
Level of Need	Style of Care	Core Service Recommendations	Number of Clients (n = 8,280)	Average Minutes Per Intake Call	Mean HEADS-ED Total Score	Standard Error	Low Need	Moderate Need	Moderate to Severe Needs	Severe or Complex Needs
General Population	Population-Based Health Promotion and Prevention	Support from Primary Care Provider	92 (1.1%)	44.9	2.01	0.156	−1.72 (p < .001)	−3.43 (p < .001)	−4.77 (p < .001)	−6.19 (p < .001)
General Population	Population-Based Health Promotion and Prevention	Prevention and Promotion	92 (1.1%)	44.9	2.01	0.156	−1.72 (p < .001)	−3.43 (p < .001)	−4.77 (p < .001)	−6.19 (p < .001)
Low Need	Early Intervention and Treatment	Brief Intervention	1,808 (21.8%)	44.9	3.73	0.036		−1.71 (p < .001)	−3.05 (p < .001)	−4.47 (p < .001)
		Counselling and Therapy (incl. Psychotherapy)
		Family Support
		Peer Support
		Targeted Prevention
Moderate Need	Targeted to Moderate Mental Health, Addictions, and Substance Use Health Needs	Addictions Treatment	4,904 (59.2%)	48.8	5.44	0.025			−1.34 (p < .001)	−2.77 (p < .001)
		Assessment and Treatment
		Brief Intervention
		Case Management
		Counselling and Therapy (incl. Psychotherapy)
		Court Supports/Diversion
		Crisis Response and Support
		Family Support
		Peer and Family Support
		Peer Support
		Specialized Consultation
		Supported Employment
		Supportive Housing
		Withdrawal Management
Moderate to Severe Needs	Intensive and Specialized Care	Addictions Treatment	1,417 (17.1%)	52.6	6.78	0.052				−1.42 (p < .001)
		Assertive Community Treatment
		Assessment and Treatment
		Case Management
		Child Youth Intensive Treatment
		Counselling and Therapy (incl. Psychotherapy)
		Court Supports/Diversion
		Crisis Response
		Early Psychosis Intervention
		Family Support
		Peer and Family Support
		Specialized Consultation
		Supportive Housing
		Withdrawal Management
Severe or Complex Needs	Highly Specialized and Intensive Care	Emergency and In-Patient Services	59 (0.7%)	57.2	8.20	0.312
		Family Support
		Forensic Psychiatry Services
		Forensic Services
		Peer Support
		Psychotherapy Services
		Withdrawal Management

Description of Level of Need used by 1Call1Click.ca. Each Level of Need includes a description of the accompanying style of care appropriate to that Level of Need, along with a list of the core service recommendations that follow from that Level of Need. This is followed by the frequencies with which each Level of Need was assigned, the time spent on intake appointments for clients at each Level of Need, and comparisons of HEADS-ED total scores between each Level of Need.

Table 3.

Correlations Between Level of Need Assignment, Presenting Problems, and Screening Tool Outcomes.

Correlations
	HEADS-ED, Presenting Problems, and Level of Need (n = 8,280)
	LoN	PP
Level of Need (LoN)
Number of Presenting Problems (PP)	.291**
HEADS-ED Total Score (TS)	.507**	.353**
	Level of Need and Substance Use
	LoN	D	CRL	#S	Max Days
Level of Need (LoN)
Drugs and alcohol (D)	.170**
CRAFFT Risk Level (CRL)	.125**(n = 5,605)	.585**(n = 5,605)
Number of Substances (#S)	.178*(n = 3,954)	.846**(n = 3,954)	.976**(n = 3,954)
Highest number of days of substance use (Max days)	.167**(n = 3,954)	.864**(n = 3,954)	.976**(n = 3,954)	.978**(n = 3,954)
Average number of days of substance use (Ave Days)	.172**(n = 3,954)	.865**(n = 3,954)	.977**(n = 3,954)	.983**(n = 3,954)	.998**(n = 3,954)
	Level of Need and ASQ
	LoN	S
Level of Need (LoN)
Suicidality (S)	.269**
ASQ Risk Level (ARL)	.254** (n = 8,183)	.897** (n = 8,183)

Spearman Rank-Order Correlations between Level of Need with presenting problems, mental health acuity, suicide and substance use. Spearman's Rho values are presented, with * indicating significance at the level of p < .05 level and ** indicating p < .001. Unless otherwise stated, each correlation includes all 8,280 clients included in the analyses.

The distribution of scores assigned along each HEADS-ED domain is shown in Figure 2. The elevated domains were Discharge or current resources and Emotions, behaviours, thought disturbances, while a score of 0 was most often assigned to the Drugs and alcohol domain.

Figure 2.

The distribution of HEADS-ED domain scores across all clients. HEADS-ED domains receive scores ranging from 0 to 2, with 0 indicating no action required. A score of 1 indicates moderate functional impairment or a need for action that is not immediate. A score of 2 indicates a need for immediate action or a severe functional impairment within that domain. The total score is obtained by adding together each of the individual domain scores.

For those screened with both the CRAFFT and HEADS-ED, we observed a 92.5% agreement in determinations of no risk (2,641/3,954) or risk (1,018/3,954). The CRAFFT risk level and the Drugs and alcohol domain of the HEADS-ED were moderately correlated (Table 3). Using responses to Part A of the CRAFFT, we found a strong positive correlation between the Drugs and alcohol domain of the HEADS-ED and both the number of substances used and the average number of days of use endorsed across these substances, and the highest use frequency (Table 3).

The risk level outcomes of the ASQ demonstrated a weak positive correlation with the overall assigned Level of Need (Table 3). The Suicidality domain of the HEADS-ED was strongly positively correlated with risk on the ASQ (Table 3). Of those assessed with the ASQ and the HEADS-ED, we observed 83.4% agreement (2,869/3,439). Clients screened with the ASQ who received a score of 0 (no intervention required) on the Suicidality domain, 76.6% (167/218) were rated as low risk on the ASQ, whereas the ASQ indicated risk on 83.9% (2,702/3,221) of those whose HEADS-ED Suicidality score indicated risk as well (1 or 2). We observed a 90.8% (7,433/8,183) agreement between the rating of the HEADS-ED Suicidality domain and the ASQ. Specifically, 57.8% (4,731/8,183) were rated at 0 on the Suicidality domain of the HEADS-ED and rated as low risk on the ASQ (167) or the intake worker elected not to use the ASQ (4,564), whereas for 33.0% (2,702/8,183) of clients the Suicidality domain indicates a need for intervention (score of 1 or 2) and the ASQ also indicates risk (moderate or high).

Use of the ASQ has not been previously reported with children under 8 years of age; 87 children under the age of 8 were screened with the ASQ at 1Call1Click.ca. Of these, 11.4% (10/87) were rated as low risk and assigned a 0 on the HEADS-ED, while risk was detected on 43.6% (38/87) of these children by both screening tools. 41.3% (36/87) were rated as low risk by the ASQ while being rated as a 1 or 2 on the HEADS-ED. If the clients in this age range who were not screened with the ASQ are included with the low-risk ASQ outcomes, the overall agreement between the tools becomes 92.8% (640/690).

Appropriateness of Referrals

1Call1Click.ca began tracking the number of referrals rejected by the referred-to services in March 2023. This includes rejections for mismatches between client acuity and scope of service, but also any other reason, including client location, age, language, school board, and others. For the period of March through December 2023, 5.6% of referrals (290/5,189) were denied by the target agency.

Discussion

Key Findings

The ASQ, CRAFFT, and HEADS-ED have been used together in practice,^34,35 but the authors are unaware of other studies describing their outcomes in relation to each other. While the ASQ and CRAFFT involve structured questions and scores, the HEADS-ED is based on the clinician's interview questions and clinical judgement to score each domain. This provides good support that structured tools align with clinical judgment to determine client needs. The HEADS-ED provides a broad picture of the client's MHASUH needs and strengths, and the domains of Suicidality and Drugs and alcohol showed substantial agreement with the outcomes of the more specialized ASQ and the CRAFFT screening tools. These suggest that the HEADS-ED can work well to determine the necessity of more specialized tools, having the potential benefit of saving time for both the client and the healthcare provider.

The ASQ is primarily used in hospital settings, and has been studied in the context of primary care or specialty outpatient settings, making this the first study to describe the use of the ASQ in the context of a virtual mental health intake and coordinated access service.^26,39 This study includes children under 8 years who were screened for suicidal ideation using both the HEADS-ED and the ASQ, an age group with whom use of the ASQ has not been previously reported. The results for children under 8 were less consistent than with older clients, suggesting that it may be worthwhile to use both screeners with younger clients to thoroughly detect any risk.

Our results suggest that the total HEADS-ED score has some utility as a tool that healthcare providers can use, along with their clinical judgement, towards deciding the level of clinical need. HEADS-ED total scores differed significantly across each Level of Need and demonstrated the strongest correlation with the intake worker assigned Level of Need. We also observed strong agreement in risk detection between the HEADS-ED domains of Suicidality and the ASQ, and the risk detection of the Drugs and alcohol domain and the CRAFFT.

The results suggest alignment between the tools in the CASH Bundle, but also that they are independently informative. While clients with higher HEADS-ED totals tend to be assigned a higher Level of Need, the total score is not predictive of Level of Need, nor are the outcomes for the ASQ and CRAFFT. While each of the measures correlates with the overall Level of Need, these correlations range from weak to moderate, indicating that these screening tools are each contributing to the intake assessment and none are sufficient in isolation. The intake workers consider this information along with the client's current supports, needs, and goals when assigning the Level of Need and in making connections to the appropriate care resources.

While we were unable to find Canadian sources on the rate of referral rejection by services, research from Scotland (17-30%)⁴⁰ and Denmark (25.3%)⁴¹ present referral rejection rates 3–6 times higher than observed here. This suggests that 5.6% rejection rate from 1Call1Click.ca is low and supports that the CASH Bundle is helping intake workers make accurate referrals.

Limitations

There were no measurements of inter-rater reliability taken, as intakes were not repeated and were conducted by a single intake worker. Likewise, there are frequent missing data where certain questions were not asked or recorded in the EHR, particularly the “Car” question of the CRAFFT. The missing data are the result of the intake workers making judgments about the necessity of those questions, and while these missing data typically fit a post hoc explanatory pattern (e.g., no substance use, clients too young to legally drive), the specific case-by-case reasons were not recorded and cannot be verified. These limitations are trade-offs against the value of analyzing the entire clinical dataset from an active coordinated access service. Previous studies have demonstrated good inter-rater reliability on the HEADS-ED^27,28 and the CRAFFT,⁴² and intake workers engage in daily team meetings to collaboratively discuss the scoring of difficult cases.

The intake workers were instructed to use the ASQ to gain further information in a standardized way for clients who scored a 1 or higher on the HEADS-ED. As the recommended use of the ASQ is predicated on an elevated Suicidality score, it is important to limit interpretation of the correlation and agreement between the Suicidality domain and the ASQ. Three of the four initial questions on the ASQ regard current suicidal ideation, where the fourth is about historical suicidal gestures. The Suicidality domain of the HEADS-ED indicates a score of 1 or higher for any current suicidal ideation, which would lead to the suggested use of the ASQ. The ASQ would only detect elevated risk differently from the HEADS-ED for clients presenting without any current suicidality and a non-recent historical suicidal gesture. Given that clients are youths who have contacted 1Call1Click.ca seeking resources for current MHASUH challenges, we speculate that there are relatively few clients to whom these conflicting outcomes would apply, though we lack the data to verify this.

Finally, we are unable at this time to describe outcomes from the services clients are connected to by 1Call1Click.ca, for instance, whether or how long clients engaged with the services following their initial connection. Future work will consider the downstream effects of a service like 1Call1Click.ca on client outcomes.

Conclusions

These results support the CASH bundle as an effective way for healthcare providers to conduct effective, clinically relevant psychosocial interviews with children and youth to assess the nature and acuity of MHASUH need and guide referrals to specialized resources.

Footnotes

Acknowledgements

The authors thank the staff and clients of 1Call1Click.ca and CIHR for funding this work. We thank Kayla Beaudin for her comments on this manuscript.

Data Access

The dataset includes potentially identifying personal health data and will not be made available for public access.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was funded in part by a 2021 operating grant from the Canadian Institutes of Health Research and the RBC Foundation.

ORCID iD

Scott J. Robson

References

Health System Resources for Mental Health and Addictions Care in Canada. July 2019.

Mental health services in Canada. Barriers and cost-effective solutions to increase access - Nicholas Moroz, Isabella Moroz, Monika Slovinec D’Angelo, 2020. [accessed 2024 Oct 11]. https://journals.sagepub.com/doi/10.1177/0840470420933911.

Whitney

Peterson

. US National and state-level prevalence of mental health disorders and disparities of mental health care use in children. JAMA Pediatr. 2019;173(4):389-391. https://doi.org/10.1001/jamapediatrics.2018.5399

Children’s Mental Health Ontario. 2020 Report on wait lists and wait times for child and youth mental health care in Ontario. https://cmho.org/wp-content/uploads/CMHO-Report-WaitTimes-2020.pdf . 2020.

More than half of young Canadians who sought mental health services said they weren’t easy to access | CIHI. [accessed 2024 Oct 11]. https://www.cihi.ca/en/news/more-than-half-of-young-canadians-who-sought-mental-health-services-said-they-werent-easy-to.

Gill

, et al. Emergency department as a first contact for mental health problems in children and youth. J Am Acad Child Adolesc Psychiatry. 2017;56(6):475-482.e4. https://doi.org/10.1016/j.jaac.2017.03.012

Malla

, et al. Youth mental health should be a top priority for health care in Canada. The Canadian Journal of Psychiatry. 2018;63(4):216-222. https://doi.org/10.1177/0706743718758968

Uhlhaas

, et al. Towards a youth mental health paradigm: a perspective and roadmap. Mol Psychiatry. 2023;28(8):3171-3181. https://doi.org/10.1038/s41380-023-02202-z

Mojtabai

Olfson

. National trends in mental health care for US adolescents. JAMA Psychiatry. 2020;77(7):703-714. [accessed 2024 Oct 11]. https://pubmed.ncbi.nlm.nih.gov/32211824/. https://doi.org/10.1001/jamapsychiatry.2020.0279 doi:10.1001/jamapsychiatry.2020.0279

10.

Phillips

. Is anxiety/depression increasing among 5-25 year-olds? A cross-sectional prevalence study in Ontario, Canada, 1997-2017. J Affect Disord. 2021;282:141-146. https://doi.org/10.1016/j.jad.2020.12.178

11.

Government of Canada SC. Leading causes of death, total population, by age group. https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310039401 . 2021 Apr 13 [accessed 2024 Oct 11].

12.

Martínez-Alés

Jiang

Keyes

Gradus

. The recent rise of suicide mortality in the United States. Annu Rev Public Health. 2022;43:99-116. https://doi.org/10.1146/annurev-publhealth-051920-123206

13.

Bechard

, Cloutier

, Lima

, Salamatmanesh

, Zemek

, Bhatt

, Suntharalingam

, Kurdyak

, Baker

, Gardner

. Cannabis-related emergency department visits by youths and their outcomes in Ontario: a trend analysis. Can Med Assoc J. 2022;10(1):E100-E108.

14.

Canada PHA of. The Chief Public Health Officer’s Report on the State of Public Health in Canada 2018. Preventing Problematic Substance Use inYouth. 2018 Oct 23 [accessed 2024 Oct 11]. https://www.canada.ca/en/public-health/corporate/publications/chief-public-health-officer-reports-state-publichealth-canada/2018-preventing-problematic-substance-use-youth.html.

15.

Government of Canada SC. Correlates of vaping among adolescents in Canada. 2022 Jul 20 [accessed 2024 Oct 11]. https://www150.statcan.gc.ca/n1/pub/82-003-x/2022007/article/00003-eng.htm.

16.

Miech

, et al. Trends in use and perceptions of nicotine vaping among US youth from 2017 to 2020. JAMA Pediatr. 2021;175(2):185-190. https://doi.org/10.1001/jamapediatrics.2020.5667

17.

Conway

, et al. Association of lifetime mental disorders and subsequent alcohol and illicit drug use: results from the national comorbidity survey–adolescent supplement. J Am Acad Child Adolesc Psychiatry. 2016;55(4):280-288. https://doi.org/10.1016/j.jaac.2016.01.006

18.

Gårdvik

, et al. Psychiatric morbidity, somatic comorbidity and substance use in an adolescent psychiatric population at 3-year follow-up. Eur Child Adolesc Psychiatry. 2021;30(7):1095-1112. https://doi.org/10.1007/s00787-020-01602-8

19.

Rioux

, et al. Substance use disorders and suicidality in youth: a systematic review and meta-analysis with a focus on the direction of the association. PLoS ONE. 2021;16(8):e0255799. https://doi.org/10.1371/journal.pone.0255799

20.

CRAFFT. [accessed 2024 Nov 15]. https://crafft.org/.

21.

HEADS-ED - Welcome to HEADS-ED. . [accessed 2024 Nov 22]. https://www.heads-ed.com/en/home.

22.

Ask Suicide-Screening Questions (ASQ) Toolkit - National Institute of Mental Health (NIMH). https://www.nimh.nih.gov/research/research-conducted-at-nimh/asq-toolkit-materials . [accessed 2024 Nov 22].

23.

Polihronis

, et al. The HEADS-ED under 6: piloting a new communimetric mental health and developmental screening and triage tool for young children. Infant Mental Health Journal: Infancy and Early Childhood. 2024;45(5):557-568. https://doi.org/10.1002/imhj.22131

24.

Knight

, et al. Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Arch Pediatr Adolesc Med. 2002;156(6):607-614. https://doi.org/10.1001/archpedi.156.6.607

25.

Shenoi

, et al. Predictive validity of the CRAFFT for substance use disorder. Pediatrics. 2019;144(2):e20183415. https://doi.org/10.1542/peds.2018-3415

26.

Aguinaldo

, et al. Validation of the ask suicide-screening questions (ASQ) with youth in outpatient specialty and primary care clinics. Gen Hosp Psychiatry. 2021;68:52-58. https://doi.org/10.1016/j.genhosppsych.2020.11.006

27.

Cappelli

, et al. The HEADS-ED: a rapid mental health screening tool for pediatric patients in the emergency department. Pediatrics. 2012;130(2):e321-e327. https://doi.org/10.1542/peds.2011-3798

28.

Cappelli

, et al. The HEADS-ED: evaluating the clinical use of a brief, action-oriented, pediatric mental health screening tool. Pediatr Emerg Care. 2020;36(1):9-15. https://doi.org/10.1097/PEC.0000000000001180

29.

Roadmap to wellness: a plan to build Ontario’s mental health and addictions system | ontario.ca. [accessed 2024 May 30]. http://www.ontario.ca/page/roadmap-wellness-plan-build-ontarios-mental-health-and-addictions-system.

30.

Knight

, et al. A new brief screen for adolescent substance abuse. Arch Pediatr Adolesc Med. 1999;153(6):591-596. https://doi.org/10.1001/archpedi.153.6.591

31.

Horowitz

, et al. Ask suicide-screening questions (ASQ): a brief instrument for the pediatric emergency department. Arch Pediatr Adolesc Med. 2012;166(12):1170-1176. https://doi.org/10.1001/archpediatrics.2012.1276

32.

DiFranza

, et al. Measuring the loss of autonomy over nicotine use in adolescents: the DANDY (development and assessment of nicotine dependence in youths) study. Arch Pediatr Adolesc Med. 2002;156(4):397-403. https://doi.org/10.1001/archpedi.156.4.397

33.

Clark

Cloutier

Polihronis

Cappelli

. Evaluating the HEADS-ED screening tool in a hospital-based mental health and addictions central referral intake system: a prospective cohort study. Hosp Pediatr. 2019;9(2):107-114. https://doi.org/10.1542/hpeds.2018-0112

34.

Jabbour

, et al. An emergency department clinical pathway for children and youth with mental health conditions. Child Adolesc Psychiatr Clin N Am. 2018;27(3):413-425. https://doi.org/10.1016/j.chc.2018.02.005

35.

Lightbody

, et al. Use of quality improvement methods to enhance implementation of a mental health care bundle in a pediatric emergency department. CJEM. 2023;25(4):326-334. https://doi.org/10.1007/s43678-023-00476-4

36.

JASP. 2024.

37.

Ialongo

. Understanding the effect size and its measures. Biochem Med (Zagreb). 2016;26(2):150. https://doi.org/10.11613/BM.2016.015

38.

Maher

Markey

Ebert-May

. The other half of the story: effect size analysis in quantitative research. CBE Life Sci Educ. 2013;12(3):345. https://doi.org/10.1187/cbe.13-04-0082

39.

Horowitz

, et al. Implementing suicide risk screening in a pediatric primary care setting: from research to practice. Acad Pediatr. 2022;22(2):217-226. https://doi.org/10.1016/j.acap.2021.10.012

40.

Ball

, et al. Inequalities in children’s mental health care: analysis of routinely collected data on prescribing and referrals to secondary care. BMC Psychiatry. 2023;23(1):22. https://doi.org/10.1186/s12888-022-04438-5

41.

Hansen

Christoffersen

Telléus

Lauritsen

. Referral patterns to outpatient child and adolescent mental health services and factors associated with referrals being rejected. A cross-sectional observational study. BMC Health Serv Res. 2021;21(1):1063. https://doi.org/10.1186/s12913-021-07114-8

42.

Carney

Myers

Louw

. Reliability of the GAIN-SS. CRAFTT and PESQ Screening Instruments for Substance use among South African Adolescents. The South African Journal of Psychiatry: SAJP: The Journal of the Society of Psychiatrists of South Africa. 2016;22(1):932. https://doi.org/10.4102/sajpsychiatry.v22i1.932