Sage Journals: Discover world-class research

Abstract

Background

Youth in contact with the juvenile legal system experience disproportionate rates of homelessness. While system contact is a critical intervening point, juvenile courts do not typically offer housing services. One solution is to refer youth to evidence-based, community-based services to meet housing-related needs, but a myriad of individual and organizational factors often impede court staff from making such referrals. Housing Stability for Youth in Courts (HSYNC) is a novel, cross-system service linkage model for court-involved youth facing housing instability. HSYNC was developed using codesign as a strategy to improve usability and incorporated evidence on juvenile court linkage strategies effective at increasing service referrals and improving outcomes.

Method

The current study utilized a multimethod approach to evaluate the usability of HSYNC from the perspective of juvenile probation counselors (JPCs) as one group of end-users. As part of the pilot implementation study, survey (N = 15) and semi-structured interview (N = 20) data were obtained from JPCs from four juvenile courts in one northwestern state.

Results

Survey results indicated good usability for HSYNC (mean implementation strategy usability scale score of 80). Themes from the interviews complemented survey findings, pointing to HSYNC as a value-added program that integrated well with existing probation practices. The housing navigator's approach was highlighted as a strong facilitator of program usability. Themes also provided greater insight into implementation barriers (i.e., individual and contextual factors) and areas for improvement (e.g., eligibility screening process).

Conclusions

Drivers of program usability and implications for program improvement to address identified implementation challenges to ensure the HSYNC model is effective and sustainable long-term are discussed.

Plain Language Summary

A study examining the key factors impacting successful implementation of a novel housing stability program to address homelessness risk for court-involved youth and their families.

Court-involved youth experience disproportionate rates of homelessness, yet housing services are not traditionally offered by juvenile courts. Court staff, such as probation staff, can refer youth to community-based services to meet their housing needs. Unfortunately, many factors complicate whether these referrals are made, including staff feeling overburdened by other work obligations and uncertainty about whether referring youth to community services fits within their job responsibilities. In this study, we surveyed and interviewed probation staff who were responsible for implementing a new referral-based housing stability program—Housing Stability for Youth in Courts (HSYNC)—to understand whether the program addressed previously identified barriers to referral-making. A unique feature of HSYNC is that there is a dedicated housing navigator, who works as a “linkage specialist” to support probation staff in making referrals while working directly with youth and families to access housing-related services in the community. Our findings suggest that, from the perspective of probation staff, HSYNC is considered to be a user-friendly program that is well-aligned with existing court practices. The housing navigator and their unique approach to working with probation staff, youth, and families was perceived as the strongest facilitator of implementation success; meanwhile, screening for program eligibility was the biggest barrier, pointing to the need for improvement before HSYNC is widely implemented. The persistent intersection of youth housing instability and court involvement calls for better coordination between juvenile courts and community agencies providing housing-related services. From the probation perspective, HSYNC offers a viable solution to meeting youths’ housing needs without creating additional workload burden for court staff.

Keywords

youth homelessness juvenile courts probation usability cross-systems

Introduction

National estimates indicate that 3%–6% of youth aged 13–17 in the United States experience unstable housing or active homelessness (McKinnon et al., 2023; Morton et al., 2018; Perlman et al., 2014). Housing instability and homelessness during adolescence are associated with a myriad of adverse social and health outcomes, including poorer physical, mental, and sexual health as well as increased risk of food insecurity, substance use, victimization, and suicidality (Gultekin et al., 2019; McKinnon et al., 2023). The harmful effects of homelessness are disproportionately felt by youth of color, those experiencing poverty, females, and gender- and sexually diverse youth (McKinnon et al., 2023). Addressing youth homelessness is a critical public health imperative with important implications for reducing broader population health disparities.

A disproportionate number of youth experiencing unstable housing or homelessness are also in contact with the juvenile legal system. Nearly 78% of runaway and homeless youth have had at least one interaction with the police (ACYF, 2016), 62% have been arrested (Morton et al., 2018), and 44%–50% have had at least one stay in juvenile detention, jail, or prison (ACYF, 2016; Morton et al., 2018). System contact itself also increases the risk of homelessness (Prince et al., 2019; Quirouette et al., 2016) as many youth exit the system into unstable housing situations (Mayfield et al., 2017). In one study, only 65% of detained youth named a parent or relative they could live with when they were released (Columbia Legal Services, 2015). Nearly 30% of youth exiting juvenile residential or adult correctional facilities may be homeless within 12 months (Danielson et al., 2020). Adolescent incarceration is also associated with earlier homelessness and longer durations of homelessness (Cox et al., 2021).

The overlap of currently unhoused and youth at risk of housing instability in the juvenile legal system is significant, making juvenile courts an important inflection point for addressing housing instability to prevent homelessness and associated adverse outcomes. As a point of contact for over half of youth entering the legal system after arrest (Puzzanchera et al., 2022, 2023), probation departments are well-positioned to reach a large proportion of youth who would otherwise remain disconnected from services. Because juvenile courts do not typically offer housing-related services internally (e.g., Walker et al., 2018), probation staff would need to identify and refer youth to these services in the community.

Previous studies find that the adoption and sustainment of cross-system referral strategies is low, typically below 50% for referrals from probation departments to non-legally mandated community-based services (Dennis et al., 2019; Knight et al., 2019; Wasserman et al., 2021). Several factors present barriers to adoption, including high workload demands (Huang et al., 2020; Wasserman et al., 2021); perceptions of referral-making as relevant (or not) to their role (Knight et al., 2019); staff adaptability or proclivity to changing current assessment and referral practices (Farrell et al., 2011; Knight et al., 2019); organizational climate surrounding implementation of health innovations (Knight et al., 2019; Scott et al., 2019; Taxman et al., 2014; Wasserman et al., 2021); degree of communication and collaboration with community providers (Elkington et al., 2020; Scott et al., 2019; Taxman et al., 2014); data sharing across systems (Dennis et al., 2019); and staff perceptions of youths’ treatment needs (Wasserman et al., 2008).

One strategy to improve the adoption of cross-system referral strategies is to increase usability (Dopp et al., 2019; Lyon & Bruns, 2019). Usability in health services describes the extent to which the intended users of a program can implement the program easily while maintaining effectiveness, efficiency, and satisfaction (International Organization for Standardization, 2018; Lyon, Coifman, et al., 2021). Usability is both a complex implementation strategy and upstream implementation determinant (Lyon & Bruns, 2019; Lyon, Coifman, et al., 2021). Improving the usability of programs can be facilitated by co-design (Clanchy et al., 2024; Dopp et al., 2020; Kushniruk & Nohr, 2016), a method that engages intended users directly in the design and development of new service innovations to increase alignment with user needs (Sanders & Stappers, 2008; Vargas et al., 2022). The current study evaluates the usability (implementation determinant) of a novel, cross-system court referral program to address youth housing instability risk, which was developed using co-design (an implementation strategy) to enhance usability among probation staff.

Co-designing HSYNC

Housing Stability for Youth in Courts (HSYNC) is a cross-system referral (“linkage”) program that uses system navigators and flexible funds to connect youth and their families to community-based resources to address housing needs (CoLab, n.d.; Walker et al., 2018). The approach used to design HSYNC mirrored the steps of the Discover, Design and Build, and Test (DDBT) framework for developing human-centered, evidence-based interventions as an implementation strategy (Lyon et al., 2019). The discover stage focuses on understanding the evidence base and potential contextual factors that could influence intervention success, often by gathering insights from stakeholders; the design and build stage focuses on collaborative intervention development, where insights are leveraged from the discovery phase to ensure the intervention and implementation aligns with stakeholder needs and practical considerations; and the test stage focuses on evaluating key implementation outcomes (Lyon et al., 2019).

The HSYNC design process involved capturing and synthesizing (1) the research evidence on health service cascade models (Gardner et al., 2011; MacCarthy et al., 2015; Mugavero et al., 2013) and behavioral health cascades in juvenile court settings specifically (Belenko et al., 2017; Elkington et al., 2023); (2) local knowledge of service needs through group and key informant interviews; and (3) systems expertise from multisector teams. The discovery stage of intervention development identified evidence that juvenile court service referral linkage strategies are effective at improving service referrals by reducing workload demands on probation staff (Elkington et al., 2020). Further, when referred to evidence-based interventions such as multidimensional treatment foster care (Ladner-Graham, 2019), evidence-based substance use disorder treatment (Fadus et al., 2019), and evidence-based family conflict interventions (Littell et al., 2023), youth recidivism outcomes improve. The HSYNC model was developed based on this evidence, with the design and build stage following a participatory co-design process (Owens et al., 2022; Sanders & Stappers, 2008) where multisector teams from two juvenile court jurisdictions were engaged in parallel development processes to identify common and unique needs across contexts. This process resulted in the development of a service linkage model for juvenile probation that integrates cash-plus service navigation (i.e., housing navigators and cash transfers) within a cascade of care model to increase cross-system referral to address housing instability risk.

The literature on service linkage models points to common barriers that can interrupt successful implementation in real-world settings, including the adoption of new screening tools into workflows (Rieder et al., 2020), low perceived relative advantage compared to usual practices (Dennis et al., 2019), and complex or difficult to understand processes (Becan et al., 2020; Dennis et al., 2019; Rieder et al., 2020). The HSYNC model was intentionally designed to reduce these barriers through the co-design process by using routine screening tools already in use by the juvenile courts for case management; training navigators to educate probation staff on HSYNC program benefits for their workload and youth outcomes; and by placing navigators in physical and/or virtual shared workspaces with probation staff to facilitate the referral process.

The current study represents the test stage of the DDBT framework. We aimed to assess whether the above program components reduced anticipated barriers using a usability evaluation framework given the recognition of intervention usability as an important determinant of implementation success (Lyon et al., 2020; Moldestad et al., 2023). Juvenile probation counselor (JPC) perceptions of the usability of the service linkage step of the broader HSYNC model is one part of a complex implementation system. However, consistent with many implementation science studies, we were specifically interested in isolating and studying usability as one potential implementation determinant. As recommended when testing the usability of health service innovations (Lyon et al., 2019; Walji et al., 2014), using multiple evaluation methods offers a more comprehensive understanding of usability challenges and successes than singular methods. As such, we quantitatively examined JPC usability ratings via survey responses to the Implementation Strategy Usability Scale (Lyon, 2016; Lyon, Pullmann, et al., 2021) and conducted qualitative interviews with JPCs to gain further insight into specific aspects of HSYNC that facilitated or hindered implementation, alongside suggestions for quality improvement.

Method

Study Procedures

All JPCs from four juvenile court pilot sites in one northwestern state with active caseloads at the time of the study were eligible to participate. Sites were approached to participate in the evaluation study approximately 1 year from the start of the program. Research staff obtained JPC email lists from court leadership and sent two emails through staff listservs to recruit for the study. The email contained a direct link to an online survey and included information about the opportunity to participate in a phone/video interview. JPCs indicating interest in the interview received a phone/video call to receive more information and complete informed consent, if willing. The semi-structured interviews were recorded by video and then transcribed using a professional transcription service. Fifteen eligible JPCs completed the online survey (response rate = 42%), and 20 JPCs elected to participate in interviews (response rate = 38%). Study procedures were approved by the University of Washington Institutional Review Board.

Participants

Table 1 provides JPC demographic characteristics for the survey sample (N = 15). On average, participants were 48 years old, 80% female, majority white (60%), and reported over 20 years of experience in the field (73%) including over 20 years in their current position (56%). We are unable to report demographics for the interview sample, as this information was destroyed prior to analysis in accordance with the study's IRB-approved data retention policy.

Table 1

Survey Sample Demographic Characteristics (N = 15)

	N (%)
Gender
Male	3 (20%)
Female	12 (80%)
Non-binary/transgender	0
Other	0
Ethnoracial identity^a
Hispanic or Latino	1 (7%)
Non-Hispanic or Latino	1 (7%)
White	8 (53%)
Asian American	1 (7%)
Native Hawaiian/Pacific Islander	1 (7%)
American Indian/Alaska Native	0
Black/African American	1 (7%)
Bi- or multiracial	2 (13%)
Years in current position
<2	1 (7%)
2–6	0
7–12	1 (7%)
12–20	5 (33%)
>20	8 (53%)
Years in the field
<2	1 (0.07%)
2–6	0
7–12	0
12–20	3 (20%)
>20	11 (73%)
	M (SD)
Age (in years)^b	47.93 (6.91)

Racial/ethnic categories are not mutually exclusive.

Range = 34–60 years.

Measures

Survey Measures

Implementation Strategy Usability Scale (ISUS). The 10-item System Usability Scale (SUS: Brooke, 1996) was adapted to create the ISUS (Lyon, 2016; Lyon, Pullmann, et al., 2021). As Lyon, Pullmann, et al. (2021), note, the term “system” was replaced with “implementation strategy” in each item without further modifications to maintain consistency with the SUS operationalization of usability. In the current study, the terms “implementation strategy” were replaced with “the HSYNC navigator program” to fit the evaluation context. ISUS items range from 1 (strongly disagree) to 5 (strongly agree), with half of the items reversed-scored so that all items reflect a greater degree of usability. As recommended by scale developers (Lyon, 2016), the total usability score was calculated by multiplying the sum of individual item scores by 2.5 (possible range: 0–100).

ISUS Follow-up Questions. Four open-ended questions were included on the survey to give respondents the opportunity to expand on their ISUS ratings. Questions focused on what HSYNC offered clients beyond typical probation offerings, who or what helped JPCs understand the benefits of HSYNC, suggestions around simplifying the referral process, and ideas for improving HSYNC services.

Semi-Structured Interview Guide

The semi-structured interview guide was informed by the ISUS and explored JPC perspectives on the screening and referral process (e.g., “What worked best about the referral process?” “What was the most challenging part about the referral process?” “How well did the routine court flags identify youth who needed services?”), facilitators and barriers to program implementation (e.g., “What did you find most helpful about the HSYNC trainings?” “Did the services appear to coordinate well or conflict with court referred services?”), and program improvement suggestions (e.g., “What concerns or suggestions do you have for the program?”).

Analytic Approach

We undertook two data collection and analysis approaches—survey and semi-structured interviews—to complete the HSYNC usability evaluation. Findings from each approach are presented independently and then used in a complimentary manner to draw implications and conclusions regarding HSYNC usability (detailed in the Discussion).

Survey Data

The analysis of survey data proceeded in several steps. First, descriptive statistics were used to assess item functioning and Cronbach's alpha was used to assess ISUS scale reliability. Missing data were present in three cases. Because the ISUS score is computed as a summative measure, item-level missing data negatively biases the total scale score. Thus, we report item functioning and scale scores from the original data (with missing values) and from a new dataset where missing values were imputed by the item mean (termed “imputed data”). We then compared the imputed ISUS score against the current quality benchmark (Bangor et al., 2009). Analyses were run using SPSS version 27.

Responses to the open-ended follow-up questions were analyzed to supplement the quantitative data using a conceptual content analysis approach (Krippendorff, 2019; Neuendorf, 2017). Specifically, responses from each question were condensed into conceptually similar categories—thematic codes—that recurred across the data by a PhD-level researcher (first author). Codes were then quantified into response rates, and theme titles were developed to reflect the focus of each set of codes.

Qualitative Interviews

JPC interviews were analyzed following a thematic analysis approach (Braun & Clark, 2022). This included a deductive, iterative process of (1) coding the data using an a-priori coding framework, (2) using research team discussions to enhance credibility and trustworthiness (i.e., researcher triangulation; Miles et al., 2020), and (3) condensing codes and naming themes. The coding framework was developed by the study PI (second author) and included concepts from usability and diffusion of innovations frameworks (Lyon et al., 2019; Rogers, 1995). The coding framework was pilot tested with two members of the research team using three transcripts. Codes included relative advantage of the program compared to services offered elsewhere, observability of the benefits of the program, implementation barriers/facilitators, and suggestions for improvement. The codes were refined in team discussions where discrepancies in code definitions and the application of codes were resolved. Two master's level research assistants were trained on the coding framework then proceeded to manually code all transcripts using the established coding framework. Disagreements were discussed and negotiated until consensus was reached. Next, a PhD-level researcher (first author) grouped and condensed codes based on conceptual consistency into broader themes. Theme titles were developed to describe the focus of each set of codes and exemplar quotes were identified to illustrate thematic findings and give voice to participants.

Results

Quantitative Usability Ratings

Scores for the individual ISUS items included the full range for Items 2, 4, 7, 8, 10 and partial range for Items 1, 6, 9 (range: 2–5), and 3, 9 (range: 3–5) (Table 2). Differences in standard deviation values for the imputed items compared to the items from the original data (with missing values) were negligible suggesting bias from mean imputation was low. Mean ratings were high across items (in the 4-point range, translating to “agree to strongly agree”), except for Item 1 (referral frequency) which had the lowest mean rating of 3.33 (closer to a “neither agree nor disagree” translation). Notably, Item 3—referring to the navigator program was easy—had the highest mean rating with the smallest degree of variation in responses (M = 4.79, SD = 0.58, range = 3–5) suggesting that the HSYNC referral process was perceived as simple and straightforward by most JPCs.

Table 2

ISUS Individual Item Descriptive Statistics and Scale Reliability for HSYNC (N = 15)

	Original data			Imputed data^a
ISUS items	N	Range	M (SD)	N	Range	M (SD)
1. I referred to the HSYNC navigator program frequently.	15	2–5	3.33 (1.18)	15	2–5	3.33 (1.18)
2. I found referring to the HSYNC navigator program unnecessarily complicated.^b	14	1–5	4.21 (1.25)	15	1–5	4.21 (1.21)
3. I thought referring to the HSYNC navigator program was easy.	14	3–5	4.79 (0.58)	15	3–5	4.79 (0.56)
4. I think that I would need the support of an expert consultant to be able to refer to the HSYNC navigator program.^b	15	1–5	4.53 (1.13)	15	1–5	4.53 (1.13)
5. I found the various components of the HSYNC navigator program were well integrated.	12	3–5	4.25 (0.75)	15	3–5	4.25 (0.67)
6. I thought there was too much inconsistency in the HSYNC navigator program.^b	14	2–5	4.07 (0.92)	15	2–5	4.07 (0.88)
7. I would imagine that most people would learn how to use the HSYNC navigator program very quickly.	14	1–5	4.14 (1.17)	15	1–5	4.14 (1.12)
8. I found the HSYNC navigator program very cumbersome to use.^b	14	1–5	4.38 (1.08)	15	1–5	4.36 (1.04)
9. I felt very confident referring to the HSYNC navigator.	15	2–5	4.13 (1.06)	15	2–5	4.13 (1.06)
10. I needed to learn a lot of things before I could get going with the HSYNC navigator program.^b	14	1–5	4.21 (1.12)	15	1–5	4.21 (1.08)
	α	Range	M (SD)	α	Range	M (SD)
Total ISUS score^c	–	–	–	.73	55–100	80.09 (13.73)

Note. For all items, 1 = strongly disagree; 2 = disagree; 3 = neither agree or disagree; 4 = agree; 5 = strongly agree.

α = Cronbach’s alpha measuring scale reliability. ISUS = Implementation Strategy Usability Scale; HSYNC = housing stability for youth in courts.

Missing values for Items 2, 3, 5, 6, 7, 8, and 10 were imputed by the item mean.

Items 2, 4, 6, 8, and 10 are reverse scored.

Calculated by multiplying the sum of individual item scores by 2.5 (possible range: 0–100). Not available for original sample due to item-level missingness.

The overall ISUS score demonstrated acceptable reliability (α = .73). ISUS ratings ranged from 55 to 100, with a mean of 80.09 (SD = 13.73) indicating excellent usability for HSYNC according to JPCs. A mean ISUS score of 80 surpasses the usability quality benchmark, where the ISUS cutoff of 70 is considered “acceptable” (Bangor et al., 2008, 2009).

Table 3 presents findings from a content analysis of the open-ended survey responses. JPCs reported that HSYNC offered a unique array of services that were not already available through the court, including housing assistance, mental health treatment, help with basic needs (health insurance, driver's license), and transportation. Approximately 60% of JPCs reported that the navigator was responsible for facilitating their knowledge and understanding of services offered through HSYNC. In line with the quantitative findings, most participants indicated that the HSYNC referral process was simple and straightforward (60%), with others recommending a shorter referral form or checklist to further simplify the referral process (27%). Several JPCs offered suggestions for improving HSYNC to better fit the court context, including providing JPCs with an updated list of services offered (20%) and case updates post referral (13%).

Table 3

Content Analysis of Participant Responses to ISUS Follow-up Open-Ended Survey Questions (N = 15)

Theme	Thematic codes	Response rate
Unique services offered by HSYNC	Housing assistance	7 (47%)
	Mental health treatment	2 (13%)
	Application help for basic needs	1 (7%)
	Transportation	1 (7%)
	Serves as a “one-stop-shop” for services	1 (7%)
	Nothing new or not sure	4 (27%)
Facilitators of JPC knowledge of HSYNC services	HSYNC navigator	9 (60%)
	Employment history with an emergency shelter	1 (7%)
	Nothing or not sure	2 (13%)
Simplifying the referral process	Not applicable; the referral process is simple	9 (60%)
	Create a shorter referral form or checklist	4 (27%)
	Create navigator directory by geographically assigned areas	1 (7%)
	Navigators attend JPC staff meetings	1 (7%)
Improving HSYNC services for juvenile courts	Provide JPCs with an updated list of services offered	3 (20%)
	Provide housing status/referral updates to JPCs	2 (13%)
	Navigator encourages youth program participation	1 (7%)
	More direct service provision	1 (7%)
	Reduce navigator turnover	1 (7%)
	Physically locate navigator in courts to facilitate youth-JPC rapport building	1 (7%)
	Nothing or not sure	4 (27%)

Note. Codes are not mutually exclusive. ISUS = Implementation Strategy Usability Scale; HSYNC = housing stability for youth in courts; JPC = juvenile probation counselor.

Qualitative Perspectives on HSYNC Usability

Six themes emerged from a thematic analysis of the JPC interviews (Table 4). Together, themes complemented survey findings while providing greater insight into implementation barriers and areas for improvement that were not readily apparent in the survey data.

Table 4

Thematic Analysis of JPC Interviews (N = 20)

Theme	Sub-theme	Exemplar quotes
Theme 1: HSYNC was value-added	Program was beneficial	“As court employees, we are not familiar with […] the ins and outs of housing programs other than the emergency shelters that we’re used to working with, so it was really helpful to have the navigator talk about what kind of housing there is and what the processes are to get a kid and a family connected to that.” (JPC 3009)“I had one pretty successful referral where the youth was hooked up with housing—pretty permanent housing—through this program […] I think it's a needed program.” (JPC 3013)
	Program offered unique services	“The [navigator] did a really amazing job attempting to connect with kids and families to provide them with resources that they might not otherwise have been able to, or even know about.” (JPC 3003)“Just being a resource for us [JPCs] was great because she [navigator] provided additional resources other than housing—places to get food, clothing, or furniture […] she came up with resources I wasn't aware of.” (JPC 3005)
	No conflict with existing court programming	“It [HSYNC] very much seemed just kind of a courthouse adjacent program.” (JPC 3002)“They [HSYNC and other court services] weren't conflicting and it was helpful.” (JPC 3017)
	Desire for sustainability	“A concern would be that there's enough traffic to sustain it [HSYNC].” (JPC 3002)“The general concern is dried up resources and what will be funded […] that's my worry. We need them [housing navigator programs] in our county, probably more so now than ever.” (JPC 3011)
Theme 2: HSYNC integrated well with probation	Easy, smooth referral process	“The actual referral is incredibly easy. It's really low barrier for us. There's nothing required on the kid or family's part that they need to follow-up on, which I think is extremely helpful.” (JPC 3009)“The referral is user-friendly and easy to operate.” (JPC 3011)
	Navigator conveniently located	“It [referral process] was really good because she [navigator] was in our building. So, a lot of conversations—I just walked into her office or she could just walk down to my office and do a follow-up about the referral.” (JPC 3004)“Having the navigator in the building to consult and ask about cases and ask about kids. I think that's probably the most helpful way to establish whether or not it's an appropriate referral or to keep in the back of our minds, as probation counselors, that this program's out there and is seeking to help kids.” (JPC 2023)
	JPC-navigator casecollaboration	“We would coordinate meetings with the kid […] I was pretty involved [in conversations about service options and referrals]. We [JPC and navigator] would meet with the kids together.” (JPC 3004)“[The youth] was hard to get in touch with so there was a lot of back and forth between myself, the grandmother, the navigator, and trying to see what we could do to help. We would work together.” (JPC 3017)
	Complimentary engagement approaches	“Unlike me, the navigators are able to reach out via social media, which is helpful because a lot of kids, even if they don't have a phone, are still able to log in to social media accounts.” (JPC 3017)
Theme 3: The navigator's approach facilitated successful implementation	Accessible to JPCs	“She [navigator] walked me through what the program was and did […] she basically came into my office and taught me about the program. And I knew I had a point person that I could ask questions to, or you know, I could call and say, ‘Hey is this kid appropriate?’ I think that was helpful.” (JPC 3002)“She's [navigator] appeared in our staff meetings multiple times to go over the referral process and how to make things better. She [also] made herself very available for one-on-one trainings if people needed them.” (JPC 3014)
	Proactive and responsive	“She [navigator] showed up to our unit meetings saying, “Remember, we’ve got this program so please refer.” (JPC 3004)“I would say the ease of being able to email her [navigator] at any time. She's pretty communicative and I think that's the key piece. (JPC 3013)
	Helped identify appropriate referrals	“She [navigator] was very easy to work with. She said, ‘If you have questions, just refer them. I can always call and find out what's going on and make the determination for myself if this kid is appropriate for the program or not.’ which I appreciated […] So that was helpful to me.” (JPC 3002)“I will definitely do a referral but I’m not sure if it completely fits the criteria. I email her [navigator] and give her the scenario and see if she thinks it's an appropriate referral […] There's an ease when I email her.” (JPC 3013)
	Connected effectively with youth and families	“Once I referred kids, I had a lot of success which I really kind of attribute to our navigator […] with her ability to connect with kids and know the services and know the programs available.” (JPC 3008)“She [the navigator] was a nice, kind person that could relate with people.” (JPC 3005)
Theme 4: Individual and contextual factors served as barriers to implementation success	Individual: JPC biases	“Often times you find out that the kid was on the street because they wanted to be […] I just know that the parents themselves aren't homeless. The kid is just identifying as homeless.” (JPC 3002)“I think it's just kids that don't want to follow the rules of the parents’ home, you know in most cases, and then they just take off.” (JPC 3005)“All of us JPCs have our own biases in place and even though we might get a hit on the PACT assessment, it might not register to us that this kid could use her [navigator] support because of our implicit biases.” (JPC 3016)
	Individual: Youth and family characteristics	“It's difficult to actually get resources for some of these families […] the struggle that I have is actually kids that we work with, you get the parent that's sort of drug impacted, their kids don't stand a chance.” (JPC 3005)“Sometimes when they live out in more rural areas […] the youth don't always stay engaged with the navigator. It can be [challenging] if the youth doesn't have access to a car.” (JPC 3008)“I think a lot of families just can't get to the phone or can't get to a computer and I mean it's tough […] they don't have a physical address, they move around a lot, sometimes you can't get ahold of them.” (JPC 3031)
	Contextual: Workplace challenges	“There's just a lack of staffing—we don't have the appropriate amount of time to dedicate to cases.” (JPC 3009)“I don't know if homelessness has been at the top of the priority. I don't think I’ve been to a training [related to] homelessness in years unfortunately.” (JPC 3013)“The real challenging piece is that we receive pressure to refer kids to all kinds of things. All of these possible referrals get put into a hierarchy for JPCs. This program [doesn't] always reach to the top of the hierarchy.” (JPC 2023)
	Contextual: Service limitations	“The challenge is that whenever you have families with minimal resources [it's difficult for them] to get to different places. I mean, fortunately, we have a transit system, but then that costs money too. So, I think that if she [navigator] was able to provide bus fare that would have been great.” (JPC 3005)“They [families] need someone with an actual ‘in’ to make resources happen. I think it's hard for families to understand why I’m making the referral when it's not like she has any secret ‘in’ with the [housing] programs themselves. She can’t fast track an application. She doesn’t have any extra funding for housing.” (JPC 3016)
	Contextual: COVID-19	“I think COVID definitely threw it [HSYNC] off a little bit. Right as we were just starting to get used to it [referral form]—or to the process of remembering to fill it out—COVID hit and we were all sent home. I think it's fallen through the cracks quite a bit because of that.” (JPC 3010)“I think COVID has been problematic—having to have changed a lot of processes and it was a new referral […] so it was hard to kind of keep that in the forefront unless there was something going on with the youth like, ‘Oh yeah, I should be referring to this program.’” (JPC 3017)
Theme 5: Screening for program eligibility was the most challenging aspect of implementation	Issues with eligibility criteria	“It's possible that there were [criteria for determining eligibility] and I’m just ignorant of them […] As is the case with a lot of things within this line of work, you don't often get those clear, extreme end of the spectrum cases. You have to move in that gray area a little bit. So, it wasn't clear to me.” (JPC 3002)“We’re supposed to do anything that has housing indicators on the PACT assessment but sometimes I think it's easier to, you know, just go through the cases and say who could use her [navigator] help with anything housing stability related or family stability related.” (JPC 3016)“As we know, kids’ circumstances change quite frequently. So, we could have done an assessment 2 months ago, and then all of a sudden, they become homeless.” (JPC 3009)
	Professional judgement used	“Obviously, using our professional judgment if we know that a kid has unstable housing, or the family is kind of in imminent risk of homelessness.” (JPC 3009)“I have made referrals just kind of out of common sense for kids that are obviously in dire need of this service, not from the PACT flag.” (JPC 3013)“There are indicators which trigger a referral that we’re supposed to do […] sometimes I don't refer every single kid probably that meets the qualification […] kids that might meet the qualifications but aren't necessarily experiencing homelessness at the moment, are not probably referred.” (JPC 3017)
	High workload demands	“We’re not always looking for the homeless issue. Sometimes there are other factors that kind of demand all of our time or all of our focus.” (JPC 3023)“I got very busy […] I haven't had the mind energy or bandwidth to figure out who would be eligible.” (JPC 3012)
Theme 6: Improvement suggestions centered on strategies for successful implementation	Increase program awareness	“We didn't realize how much they could help with educational pieces and other things. So, I think one thing that might be helpful is just like, ‘this is what it is. These are some things we’ve helped kids out with.’ […] maybe a little flyer or pamphlet with bullet points of things that they can do to be able to pass along.” (JPC 3017)“It would maybe be good to come up with a list of other things she [navigator] can do besides simply getting them into housing.” (JPC 3019)
	Streamline referral process	“It would be really helpful if the supervisors or managers could just run a query and tell us a list that we need to refer to her and for us to streamline that process that way.” (JPC 3016)“I would like to see this program not rely on us for the referrals but go after the referrals themselves and then collaborate with us. Going around us to start engaging with these kids might be more advantageous.” (JPC 3023)
	Improve screening practices	We’ve talked about our supervisors printing off a spreadsheet that has a list of all the kids on our caseloads who are eligible. I think that is going to be really instrumental in making sure we are referring appropriately.” (JPC 3010)“It might be a better use of time to just send contact information via email and then for us to do essentially the clinical case staffing over the phone and for her to gather the information that she feels is relevant.” (JPC 3016)
	Increase services offered and accessibility	“I think helping kids get ID cards and helping them […] contact the actual shelter.” (JPC 3004)“The navigator being able to go into the community itself and meeting them at those exact locations and getting them connected with the right people […] to provide assistance with housing.” (JPC 3007)“I just wish there were more placements for kids.” (JPC 3030)
	Provide JPCs with case updates	“I just want the most information […] I want to know what's going on with the youth, good or bad.” (JPC 3013)“It would be beneficial to know specifically what case management they’re doing and to be able to communicate with that person so that we’re not duplicating services.” (JPC 3029)

Note. JPC = juvenile probation counselor; HSYNC = housing stability for youth in courts.

Theme 1: HSYNC Was Value-Added

JPCs recognized the value of a housing stability program for court-involved youth and their families. All participants indicated that the program was beneficial and needed: “I think the reason it [HSYNC] works so good is because it's a needed program.” JPCs indicated that HSYNC offered a unique suite of services that were not available to youth and families otherwise. HSYNC services were also perceived as not conflicting with those already being offered by the court. One participant commented on how the navigator herself—beyond the services she offered—played an important therapeutic role: “When I make a referral, I know she [navigator] will be just another positive protective factor in a youth's life.” Several JPCs also referenced concern for program sustainability. As perceived value is a defining characteristic of program sustainability (Palinkas et al., 2020), JPC concerns about sustainability provide further evidence of the program as value-added from the JPC perspective.

Theme 2: HSYNC Integrated Well With Probation

HSYNC was perceived as well-integrated with existing probation practices. Most JPCs indicated that the referral process was smooth and easy, and it was not generally considered an additional workload burden: “The actual referral is incredibly easy […] it's really low barrier for us.” Having the navigator co-located with probation (in the same building) was perceived as convenient and facilitated regular communication and collaboration between JPCs and navigators. Two JPCs also reported that, because navigators operated independently from the court, they could be more flexible in the approaches they used to contact youth (e.g., social media), which was helpful to JPCs, particularly when traditional forms of engagement (e.g., phone calls) were difficult because the youth was experiencing active homelessness.

Theme 3: The Navigator's Approach Facilitated Successful Implementation

The navigator—and their unique approach to working with JPCs, youth, and families—was one of the strongest facilitators of implementation success from the JPC perspective. Most participants reported that navigators were accessible to JPCs, including making themselves available to answer questions and provide guidance on the referral process. JPCs also indicated that the navigators were responsive and timely in their communication, as well as proactive in providing implementation support to JPCs while ensuring the program was operating smoothly. Nearly half of the participants indicated that their navigator played a role in identifying appropriate referrals, which JPCs found helpful. JPCs also perceived the navigators as successful in their ability to connect effectively with youth and their families: “Once I referred kids, I had a lot of success which I really attribute to our navigator, with her ability to connect with kids.” JPCs attributed this success to the navigators making themselves available to youth and families, as well as certain characteristics and traits particular to their navigators (e.g., relatable, kind, dedicated): “She [navigator] was a nice, kind person that could relate with people.”

Theme 4: Individual and Contextual Factors Served as Barriers to Implementation Success

Perceived challenges to program implementation and service uptake included individual and contextual factors. At the individual level, assumptions and biases regarding why youth were experiencing homelessness (e.g., “they don't want to follow the rules of the parents’ home”) impacted whether JPCs believed youth met the eligibility criteria and/or whether they should be referred to the program: “even though we might get a hit on the PACT assessment, it might not register to us that this kid could use her [navigator] support because of whatever our implicit biases are.” Most JPCs identified youth and family characteristics as potential barriers to program engagement and service uptake. These included: youth disinterest in the program, problematic youth behavior, case type and status, criminal history, poverty, homelessness, parental substance abuse, and living in a rural area.

Contextual factors at the court and community levels also emerged as barriers to implementation success. JPCs highlighted several features of the court workplace environment that made assessment and referral to HSYNC challenging, including staff shortages, high caseloads, competing service priorities, and feeling pressured to refer to multiple programs. JPCs were also aware of service limitations that youth and families face, both in terms of what was available in the community and access issues. For instance, several JPCs indicated that there was a misconception (on the part of JPCs and youth/families) that HSYNC could provide actual housing: “I think that there was a common misconception—making this referral to the navigator does not have housing attached to it.” Additionally, JPCs commented on accessibility issues such as transportation and lack of services in rural areas as another challenge to service uptake. Most JPCs referenced COVID-19 as a key contextual factor that created implementation challenges, including negative impacts to the screening and referral process (resulting from higher workload demands), youth engagement, communication and collaboration with the navigators, and the ability to provide warm handoffs.

Theme 5: Screening for Program Eligibility Was the Most Challenging Aspect of Implementation

The most challenging aspect of HSYNC implementation was screening for program eligibility. Most JPCs discussed issues with the eligibility criteria, including not knowing that criteria existed or experiencing challenges determining whether the criteria applied to specific cases (e.g., youth with no current housing instability; non-probation cases such as those in detention or pre-trial, at-risk youth, or child in need of services petitions). Other participants noted that the eligibility criteria were too broad or not dynamic enough to capture changes in a youth's living status over time, leading JPCs to feel like the current criteria may result in over- or under-referring. Many JPCs reported using professional judgment in lieu of, or in addition to, the established criteria to determine referral appropriateness: “I have made referrals out of common sense for kids that are obviously in dire need of this service, not from the PACT flag.” Notably, only 12 participants explicitly named at least one of the correct eligibility criteria (i.e., run away or kicked out of home; family conflict); meanwhile, most indicated that active homelessness or the perceived risk of homelessness were clear indicators of service need despite these items not being included on the standardized risk assessment that was (in theory) used to flag instability risk. In addition to issues with eligibility determination, a demanding workload was commonly cited as impacting screening practices: “I haven't had the mind energy or bandwidth to try and figure out who would be eligible.”

Theme 6: Improvement Suggestions Centered on Strategies for Successful Implementation

Building on perceived barriers to implementation and programmatic success, most JPCs offered suggestions for program improvement. JPCs felt that the program could benefit from a greater level of awareness of the program among court staff, and specifically that it offers unique services to youth/families. Suggestions included developing and distributing written materials about the program and services offered (e.g., flyers, pamphlets) and providing more trainings. JPCs also felt that screening practices and the referral process could be further streamlined to increase efficiency and effectiveness. Suggestions included replacing the referral form with less formal means of communication (emails, phone calls, in-person conversations); relying on the navigator to complete the referrals; integrating a mechanism in the court database to help JPCs quickly assess case eligibility; and implementing a referral reminder system. In response to their own awareness of the current limitations of the service landscape, JPCs also indicated that HSYNC should offer more services (e.g., basic needs, housing) and better address accessibility issues (e.g., provide bus fare). Finally, several JPCs indicated a preference that navigators provide more case updates following a referral.

Discussion

This cross-sectional study evaluated the usability of the HSYNC program from the perspective of JPCs. Usability testing is crucial in early program implementation to identify necessary improvements for functionality and efficiency prior to taking a new program to scale (Corrao et al., 2010; Lyon et al., 2019). Following usability testing recommendations (Walji et al., 2014), we used multiple methods to capture a comprehensive view of HSYNC's implementation challenges and successes from the JPC perspective. Survey results showed good overall usability, with a mean ISUS score of 80 and generally high mean ratings on ISUS items (all but one item rated above 4.0, which indicates strong agreement). The lowest rated item was related to referral frequency, while the highest ratings were for ease of the referral process and lack of need for external implementation support. Open-ended survey responses supported these findings, highlighting the navigator and the simple referral process as key facilitators of usability.

While a mean ISUS score of 80 suggests good usability, it also indicates room for improvement. Information gleaned from the interviews provided greater insight into implementation barriers and areas for improvement that were not readily apparent in the survey data. For example, screening for program eligibility and contextual factors like the COVID-19 pandemic were identified as common implementation challenges not captured via the ISUS that likely impacted referral frequency. Interviews revealed additional individual and contextual factors that clarified why certain aspects of the program were more challenging for JPCs to implement, alongside concrete strategies for addressing implementation barriers and increasing program usability. Interviews also complemented survey findings, indicating that HSYNC and the navigator integrated well with probation practices and were seen as valuable additions to existing court services. Together, findings from these complementary data hold important implications for program usability and sustainability (discussed further below).

Drivers of Program Usability

The high degree of usability reported by JPCs via survey and interview responses was particularly noteworthy given common barriers court staff face when adopting new programs (e.g., high workload burden, low perceived value of new programs compared to usual practices). The HSYNC model was designed in collaboration with court staff and community providers to reduce barriers that could interrupt successful implementation. Although our analysis focused on evaluating perceived usability and not the impact of co-design per se, our findings suggest that the use of a co-design approach may have reduced anticipated barriers (e.g., program was perceived as value added, referral process was low-barrier and integrated well with existing practices, having a designated linkage navigator meant little-to-no workload increase for JPCs) and likely contributed to a higher degree of usability and acceptability than would be the case if the program was not designed collaboratively with court staff. Research evaluating the effectiveness of co-design as an implementation strategy for improving HSYNC usability could support these conclusions.

The housing navigators—and their unique approach to working with JPCs, youth, and families—was key to implementation success. Sixty percent of JPCs credited the navigator with their knowledge and understanding of program processes and services offered. In the interviews, JPCs highlighted the navigator's accessibility, responsive and timely communication, proactive support (e.g., helping JPCs identify appropriate referrals), and collaboration on cases as crucial factors. These findings align with prior studies showing that similar navigator practices enhance implementation success and positive client outcomes (Dir et al., 2023; Elkington et al., 2023; Markoulakis et al., 2019). Research also underscores the importance of navigator personality and relational qualities for program effectiveness (Phillips et al., 2014), a theme reaffirmed in our study. Courts considering adopting HSYNC or similar linkage programs should consider an individual's practice philosophy as well as personal attributes—like relatability, kindness, and dedication to helping others—when selecting navigators/linkage specialists.

The referral process was another program component that emerged as a strong facilitator of implementation success from the JPC perspective. Most JPCs agreed that the referral process was low-barrier—a simple, easy, and smooth process with a user-friendly referral form. The perceived ease of the referral process was likely due, in part, to having the navigator physically located in the same building as JPCs. In the qualitative interviews, JPCs noted that having the navigator co-located with probation facilitated regular communication about the referral process and identification of appropriate referrals. Combining a court-located linkage specialist with a low-barrier referral process that is well-aligned with existing practice can increase the likelihood that JPCs will make referrals, which in turn, is likely to increase service uptake.

The perceived usability and acceptability documented here have since been further evidenced by the collective action the pilot courts and their community partner organizations took to sustain funding for the program with the Washington State legislature after the pilot period ended. When HSYNC was initially implemented, it was funded by a short-term mix of philanthropic and public dollars. As a result of collective efforts, however, HSYNC is now funded in six counties, and the state is considering scaling the program to all counties. Prior to taking the program to scale, program improvements to address identified implementation challenges are needed. This will ensure the program is feasible, acceptable, and effective for all users (i.e., probation counselors, navigators, community providers, youth, caregivers).

Implications for Program Improvement

We found that the ISUS item on referral frequency had the lowest mean rating, indicating infrequent referrals to HSYNC. Interviews revealed that JPCs struggled with referrals, in part, due to organizational factors such as staff shortages, high caseloads, and competing service/referral priorities. These common challenges for court staff (Huang et al., 2020; Knight et al., 2019; Wasserman et al., 2021) have only worsened since the onset of the COVID-19 pandemic (Lockwood et al., 2023). In our study, most JPCs reported that COVID-19 hindered their ability to prioritize HSYNC referrals. To increase referrals, JPCs recommended distributing more information about the program, offering more trainings, and implementing a referral reminder system. Efforts to reduce caseloads and emphasize housing needs as a service priority may also increase referrals (Knight et al., 2019; Wasserman et al., 2021). Aligning HSYNC processes with universal design principles (Burgstahler, 2021), such as offering flexible training modalities, can reduce implementation barriers caused by rapid, unpredictable changes like those resulting from the COVID-19 pandemic.

Improving referral practices requires effective screening and assessment of service needs (Belenko et al., 2017). Screening for program eligibility emerged as a significant challenge, with over half of JPCs uncertain about eligibility criteria and program offerings. Professional biases and deviations from the standardized risk assessment may have compromised eligibility determinations. For instance, some JPCs used perceived risk of homelessness as a criterion, while others viewed homelessness as a choice by unhappy youth, leading to uncertainty about program fit. Proper screening and need identification are strong predictors of referral, which impacts service and treatment uptake (Dennis et al., 2019). To improve program success, JPCs need further education on program offerings and training/support to ensure correct use of the standardized assessment tool while guarding against implicit bias. Staff may also benefit from professional development opportunities to ensure clinical decision-making, when used, aligns with best practices. To reduce workload burden, JPCs suggested automating risk flagging in the data management system, which aligns with recommendations for streamlined program implementation in court settings (Dennis et al., 2019).

There is growing recognition that referral-based linkage programs are needed to address youth housing needs beyond traditional court services. However, programs like HSYNC can only be effective if community services are available. Housing shortages, unaffordable housing, and lack of shelter beds are representative of a crisis that makes it impossible to meet housing needs. Most JPCs were aware of current housing limitations and expressed a desire for HSYNC to provide actual housing, not just housing-adjacent services. Ultimately, increasing housing availability and removing structural barriers like poverty and rental discrimination are essential to reduce homelessness among court-involved youth. While this is beyond HSYNC's scope, ensuring navigators are knowledgeable and connected to community housing agencies will improve housing connections for youth.

Study Limitations

This study had several limitations. First, the sample size was small comprising participants from four pilot sites in one northwestern state, which may limit generalizability of findings to other probation counselors or juvenile court contexts. It is possible that individual, organizational, and systemic factors influencing JPC perspectives of HSYNC in our study differ from those in other states or jurisdictions. Additionally, the limited number of JPCs at each site, with one site having only one participant, prevented examination or reporting of differences in implementation experiences across sites due to participant confidentiality concerns in the qualitative analysis. Increasing the sample size in future evaluations would help clarify whether reported implementation challenges are specific to certain court contexts or prevalent across sites, thereby providing valuable insights for program improvement efforts. Furthermore, the participation rate was less than 50% and the sampling method might have yielded a sample that was more engaged with or had positive attitudes towards court service innovations, including the HSYNC program, potentially introducing bias. Additionally, we were unable to report the demographic characteristics of the qualitative sample and could not ascertain the extent of overlap between the survey and interview samples, which may have influenced differences in findings between the quantitative and qualitative portions of the evaluation. However, there was likely significant overlap because we targeted JPCs who were part of the HSYNC pilot for both survey and interview participation.

Conclusion

The persistent intersection of youth housing instability and court involvement calls for better coordination between juvenile courts and community agencies providing housing-related services. Our study suggests that, from the perspective of JPCs, HSYNC offers a promising cross-system approach to address housing-related needs by linking court-involved youth and their families to community-based services. An evaluation of the effectiveness of HSYNC for improving youth outcomes will be needed alongside program improvements to address identified implementation challenges to ensure the program is feasible, acceptable, and effective for all users (i.e., probation counselors, navigators, community providers, youth, caregivers) prior to taking the program to scale.

Footnotes

Acknowledgments

The authors are grateful to the King County, Kitsap County, Okanogan County, and Snohomish County juvenile courts for their willingness to pilot test the HSYNC program and for assisting the evaluation team with participant recruitment. This study would not have been possible without the individual JPCs who graciously agreed to share their experiences and perspectives of HSYNC for this evaluation. The authors would also like to thank Reed Klein, MSW for his feedback and suggestions on earlier versions of this manuscript, and Joane Gonzalez for her technical support during manuscript preparation.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Raikes Foundation, Block-Leavitt Foundation, and the Washington State Department of Commerce.

ORCID iDs

Asia S. Bishop

Sarah C. Walker

Kristin Vick

Taquesha Dean

References

ACYF. (2016). Administration for Children and Families Family and Youth Services Bureau: Street Outreach Program. U. S. Department of Health and Human Services. https://www.acf.hhs.gov/sites/default/files/documents/fysb/data_collection_study_final_report_street_outreach_program.pdf

Bangor

Kortum

P. T.

Miller

J. T.

(2008). An empirical evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574–594. https://doi.org/10.1080/10447310802205776

Bangor

Kortum

Miller

(2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of User Experience, 4(3), 114–123. https://uxpajournal.org/determining-what-individual-sus-scores-mean-adding-an-adjective-rating-scale/

Becan

J. E.

Fisher

J. H.

Johnson

I. D.

Bartkowski

J. P.

Seaver

Gardner

S. K.

Aarons

G. A.

Renfro

T. L.

Muiruri

Blackwell

Piper

K. N.

Wiley

T. A.

Knight

D. K.

(2020). Improving substance use services for juvenile justice-involved youth: Complexity of process improvement plans in a large scale multi-site study. Administration and Policy in Mental Health and Mental Health Services Research, 47(4), 501–514. https://doi.org/10.1007/s10488-019-01007-z

Belenko

Knight

Wasserman

G. A.

Dennis

M. L.

Wiley

Taxman

F. S.

Oser

Dembo

Robertson

A. A.

Sales

(2017). The juvenile justice behavioral health services cascade: A new framework for measuring unmet substance use treatment services needs among adolescent offenders. Journal of Substance Abuse Treatment, 74, 80–91. https://doi.org/10.1016/j.jsat.2016.12.012

Braun

Clarke

(2022). Thematic analysis: A practical guide. SAGE.

Brooke

(1996). SUS: A quick and dirty usability scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability evaluation in industry (pp. 184–194). CRC Press.

Burgstahler

(2021). Universal design: Process, principles, and applications. https://www.washington.edu/doit/sites/default/files/atoms/files/Universal_Design_04_12_21.pdf

Clanchy

Mitchell

Mulholland

Jurd

Kendall

Lloyd

D. G.

Palipana

Pizzolato

Shirota

(2024). Towards co-design of rehabilitation technologies: A collaborative approach to prioritize usability issues. Frontiers in Rehabilitation Sciences, 5, 1–14. https://doi.org/10.3389/fresc.2024.1302179

10.

CoLab. (n.d.). Research brief: Housing stability for youth in courts: A “cash-plus” model for legally involved youth. https://static1.squarespace.com/static/5935ee95893fc011586f1304/t/6572115cf0d03278310eb080/1701974367879/Housing+Stability+for+Youth+in+Courts_brief.pdf

11.

Columbia Legal Services. (2015). Falling through the gaps: How a stay in detention can lead to youth homelessness. https://docplayer.net/80908085-Falling-through-the-gaps-how-a-stay-in-detention-can-lead-to-youth-homelessness-december-2015.html

12.

Corrao

N. J.

Robinson

A. G.

Swiernik

M. A.

Naeim

(2010). Importance of testing for usability when selecting and implementing an electronic health or medical record system. JCO Oncology Practice, 6(3), 120–124. https://doi.org/10.1200/JOP.200017

13.

Cox

Rhoades

Wenzel

Lahey

Henwood

(2021). Does the timing of incarceration impact the timing and duration of homelessness? Evidence from “The Transitions to Housing” study. Justice Quarterly, 38(6), 1070–1094. https://doi.org/10.1080/07418825.2019.1709883

14.

Danielson

Mayfield

Callie

Felver

(2020). Homelessness among youth exiting systems of care in Washington state. https://www.dshs.wa.gov/ffa/rda/research-reports/homelessness-among-youth-exiting-systems-care-washington-state

15.

Dennis

M. L.

Smith

C. N.

Belenko

Knight

McReynolds

Rowan

Dembo

DiClemente

Robertson

Wiley

(2019). Operationalizing a behavioral health services cascade of care model: Lessons learned from a 33-site implementation in juvenile justice community supervision. Federal Probation, 83(2), 52–64. https://www.uscourts.gov/federal-probation-journal/2019/09/operationalizing-behavioral-health-services-cascade-care-model

16.

Dir

A. L.

Pederson

Khazvand

Schwartz

Wiehe

S. E.

Aalsma

M. C.

(2023). Caregiver and juvenile justice personnel perspectives on challenges and importance of caregiver engagement and the potential utility of a peer navigator program in the juvenile justice system. Health & Justice, 11(1), Article 30. https://doi.org/10.1186/s40352-023-00231-y

17.

Dopp

Parisi

Munson

Lyon

(2019). A glossary of user-centered design strategies for implementation experts. Translational Behavioral Medicine, 9(6), 1057–1064. https://doi.org/10.1093/tbm/iby119

18.

Dopp

Parisi

Munson

Lyon

(2020). Aligning implementation and user-centered design strategies to enhance the impact of health services: Results from a concept mapping study. Implementation Science Communications, 1(17), 1–13. https://doi.org/10.1186/s43058-020-00020-w

19.

Elkington

K. S.

Lee

Brooks

Watkins

Wasserman

G. A.

(2020). Falling between two systems of care: Engaging families, behavioral health and the justice systems to increase uptake of substance use treatment in youth on probation. Journal of Substance Abuse Treatment, 112, 49–59. https://doi.org/10.1016/j.jsat.2020.01.008

20.

Elkington

K. S.

Robson

Sichel

C. E.

Lee

Wasserman

G. A.

(2023). Family connect: The pilot test of a cross-systems behavioral health treatment referral and linkage intervention for youth on probation. Criminal Justice and Behavior, 50(1), 22–39. https://doi.org/10.1177/00938548221121130

21.

Fadus

Squeglia

Valadez

Tomko

Bryant

Gray

(2019). Adolescent substance use disorder treatments: An update on evidence-based strategies. Current Psychiatry Reports, 21(96), 1–10. https://doi.org/10.1007/s11920-019-1086-0

22.

Farrell

J. L.

Young

D. W.

Taxman

F. S.

(2011). Effects of organizational factors on use of juvenile supervision practices. Criminal Justice and Behavior, 38(6), 565–583. https://doi.org/10.1177/0093854811401786

23.

Gardner

E. M.

McLees

M. P.

Steiner

J. F.

del Rio

Burman

W. J.

(2011). The spectrum of engagement in HIV care and its relevance to test-and-treat strategies for prevention of HIV infection. Clinical Infectious Diseases, 52(6), 793–800. https://doi.org/10.1093/cid/ciq243

24.

Gultekin

L. E.

Brush

B. L.

Ginier

Cordom

Dowdell

E. B.

(2019). Health risks and outcomes of homelessness in school-age children and youth: A scoping review of the literature. The Journal of School Nursing, 36(1), 10–18. https://doi.org/10.1177/1059840519875182

25.

Huang

Gardner

Piper

K. N.

Coleman

A. S.

Becan

J. E.

Robertson

A. A.

Elkington

K. S.

(2020). Bridging systems to implement HIV/STI programming within the juvenile justice system: Strengths and challenges of a local change team approach. AIDS Education and Prevention, 32(5), 432–453. https://doi.org/10.1521/aeap.2020.32.5.432

26.

International Organization for Standardization. (2018). Ergonomic of human-system interaction—Part 11: Usability definitions and concepts. https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en

27.

Knight

D. K.

Joe

G. W.

Morse

D. T.

Smith

Knudsen

Johnson

Wasserman

G. A.

Arrigona

McReynolds

L. S.

Becan

J. E.

Leukefeld

Wiley

T. R. A.

(2019). Organizational context and individual adaptability in promoting perceived importance and use of best practices for substance use. The Journal of Behavioral Health Services & Research, 46(2), 192–216. https://doi.org/10.1007/s11414-018-9618-7

28.

Krippendorff

(2019). Content analysis: An introduction to its methodology. SAGE.

29.

Kushniruk

Nohr

(2016). Participatory design, user involvement and health IT evaluation. Evidence-Based Health Informatics, 222, 139–151. https://doi.org/10.3233/978-1-61499-635-4-139

30.

Ladner-Graham

(2019). Empirical support for Multidimensional Treatment Foster Care (MTFC): A critical review (Publication No. 1714) [Doctoral dissertation]. University of Mississippi]. Electronic Theses and Dissertations. https://egrove.olemiss.edu/etd/1714

31.

Littell

Pigott

Nilsen

Roberts

Labrum

(2023). Functional family therapy for families of youth (age 11-18) with behaviour problems: A systematic review and meta-analysis. Campbell Systematic Review, 19(3), Article e1324. https://doi.org/10.1002/cl2.1324

32.

Lockwood

Viglione

Peck

J. H.

(2023). COVID-19 and juvenile probation: A qualitative examination of emergent challenges and useful strategies. Criminal Justice and Behavior, 50(1), 56–75. https://doi.org/10.1177/00938548211046977

33.

Lyon

A. R.

(2016). Intervention Usability Scale (IUS). University of Washington.

34.

Lyon

A. R.

Bruns

E. J.

(2019). User-centered redesign of evidence-based psychosocial interventions to enhance implementation – hospitable soil or better seeds? JAMA Psychiatry, 761(1), 3–4. https://doi.org/10.1001/jamapsychiatry.2018.3060

35.

Lyon

A. R.

Coifman

Cook

McRee

Liu

F. F.

Ludwig

Dorsey

Koerner

Munson

S. A.

McCauley

(2021). The cognitive walkthrough for implementation strategies (CWIS): A pragmatic method for assessing implementation strategy usability. Implementation Science Communications, 2(1), Article 78. https://doi.org/10.1186/s43058-021-00183-0

36.

Lyon

A. R.

Koerner

Chung

(2020). Usability evaluation for evidence-based psychosocial interventions (USE-EBPI): A methodology for assessing complex intervention implementability. Implementation Research and Practice, 1, 1–17. https://doi.org/10.1177/2633489520932924

37.

Lyon

A. R.

Munson

S. A.

Renn

B. N.

Atkins

D. C.

Pullman

M. D.

Friedman

Arean

P. A.

(2019). Use of human-centered design to improve implementation of evidence-based psychotherapies in low-resource communities: Protocol for studies applying a framework to assess usability. JMIR Research Protocols, 8(10), Article e14990. https://doi.org/10.2196/14990

38.

Lyon

A. R.

Pullmann

M. D.

Jacobson

Osterhage

Al Achkar

Renn

B. N.

Munson

S. A.

Arean

P. A.

(2021). Assessing the usability of complex psychosocial interventions: The intervention usability scale. Implementation Research and Practice, 2, 1–9. https://doi.org/10.1177/2633489520987828

39.

MacCarthy

Hoffmann

Ferguson

Nunn

Irvin

Bangsberg

Gruskin

Dourado

(2015). The HIV care cascade: Models, measures and moving forward. Journal of the International AIDS Society, 18(1), Article 19395. https://doi.org/10.7448/IAS.18.1.19395

40.

Markoulakis

Chan

Levitt

(2019). Identifying the key features and outcomes of family navigation services for mental health and/or addictions concerns: A Delphi study. BMC Health Services Research, 19(1), 137–137. https://doi.org/10.1186/s12913-019-3968-6

41.

Mayfield

Black

Sprague

Felver

B. E. M.

(2017). Housing status of youth exiting foster care, behavioral health and criminal justice systems. https://www.dshs.wa.gov/ffa/rda/research-reports/housing-status-youth-exiting-foster-care-behavioral-health-and-criminal-justice-systems

42.

McKinnon

I. I.

Krause

K. H.

Robin

King

Leon-Nguyen

Zavala

Suarez

N. A.

Lim

Smith-Grant

Underwood

J. M.

(2023). Experiences of unstable housing among high school students—Youth risk behavior survey, United States, 2021. MMWR Supplements, 72(1), 29–36. https://doi.org/10.15585/mmwr.su7201a4

43.

Miles

M. B. H.

Huberman

A. M.

Saldana

(2020). Qualitative data analysis: A methods sourcebook (4th ed.). Sage Publications.

44.

Moldestad

Petrova

Tirtanadi

Mishra

Rajan

Sayre

Fortney

Schacht Reisinger

(2023). Improving the usability of written exposure therapy for therapists in the Department of Veterans Affairs Telemental Health: Formative study using qualitative and user-centered design methods. JMIR Formative Research, 7, Article e47189. https://doi.org/10.2196/47189

45.

Morton

M. H.

Dworsky

Matjasko

J. L.

Curry

S. R.

Schlueter

Chavez

Farrell

A. F.

(2018). Prevalence and correlates of youth homelessness in the United States. Journal of Adolescent Health, 62(1), 14–21. https://doi.org/10.1016/j.jadohealth.2017.10.006

46.

Mugavero

M. J.

Amico

K. R.

Horn

Thompson

M. A.

(2013). The state of engagement in HIV care in the United States: From cascade to continuum to control. Clinical Infectious Diseases, 57(8), 1164–1171. https://doi.org/10.1093/cid/cit420

47.

Neuendorf

K. A.

(2017). The content analysis guidebook (2nd ed.). SAGE.

48.

Owens

Ngo

Grinnell

Pearlman

Bekemeier

Walker

(2022). Co-producing evidence-informed criminal legal re-entry policy with the community: An application of policy codesign. Bristol University Press Digital, 18(2), 356–375. https://doi.org/10.1332/174426421X16445109542161

49.

Palinkas

Spear

Mendon

Villamar

Reynolds

Green

Olson

Adade

Brown

C. H.

(2020). Conceptualizing and measuring sustainability of prevention programs, policies, and practices. Translational Behavioral Medicine, 10(1), 136–145. https://doi.org/10.1093/tbm/ibz170

50.

Perlman

Willard

Herbers

J. E.

Cutuli

J. J.

Eyrich Garg

K. M.

(2014). Youth homelessness: Prevalence and mental health correlates. Journal of the Society for Social Work & Research, 5(3), 361–377. https://doi.org/10.1086/677757

51.

Phillips

Nonzee

Tom

Murphy

Hajjar

Bularzik

Dong

Simon

M. A.

(2014). Patient navigators’ reflections on the navigator-patient relationship. Journal of Cancer Education, 29(2), 337–344. https://doi.org/10.1007/s13187-014-0612-3

52.

Prince

D. M.

Vidal

Okpych

Connell

C. M.

(2019). Effects of individual risk and state housing factors on adverse outcomes in a national sample of youth transitioning out of foster care. Journal of Adolescence, 74, 33–44. https://doi.org/10.1016/j.adolescence.2019.05.004

53.

Puzzanchera

Hockenberry

Sickmund

(2022). Youth and the Juvenile Justice System: 2022 National Report. Office of Juvenile Justice and Delinquency Prevention. https://ojjdp.ojp.gov/library/publications/youth-and-juvenile-justice-system-2022-national-report

54.

Puzzanchera

Sladky

Kang

(2023). Easy access to juvenile court statistics: 1985-2021. Office of Juvenile Justice and Delinquency Prevention.

55.

Quirouette

Frederick

Hughes

Karabanow

Kidd

(2016). ‘Conflict with the Law’: Regulation & homeless youth trajectories toward stability. Canadian Journal of Law and Society, 31(3), 383–404. https://doi.org/10.1017/cls.2016.26

56.

Rieder

J. M. D. M. S.

Meissner

P. M.

Shankar

V. P.

Moon

J.-Y. P.

Wylie-Rosett

J. E.

(2020). A cascade model for school wellness programming: A feasibility case study. Health Behavior and Policy Review, 7(1), 38–50. https://doi.org/10.14485/HBPR.7.1.5

57.

Rogers

E. M.

(1995). Diffusion of innovations (4th ed.). Simon & Schuster.

58.

Sanders

Stappers

(2008). Co-creation and the new landscapes of design. International Journal of CoCreation in Design and the Arts, 4(1), 5–18. https://doi.org/10.1080/15710880701875068

59.

Scott

C. K.

Dennis

M. L.

Grella

C. E.

Funk

R. R.

Lurigio

A. J.

(2019). Juvenile justice systems of care: Results of a national survey of community supervision agencies and behavioral health providers on services provision and cross-system interactions. Health & Justice, 7(1), Article 11. https://doi.org/10.1186/s40352-019-0093-x

60.

Taxman

F. S.

Henderson

Young

Farrell

(2014). The impact of training interventions on organizational readiness to support innovations in juvenile justice offices. Administration and Policy in Mental Health and Mental Health Services Research, 41(2), 177–188. https://doi.org/10.1007/s10488-012-0445-5

61.

Vargas

Whelan

Brimblecombe

Allender

(2022). Co-creation, co-design, and co-production for public health: A perspective on definitions and distinctions. Public Health Research and Practice, 32(2), Article e3222211. https://doi.org/10.177061/phrp3222211

62.

Walji

Kalenderian

Piotrowski

Tran

Kookal

Tokede

White

Vaderhobli

Ramoni

Stark

Kimmes

Lagerweij

Patel

(2014). Are three methods better than one? A comparative assessment of usability evaluation methods in an EHR. International Journal of Medical Informatics, 83(5), 361–367. https://doi.org/10.1016/j.ijmedinf.2014.01.010

63.

Walker

S. C.

Valencia

Bishop

Irons

Gertseva

(2018). Developing a coordinated youth housing stability program for juvenile courts. Cityscape, 20(3), 117–137. https://www.jstor.org/stable/26524875

64.

Wasserman

G. A.

McReynolds

L. S.

Taxman

F. S.

Belenko

Elkington

K. S.

Robertson

A. A.

Dennis

M. L.

Knight

D. K.

Knudsen

H. K.

Dembo

Ciarleglio

Wiley

T. R. A.

(2021). The missing link(age): Multilevel contributors to service uptake failure among youths on community justice supervision. Psychiatric Services, 72(5), 546–554. https://doi.org/10.1176/appi.ps.202000163

65.

Wasserman

G. A.

McReynolds

L. S.

Whited

A. L.

Keating

J. M.

Musabegovic

Huo

(2008). Juvenile probation officers’ mental health decision making. Administration and Policy in Mental Health and Mental Health Services Research, 35(5), 410–422. https://doi.org/10.1007/s10488-008-0183-x

Usability of a cross-system housing stability program for juvenile courts: A multimethod study of probation counselor perspectives

Abstract

Background

Method

Results

Conclusions

Plain Language Summary

Keywords

Introduction

Co-designing HSYNC

Method

Study Procedures

Participants

Measures

Survey Measures

Semi-Structured Interview Guide

Analytic Approach

Survey Data

Qualitative Interviews

Results

Quantitative Usability Ratings

Qualitative Perspectives on HSYNC Usability

Theme 1: HSYNC Was Value-Added

Theme 2: HSYNC Integrated Well With Probation

Theme 3: The Navigator's Approach Facilitated Successful Implementation

Theme 4: Individual and Contextual Factors Served as Barriers to Implementation Success

Theme 5: Screening for Program Eligibility Was the Most Challenging Aspect of Implementation

Theme 6: Improvement Suggestions Centered on Strategies for Successful Implementation

Discussion

Drivers of Program Usability

Implications for Program Improvement

Study Limitations

Conclusion

Footnotes

Acknowledgments

Declaration of Conflicting Interests

Funding

ORCID iDs

References