Abstract
Background
Youth in contact with the juvenile legal system experience disproportionate rates of homelessness. While system contact is a critical intervening point, juvenile courts do not typically offer housing services. One solution is to refer youth to evidence-based, community-based services to meet housing-related needs, but a myriad of individual and organizational factors often impede court staff from making such referrals. Housing Stability for Youth in Courts (HSYNC) is a novel, cross-system service linkage model for court-involved youth facing housing instability. HSYNC was developed using codesign as a strategy to improve usability and incorporated evidence on juvenile court linkage strategies effective at increasing service referrals and improving outcomes.
Method
The current study utilized a multimethod approach to evaluate the usability of HSYNC from the perspective of juvenile probation counselors (JPCs) as one group of end-users. As part of the pilot implementation study, survey (N = 15) and semi-structured interview (N = 20) data were obtained from JPCs from four juvenile courts in one northwestern state.
Results
Survey results indicated good usability for HSYNC (mean implementation strategy usability scale score of 80). Themes from the interviews complemented survey findings, pointing to HSYNC as a value-added program that integrated well with existing probation practices. The housing navigator's approach was highlighted as a strong facilitator of program usability. Themes also provided greater insight into implementation barriers (i.e., individual and contextual factors) and areas for improvement (e.g., eligibility screening process).
Conclusions
Drivers of program usability and implications for program improvement to address identified implementation challenges to ensure the HSYNC model is effective and sustainable long-term are discussed.
Plain Language Summary
Court-involved youth experience disproportionate rates of homelessness, yet housing services are not traditionally offered by juvenile courts. Court staff, such as probation staff, can refer youth to community-based services to meet their housing needs. Unfortunately, many factors complicate whether these referrals are made, including staff feeling overburdened by other work obligations and uncertainty about whether referring youth to community services fits within their job responsibilities. In this study, we surveyed and interviewed probation staff who were responsible for implementing a new referral-based housing stability program—Housing Stability for Youth in Courts (HSYNC)—to understand whether the program addressed previously identified barriers to referral-making. A unique feature of HSYNC is that there is a dedicated housing navigator, who works as a “linkage specialist” to support probation staff in making referrals while working directly with youth and families to access housing-related services in the community. Our findings suggest that, from the perspective of probation staff, HSYNC is considered to be a user-friendly program that is well-aligned with existing court practices. The housing navigator and their unique approach to working with probation staff, youth, and families was perceived as the strongest facilitator of implementation success; meanwhile, screening for program eligibility was the biggest barrier, pointing to the need for improvement before HSYNC is widely implemented. The persistent intersection of youth housing instability and court involvement calls for better coordination between juvenile courts and community agencies providing housing-related services. From the probation perspective, HSYNC offers a viable solution to meeting youths’ housing needs without creating additional workload burden for court staff.
Introduction
National estimates indicate that 3%–6% of youth aged 13–17 in the United States experience unstable housing or active homelessness (McKinnon et al., 2023; Morton et al., 2018; Perlman et al., 2014). Housing instability and homelessness during adolescence are associated with a myriad of adverse social and health outcomes, including poorer physical, mental, and sexual health as well as increased risk of food insecurity, substance use, victimization, and suicidality (Gultekin et al., 2019; McKinnon et al., 2023). The harmful effects of homelessness are disproportionately felt by youth of color, those experiencing poverty, females, and gender- and sexually diverse youth (McKinnon et al., 2023). Addressing youth homelessness is a critical public health imperative with important implications for reducing broader population health disparities.
A disproportionate number of youth experiencing unstable housing or homelessness are also in contact with the juvenile legal system. Nearly 78% of runaway and homeless youth have had at least one interaction with the police (ACYF, 2016), 62% have been arrested (Morton et al., 2018), and 44%–50% have had at least one stay in juvenile detention, jail, or prison (ACYF, 2016; Morton et al., 2018). System contact itself also increases the risk of homelessness (Prince et al., 2019; Quirouette et al., 2016) as many youth exit the system into unstable housing situations (Mayfield et al., 2017). In one study, only 65% of detained youth named a parent or relative they could live with when they were released (Columbia Legal Services, 2015). Nearly 30% of youth exiting juvenile residential or adult correctional facilities may be homeless within 12 months (Danielson et al., 2020). Adolescent incarceration is also associated with earlier homelessness and longer durations of homelessness (Cox et al., 2021).
The overlap of currently unhoused and youth at risk of housing instability in the juvenile legal system is significant, making juvenile courts an important inflection point for addressing housing instability to prevent homelessness and associated adverse outcomes. As a point of contact for over half of youth entering the legal system after arrest (Puzzanchera et al., 2022, 2023), probation departments are well-positioned to reach a large proportion of youth who would otherwise remain disconnected from services. Because juvenile courts do not typically offer housing-related services internally (e.g., Walker et al., 2018), probation staff would need to identify and refer youth to these services in the community.
Previous studies find that the adoption and sustainment of cross-system referral strategies is low, typically below 50% for referrals from probation departments to non-legally mandated community-based services (Dennis et al., 2019; Knight et al., 2019; Wasserman et al., 2021). Several factors present barriers to adoption, including high workload demands (Huang et al., 2020; Wasserman et al., 2021); perceptions of referral-making as relevant (or not) to their role (Knight et al., 2019); staff adaptability or proclivity to changing current assessment and referral practices (Farrell et al., 2011; Knight et al., 2019); organizational climate surrounding implementation of health innovations (Knight et al., 2019; Scott et al., 2019; Taxman et al., 2014; Wasserman et al., 2021); degree of communication and collaboration with community providers (Elkington et al., 2020; Scott et al., 2019; Taxman et al., 2014); data sharing across systems (Dennis et al., 2019); and staff perceptions of youths’ treatment needs (Wasserman et al., 2008).
One strategy to improve the adoption of cross-system referral strategies is to increase usability (Dopp et al., 2019; Lyon & Bruns, 2019). Usability in health services describes the extent to which the intended users of a program can implement the program easily while maintaining effectiveness, efficiency, and satisfaction (International Organization for Standardization, 2018; Lyon, Coifman, et al., 2021). Usability is both a complex implementation strategy and upstream implementation determinant (Lyon & Bruns, 2019; Lyon, Coifman, et al., 2021). Improving the usability of programs can be facilitated by co-design (Clanchy et al., 2024; Dopp et al., 2020; Kushniruk & Nohr, 2016), a method that engages intended users directly in the design and development of new service innovations to increase alignment with user needs (Sanders & Stappers, 2008; Vargas et al., 2022). The current study evaluates the usability (implementation determinant) of a novel, cross-system court referral program to address youth housing instability risk, which was developed using co-design (an implementation strategy) to enhance usability among probation staff.
Co-designing HSYNC
Housing Stability for Youth in Courts (HSYNC) is a cross-system referral (“linkage”) program that uses system navigators and flexible funds to connect youth and their families to community-based resources to address housing needs (CoLab, n.d.; Walker et al., 2018). The approach used to design HSYNC mirrored the steps of the Discover, Design and Build, and Test (DDBT) framework for developing human-centered, evidence-based interventions as an implementation strategy (Lyon et al., 2019). The discover stage focuses on understanding the evidence base and potential contextual factors that could influence intervention success, often by gathering insights from stakeholders; the design and build stage focuses on collaborative intervention development, where insights are leveraged from the discovery phase to ensure the intervention and implementation aligns with stakeholder needs and practical considerations; and the test stage focuses on evaluating key implementation outcomes (Lyon et al., 2019).
The HSYNC design process involved capturing and synthesizing (1) the research evidence on health service cascade models (Gardner et al., 2011; MacCarthy et al., 2015; Mugavero et al., 2013) and behavioral health cascades in juvenile court settings specifically (Belenko et al., 2017; Elkington et al., 2023); (2) local knowledge of service needs through group and key informant interviews; and (3) systems expertise from multisector teams. The discovery stage of intervention development identified evidence that juvenile court service referral linkage strategies are effective at improving service referrals by reducing workload demands on probation staff (Elkington et al., 2020). Further, when referred to evidence-based interventions such as multidimensional treatment foster care (Ladner-Graham, 2019), evidence-based substance use disorder treatment (Fadus et al., 2019), and evidence-based family conflict interventions (Littell et al., 2023), youth recidivism outcomes improve. The HSYNC model was developed based on this evidence, with the design and build stage following a participatory co-design process (Owens et al., 2022; Sanders & Stappers, 2008) where multisector teams from two juvenile court jurisdictions were engaged in parallel development processes to identify common and unique needs across contexts. This process resulted in the development of a service linkage model for juvenile probation that integrates cash-plus service navigation (i.e., housing navigators and cash transfers) within a cascade of care model to increase cross-system referral to address housing instability risk.
The literature on service linkage models points to common barriers that can interrupt successful implementation in real-world settings, including the adoption of new screening tools into workflows (Rieder et al., 2020), low perceived relative advantage compared to usual practices (Dennis et al., 2019), and complex or difficult to understand processes (Becan et al., 2020; Dennis et al., 2019; Rieder et al., 2020). The HSYNC model was intentionally designed to reduce these barriers through the co-design process by using routine screening tools already in use by the juvenile courts for case management; training navigators to educate probation staff on HSYNC program benefits for their workload and youth outcomes; and by placing navigators in physical and/or virtual shared workspaces with probation staff to facilitate the referral process.
The current study represents the test stage of the DDBT framework. We aimed to assess whether the above program components reduced anticipated barriers using a usability evaluation framework given the recognition of intervention usability as an important determinant of implementation success (Lyon et al., 2020; Moldestad et al., 2023). Juvenile probation counselor (JPC) perceptions of the usability of the service linkage step of the broader HSYNC model is one part of a complex implementation system. However, consistent with many implementation science studies, we were specifically interested in isolating and studying usability as one potential implementation determinant. As recommended when testing the usability of health service innovations (Lyon et al., 2019; Walji et al., 2014), using multiple evaluation methods offers a more comprehensive understanding of usability challenges and successes than singular methods. As such, we quantitatively examined JPC usability ratings via survey responses to the Implementation Strategy Usability Scale (Lyon, 2016; Lyon, Pullmann, et al., 2021) and conducted qualitative interviews with JPCs to gain further insight into specific aspects of HSYNC that facilitated or hindered implementation, alongside suggestions for quality improvement.
Method
Study Procedures
All JPCs from four juvenile court pilot sites in one northwestern state with active caseloads at the time of the study were eligible to participate. Sites were approached to participate in the evaluation study approximately 1 year from the start of the program. Research staff obtained JPC email lists from court leadership and sent two emails through staff listservs to recruit for the study. The email contained a direct link to an online survey and included information about the opportunity to participate in a phone/video interview. JPCs indicating interest in the interview received a phone/video call to receive more information and complete informed consent, if willing. The semi-structured interviews were recorded by video and then transcribed using a professional transcription service. Fifteen eligible JPCs completed the online survey (response rate = 42%), and 20 JPCs elected to participate in interviews (response rate = 38%). Study procedures were approved by the University of Washington Institutional Review Board.
Participants
Table 1 provides JPC demographic characteristics for the survey sample (N = 15). On average, participants were 48 years old, 80% female, majority white (60%), and reported over 20 years of experience in the field (73%) including over 20 years in their current position (56%). We are unable to report demographics for the interview sample, as this information was destroyed prior to analysis in accordance with the study's IRB-approved data retention policy.
Survey Sample Demographic Characteristics (N = 15)
Racial/ethnic categories are not mutually exclusive.
Range = 34–60 years.
Measures
Survey Measures
Implementation Strategy Usability Scale (ISUS). The 10-item System Usability Scale (SUS: Brooke, 1996) was adapted to create the ISUS (Lyon, 2016; Lyon, Pullmann, et al., 2021). As Lyon, Pullmann, et al. (2021), note, the term “system” was replaced with “implementation strategy” in each item without further modifications to maintain consistency with the SUS operationalization of usability. In the current study, the terms “implementation strategy” were replaced with “the HSYNC navigator program” to fit the evaluation context. ISUS items range from 1 (strongly disagree) to 5 (strongly agree), with half of the items reversed-scored so that all items reflect a greater degree of usability. As recommended by scale developers (Lyon, 2016), the total usability score was calculated by multiplying the sum of individual item scores by 2.5 (possible range: 0–100).
ISUS Follow-up Questions. Four open-ended questions were included on the survey to give respondents the opportunity to expand on their ISUS ratings. Questions focused on what HSYNC offered clients beyond typical probation offerings, who or what helped JPCs understand the benefits of HSYNC, suggestions around simplifying the referral process, and ideas for improving HSYNC services.
Semi-Structured Interview Guide
The semi-structured interview guide was informed by the ISUS and explored JPC perspectives on the screening and referral process (e.g., “What worked best about the referral process?” “What was the most challenging part about the referral process?” “How well did the routine court flags identify youth who needed services?”), facilitators and barriers to program implementation (e.g., “What did you find most helpful about the HSYNC trainings?” “Did the services appear to coordinate well or conflict with court referred services?”), and program improvement suggestions (e.g., “What concerns or suggestions do you have for the program?”).
Analytic Approach
We undertook two data collection and analysis approaches—survey and semi-structured interviews—to complete the HSYNC usability evaluation. Findings from each approach are presented independently and then used in a complimentary manner to draw implications and conclusions regarding HSYNC usability (detailed in the Discussion).
Survey Data
The analysis of survey data proceeded in several steps. First, descriptive statistics were used to assess item functioning and Cronbach's alpha was used to assess ISUS scale reliability. Missing data were present in three cases. Because the ISUS score is computed as a summative measure, item-level missing data negatively biases the total scale score. Thus, we report item functioning and scale scores from the original data (with missing values) and from a new dataset where missing values were imputed by the item mean (termed “imputed data”). We then compared the imputed ISUS score against the current quality benchmark (Bangor et al., 2009). Analyses were run using SPSS version 27.
Responses to the open-ended follow-up questions were analyzed to supplement the quantitative data using a conceptual content analysis approach (Krippendorff, 2019; Neuendorf, 2017). Specifically, responses from each question were condensed into conceptually similar categories—thematic codes—that recurred across the data by a PhD-level researcher (first author). Codes were then quantified into response rates, and theme titles were developed to reflect the focus of each set of codes.
Qualitative Interviews
JPC interviews were analyzed following a thematic analysis approach (Braun & Clark, 2022). This included a deductive, iterative process of (1) coding the data using an a-priori coding framework, (2) using research team discussions to enhance credibility and trustworthiness (i.e., researcher triangulation; Miles et al., 2020), and (3) condensing codes and naming themes. The coding framework was developed by the study PI (second author) and included concepts from usability and diffusion of innovations frameworks (Lyon et al., 2019; Rogers, 1995). The coding framework was pilot tested with two members of the research team using three transcripts. Codes included relative advantage of the program compared to services offered elsewhere, observability of the benefits of the program, implementation barriers/facilitators, and suggestions for improvement. The codes were refined in team discussions where discrepancies in code definitions and the application of codes were resolved. Two master's level research assistants were trained on the coding framework then proceeded to manually code all transcripts using the established coding framework. Disagreements were discussed and negotiated until consensus was reached. Next, a PhD-level researcher (first author) grouped and condensed codes based on conceptual consistency into broader themes. Theme titles were developed to describe the focus of each set of codes and exemplar quotes were identified to illustrate thematic findings and give voice to participants.
Results
Quantitative Usability Ratings
Scores for the individual ISUS items included the full range for Items 2, 4, 7, 8, 10 and partial range for Items 1, 6, 9 (range: 2–5), and 3, 9 (range: 3–5) (Table 2). Differences in standard deviation values for the imputed items compared to the items from the original data (with missing values) were negligible suggesting bias from mean imputation was low. Mean ratings were high across items (in the 4-point range, translating to “agree to strongly agree”), except for Item 1 (referral frequency) which had the lowest mean rating of 3.33 (closer to a “neither agree nor disagree” translation). Notably, Item 3—referring to the navigator program was easy—had the highest mean rating with the smallest degree of variation in responses (M = 4.79, SD = 0.58, range = 3–5) suggesting that the HSYNC referral process was perceived as simple and straightforward by most JPCs.
ISUS Individual Item Descriptive Statistics and Scale Reliability for HSYNC (N = 15)
Note. For all items, 1 = strongly disagree; 2 = disagree; 3 = neither agree or disagree; 4 = agree; 5 = strongly agree.
α = Cronbach’s alpha measuring scale reliability. ISUS = Implementation Strategy Usability Scale; HSYNC = housing stability for youth in courts.
Missing values for Items 2, 3, 5, 6, 7, 8, and 10 were imputed by the item mean.
Items 2, 4, 6, 8, and 10 are reverse scored.
Calculated by multiplying the sum of individual item scores by 2.5 (possible range: 0–100). Not available for original sample due to item-level missingness.
The overall ISUS score demonstrated acceptable reliability (α = .73). ISUS ratings ranged from 55 to 100, with a mean of 80.09 (SD = 13.73) indicating excellent usability for HSYNC according to JPCs. A mean ISUS score of 80 surpasses the usability quality benchmark, where the ISUS cutoff of 70 is considered “acceptable” (Bangor et al., 2008, 2009).
Table 3 presents findings from a content analysis of the open-ended survey responses. JPCs reported that HSYNC offered a unique array of services that were not already available through the court, including housing assistance, mental health treatment, help with basic needs (health insurance, driver's license), and transportation. Approximately 60% of JPCs reported that the navigator was responsible for facilitating their knowledge and understanding of services offered through HSYNC. In line with the quantitative findings, most participants indicated that the HSYNC referral process was simple and straightforward (60%), with others recommending a shorter referral form or checklist to further simplify the referral process (27%). Several JPCs offered suggestions for improving HSYNC to better fit the court context, including providing JPCs with an updated list of services offered (20%) and case updates post referral (13%).
Content Analysis of Participant Responses to ISUS Follow-up Open-Ended Survey Questions (N = 15)
Note. Codes are not mutually exclusive. ISUS = Implementation Strategy Usability Scale; HSYNC = housing stability for youth in courts; JPC = juvenile probation counselor.
Qualitative Perspectives on HSYNC Usability
Six themes emerged from a thematic analysis of the JPC interviews (Table 4). Together, themes complemented survey findings while providing greater insight into implementation barriers and areas for improvement that were not readily apparent in the survey data.
Thematic Analysis of JPC Interviews (N = 20)
Note. JPC = juvenile probation counselor; HSYNC = housing stability for youth in courts.
Theme 1: HSYNC Was Value-Added
JPCs recognized the value of a housing stability program for court-involved youth and their families. All participants indicated that the program was beneficial and needed: “I think the reason it [HSYNC] works so good is because it's a needed program.” JPCs indicated that HSYNC offered a unique suite of services that were not available to youth and families otherwise. HSYNC services were also perceived as not conflicting with those already being offered by the court. One participant commented on how the navigator herself—beyond the services she offered—played an important therapeutic role: “When I make a referral, I know she [navigator] will be just another positive protective factor in a youth's life.” Several JPCs also referenced concern for program sustainability. As perceived value is a defining characteristic of program sustainability (Palinkas et al., 2020), JPC concerns about sustainability provide further evidence of the program as value-added from the JPC perspective.
Theme 2: HSYNC Integrated Well With Probation
HSYNC was perceived as well-integrated with existing probation practices. Most JPCs indicated that the referral process was smooth and easy, and it was not generally considered an additional workload burden: “The actual referral is incredibly easy […] it's really low barrier for us.” Having the navigator co-located with probation (in the same building) was perceived as convenient and facilitated regular communication and collaboration between JPCs and navigators. Two JPCs also reported that, because navigators operated independently from the court, they could be more flexible in the approaches they used to contact youth (e.g., social media), which was helpful to JPCs, particularly when traditional forms of engagement (e.g., phone calls) were difficult because the youth was experiencing active homelessness.
Theme 3: The Navigator's Approach Facilitated Successful Implementation
The navigator—and their unique approach to working with JPCs, youth, and families—was one of the strongest facilitators of implementation success from the JPC perspective. Most participants reported that navigators were accessible to JPCs, including making themselves available to answer questions and provide guidance on the referral process. JPCs also indicated that the navigators were responsive and timely in their communication, as well as proactive in providing implementation support to JPCs while ensuring the program was operating smoothly. Nearly half of the participants indicated that their navigator played a role in identifying appropriate referrals, which JPCs found helpful. JPCs also perceived the navigators as successful in their ability to connect effectively with youth and their families: “Once I referred kids, I had a lot of success which I really attribute to our navigator, with her ability to connect with kids.” JPCs attributed this success to the navigators making themselves available to youth and families, as well as certain characteristics and traits particular to their navigators (e.g., relatable, kind, dedicated): “She [navigator] was a nice, kind person that could relate with people.”
Theme 4: Individual and Contextual Factors Served as Barriers to Implementation Success
Perceived challenges to program implementation and service uptake included individual and contextual factors. At the individual level, assumptions and biases regarding why youth were experiencing homelessness (e.g., “they don't want to follow the rules of the parents’ home”) impacted whether JPCs believed youth met the eligibility criteria and/or whether they should be referred to the program: “even though we might get a hit on the PACT assessment, it might not register to us that this kid could use her [navigator] support because of whatever our implicit biases are.” Most JPCs identified youth and family characteristics as potential barriers to program engagement and service uptake. These included: youth disinterest in the program, problematic youth behavior, case type and status, criminal history, poverty, homelessness, parental substance abuse, and living in a rural area.
Contextual factors at the court and community levels also emerged as barriers to implementation success. JPCs highlighted several features of the court workplace environment that made assessment and referral to HSYNC challenging, including staff shortages, high caseloads, competing service priorities, and feeling pressured to refer to multiple programs. JPCs were also aware of service limitations that youth and families face, both in terms of what was available in the community and access issues. For instance, several JPCs indicated that there was a misconception (on the part of JPCs and youth/families) that HSYNC could provide actual housing: “I think that there was a common misconception—making this referral to the navigator does not have housing attached to it.” Additionally, JPCs commented on accessibility issues such as transportation and lack of services in rural areas as another challenge to service uptake. Most JPCs referenced COVID-19 as a key contextual factor that created implementation challenges, including negative impacts to the screening and referral process (resulting from higher workload demands), youth engagement, communication and collaboration with the navigators, and the ability to provide warm handoffs.
Theme 5: Screening for Program Eligibility Was the Most Challenging Aspect of Implementation
The most challenging aspect of HSYNC implementation was screening for program eligibility. Most JPCs discussed issues with the eligibility criteria, including not knowing that criteria existed or experiencing challenges determining whether the criteria applied to specific cases (e.g., youth with no current housing instability; non-probation cases such as those in detention or pre-trial, at-risk youth, or child in need of services petitions). Other participants noted that the eligibility criteria were too broad or not dynamic enough to capture changes in a youth's living status over time, leading JPCs to feel like the current criteria may result in over- or under-referring. Many JPCs reported using professional judgment in lieu of, or in addition to, the established criteria to determine referral appropriateness: “I have made referrals out of common sense for kids that are obviously in dire need of this service, not from the PACT flag.” Notably, only 12 participants explicitly named at least one of the correct eligibility criteria (i.e., run away or kicked out of home; family conflict); meanwhile, most indicated that active homelessness or the perceived risk of homelessness were clear indicators of service need despite these items not being included on the standardized risk assessment that was (in theory) used to flag instability risk. In addition to issues with eligibility determination, a demanding workload was commonly cited as impacting screening practices: “I haven't had the mind energy or bandwidth to try and figure out who would be eligible.”
Theme 6: Improvement Suggestions Centered on Strategies for Successful Implementation
Building on perceived barriers to implementation and programmatic success, most JPCs offered suggestions for program improvement. JPCs felt that the program could benefit from a greater level of awareness of the program among court staff, and specifically that it offers unique services to youth/families. Suggestions included developing and distributing written materials about the program and services offered (e.g., flyers, pamphlets) and providing more trainings. JPCs also felt that screening practices and the referral process could be further streamlined to increase efficiency and effectiveness. Suggestions included replacing the referral form with less formal means of communication (emails, phone calls, in-person conversations); relying on the navigator to complete the referrals; integrating a mechanism in the court database to help JPCs quickly assess case eligibility; and implementing a referral reminder system. In response to their own awareness of the current limitations of the service landscape, JPCs also indicated that HSYNC should offer more services (e.g., basic needs, housing) and better address accessibility issues (e.g., provide bus fare). Finally, several JPCs indicated a preference that navigators provide more case updates following a referral.
Discussion
This cross-sectional study evaluated the usability of the HSYNC program from the perspective of JPCs. Usability testing is crucial in early program implementation to identify necessary improvements for functionality and efficiency prior to taking a new program to scale (Corrao et al., 2010; Lyon et al., 2019). Following usability testing recommendations (Walji et al., 2014), we used multiple methods to capture a comprehensive view of HSYNC's implementation challenges and successes from the JPC perspective. Survey results showed good overall usability, with a mean ISUS score of 80 and generally high mean ratings on ISUS items (all but one item rated above 4.0, which indicates strong agreement). The lowest rated item was related to referral frequency, while the highest ratings were for ease of the referral process and lack of need for external implementation support. Open-ended survey responses supported these findings, highlighting the navigator and the simple referral process as key facilitators of usability.
While a mean ISUS score of 80 suggests good usability, it also indicates room for improvement. Information gleaned from the interviews provided greater insight into implementation barriers and areas for improvement that were not readily apparent in the survey data. For example, screening for program eligibility and contextual factors like the COVID-19 pandemic were identified as common implementation challenges not captured via the ISUS that likely impacted referral frequency. Interviews revealed additional individual and contextual factors that clarified why certain aspects of the program were more challenging for JPCs to implement, alongside concrete strategies for addressing implementation barriers and increasing program usability. Interviews also complemented survey findings, indicating that HSYNC and the navigator integrated well with probation practices and were seen as valuable additions to existing court services. Together, findings from these complementary data hold important implications for program usability and sustainability (discussed further below).
Drivers of Program Usability
The high degree of usability reported by JPCs via survey and interview responses was particularly noteworthy given common barriers court staff face when adopting new programs (e.g., high workload burden, low perceived value of new programs compared to usual practices). The HSYNC model was designed in collaboration with court staff and community providers to reduce barriers that could interrupt successful implementation. Although our analysis focused on evaluating perceived usability and not the impact of co-design per se, our findings suggest that the use of a co-design approach may have reduced anticipated barriers (e.g., program was perceived as value added, referral process was low-barrier and integrated well with existing practices, having a designated linkage navigator meant little-to-no workload increase for JPCs) and likely contributed to a higher degree of usability and acceptability than would be the case if the program was not designed collaboratively with court staff. Research evaluating the effectiveness of co-design as an implementation strategy for improving HSYNC usability could support these conclusions.
The housing navigators—and their unique approach to working with JPCs, youth, and families—was key to implementation success. Sixty percent of JPCs credited the navigator with their knowledge and understanding of program processes and services offered. In the interviews, JPCs highlighted the navigator's accessibility, responsive and timely communication, proactive support (e.g., helping JPCs identify appropriate referrals), and collaboration on cases as crucial factors. These findings align with prior studies showing that similar navigator practices enhance implementation success and positive client outcomes (Dir et al., 2023; Elkington et al., 2023; Markoulakis et al., 2019). Research also underscores the importance of navigator personality and relational qualities for program effectiveness (Phillips et al., 2014), a theme reaffirmed in our study. Courts considering adopting HSYNC or similar linkage programs should consider an individual's practice philosophy as well as personal attributes—like relatability, kindness, and dedication to helping others—when selecting navigators/linkage specialists.
The referral process was another program component that emerged as a strong facilitator of implementation success from the JPC perspective. Most JPCs agreed that the referral process was low-barrier—a simple, easy, and smooth process with a user-friendly referral form. The perceived ease of the referral process was likely due, in part, to having the navigator physically located in the same building as JPCs. In the qualitative interviews, JPCs noted that having the navigator co-located with probation facilitated regular communication about the referral process and identification of appropriate referrals. Combining a court-located linkage specialist with a low-barrier referral process that is well-aligned with existing practice can increase the likelihood that JPCs will make referrals, which in turn, is likely to increase service uptake.
The perceived usability and acceptability documented here have since been further evidenced by the collective action the pilot courts and their community partner organizations took to sustain funding for the program with the Washington State legislature after the pilot period ended. When HSYNC was initially implemented, it was funded by a short-term mix of philanthropic and public dollars. As a result of collective efforts, however, HSYNC is now funded in six counties, and the state is considering scaling the program to all counties. Prior to taking the program to scale, program improvements to address identified implementation challenges are needed. This will ensure the program is feasible, acceptable, and effective for all users (i.e., probation counselors, navigators, community providers, youth, caregivers).
Implications for Program Improvement
We found that the ISUS item on referral frequency had the lowest mean rating, indicating infrequent referrals to HSYNC. Interviews revealed that JPCs struggled with referrals, in part, due to organizational factors such as staff shortages, high caseloads, and competing service/referral priorities. These common challenges for court staff (Huang et al., 2020; Knight et al., 2019; Wasserman et al., 2021) have only worsened since the onset of the COVID-19 pandemic (Lockwood et al., 2023). In our study, most JPCs reported that COVID-19 hindered their ability to prioritize HSYNC referrals. To increase referrals, JPCs recommended distributing more information about the program, offering more trainings, and implementing a referral reminder system. Efforts to reduce caseloads and emphasize housing needs as a service priority may also increase referrals (Knight et al., 2019; Wasserman et al., 2021). Aligning HSYNC processes with universal design principles (Burgstahler, 2021), such as offering flexible training modalities, can reduce implementation barriers caused by rapid, unpredictable changes like those resulting from the COVID-19 pandemic.
Improving referral practices requires effective screening and assessment of service needs (Belenko et al., 2017). Screening for program eligibility emerged as a significant challenge, with over half of JPCs uncertain about eligibility criteria and program offerings. Professional biases and deviations from the standardized risk assessment may have compromised eligibility determinations. For instance, some JPCs used perceived risk of homelessness as a criterion, while others viewed homelessness as a choice by unhappy youth, leading to uncertainty about program fit. Proper screening and need identification are strong predictors of referral, which impacts service and treatment uptake (Dennis et al., 2019). To improve program success, JPCs need further education on program offerings and training/support to ensure correct use of the standardized assessment tool while guarding against implicit bias. Staff may also benefit from professional development opportunities to ensure clinical decision-making, when used, aligns with best practices. To reduce workload burden, JPCs suggested automating risk flagging in the data management system, which aligns with recommendations for streamlined program implementation in court settings (Dennis et al., 2019).
There is growing recognition that referral-based linkage programs are needed to address youth housing needs beyond traditional court services. However, programs like HSYNC can only be effective if community services are available. Housing shortages, unaffordable housing, and lack of shelter beds are representative of a crisis that makes it impossible to meet housing needs. Most JPCs were aware of current housing limitations and expressed a desire for HSYNC to provide actual housing, not just housing-adjacent services. Ultimately, increasing housing availability and removing structural barriers like poverty and rental discrimination are essential to reduce homelessness among court-involved youth. While this is beyond HSYNC's scope, ensuring navigators are knowledgeable and connected to community housing agencies will improve housing connections for youth.
Study Limitations
This study had several limitations. First, the sample size was small comprising participants from four pilot sites in one northwestern state, which may limit generalizability of findings to other probation counselors or juvenile court contexts. It is possible that individual, organizational, and systemic factors influencing JPC perspectives of HSYNC in our study differ from those in other states or jurisdictions. Additionally, the limited number of JPCs at each site, with one site having only one participant, prevented examination or reporting of differences in implementation experiences across sites due to participant confidentiality concerns in the qualitative analysis. Increasing the sample size in future evaluations would help clarify whether reported implementation challenges are specific to certain court contexts or prevalent across sites, thereby providing valuable insights for program improvement efforts. Furthermore, the participation rate was less than 50% and the sampling method might have yielded a sample that was more engaged with or had positive attitudes towards court service innovations, including the HSYNC program, potentially introducing bias. Additionally, we were unable to report the demographic characteristics of the qualitative sample and could not ascertain the extent of overlap between the survey and interview samples, which may have influenced differences in findings between the quantitative and qualitative portions of the evaluation. However, there was likely significant overlap because we targeted JPCs who were part of the HSYNC pilot for both survey and interview participation.
Conclusion
The persistent intersection of youth housing instability and court involvement calls for better coordination between juvenile courts and community agencies providing housing-related services. Our study suggests that, from the perspective of JPCs, HSYNC offers a promising cross-system approach to address housing-related needs by linking court-involved youth and their families to community-based services. An evaluation of the effectiveness of HSYNC for improving youth outcomes will be needed alongside program improvements to address identified implementation challenges to ensure the program is feasible, acceptable, and effective for all users (i.e., probation counselors, navigators, community providers, youth, caregivers) prior to taking the program to scale.
Footnotes
Acknowledgments
The authors are grateful to the King County, Kitsap County, Okanogan County, and Snohomish County juvenile courts for their willingness to pilot test the HSYNC program and for assisting the evaluation team with participant recruitment. This study would not have been possible without the individual JPCs who graciously agreed to share their experiences and perspectives of HSYNC for this evaluation. The authors would also like to thank Reed Klein, MSW for his feedback and suggestions on earlier versions of this manuscript, and Joane Gonzalez for her technical support during manuscript preparation.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the Raikes Foundation, Block-Leavitt Foundation, and the Washington State Department of Commerce.
