Usability Evaluation of an Asynchronous Online Training Platform for Ready for Wages : Implications for Rehabilitation Counselors Serving Justice-Involved Youth

Abstract

This study evaluated the usability of Ready for Wages, an asynchronous online training platform that prepares juvenile service professionals to deliver employability social skills curricula to justice-involved youth. Using a convergent parallel mixed-methods design, 33 professionals (11 Cognitive Lab, 22 asynchronous) completed the System Usability Scale (SUS) and provided qualitative feedback through cognitive interviews and open-ended surveys. The training platform achieved an excellent usability score (M = 89.39, SD = 8.32). No significant differences emerged between cognitive-lab and asynchronous groups, t(18.44) = .60, p = .554, d = .23, though there were differences in those who had prior experience with the curriculum and those who did not, t(10.75) = −3.44, p = .006, d = 1.07. Qualitative themes converged with quantitative results, highlighting ease of navigation, utility of embedded teaching tips, and the need for printable materials and greater content diversity. Findings suggest that Ready for Wages offers a scalable, low-cost, and highly usable training solution for implementers. Implications for rehabilitation counseling practice include extending access to professional development in correctional facilities and supporting workforce reentry outcomes for youth.

Keywords

juvenile justice prevention incarceration vocational training social skills employment career readiness

Get full access to this article

View all access options for this article.

References

Ashford

Gallagher

(2019). Preventing juvenile transitions to adult crime: A pilot study of probation interventions for older, high-risk juvenile delinquents. Criminal Justice and Behavior, 46(8), 1148–1164. https://doi.org/10.1177/0093854819835277

Bandura

(1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295X.84.2.191

Brame

Mulvey

E. P.

Schubert

C. A.

Piquoro

A. R.

(2018). Recidivism in a sample of serious adolescent offenders. Journal of Quantitative Criminology, 24, 167–187. https://doi.org/10.1007/s10940-016-9329-2

Brooke

(1996). SUS: A “quick and dirty” usability scale. In Jordan

Thomas

Weerdmeester

(Eds.), Usability evaluation in industry (pp. 189–194). Taylor & Francis.

Calleja

N. G.

(2020). Assessing and treating trauma in detained adolescents: A pre–post within subjects evaluation. Journal of Child and Family Studies, 29(4), 934–941. https://doi.org/10.1007/s10826-019-01564-9

Creswell

J. W.

(2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Sage.

Creswell

J. W.

Plano Clark

V. L.

(2018). Designing and conducting mixed methods research (3rd ed.). Sage.

Danaher

B. G.

Milgrom

Seeley

J. R.

Stuart

Schembri

Tyler

M. S.

Ericksen

Lester

Gemmill

A. W.

Lewinsohn

(2012). Web-Based intervention for postpartum depression: Formative research and design of the MomMoodBooster program. JMIR Research Protocols, 1(2), e18. https://doi.org/10.2196/resprot.2329

Dedoose (Version 9.0.107) [Computer software]. SocioCultural Research Consultants, LLC. https://www.dedoose.com

10.

Dishion

Patterson

G. R.

(2006). The development and ecology of antisocial behavior in children and adolescents. In Cicchetti

Cohen

D. J.

Eds., Developmental psychopathology: Volume 3. Risk, disorder, and adaptation (2nd ed., pp. 503–541). Wiley.

11.

Ericsson

K. A.

Simon

H. A.

(1993). Protocol analysis: Verbal reports as data (Revised edition). MIT Press.

12.

Fishman

Konstantopoulos

Kubitskey

B. W.

Vath

Park

Johnson

Edelson

D. C.

(2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64(5), 426–438. https://doi.org/10.1177/0022487113494413

13.

Gagnon

J. C.

Steinberg

M. A.

Crockett

Murphy

K. M.

Gaddis

(2013). IDEA-related professional development in juvenile corrections schools. Journal of Special Education Leadership, 26(2), 93–105. https://eric.ed.gov/?id=EJ1088907

14.

Gagnon

J. C.

Swank

J. M.

(2021). A national survey on mental health professional development in juvenile justice facilities: Implications for youth reentry. Behavioral Disorders, 46(3), 149–162. https://doi.org/10.1177/0198742920911183

15.

Griller Clark

Mathur

S. R.

Ott

McTier

T. S

. (2020). Employers’ perceptions of hiring juvenile offenders. Criminal Justice and Behavior, 47(9), 1156–1175. https://doi.org/10.1177/0093854820919914

16.

Houchins

D. E.

Shippen

M. E.

Murphy

K. M.

(2012). Evidence-based professional development considerations along the school-to-prison pipeline. Teacher Education and Special Education, 35(4), 271–283. https://doi.org/10.1177/0888406411412396

17.

Individuals With Disabilities Education Act, 20 U.S.C. § 1400. (2004). https://www.congress.gov/108/plaws/publ446/PLAW-108publ446.pdf

18.

Johnstone

C. J.

Bottsford-Miller

N. A.

Thompson

S. J.

(2006). Using the think aloud method (cognitive labs) to evaluate test design for students with disabilities and English language learners (Technical Report 44). University of Minnesota, National Center on Educational Outcomes. https://eric.ed.gov/?id=ED495909

19.

Kutscher

E. L.

Mazzotti

V. L.

McDaniel

S. L.

Zeng

Reardon

Bumble

J. L.

Voggt

Harris

(2024). Mixed methods research in secondary transition: A systematic literature review. Career Development and Transition for Exceptional Individuals, 49(1), 7–20. https://doi.org/10.1177/21651434241268003

20.

Leko

M. M.

Hitchcock

J. H.

Love

H. R.

Houchins

D. E.

Conroy

M. A.

(2023). Quality indicators for mixed-methods research in special education. Exceptional Children, 89(4), 432–448. https://doi.org/10.1177/00144029221141031

21.

Mathur

S. R.

Griller Clark

(2014). Community engagement for reentry success of youth from juvenile justice: Challenges and opportunities. Education and Treatment of Children, 37(4), 713–734. https://www.jstor.org/stable/44683944. https://doi.org/10.1353/etc.2014.0034

22.

Mathur

S. R.

Griller Clark

Grayson

Robertson

K. S.

(2023). Preparing justice involved youth for transition: Employer perceptions. Children and Youth Services Review, 155, 1–9. https://doi.org/10.1016/j.childyouth.2023.107270

23.

Maxwell

J. A.

(2005). Qualitative research design: An interactive approach (2nd ed.). Sage.

24.

Mazzotti

Rowe

Kwiatek

Chang

Fowler

Poppen

Sinclair

Test

(2021). Secondary transition predictors of postschool success: An update to the research base. Career Development and Transition for Exceptional Individuals, 44(1), 47–64. https://doi.org/10.1177/2165143420959793

25.

Mendoza

Blake

J. J.

Marchbanks

M. P.

Ragan

(2020). Race, gender, and disability and the risk for juvenile justice contact. The Journal of Special Education, 53(4), 226–235. https://doi.org/10.1177/0022466919845113

26.

Merriam

S. B.

(2002). Qualitative research in practice: Examples for discussion and analyses. Jossey-Bass.

27.

Miller

A. A.

(2019). Navigating barriers to special education in a juvenile detention center. The Journal of Correctional Education, 70(2), 2–30. https://eric.ed.gov/?id=EJ1335464

28.

Murray

Doren

(2013). The effects of working at gaining employment skills on the social and vocational skills of adolescents with disabilities: A school-based intervention. Rehabilitation Counseling Bulletin, 56(2), 96–107. https://doi.org/10.1177/003435521245261

29.

(2017). The consequences of school dropout among serious adolescent offenders: More offending? More arrest? Both? Journal of Research in Crime and Delinquency, 54(1), 78–110. https://doi.org/10.1177/0022427816664118

30.

Nelson

C. M.

Jolivette

Leone

P. E.

Mathur

S. R.

(2017). Meeting the needs of at-risk and adjudicated youth with behavioral challenges: The promise of juvenile justice. Behavioral Disorders, 36(1), 70–80. https://doi.org/10.1177/019874291003600108

31.

Nielsen

(2012). How many test users in a usability study? https://www.nngroup.com/articles/how-many-test-users/

32.

Nielsen-Pheiffer

Lindstrom

(2023). Teacher use of evidence-based practices in juvenile justice settings: A national survey. Journal of Correctional Education, 74(1), 60–88. https://eric.ed.gov/?id=EJ1367743

33.

R Core Team. (2024). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/

34.

Shippen

M. E.

Houchins

D. E.

Lockwood

(2014). Juvenile correctional professional development: From conceptualization to evaluation. The Journal of Correctional Education, 65(1), 68–87. https://eric.ed.gov/?id=EJ1185234

35.

Snydman

(2022). Unlocking futures: Youth with learning disabilities and the juvenile justice system. National Center for Learning Disabilities.

36.

Swank

J. M.

Gagnon

J. C.

(2016). Mental health services in juvenile correctional facilities: A national survey of clinical staff. Journal of Child and Family Studies, 25, 2862–2872. https://doi.org/10.1007/s10826-016-0436-3

37.

Test

D. W.

Mazzotti

V. L.

Mustian

A. L.

Fowler

C. H.

Kortering

L. J.

Kohler

P. H.

(2009). Evidence-based secondary transition predictors for improving post-school outcomes for students with disabilities. Career Development for Exceptional Individuals, 32, 160–181. https://doi.org/10.1177/0885728809346960

38.

Underwood

L. A.

Washington

(2016). Mental illness and juvenile offenders. International Journal of Environmental Research and Public Health, 13(2), 228. https://doi.org/10.3390/ijerph13020228

39.

Unruh

D. K.

Johnson

Waintrup

Sinclair

(2014). READY for W.A.G.E.S. University of Oregon.

40.

Unruh

D. K.

Reardon

Strycker

(2023). Employer perspectives on hiring youth previously involved with juvenile justice: A national survey. Education and Treatment of Children, 46, 337–349. https://doi.org/10.1007/s43494-023-00110-z

41.

Unruh

D. K.

Smolkowski

Gau

Reardon

Bromley

K. W.

Seeley

(2024). Evaluation of an employability social skills intervention to im prove community engagement skills among justice-involved youth: Ready for Wages. Journal of Correctional Education, 75(1), 37–57. https://uoregon.idm.oclc.org/login?url=https://www.proquest.com/scholarly-journals/ready-wages-social-skills-intervention-improve/docview/2933149714/se-2

42.

U.S. Department of Commerce, Census Bureau, American Community Survey (ACS), 1-Year Public Use Microdata Sample (PUMS) data. (2022). See Digest of Education Statistics 2023, table 219.80. https://nces.ed.gov/programs/coe/indicator/coj/status-dropout-rates

43.

Varghese

F. P.

Hardin

E. E.

Bauer

R. L.

Morgan

R. D.

(2010). Attitudes toward hiring offenders: The roles of criminal history, job qualifications, and race. International Journal of Offender Therapy and Comparative Criminology, 54(5), 769–782. https://doi.org/10.1177/0306624X09344960

44.

Walker

S. C.

Herting

J. R.

(2020). The impact of pretrial juvenile detention on 12-month recidivism: A matched comparison study. Crime & Delinquency, 66(13–14), 1865–1887. https://doi.org/10.1177/0011128720926115

45.

Wilson

S. J.

Lipsey

M. W.

Soydan

(2003). Are mainstream programs for juvenile delinquency less effective with minority youth than majority youth? A meta-analysis of outcomes research. Research on Social Work Practice, 13, 3–26. https://doi.org/10.1177/1049731502238754

46.

Workforce Innovation and Opportunity Act, 128 U.S.C. § 1426 et seq. (2014).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.03 MB

0.00 MB