Revalidating the Positive Achievement Change Tool (PACT) in a Major City: A Survival Analysis

Abstract

The Positive Achievement Change Tool (PACT) is a popular risk-assessment tool used in juvenile justice systems across the country. This study is an effort at revalidating the PACT in the context of a large Southern city by utilizing days to recidivism in the context of survival analysis. The current research is concerned with whether the PACT’s risk classifications are effective in predicting recidivism. Findings indicate that the PACT effectively predicts recidivism better than chance and that recidivism is especially common early on in probation, with most incidents taking place within the first 200 days. However, the predictive validity of the PACT is not strong for the study population, and its implementation may benefit from context-specific reforms.

Keywords

risk assessment recidivism juvenile delinquency juvenile justice

Get full access to this article

View all access options for this article.

References

Allison

P. D.

(1984). Event history analysis: Regression for longitudinal event data. SAGE.

Andretta

J. R.

Worrell

F. C.

Watkins

K. M.

Sutton

R. M.

Thompson

A. D.

Woodland

M. H.

(2019). Race and stereotypes matter when you ask about conduct problems: Implications for violence risk assessment in juvenile justice settings. Journal of Black Psychology, 45(1), 26–51.

Andrews

D. A.

Bonta

(2010). The psychology of criminal conduct. Routledge.

Andrews

D. A.

Bonta

Wormith

J. S.

(2006). The recent past and near future of risk and/or need assessment. Crime & Delinquency, 52(1), 7–27.

Andrews

D. A.

Guzzo

Raynor

Rowe

R. C.

Rettinger

L. J.

Brews

Wormith

J. S.

(2012). Are the major risk/need factors predictive of both female and male re-offending? A test with the eight domains of the Level of Service/Case Management Inventory. International Journal of Offender Therapy and Comparative Criminology, 56(1), 113–133.

Arnold

Flack

Heilbrun

(2018). Risk assessment and juvenile resentencing: A critical analysis. Behavioral Sciences & the Law, 36(5), 576–586.

Ashford

J. B.

Gallagher

J. M.

(2019). Preventing juvenile transitions to adult crime: A pilot study of probation interventions for older, high-risk juvenile delinquents. Criminal Justice and Behavior, 46(8), 1148–1164.

Baglivio

M. T.

(2009). The assessment of risk to recidivate among a juvenile offending population. Journal of Criminal Justice, 37(6), 596–607.

Baglivio

M. T.

(2014). Briefing report: Recidivism time to failure by placement and risk level. Florida Department of Juvenile Justice. http://www.djj.state.fl.us/docs/research2/briefing-report-recidivism-time-to-failure.pdf

10.

Baglivio

M. T.

(2015). Examining the validity of the positive achievement change tool (PACT) for community probation youth in Texas [Unpublished manuscript]. G4S Youth Services, LLC.

11.

Baglivio

M. T.

Jackowski

(2013). Examining the validity of a juvenile offending risk assessment instrument across gender and race/ethnicity. Youth Violence and Juvenile Justice, 11(1), 26–43.

12.

Baglivio

M. T.

Wolff

K. T.

(2019). Predicting juvenile reentry success: Developing a global risk score and risk classification levels using the residential positive achievement change tool. Youth Violence and Juvenile Justice, 17(3), 241–268.

13.

Barnes

A. R.

Campbell

N. A.

Anderson

V. R.

Campbell

C. A.

Onifade

Davidson

W. S.

(2016). Validity of initial, exit, and dynamic juvenile risk assessment: An examination across gender and race/ethnicity. Journal of Offender Rehabilitation, 55(1), 21–38.

14.

Barnoski

(2004). Washington State juvenile court assessment manual, version 2.1. Washington State Institute for Public Policy.

15.

Campbell

Papp

Barnes

Onifade

Anderson

(2018). Risk assessment and juvenile justice: An interaction between risk, race, and gender. Criminology & Public Policy, 17(3), 525–545.

16.

Cohen

T. H.

Vanbenschoten

S. W.

(2014). Does the risk of recidivism for supervised offenders improve over time? Examining changes in the dynamic risk characteristics for offenders under federal supervision. Federal Probation, 78(2), 41–56.

17.

Enebrink

Långström

Gumpert

C. H.

(2006). Predicting aggressive and disruptive behavior in referred 6-to 12-year-old boys: Prospective validation of the EARL-20B risk/needs checklist. Assessment, 13(3), 356–367.

18.

Farrington

D. P.

(1986). Age and crime. In Tonry

Morris

(Eds.), Crime and justice: An annual review of research (Vol. 7, pp. 189–250). University of Chicago Press.

19.

Fawcett

(2006). An introduction to ROC analysis. Pattern Recognition Letters, 27, 861–874. Florida Positive Achievement Change Tool (PACT) risk and needs assessment juris Florida Showcase of Texas, Archives, Research & Scholarship. (2459)

20.

Glen

(2020). C-Statistic: Definitions, examples, weighting, and significance. Statistics How To. https://www.statisticshowto.com/c-statistic/

21.

Gottfredson

M. R.

Hirschi

(1990). A general theory of crime. Stanford University Press.

22.

Gottfredson

S. D.

Moriarty

L. J.

(2006). Statistical risk assessment: Old problems and new applications. Crime & Delinquency, 52(1), 178–200.

23.

Hamilton

Kigerl

Kowalski

M. A.

Routh

(2019). Optimizing youth risk assessment performance: Development of the modified positive achievement change tool in Washington state. Criminal Justice and Behavior, 46(8), 1106–1127.

24.

Hamilton

van Wormer

Barnoski

(2015). PACT validation and weighting results: Technical report. Washington Association of Juvenile Court Administration.

25.

Harrell

Califf

R. M.

Pryor

D. B.

Lee

K. L.

Rosati

R. A.

(1982). Evaluating the yield of medical tests. JAMA, 247(18), 2543–2546.

26.

Hempel

Buck

Cima

Van Marle

(2013). Review of risk assessment instruments for juvenile sex offenders: What is next? International Journal of Offender Therapy and Comparative Criminology, 57(2), 208–228.

27.

Hilterman

E. L.

Bongers

Nicholls

T. L.

Van Nieuwenhuizen

(2016). Identifying gender specific risk/need areas for male and female juvenile offenders: Factor analyses with the structured assessment of violence risk in youth (Savry). Law and Human Behavior, 40(1), 82–96.

28.

Rocheleau

G. C.

(2021). Gender differences in risk assessment among youth adjudicated to juvenile correctional facilities. Women and Criminal Justice, 31, 463–475.

29.

Holtfreter

Cupp

(2007). Gender and risk assessment the empirical status of the LSI-R for women. Journal of Contemporary Criminal Justice, 23(4), 363–382.

30.

Huebner

B. M.

Berg

M. T.

(2011). Examining the sources of variation in risk for recidivism. Justice Quarterly, 28(1), 146–173.

31.

Hutchins

E. P.

(2019). Assessing the predictive utility of the positive achievement change tool at a Texas Juvenile Justice Agency. The University of Texas at El Paso.

32.

Kleeven

A. T.

de Vries Robbé

Mulder

E. A.

Popma

(2022). Risk assessment in juvenile and young adult offenders: Predictive validity of the SAVRY and SAPROF-YV. Assessment, 29, 181–197.

33.

Leiber

M. J.

Fix

(2019). Reflections on the impact of race and ethnicity on juvenile court outcomes and efforts to enact change. American Journal of Criminal Justice, 44(4), 581–608.

34.

Luong

Wormith

J. S.

(2011). Applying risk/need assessment to probation practice and its impact on the recidivism of young offenders. Criminal Justice and Behavior, 38(12), 1177–1199.

35.

Martin

J. H.

(2012). Predicting risk to re-offend: Establishing the validity of the positive achievement change tool [Doctoral dissertation]. University of Central Florida.

36.

McCafferty

J. T.

(2016). The importance of counties: Examining the predictive validity of a state juvenile risk assessment instrument. Journal of Offender Rehabilitation, 55(6), 377–395.

37.

McCafferty

J. T.

(2017). Professional discretion and the predictive validity of a juvenile risk assessment instrument: Exploring the overlooked principle of effective correctional classification. Youth Violence and Juvenile Justice, 15(2), 103–118.

38.

McKenzie

(2018). A validation study of the Postive Achievement Change Tool (PACT) [Doctoral dissertation]. Sam Houston State University. https://shsu-ir.tdl.org/bitstream/handle/20.500.11875/2382/MCKENZIE-DISSERTATION-2018.pdf?sequence=1&isAllowed=y

39.

McNeill

Robinson

(2013). Liquid legitimacy and community sanctions. In Crawford

Hucklesby

(Eds.), Legitimacy and compliance in criminal justice (pp. 116–137). Routledge.

40.

Mehmetoglu

Jakobsen

T. G.

(2016). Applied statistics using Stata: A guide for the social sciences. SAGE.

41.

Miller

W. R.

Rollnick

(2002). Motivational interviewing: Preparing people for change (2nd ed.). The Guilford Press.

42.

Moffitt

T. E.

(1993). Adolescence-limited and life-course-persistent antisocial behavior: A developmental taxonomy. Psychological Review, 100(4), 674.

43.

Moore

L. D.

Padavic

(2011). Risk assessment tools and racial/ethnic disparities in the juvenile justice system. Sociology Compass, 5(10), 850–858.

44.

Mossman

(1994). Assessing predictions of violence: Being accurate about accuracy. Journal of Consulting and Clinical Psychology, 62(4), 783–792.

45.

Mulvey

E. P.

Schubert

C. A.

Pitzer

Hawes

Piquero

Cardwell

(2016). An examination of change in dynamic risk of offending over time among serious juvenile offenders. Journal of Criminal Justice, 45, 48–53.

46.

Newson

R. B.

(2010). Comparing the predictive powers of survival models using Harrell’s C or Somers’ D. The Stata Journal, 10(3), 339–358.

47.

Obuchowski

N. A.

(2005). ROC analysis. American Journal of Roentgenology, 184(2), 364–372.

48.

Phillips

H. K.

Gray

N. S.

MacCulloch

S. I.

Taylor

Moore

S. C.

Huckle

MacCulloch

M. J.

(2005). Risk assessment in offenders with mental disorders: Relative efficacy of personal demographic, criminal history, and clinical variables. Journal of Interpersonal Violence, 20(7), 833–847.

49.

Pusch

Holtfreter

(2018). Gender and risk assessment in juvenile offenders: A meta-analysis. Criminal Justice and Behavior, 45(1), 56–81.

50.

Rice

M. E.

Harris

G. T.

(1995). Violent recidivism: Assessing predictive validity. Journal of Consulting and Clinical Psychology, 63(5), 737.

51.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29(5), 615–620.

52.

Rich

J. T.

Neely

J. G.

Paniello

R. C.

Voelker

C. C.

Nussenbaum

Wang

E. W.

(2010). A practical guide to understanding Kaplan-Meier curves. Otolaryngology: Head and Neck Surgery, 143(3), 331–336.

53.

Rollnick

Allison

(Eds.) (2004). Motivational interviewing. In The essential handbook of treatment and prevention of alcohol problems (pp. 105–116). John Wiley & Sons.

54.

Rollnick

Miller

W. R.

(1995). What is motivational interviewing? Behavioural and Cognitive Psychotherapy, 23(4), 325–334.

55.

Schmidt

Heffernan

Ward

(2020). Why we cannot explain cross-cultural differences in risk assessment. Aggression and Violent Behavior, 50, 101346.

56.

Schwalbe

C. S.

(2007). Risk assessment for juvenile justice: A meta-analysis. Law and Human Behavior, 31(5), 449.

57.

Schwalbe

C. S.

(2008). A meta-analysis of juvenile justice risk assessment instruments: Predictive validity by gender. Criminal Justice and Behavior, 35(11), 1367–1381.

58.

Schwalbe

C. S.

Fraser

M. W.

Day

S. H.

Cooley

(2006). Classifying juvenile offenders according to risk of recidivism: Predictive validity, race/ethnicity, and gender. Criminal Justice and Behavior, 33(3), 305–324.

59.

Scott

C. K.

Grella

C. E.

Dennis

M. L.

Funk

R. R.

(2014). Predictors of recidivism over 3 years among substance-using women released from jail. Criminal Justice and Behavior, 41(11), 1257–1289.

60.

Shapiro

C. J.

Malone

P. S.

Gavazzi

S. M.

(2018). Modifying a risk assessment instrument for youthful offenders. International Journal of Offender Therapy and Comparative Criminology, 62(2), 482–503.

61.

Skeem

J. L.

Kennealy

P. J.

Tatar

J. R.

Hernandez

I. R.

Keith

F. A.

(2017). How well do juvenile risk assessments measure factors to target in treatment? examining construct validity. Psychological Assessment, 29(6), 679–691.

62.

Vieira

T. A.

Skilling

T. A.

Peterson-Badali

(2009). Matching court-ordered services with treatment needs: Predicting treatment success with young offenders. Criminal Justice and Behavior, 36(4), 385–401.

63.

Vincent

G. M.

Guy

L. S.

Perrault

R. T.

Gershenson

(2016). Risk assessment matters, but only when implemented well: A multisite study in juvenile probation. Law and Human Behavior, 40(6), 683–696.

64.

Vose

(2016). Risk assessment and reassessment: An evidence-based approach to offender management. Criminology & Public Policy, 15(2), 301–308.

65.

Winokur-Early

Hand

G. A.

Blankenship

J. L.

(2012). Validity and reliability of the Florida Positive Achievement Change Tool (PACT) risk and needs assessment instrument: A three-phase evaluation (validation study, factor analysis, inter-rater reliability). Justice Research Center.