Best Practices for Neglected Assumptions: Multi-Site Confirmation of the MPACT-6

Abstract

In the assessment world, risk determines “who” to treat and needs determine “what” (Bonta & Andrews, 2016). Yet, for youth, greater emphasis is placed on needs that target recidivism reduction interventions. Unlike risk, needs represent dynamic domains, or latent constructs, requiring testing to assure adequate measurement. We conducted a multi-level, multi-group analysis of the Modified Positive Achievement Change Tool (MPACT) with a 10-state sample of youth (N = 258,464). Findings confirm the validity and reliability of needs domains, and the development of a novel “Global Needs Factor,” a composite summary of needs. Created without criminal history/static measures, needs scales demonstrate predictive accuracy. Further, measurement invariance and aspects of gender and race/ethnicity prediction parity are observed.

Keywords

risk-needs-assessment juvenile justice overclassification measurement invariance

Get full access to this article

View all access options for this article.

References

American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

Andrews . (1982). A personal, interpersonal and community-reinforcement perspective on devoant behaviour (PIC-R). Ontario Ministry of Correctional Services.

Andrews

D. A.

Bonta

(1995). The level of service inventory—Revised. Multi-Health Systems.

Andrews

D. A.

Bonta

(2010). The psychology of criminal conduct (5th ed.). Lexis Nexis/Anderson Pub.

Andrews

D. A.

Bonta

Wormith

J. S.

(2006). The recent past and near future of risk and/or need assessment. Crime and Delinquency, 52(1), 7–27.

Angwin

Larson

Mattu

Kirchner

(2016). Machine bias. In Martin K. (Ed.), Ethics of data and analytics (pp. 254–264). Auerbach Publications.

Baglivio

M. T.

Jackowski

(2013). Examining the validity of a juvenile offending risk assessment instrument across gender and race/ethnicity. Youth Violence and Juvenile Justice, 11(1), 26–43.

Barnoski

(1997). Washington State juvenile court recidivism estimates: Fiscal year 1994 youth. Washington Institute for Public Policy.

Barnoski

(2004a). Assessing risk for re-offense: Validating the Washington State Juvenile Court Assessment (Report No. 04-03-1201). Washington State Institute for Public Policy.

10.

Barnoski

(2004b). Washington State Juvenile Court Assessment manual, Version 2.1 (Report No. 04-03-1203). Washington State Institute for Public Policy.

11.

Belisle

L. A.

Salisbury

E. J.

(2021). Starting with girls and their resilience in mind: Reconsidering risk/needs assessments for system-involved girls. Criminal Justice and Behavior, 48(5), 596–616.

12.

Bonta

(2002). Offender risk assessment: Guidelines for selection and use. Criminal Justice and Behavior, 29(4), 355–379.

13.

Bonta

Andrews

D. A.

(2016). The psychology of criminal conduct. Routledge.

14.

Brennan

Dieterich

Ehret

(2009). Evaluating the predictive validity of the COMPAS risk and needs assessment system. Criminal Justice and Behavior, 36(1), 21–40.

15.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research. New York, NY: Routledge.

16.

Butler

L. C.

Hamilton

Krushas

A. E.

Kigerl

Kowalski

(2023). Racial bias and amelioration strategies for juvenile risk assessment. In Ahlin

E. M.

Mitchell

Atkin-Plunk

(Eds.), Handbook on inequalities in sentencing and corrections among marginalized populations (Division on Corrections Sentencing Handbook, Vol. 7, pp. 70–118). New York, NY: Routledge.

17.

Byrne

B. M.

Stewart

S. M.

(2006). Teacher's corner: The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling, 13(2), 287–321.

18.

Campbell

C. A.

Barnes

Papp

D’amato

Anderson

V. R.

Moses

(2020). Understanding the role of neighborhood typology and sociodemographic characteristics on time to recidivism among adjudicated youth. Criminal Justice and Behavior, 47(9), 1079–1096.

19.

Campbell

D. T.

Fiske

D. W.

(1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105.

20.

Catalán

H. E. N.

(2019). Reliability, population classification and weighting in multidimensional poverty measurement: A Monte Carlo Study. Social Indicators Research, 142, 887–910. 2019.

21.

Caudy

M. S.

Durso

J. M.

Taxman

F. S.

(2013). How well do dynamic needs predict recidivism? Implications for risk assessment and risk reduction. Criminal Justice Journal, 41(6), 458–466.

22.

Chen

F. F.

Sousa

K. H.

West

S. G.

(2005). Teacher's Corner: Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12, 471–492.

23.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modelling, 9(2), 233–255.

24.

Childs

K. K.

Peck

J. H.

Brady

C. M.

(2022). Predictive bias in juvenile risk assessment: Considering race/ethnicity and sex. Crime and Delinquency, 00111287221143936.

25.

Deng

Chan

(2017). Testing the difference between reliability coefficients alpha and omega. Educational and Psychological Measurement, 77(2), 185–203. https://doi.org/10.1177/0013164416658325

26.

Elkington

K. S.

Robson

Sichel

C. E.

Lee

Wasserman

G. A.

(2023). Family Connect: The Pilot Test of a cross-systems behavioral health treatment referral and linkage intervention for youth on probation. Criminal Justice and Behavior, 50(1), 22–39.

27.

Hamilton

Campagna

Tollefsbol

van Wormer

Barnoski

(2017). A more consistent application of the RNR model: The STRONG-R needs assessment. Criminal Justice and Behavior, 44(2), 261–292.

28.

Hamilton

Kigerl

Kowalski

(2022). Prediction is local: The benefits of risk assessment optimization. Justice Quarterly, 39(4), 722–744.

29.

Hamilton

Kowalski

M. A.

Kigerl

Routh

(2019). Optimizing youth risk assessment performance: Development of the modified positive achievement change tool in Washington State. Criminal Justice and Behavior, 46(8), 1106–1127.

30.

Henning

K. N.

(2012). Criminalizing normal adolescent behavior in communities of color: The role of prosecutors in juvenile justice reform. Cornell Law Review, 98(2), 383–462.

31.

Herz

D. C.

Lee

Lutz

Stewart

Tuell

Wiig

(2012). Addressing the needs of multi-system youth: Strengthening the connection between child welfare and juvenile justice. Center for Juvenile Justice Reform and Robert F. Kennedy Children's Action Corps. https://cjjr.georgetown.edu/wpcontent/uploads/2015/03/MultiSystemYouth_March2012.pdf

32.

Hollin

C. R.

Palmer

E. J.

Clark

(2003). The level of service inventory-revised profile of English prisoners: A needs analysis. Criminal Justice and Behavior, 30, 422–440.

33.

Hsu

C. I.

Caputi

Byrne

M. K.

(2010). Level of Service Inventory–Revised: Assessing the risk and need characteristics of Australian indigenous offenders. Psychiatry, Psychology and Law, 17(3), 355–367.

34.

Jung

Rawana

E. P.

(1999). Risk and need assessment of juvenile offenders. Criminal Justice and Behavior, 26(1), 69–89.

35.

Juvenile Justice Geography Policy Practice Statistics. (2020, Nov 23). The juvenile justice GPS updates risk assessment in probation. National Center for Juvenile Justice. http://www.jjgps.org/news/article/80/the-juvenile-justice-gps-updates-risk-as.

36.

Kapoor

Peterson-Badali

Skilling

(2018). Barriers to service provision for justice-involved youth. Criminal Justice and Behavior, 45(12), 1832–1851.

37.

Kitzmiller

M. K.

Paruk

J. K.

Cavanagh

(2022). Criminogenic risk score trajectories of Justice-involved youth: An investigation across Race/ethnicity (p. 00938548221098985). Criminal Justice and Behavior.

38.

Kroner

D. G.

(2005). Issues in violent risk assessment: Lessons learned and future directions. Journal of Interpersonal Violence, 20(2), 231–235.

39.

Krushas

Hamilton

Mei

(2023). Assessing the Construct validity of the M-PACT 6 (Document # 02-08-001). Nebraska Center for Justice Research. https://www.unomaha.edu/college-of-public-affairs-and-community-service/nebraska-center-for-justice-research/documents/multi-state-confirmation-mpact-2-2023.pdf

40.

Latessa

E. J.

Lovins

(2010). The role of offender risk assessment: A policy maker guide. Victims and Offenders, 5(3), 203–219.

41.

Little

(2013). Longitudinal structural equation modeling. Guilford Press.

42.

Matsueda

R. L.

Drakulich

K. M.

(2016). Measuring collective efficacy: A multilevel measurement model for nested data. Sociological Methods & Research, 45(2), 191–230.

43.

Mei

Hamilton

Kowalski

Kigerl

(2021). Redesigning the Central Eight: Introducing the M-PACT Six. Youth Violence and Juvenile Justice, 19(4), 445–470.

44.

Miller

Maloney

(2013). Practitioner compliance with risk/needs assessment tools: A theoretical and empirical assessment. Criminal Justice and Behavior, 40(7), 716–736.

45.

Miller

W. T.

Campbell

C. A.

Papp

Ruhland

(2022). The contribution of static and dynamic factors to recidivism prediction for black and White youth offenders. International Journal of Offender Therapy and Comparative Criminology, 66, 1779–1795.

46.

Millsap

R. E.

Yun-Tein

(2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479–515.

47.

National Academies of Sciences, Engineering, and Medicine. (2022). The limits of recidivism: Measuring success after prison.

48.

Olver

M. E.

Stockdale

K. C.

Wormith

J. S.

(2014). Thirty years of research on the level of service scales: A meta-analytic examination of predictive accuracy and sources of variability. Psychological Assessment, 26(1), 156–176.

49.

Onifade

Davidson

Campbell

(2009). Risk assessment: The predictive validity of the youth level of service case management inventory with African Americans and girls. Journal of Ethnicity in Criminal Justice, 7(3), 205–221.

50.

Palmer

E. J.

Hollin

C. R.

(2007). The level of service inventory— Revised with English women prisoners. Criminal Justice and Behavior, 34(8), 971–984.

51.

Pardoel

(2020). An examination of the influence of gender and race on dynamic risk assessment [Doctoral dissertation, Carleton University].

52.

Pedersen

Hagtvet

K. A.

Karterud

(2007). Generalizability studies of the Global Assessment of functioning-split version. Comprehensive Psychiatry, 48(1), 88–94.

53.

Putnick

D. L.

Bornstein

M. H.

(2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90.

54.

Rice

M. E.

Harris

G. T.

(2005). Comparing effect sizes in follow-up studies: ROC area, Cohen’s d, and r. Law and Human Behavior, 29(5), 615–620. https://doi.org/10.1007/s10979-005-6832-7

55.

Rios

Wells

(2014). Validity evidence based on internal structure. Psicothema, 26(1), 108–116.

56.

Rudnev

Lytkina

Davidov

Schmidt

Zick

(2017). Testing measurement invariance for a second-order factor: A cross-national test of the alienation scale. Methods, data, analyses, 12(1), 47–76.

57.

Schmidt

Hoge

R. D.

Gomes

(2005). Reliability and validity analyses of the youth level of service/case management inventory. Criminal Justice and Behavior, 32(3), 329–344.

58.

Schmidt

Lien

Vaughan

Huss

M. T.

(2017). An examination of individual differences and factor structure on the LS/CMI: Does this popular risk assessment tool measure up? Deviant Behavior, 38(3), 306–317.

59.

Schmidt

Heffernan

Ward

(2020). Why we cannot explain cross-cultural differences in risk assessment. Aggression and Violent Behavior, 50, 101346.

60.

Schmitt

Kuljanin

(2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18(4), 210–222.

61.

Scott

Brown

S. L.

Skilling

T. A.

(2019). Predictive and convergent validity of the youth assessment and screening instrument in a sample of male and female justice-involved youth. Criminal Justice and Behavior, 46(6), 811–831. https://doi.org/10.1177/0093854819842585

62.

Singh

(2013). Predictive validity performance indicators in violence risk assessment: A methodological primer. Behavioral Sciences & the Law, 31.

63.

Singh

J. P.

Desmarais

S. L.

Sellers

B. G.

Hylton

Tirotti

Van Dorn

R. A.

(2014). From risk assessment to risk management: Matching interventions to adolescent offenders’ strengths and vulnerabilities. Children and Youth Services Review, 47, 1–9.

64.

Skeem

Monahan

Lowenkamp

(2016). Gender, risk assessment, and sanctioning: The cost of treating women like men. Law and Human Behavior, 40(5), 580–593.

65.

Skeem

J. L.

Kennealy

P. J.

Hernandez

(2013). CA-YASI construct validity: To what extent do the domains measure the risk factors they’re supposed to measure? Division of Juvenile Justice. California Department of Corrections and Rehabilitation. Rsilience.berkeley.edu/content/publications-1.

66.

Stevenson

H. E.

Wormith

J. S.

(1987). Psychopathy and the level of supervision inventory (User Report 1987-25). Ministry of the Solicitor General of Canada.

67.

Steyerberg

E. W.

Bleeker

S. E.

Moll

H. A.

Grobbee

D. E.

Moons

K. G.

(2003). Internal and external validation of predictive models: A simulation study of bias and precision in small samples. Journal of Clinical Epidemiology, 56(5), 441–447.

68.

Sullivan

C. J.

McCafferty

Newsome

Mandalari

(2022). Predictive validity and measurement invariance in juvenile risk assessment: Implications for racial and ethnic disparities in juvenile justice. Journal of Crime and Justice, 45, 409–429.

69.

Sullivan

C. J.

Strange

Sullivan

Newsome

Lugo

Mueller

McCafferty

(2019). Multi-method study on risk assessment implementation and youth outcomes in the juvenile justice system. University of Cincinnati, Center for Criminal Justice Research.

70.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed.). Allyn & Bacon/Pearson Education.

71.

Taxman

F. S.

(2017). Handbook of recidivism risk/needs assessment tools . In Singh

J. P.

Kroner

Wormith

Desmarais

Hamilton

& Z. (Eds.), Risk assessment: Where do we go from here? (pp. 269–284). Wiley-Blackwell.

72.

Taxman

F. S.

Caudy

M. S.

(2015). Risk tells us who, but not what or how: Empirical assessment of the complexity of criminogenic needs to inform correctional programming. Criminology & Public Policy, 14(1), 71–103.

73.

Taxman

F. S.

Smith

(2021). Risk-need-responsivity (RNR) classification models: Still evolving. Aggression and Violent Behavior, 59, 101459. https://doi.org/10.1016/j.avb.2020.101459

74.

Thompson

(2004). The value of item response theory in clinical assessment: A review. Assessment, 18(3), 291–307.

75.

Van Voorhis

Wright

E. M.

Salisbury

Bauman

. (2010). Women’s risk factors and their contributions to existing risk/needs assessment: The current status of a gender-responsive supplement. Criminal Justice and Behavior, 37(3), 261–288.

76.

Viglione

Rudes

D. S.

Taxman

F. S.

(2015). The myriad of challenges with correctional change: From goals to culture. European Journal of Probation, 7(2), 103–123.

77.

Vincent

G. M.

Chapman

Cook

N. E.

(2011). Risk-needs assessment in juvenile justice: Predictive validity of the SAVRY, racial differences, and the contribution of needs factors. Criminal Justice and Behavior, 38(1), 42–62.

78.

Vincent

G. M.

Guy

L. S.

Grisso

(2012). Assessment in juvenile justice: A guidebook for implementation. John D. and Catherine T. MacArthur Foundation.

79.

Wong

S. C. P.

Gordon

(2006). The validity and reliability of the Violence Risk Scale: A treatment-friendly violence risk assessment tool. Psychology Public Policy and Law, 12(3), 279–309.

80.

Wormith

S. J.

Bonta

(2018). The level of service (LS) instruments. Handbook of recidivism risk/needs assessment tools (pp. 117–145). John Wiley & Sons.

81.

Zottola

S. A.

Desmarais

S. L.

Lowder

E. M.

Duhart Clarke

S. E.

(2022). Evaluating fairness of algorithmic risk assessment instruments: The problem with forcing dichotomies. Criminal Justice and Behavior, 49(3), 389–410.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB