I read the Williams, Wormith, Bonta, and Sitarenios (2017) critique of the Singh, Grann, and Fazel (2011) meta-analysis as someone who has struggled with conceptual and methodological issues when conducting risk assessment meta-analyses. I offer four lessons for risk assessment meta-analysts, based on my experiences as a risk assessment meta-analyst and consumer of risk assessment research. Lesson 1: We should accept the Dodo bird verdict in risk assessment instrument research and then move forward. Lesson 2: If you must compare, compare within samples. Lesson 3: Meta-analyses are often more useful for showing how much we don't know than how much we do know. Lesson 4: There are not enough risk assessment instrument studies examining the predictive validity of commonly used score interpretations.
Get full access to this article
View all access options for this article.
References
1.
AndrewsD. A.BontaJ. (1995). Levels of Service Inventory—Revised: User's manual. Toronto, Canada: Multi-Health Systems.
2.
BlairP.MarcusD.BoccacciniM.T. (2008). Is there an allegiance effect for assessment instruments? Actuarial risk assessment as an exemplar. Clinical Psychology: Science and Practice, 15, 346–360.
3.
BoccacciniM. T.MurrieD. C.CapertonJ. D.HawesS. W. (2009). Field validity of the STATIC-99 and MnSOST-R among sex offenders evaluated for commitment as sexually violent predators. Psychology, Public Policy, and Law, 15, 278–314. doi: 10.1037/a0017232
4.
ChevalierC.BoccacciniM. T.MurrieD. C.VarelaJ. G. (2015). Static-99R reporting practices in sexually violent predator cases: Does norm selection reflect adversarial allegiance? Law and Human Behavior, 39, 209–218. doi:10.1037/lhb0000114
5.
GardnerB. O.BoccacciniM. T.BittingB.EdensJ. F. (2015). Personality Assessment Inventory scores as predictors of misconduct, recidivism, and violence: A meta-analytic review. Psychological Assessment, 27, 534–544. doi: 10.1037/pas0000065
6.
HansonR. K.Morton-BourgonK. E. (2009). The accuracy of recidivism risk assessments for sexual offenders: A meta-analysis of 118 prediction studies. Psychological Assessment, 21, 1–21. doi: 10.1037/a0014421
7.
HansonR. K.ThorntonD. (2000). Improving risk assessments for sex offenders: a comparison of three actuarial scales. Law and Human Behavior, 24, 119. doi: 10.1023/A:1005482921333
8.
HareR. D. (2003). The Hare Psychopathy Checklist—Revised (2nd ed.). Toronto, Ontario, Canada: Multi-Health Systems.
9.
HarrisG. T.RiceM. E.QuinseyV. L. (2010). Allegiance or fidelity? A clarifying reply. Clinical Psychology: Science and Practice, 17, 82–89.
10.
HawesS.BoccacciniM. T. (2009). Detection of overreporting of psychopathology on the Personality Assessment Inventory: A meta-analytic review. Psychological Assessment, 21, 112–124. doi: 10.1037/a0015036
11.
HawesS. M.BoccacciniM. T.MurrieD. C. (2013). Psychopathy and the combination of psychopathy and sexual deviance as predictors of sexual recidivism: Meta-analytic findings using the Psychopathy Checklist-Revised. Psychological Assessment, 25, 233–243. doi: 10.1037/a0030391
12.
HowardP. D. (2016). The effect of sample heterogeneity and risk categorization on area under the curve predictive validity metrics. Criminal Justice and Behavior. Advanced online publication. doi: 10.1177/0093854816678899
13.
LuborskyL.DiguerL.SeligmanD. A.RosenthalR.KrauseE. D.JohnsonS. (1999). The researcher's own therapy allegiances: A “wild card” in comparisons of treatment efficacy. Clinical Psychology: Science and Practice, 6, 95–106.
14.
LuborskyL.RosenthalR.DiguerL.AndrusynaT. P.BermanJ. S.LevittJ. T. (2002). The dodo bird verdict is alive and well—mostly. Clinical Psychology: Science and Practice, 9, 2–12. doi: 10.1093/clinpsy.9.1.2
15.
MurrieD. C.BoccacciniM. T.RufinoK.CapertonJ. (2012). Field validity of the Psychopathy Checklist-Revised in sex offender risk assessment. Psychological Assessment, 24, 524–229. doi: 10.1037/a0026015
16.
RosenzweigS. (1936). Some implicit common factors in diverse methods of psychotherapy: “At last the Dodo said, ‘Everybody has won and all must have prizes.’” American Journal of Orthopsychiatry, 6, 412–415.
17.
SinghJ. P.GrannM.FazelS. (2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31, 499–513. doi: 10.1016/j.cpr.2010.11.009
18.
VarelaJ. G.BoccacciniM. T.MurrieD. C.CapertonJ.GonzalezE.Jr. (2013). Do the Static-99 and Static-99R perform similarly for White, Black, and Latino offenders?International Journal of Forensic Mental Health, 12, 231–243. doi: 10.1080/14999013.2013.846950
19.
WilliamsWormithJ. S.BontaJ.Sitarenios (2017). The use of meta-analysis to compare and select offender risk instruments: A commentary on Singh, Grann, and Fazel (2011). International Journal of Forensic Mental Health. Advance online publication.
