This article examines the design of evaluations in settings where there is a choice as to how an intervention is to be introduced and evaluated. It uses data from a supervision program for offenders on probation in the UK (Bruce and Hollin forthcoming) that had been indicated by a pilot evaluation in one probation area to merit wider-scale implementation and evaluation. For the remaining two probation areas in the region, a randomized controlled allocation of participants to conditions was recommended. One of the areas adopted a stepped wedge design, in which probation offices were randomly allocated sequentially to the program. The second area opted to launch the program across the whole area simultaneously, with a retrospective sample as control group. The article compares the results of implementation in each probation area and seeks to draw wider inferences about the management of program implementation and the randomized controlled designs appropriate for similar field studies.
Andrews, Don A., Ivan Zinger, Robert D.Hoge, JamesBonta, Paul Gendreau, and Francis T. Cullen .1990. Does correctional treatment work? A clinically relevant and psychologically informed meta-analysis. Criminology28 (3): 369-404.
2.
Antonowicz, Daniel H., and Robert R. Ross.1994. Essential components of successful rehabilitation programmes for offenders. International Journal of Offender Therapy and Comparative Criminology38:97-104.
3.
Briggs, Sarah, and Russell Turner.2003. Barriers to starting programmes: Second phase report. Research report, National Probation Service , West Yorkshire, UK.
4.
Brown, Celia A., and Richard J. Lilford .2006. The stepped wedge trial design: A systematic review. BMC Medical Research Methodology6:54-62.
5.
Bruce, Russell, and Clive R. Hollin. Forthcoming. Developing citizenship. Vista.
6.
Campbell, Siobhan.2003. The feasibility of conducting an RCT at HMP Grendon . Home Office Online Report. http://rds.homeoffice.gov.uk/rds/pdfs2/rdsolr0303.pdf (accessed May 27, 2009).
7.
Cann, Jenny, Louise Falshaw, Francis Nugent, and Caroline Friendshi p. 2003. Understanding what works: Accredited cognitive skills programmes for adult men and young offenders. Home Office Research Findings226. London: Home Office.
8.
Clarke, Alan, Rosemary Simmonds, and Sarah Wydall.2004. Delivering cognitive skills programmes in prison: A qualitative study. Home Office Research Findings242. London: Home Office.
9.
Clarke, Ronald V. G., and Derek B. Cornish.1972. The controlled trial in institutional research: Paradigm or pitfall for penal evaluators?Home Office Research Study16. London: Home Office.
10.
Cornish, Derek B., and Ronald V.G. Clarke.1975. Residential treatment and its effects on delinquency . Home Office Research Study32. London: Home Office.
11.
Debidin, Mia, and Jorgen Lovbakke .2005. Offending behaviour programmes in prison and probation. In The impact of corrections on reoffending: A review of "what works," ed. G. Harper and C. Chitty, 31-54. Home Office Research Study291 (2). London : Home Office.
12.
Dowden, Craig, and Don A. Andrews.2004. The importance of staff practice in delivering effective correctional treatment: A meta-analytic review of core correctional practice. International Journal of Offender Therapy and Comparative Criminology48:180-87.
13.
Falshaw, Louise, Caroline Friendship, Rosie Travers, and Francis Nugent.2004. Searching for "what works": HM Prison Service accredited cognitive skills programmes. British Journal of Forensic Practice6:3-13.
14.
Farrington, David P.2003a. British randomised experiments on crime and justice . The Annals of the American Academy of Political and Social Science589:150-67.
15.
Farrington, David P.2003b. A short history of randomised experiments in criminology: A meagre feast. Evaluation Review27 (3): 218-27.
16.
Farrington, David P., and Darrick Jolliffe .2002. A feasibility study into using a randomised controlled trial to evaluate treatment pilots at HMP Whitemoor . Home Office Online Report14/02. London: Home Office. www.homeoffice.gov.uk/rds/pdfs2/rdsolr1402.pdf (accessed May 28, 2009).
17.
Friendship, Caroline, Linda Blud, Matthew Erikson, Rosie Travers, and David Thornton.2003. Cognitive-behavioural treatment for imprisoned offenders: An evaluation of HM Prison Service’s cognitive skills programmes. Legal and Criminological Psychology8:103-14.
18.
Hewitt, Catherine J., David J. Torgerson, and Jeremy N.V. Miles.2006. Taking account of non-compliance in randomised trials . Canadian Medical Association Journal175:347-48.
19.
Hollin, Clive R.1995. The meaning and implications of programme integrity . In What works: Reducing reoffending: Guidelines from research and practice , ed. J. McGuire, 195-208. Chichester , UK: Wiley.
20.
Lipsey, Mark W.1995. What do we learn from 400 research studies on the effectiveness of treatment with juvenile delinquents? In What works: Reducing reoffending: Guidelines from research and practice, ed. J. McGuire, 63-78. Chichester, UK: Wiley.
21.
Lipsey, Mark W.1999a. Can intervention rehabilitate serious delinquents?The Annals of the American Academy of Political and Social Science564:142-66.
22.
Lipsey, Mark W.1999b. Can rehabilitative programs reduce the recidivism of juvenile offenders? An enquiry into the effectiveness of practical programs . Virginia Journal of Social Policy & the Law6 (3): 610-41.
23.
Lipsey, Mark W., David B. Wilson, and Lynn Cothern.1998. Effective intervention for serious juvenile offenders: A synthesis of research. In Serious and violent juvenile offenders: Risk factors and successful interventions, ed. Ralph Loeber and David P. Farrington.Thousand Oaks, CA: Sage. National Institute of Justice database, http:/www.ncjrs.gov/pdffiles1/ojjdp/181201.pdf (accessed May 28, 2009).
24.
McDougall, Cynthia, Jane Clarbour, Amanda E. Perry, Roger Bowles, and Gillian Worthy.2009. Evaluation of HM Prison Service Enhanced Thinking Skills Programme: Report on the implementation of a randomised controlled trial. Ministry of Justice Research Series4/09.
25.
McDougall, Cynthia, Amanda E. Perry, Jane Clarbour, Roger Bowles, and Gillian Worthy.2009. Evaluation of HM Prison Service Enhanced Thinking Skills Programme: Report on the outcomes from a randomised controlled design . Ministry of Justice Research Series3/09.
26.
McDougall, Cynthia, Amanda E. Perry, and David P. Farrington.2006. Reducing crime. In Reducing crime: The effectiveness of criminal justice interventions , ed. Amanda E. Perry, Cynthia McDougall, and David P. Farrington , 1-11. Chichester, UK: Wiley.
27.
Pawson, Ray, and Nick Tilley.1998. Caring communities, paradigm polemics, design debates. Evaluation4:73-90.
28.
Redondo, Santiago, Julio Sanchez-Meca, and Vicente Garrido .1999. The influence of treatment programmes on the recidivism of juvenile and adult offenders: A European meta-analytic review. Psychology, Crime and Law5:251-78.
29.
Sherman, Lawrence W., Denise Gottfredson, Doris MacKenzie, John Eck, Peter Reuler, and Shawn Bushay.1997. Preventing crime: What works, what doesn’t, what’s promising: A report to the United States Congress. National Institute of Justice database, http://www.ncjrs.gov/pdffiles/171676.pdf .
30.
van Voorhis, Patricia, Lisa M.Spruance, P.Neal Ritchey, Shelley J. Listwan, and Renita Seabrook.2004. The Georgia cognitive skills experiment: A replication of Reasoning and Rehabilitation. Criminal Justice and Behavior31:282-305.
31.
Weisburd, David, Cynthia M. Lum, and Anthony Petrosino .2001. Does research design affect study outcomes in criminal justice?The Annals of the American Academy of Political and Social Science578:50-70.
32.
Williams, Mark.1975. Aspects of the psychology of imprisonment. In The use of imprisonment, ed. S. McConville , 32-42. London: Routledge and Kegan Paul.
