Multimethod Strategy for Assessing Program Fidelity: The National Evaluation of the Revised G.R.E.A.T. Program

Abstract

This study reports the results of the process evaluation component of the Process and Outcome Evaluation of the Gang Resistance Education and Training (G.R.E.A.T.) program. The process evaluation consisted of multiple methods to assess program fidelity: (a) observations of G.R.E.A.T. Officer Trainings (G.O.T); (b) surveys and interviews of G.R.E.A.T.-trained officers and supervisors; (c) surveys of school personnel; and (d) ‘‘on-site,’’ direct observations of officers delivering the G.R.E.A.T. program in the study sites. Results illustrate a high level of program fidelity, providing greater confidence in any subsequent outcome results.

Keywords

fidelity process evaluation program evaluation prevention

Get full access to this article

View all access options for this article.

References

Basch, C.E. , E.M. Sliepcevich , R.S. Gold , D.F. Duncan , and L.J. Kolbe. 1985. ‘‘Avoiding Type III Errors in Health Education Program Evaluations: A Case Study.’’ Health Education and Behavior 12:315-31.

Dane, A.V. , and B.H. Schneider. 1998. ‘‘Program Integrity in Primary and Early Secondary Prevention: Are Implementation Effects out of Control ?’’ Clinical Psychology Review 18:23-4.

Dobson, D. , and T.J. Cook. 1980. ‘‘Avoiding Type III Error in Program Evaluation: Results From a Field Experiment.’’ Evaluation and Program Planning 3:269-76.

Dumas, J.E. , A.M. Lynch , J.E. Laughlin , E.P. Smith , and R.J. Prinz. 2001. ‘‘Promoting Intervention Fidelity: Conceptual Issues, Methods, and Preliminary Results From the Early Alliance Prevention Program.’’ American Journal of Preventive Medicine 20:38-47.

Durlak, J.A. , and E.P. DuPre. 2008. ‘‘Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation.’’ American Journal of Community Psychology 41:327-50.

Dusenbury, L. , R. Brannigan , M. Falco , and W.B. Hansen. 2003. ‘‘A Review of Research on Fidelity of Implementation: Implications for Drug Abuse Prevention in School Settings.’’ Health Education Research 18:237-56.

Elliott, D.S. , and S. Mihalic. 2004. ‘‘Issues in Disseminating and Replicating Effective Prevention Programs.’’ Prevention Science 5:47-53.

Emshoff, J.G. , C. Blakely , R. Gottschalk , J. Mayer , W.S. Davidson , and S. Erickson. 1987. ‘‘Innovation in Education and Criminal Justice: Measuring Fidelity of Implementation and Program Effectiveness.’’ Educational Evaluation and Policy Analysis 9:300-11.

Esbensen, F.A. , A. Freng , T.J. Taylor , D. Peterson , and D.W. Osgood. 2002. ‘‘Putting Research Into Practice: The National Evaluation of the Gang Resistance Education and Training (G.R.E.A.T.) Program.’’ In Responding to Gangs: Evaluation and Research, edited by W. L. Reed and S. H. Decker , 139-67. Washington, DC: U.S. Department of Justice, National Institute of Justice.

10.

Esbensen, F.A. , and D.W. Osgood. 1999. ‘‘Gang Resistance Education and Training (G.R.E.A.T.): Results From the National Evaluation. ’’ Journal of Research in Crime and Delinquency 36:194-225.

11.

Esbensen, F.A. , D.W. Osgood , T.J. Taylor , D. Peterson , and A. Freng. 2001. ‘‘How Great is G.R.E.A.T.? Results From a Quasi-Experimental Design.’’ Criminology and Public Policy 1:87-118.

12.

Esbensen, F.A. , D. Peterson , T.J. Taylor , A. Freng , D.W. Osgood , D.C. Carson , and K.N. Matsuda. 2011. ‘‘Evaluation and Evolution of the Gang Resistance Education and Training Program.’’ Journal of School Violence 10:53-70.

13.

Fagan, A.A. , K. Hanson , J.D. Hawkins , and M.W. Arthur. 2008. ‘‘Bridging Science to Practice: Achieving Prevention Program Fidelity in the Community Youth Development Study.’’ American Journal of Community Psychology 41:235-49.

14.

Fagan, A.A. , and S. Mihalic. 2003. ‘‘Strategies for Enhancing the Adoption of School-Based Prevention Programs: Lessons Learned From the Blueprints for Violence Prevention Replications of the Life Skills Training Program.’’ Journal of Community Psychology 31:235-54.

15.

Gottfredson, D. 2001. Schools and Delinquency. New York, NY: Cambridge University Press.

16.

Heller, K. 1996. ‘‘Coming of Age in Prevention Science: Comments on the 1994 National Institute of Mental Health-Institute of Medicine Prevention Reports.’’ American Psychologist 51:1123-7.

17.

Kovaleski, J.H. , E.E. Gickling , H. Morrow , and P.R. Swank. 1999. ‘‘High Versus Low Implementation of Instructional Support Teams: A Case for Maintaining Program Fidelity. ’’ Remedial and Special Education 20:170-83.

18.

Lillehoj, C.J. , K.W. Griffin , and R. Spoth. 2004. ‘‘Program Provider and Observer Ratings of School-Based Preventive Intervention Implementation: Agreement and Relation to Youth Outcomes.’’ Health Education and Behavior 31:242-57.

19.

Lipsey, M. 2009. ‘‘The Primary Factors That Characterize Effective Interventions With Juvenile Offenders: A Meta-Analytic Overview.’’ Victims and Offenders 4:124-47.

20.

McHugo, G.J. , R.E. Drake , R. Whitley , G.R. Bond , K. Campbell , C.A. Rapp , H.H. Goldman , W.J. Lutz , and M.T. Finnerty. 2007. ‘‘Fidelity Outcomes in the National Implementing Evidence-Based Practices Project.’’ Psychiatric Services 58:1279-84.

21.

Melde, C. , F.A. Esbensen , and K. Tusinski. 2006. ‘‘Addressing Program Fidelity Using Onsite Observations and Program Provider Descriptions of Program Delivery .’’ Evaluation Review 30:714-30.

22.

Mihalic, S.F. , and K. Irwin. 2003. ‘‘Blueprints for Violence Prevention: From Research to Real-World Settings-Factors Influencing the Successful Replication of Model Programs.’’ Youth Violence and Juvenile Justice 1:307-29.

23.

Moncher, F.J. , and R.J. Prinz. 1991. ‘‘Treatment Fidelity in Outcome Studies.’’ Clinical Psychology Review 11:247-66.

24.

Peterson, D. , and F.A. Esbensen. 2004. ‘‘The Outlook is G.R.E.A.T.: What Educators Say About School-Based Prevention and the Gang Resistance Education and Training (G.R.E.A.T.) Program.’’ Evaluation Review 28:218-45.

25.

Rossi, P.H. , H.E. Freeman , and M.W. Lipsey. 1999. Evaluation: A Systematic Approach.’’ 6th ed. Thousand Oaks, CA: SAGE.

26.

Taylor, T.J. , F.A. Esbensen , and D. Peterson. 2009. National Evaluation of the Gang Resistance Education and Training (G.R.E.A.T.) Program: G.R.E.A.T. Officer Training (G.O.T.) Report. St. Louis, MO: University of Missouri-St. Louis.

27.

Teague, G.B. , G.R. Bond , and R.E. Drake. 1998. ‘‘Program Fidelity in Assertive Community Treatment: Development and Use of a Measure.’’ American Journal of Orthopsychiatry 68:216-32.

28.

Winfree, L.T., Jr. , D. Peterson Lynskey , and J.R. Maupin. 1999. ‘‘Developing Local Police and Federal Law Enforcement Partnerships: G.R.E.A.T. as a Case Study of Policy Implementation.’’ Criminal Justice Review 24:145-68.