Important Methodological Issues in Evaluating Community-Based Interventions

Abstract

Background:

Evaluation of offending behavior programs has become reasonably controversial in recent times as some researchers have advocated that evaluations should use a treatment as received (TR) model, which ignores dropouts and refusers, despite criticism that this removes the ability to make inferences about the effectiveness of the program. Some of those arguing in favor of a TR methodology suggest that the large number of noncompleters in community-based evaluations will make it impossible to detect treatment effects. A further issue with evaluation of community-based offender behavior programs is the fact that many programs are reasonably long. Evaluation methodologies inherited from custodial programs count recidivism from day one—which is before any benefits from the program are hypothesized to occur.

Objectives:

This article explores these issues to find solutions that are scientifically robust but also sensitive to actual program effects.

Conclusion:

The current authors argue that evaluations should use an intention-to-treat (ITT) model which includes noncompleters and that, using this methodology, an effective program, delivered with high integrity, should be able to demonstrate an impact on offending. However, the time period selected for evaluation should relate to the hypothesized effects of the program, not just begin from the start of supervision or commencement of the program. In doing so, the evaluators should ensure that both treatment and control groups are treated equally.