Abstract
The short history of evaluation research can be described in terms of four developmental stages. The differences between the various approaches and stages of evaluation reflect, among other things, differences in understanding the importance of causality. The paper makes an attempt to argue that whilst it is important not to ‘let go’ the goal of discovering causal explanations, evaluation of such complex efforts as local prevention of alcohol and drug-related problems, needs to be based on a more naturalistic approach than the one provided by the classic experimental setting. This argument is based on previous literature and the experiences of evaluation of a Finnish project concerned with the community-based prevention of alcohol and drug problems. Evaluation can also assume a function more complementary to the partnerships, which are at the heart of community action through its formative role. Research does not have to limit itself to evaluating the effectiveness through the eyes of an outside ‘inspector’.
One possible step forward in solving the complex problems of evaluation in this context could be an approach which combines the careful analysis of the context and the creation of a theory-based narrative of the action taking place in the ‘black box’, with measurement of changes. The precondition for being able to do this is that causality is understood in a new way, as generative causality. The theory-based narratives would also be useful in generalising the results of separate evaluation studies. Research which is relevant to questions of effectiveness, should pay more attention to the ‘black box’ of prevention. The nature of the ‘black box’ is determined not only by its inner qualities but also by being studied in relation to its wider context.
Reaching generalisations is also one of the basic problems. The most reliable results are often derived from a synthesis of several studies focusing on the same topic rather than from a single study. In this respect evaluations resemble any other form of social study. Making a synthesis has become easier with the development of methods of meta-evaluation during the last 15 years. Another possible way of assisting generalisation would be to make use of theory-based narratives concerning the inner mechanisms of the communities in action.
As an example of a project attempting to carry out research along these lines, the paper describes a project carried out in two small communities in the metropolitan area of Helsinki, Finland. The project attempted to move away from the use of standardised programmes and curricula towards a meaningful involvement of local citizens and professionals in the design and implementation of the community development. The evaluation of the project focuses on the question: is it feasible and effective to give the responsibility for alcohol and drug prevention to people at the local level? What does this mean in practice? Related to this question is the importance of national or regional level support for local communities. The project's finding was that while public discussion of community-based action can change people's opinions and make them more positive in their attitude to community-based prevention, local non-professional interest in dealing with alcohol-related problems is small. Without the professionals' input hardly anything would take place.