The Judgement Process in Management Students' Evaluation of their Learning Experiences

Abstract

The ideal approach to evaluating management development programmes has often been held to involve objective measurement of achievement of criteria, carried out within an experimental design which allows any chance discovered to be attributed to the programme itself rather than any other factor. In practice many evaluation studies rely on the collection, after the programme, of evaluative opinions from the participants. Campbell et al. (1970) have pointed out that such opinionbased follow up studies may give valid results if the judgements made by the participants, which underlie their expressed opinions, themselves embody the logical and scientific procedures that give validity to the experimental type of study.

This research investigates the process of judgement underlying the formation of evaluative opinions by programme participants. The methodology of 'protocol analysis', used so far mainly in laboratory studies, is employed. This calls for an inductive analysis of the content of recordings made when a person is asked to 'think aloud' while making a decision. Although this is a controversial methodology from some epistemological standpoints, it is also one with many potentially useful applications in the applied social sciences.

The results show that many of the features of valid judgement, like the use of objectifiable criteria, and 'control group' logic are present, to some extent, in the judgement process underlying evaluative opinions. This suggests that a certain amount of validity can be attached to opinion based follow-up studies, and that there is the possibility of designing such studies to encourage these processes to occur, and hence to enhance the validity of the conclusions.