Evaluation Journal of Australasia

An organisation wishes to evaluate one of its programs. It can ask a staff member or hire someone outside the organisation. Which should it choose?

Surprisingly little guidance is available for this common scenario. A review of 30 texts dealing with organisational performance and evaluation shows that too often the issue is assumed one way or the other.

Management texts aimed at business and organisational audiences tend to presume that evaluation is conducted by internal evaluators, usually managers. By contrast the specialist evaluation literature almost always proceeds from the opposite assumption: that evaluation is undertaken by external evaluators.

This paper proposes a series of measures for comparing the strengths and weaknesses of internal and external evaluators. These include cost, knowledge, flexibility, objectivity, accountability, willingness to criticise, ethics and utilisation of results.

A set of guidelines is offered to assist organisations in choosing between internal and external evaluation in each particular case.

Attempting to balance multiple stakeholder interests in program evaluation presents many challenges. The views of different stakeholders as part of a reference group, or multiple stakeholder voices within a sector, are often diverse and reflect different political and organisational interests. In order to ensure that the evaluation product is widely accepted, and thereby utilised, these differences need to be recognised, and mediated.

To work effectively with multiple stakeholders in this manner, the evaluator requires well developed negotiation skills. This paper argues that negotiation is an essential component to the planning stage of an evaluation, and that strategic steps need to be taken early in the evaluation process to ensure consensus is developed in stakeholder expectations regarding methodology and outcomes from the evaluation.

This article will put forward a number of negotiation principles for evaluation practice which view the evaluator as enabling stakeholders to appreciate all positions, including that of the evaluator, with consensus emerging from increased understanding and consciousness raising.

This article examines how the latent potential of communities of practice (CoPs) can be harnessed as a strategic resource for building capacity and improving organisational performance. In particular it focuses on a case study of developing communities of practice in the Philippines Department of Education (DepEd) in three Regions (XI, XII and the Autonomous Region of Muslim Mindanao (ARMM)), which have been supported under the Philippines-Australia Basic Education Assistance for Mindanao (BEAM) Project.

This article describes community-building activities sponsored by BEAM to strengthen local Department of Education (DepEd) management capacity to assume greater responsibilities following the passing of the Governance of Basic Education Act 2001. It maps and examines the process of connecting staff who were previously performing the same function independently, to come together to form knowledge sharing networks. Real examples of the changes that emerging CoPs are delivering are also provided. Discussion then turns to the new challenges CoPs face in sustaining the momentum and benefits that these professional networks are yielding.

The use of Capability Maturity Models in financial management, project management, people management and information systems management in a wide variety of organisations indicates the potential for an Organisational Evaluation Capability Hierarchy to guide the self-diagnosis of organisations in building their evaluation maturity. This paper is about the theory behind this growing trend in organisational governance and organisational diagnosis, and explores its relevance to evaluation theory and practice. This theoretical analysis may have long-term practical benefits for evaluation practitioners, as is being developed in the fields of project management, financial management, and people management in a wide range of organisations.

This paper explores the dynamics of peoples' responses to questions and uses case studies to explore ways in which the validity of data can be improved. Much social research takes for granted that the process of asking questions through interview, survey or focus groups provides accurate data about behaviour, perceptions and attitudes. However, the literature suggests that many questionnaires produce inaccurate data. Cognitive psychologists report that people tend to minimise the difficult recall or imaginative tasks when answering questions. Instead, respondents use ‘schemas’ or ‘scripts’ to interpret and respond to their immediate situation. A schema provides a ‘logic’ or ‘rationality’ that informs their responses. Some practitioners have found that they get more robust results by asking respondents to recreate mentally, specific events before asking questions based on those events.

This article reviews the use of photographs as data within the social sciences as well as defining related terminology used over the past century. It then examines the use of photos as stimuli for talking about health settings before presenting three recent case studies where photo-interviewing has been used successfully in health evaluation and research. Advantages and limitations of the method are considered.