Abstract
Introduction
As global demand for evaluation grows, advancing individual professional development is essential for the evaluation profession. Self-reflection and self-assessment on essential competencies are powerful ways to focus learning pathways for evaluators; they can help foster a culture of continuous learning for those in the field, which in turn supports the strengthening of evaluation systems. Collaboration is another key strategy essential for strengthening evaluation systems. Collaboration leverages diverse expertise, promotes shared learning, maximises resource efficiency, ensures the inclusion of diverse perspectives and, most importantly for this research project, fosters innovation in evaluation methods and tools. The Learn Evaluation Assessment Portal (LEAP) was developed and piloted by the research group (learnevaluation.org) explicitly to advance evaluation professional development through self-assessment, collaboration, and reciprocity.
Creating the LEAP involved collaboration between learnevaluation.org (three academics from the Universities of Melbourne (Australia), Mississippi (United Sates of America) and Stellenbosch (South Africa)), a private service provider (Matilda Tech), and the Australian Evaluation Society (AES) as the first Voluntary Organisation of Professional Evaluators. The research group was interested in understanding more about those who participated in the self-assessment – their evaluation competencies, their academic and work backgrounds, and other demographic characteristics, to help inform pathways for evaluation training. Rather than attempting to collect that information from a one-time survey, the researchers created a tool that could be used by individuals, any Voluntary Organisation of Professional Evaluators, or organisations globally to get a visual map of their (or their members’) self-assessed expertise on evaluation competencies. Evaluators who complete their self-assessment on the LEAP and consent to sharing their data contribute to evaluation research, as the LEAP research team uses it to explore evaluators’ learning needs, their levels of experience, and the sectors in which they work.
This Special Issue describes the development and piloting processes of the LEAP with AES and additional use and analysis. The processes are described in detail so that other Voluntary Organisations of Professional Evaluators, academic programmes, government departments, non-governmental organisations, and other organisations involved in evaluation can emulate the processes described.
The four articles featured in this issue each address critical aspects of the self-assessment tool and competency frameworks (the AES framework specifically but also competency frameworks generally) and their practical applications within the field of evaluation.
Key issues raised in the articles
This Special Issue is structured around the central theme of building evaluation capacity using self-assessment and competency frameworks. The first article, ‘Making the Learn Evaluation Assessment Portal: Developing a reciprocal tool for learning and research’, outlines the development and piloting of the Learn Evaluation Assessment Portal. The article addresses the gap, as identified by the International Society for Evaluation Education, in publicly available instruments that allow evaluators to assess their competencies. Despite challenges, such as the Dunning–Kruger effect (Kruger & Dunning, 1999), which causes novices to overestimate and experts to underestimate their skills, self-assessment remains a valuable tool for evaluators.
The second article, ‘Moving from guideline to measure to findings: The Australian Evaluation Society and the Learn Evaluation Assessment Portal’, chronicles the evolution of the AES Competency Framework (AES Professional Learning Committee, 2013) and reports on findings from the self-assessment process. The article discusses the revisions necessary to adapt the framework for self-assessment and presents data collected through its use. The findings reveal that self-assessment serves as a useful tool for both individual evaluators and organisations, enabling more targeted evaluation capacity building.
The third article, ‘Framing the future: Exploring the values that underpin the Australian Evaluation Society’s Evaluators’ Professional Learning Competencies’, provides a critique of the AES Competency Framework, exploring the philosophical and theoretical assumptions underpinning it. The article highlights how competency frameworks shape the way evaluation is conceptualised and practised, categorising the AES competencies into theoretical paradigms. While the framework is praised for its focus on values, stakeholders, and rigorous research methods, there is further work needed to address cultural safety, especially regarding First Nations contexts (Mertens & Wilson, 2018; Schwandt & Gates, 2021). The symbiotic relationship between the LEAP tool and competency frameworks means that the tool is only as good as the frameworks that are used in it.
The fourth article, ‘Evaluative attitude in action: Using the Learn Evaluation Assessment Portal in graduate education’, is a practice article that discusses how the LEAP tool has been incorporated into a university’s evaluation courses. The article describes the valuable role of self-assessment in learning to support the professional development of evaluation students. It underscores the importance of students understanding their evaluation skills and knowledge and how self-assessment can help shape their learning experiences.
The importance of this issue
This issue contributes to the ongoing conversation around the professionalisation of evaluation and the role that competencies play in that process. As the field continues to grow and diversify, the development of robust competency frameworks and self-assessment tools is crucial. These tools not only support evaluators in assessing their skills and knowledge but also contribute to the broader goals of capacity building within the profession. The articles in this issue provide theoretical insights and practical examples of how competency frameworks and self-assessment tools can enhance the evaluation profession. By addressing both the challenges and benefits of self-assessment, this issue offers valuable lessons for evaluators, educators, and organisations dedicated to fostering a learning environment in evaluation. As competency frameworks become more widely adopted, their impact on the professionalisation of the field will continue to grow, ensuring that evaluators possess the necessary knowledge, skills, and values to deliver high-quality evaluations that meet the needs of stakeholders.
As of 2024, the LEAP can use competency frameworks from any country or organisation which makes it a critical tool for developing the profession. In addition, the public nature of the LEAP has made it possible for all organisations to use it for their own ends. At the time of writing this editorial, the New South Wales Department of Education and the South African Monitoring and Evaluation Association had used LEAP to assess the learning needs of their staff and members.
If you are interested in using LEAP for your organisation, at the time of this publication, there are two options: (1) do it free through the public portal at https://learnevaluation.org/, which uses the Australian Evaluation Society Evaluator Competencies and South African Evaluator Competencies, or (2) work with the learnevaluation.org team for a fee to create a portal for your organisation or group based on your competency set.
