Evaluator Perspective: A Conversation with Australian Evaluation Society Fellow – Theo Nabben

Members who have made a long-term and wide-ranging contribution to the AES and the field of evaluation are recognised through a special category of membership called ‘Fellow of the AES’.

Theo is an AES member from Perth, Western Australia (WA). His current role is the Monitoring and Evaluation and Learning Manager for the Grower Group Alliance which is the peak body for about 90 agriculture groups in WA. It’s a not-for-profit organisation and is a fairly small group of about 20 people. Theo splits his time between monitoring and evaluation (M&E), as well as providing M&E support to the other projects and working in the agriculture extension space. He still does some work outside of the organisation in terms of training in Most Significant Change (MSC) and since 2025 training via AES and the Institute of Public Administration in M&E, Program Logic and Policy Evaluation.

I asked Theo about the influences which brought you into evaluation (whether it was people or events, choices etc.), and added that these could be very broad

While working at the Department of Agriculture, Western Australia, I undertook a graduate diploma related to agriculture extension, and one of the units was evaluation. Jeff Couts developed the course, and Jess Dart delivered it. I really enjoyed both the course and the material, so when it was finished, I thought I would try some of the logic approaches with my team. My next interaction with evaluation was as the National Landcare Facilitator in South Africa when some evaluators were brought into the program, and I realised this could be a great way to work across many sectors in the international development space. When I returned to WA, we started using logic to design projects; in fact, there was a huge emphasis on program logic, and we contracted Jess Dart to deliver training. After that I suppose you could say that I shadowed Jess Dart and co-delivered some of the training – I certainly learnt heaps from her when she delivered training for us in this area. Then it became difficult for her to come over to the West to deliver the program, so she shared the material with myself and Carina Calzoni, and we took over the delivery of the program from 2005 to 2010. Increasingly, I could see the benefit in evaluation and enjoyed putting the new skills into action. I was lucky enough to get the opportunity to undertake small consultancies, including pro bono work to build my experience. I also undertook training in MSC via Clear Horizon and developed a link to Rick Davies who developed MSC, and so I also developed my expertise in MSC and other evaluation approaches. After several years I decided to take redundancy from the Department, and work on facilitation in the areas of MSC and M&E.

In terms of your evaluation expertise, what do you regard as your main areas of interest, and has that changed over time?

When I started in the evaluation area, my initial interest was in program logic, and to develop my evaluation skills. I have always been interested in qualitative methods. My interest in MSC peaked after initial training in 2006 and has stayed with me and developed over time. I have also developed an interest in complexity aware M&E approaches after working on a project to develop USAIDs capacity in that area. I guess you could say that I differ from more traditional evaluators in WA because of my interest in other models and approaches – ones often used internationally. I have always kept my interest in International Development, mainly because in that arena they seem to try innovative approaches to problems.

Another area which I really enjoy is the participatory evaluation approaches – particularly referring to Most Significant Change process – because it involves stakeholders in making an assessment on what has been achieved and places a strong emphasis on surfacing different perceptions about what’s important for people.

Looking back over your career, thus far, what are some of the major challenges you have faced?

If I think back to early days when I tried my hand at being a sole trader in a regional town in WA, I had a lot of problems getting the business happening. There was a lot of knocking on doors. One of the problems I had was that I had a foot in two camps – domestic and international evaluation. I spent a lot of time overseas on projects, so it was difficult to concentrate on developing the business locally. For example, I might get on a panel, but I would be overseas so couldn’t take the local opportunity up. To be honest, my preference was to work overseas. Also, because I had a range of skills, I was not an evaluator all the time and often did facilitation work—however, that did help with the mortgage! Another challenge was getting across to people that projects needed people with evaluation skills. For example, if there was an evaluation project in the sports field, it would certainly be useful if an evaluator had experience in that area, but you don’t take on someone with sports interest only (which was common) – they need to have evaluation skills.

Another frustration that I found was that within Government there is a tendency to evaluate programs that really do not need to be evaluated. Another issue is that it is often difficult to get across to people what evaluation actually is. Working in Government has its challenges because they are often risk averse and operate in a traditional mode and are not keen on doing anything different in terms of evaluation approaches. Another frustration is that often the results of the evaluation do not go anywhere, documents are not public and there can be limited accountability in terms of justifying (in)action on recommendations or findings. Often it is quite separate from the decision making. Another challenge which probably faces a lot of evaluators is that so many times, evaluation is an afterthought not considered in the early stages of a design nor in program delivery/implementation. Another challenge facing most evaluators is the unrealistic timeframes presented by the client as well as the unrealistic budgets!

Tell me about some of the highlights?

For me, it is the work around MSC which provided me with an extra string to my bow. The knowledge of MSC has opened doors for me, particularly overseas. I have found it to be a great honour to work across different sectors, and in particular the personal interaction the technique provides. The first few times I went overseas with evaluation, I delivered pro bono training in MSC for World Vision or I worked with Clear Horizon so that started to build some credible experience as well as my CV. After that, I took many opportunities in a wide range countries and sectors to build my expertise. For me, the highlight is the close sharing of people’s stories – that personal interaction is a privilege and very special.

Another highlight for me was participation in an International Resource Panel piloting complexity aware M&E for USAID projects. This was a great experience as the range of people was large and my fellow Panel members included international M&E experts like Ricardo-Wilson Grau, Bob Williams and Richard Hummelbrunner. Doing the training in different cultures and languages was an extraordinary experience. Fortunately, there was always someone there to assist in terms of language and cultural norms – very essential! It keeps you on your toes as you have to unpack what is going on – as well as being aware of people’s reactions. You learn to be adaptable and to redesign processes as required. My work has taken me across Africa, the Pacific and Asia. It is essential to work with a local facilitator as they can interpret what is being said. Overall, the highlight has been working with NGOs as I really enjoy the international work. You are always looking at people’s reactions and working out what needs to be adjusted to make it relevant.

Let’s look at the field of evaluation itself. Has it changed during your career – and if yes, in what ways?

Yes, it has changed during the last 20 years. We have moved from a traditional approach where you examine a program and make an assessment as to whether it’s working or not. We have now moved to a much broader approach, a systems approach: how it fits together at a strategy level. Linked to that is that there is an increasing emphasis on values and being upfront about them. If you think about Patton, let’s look at big world transformative issues. There is increasing interest in more of a predictive and real time evaluation emphasis – especially since COVID-19. Another focus here as well as in the rest of the world is the focus on First Nations and valuing those differences. First Nations are clear about their space and the importance of data sovereignty.

If you look at the international development area, it has moved to: ‘we do not bring in the “western expert”’ but ‘let’s bring in a local!’ There is a greater emphasis on evaluation for learning and improvement rather than ‘was it successful’. It was always there but coming across more as is the emphasis on what works for whom, or doesn’t, and under what circumstances.

What are the main skills or competencies that evaluators need to have or need to develop to keep pace with emerging trends in evaluation practice?

There are the normal research skills. You need to have a bit of curiosity – as well as keeping pace with emerging trends. Be curious, know where to look, pick the right websites. There’s a lot of stuff out there. I use LinkedIn since it’s useful for keeping abreast of trends. You need to look at those broader systems, work out who to follow because there is also some rubbish out there: conflation of terms and jumping on bandwagons, many diagrams purport to be a Theory of Change without any theory! Making a real effort to come across and understand the approaches and ways of doing. Good questioning is important, digging deep and finding out what is needed. Of course, the standard skill is a good logical thinking, evaluative thinking. A useful skill is the ability to be able to influence upwards – to the commissioners, or managers. Then it’s about taking some steps to follow through and putting it into practice.

What do you see as the main social issues/problems that evaluators ought to be thinking about and seeking to resolve in the next decade? (That is, if you think that we should be involved – and indeed have a role?)

For me – one of the issues is inequality. I would like to be involved in that particular area. I’m not sure in what way but it would suit my values. The other thing is ‘who’s voice do we bring to the table?’.

I think all of us need to be cognisant of the effects of climate change and sustainability. It has to be in the forefront of people’s minds all of the time.

Tell me about your involvement in the AES?

I became a member of the AES in 1999 and did my first evaluation course with Jess Dart in 1998. I spent a lot of time poring through the Evaluation Journal of Australasia. I also attended the workshops in WA and loved organising local workshops/lunchtime sessions for the AES.

Has your involvement in the Society been beneficial to you, and in what ways?

It’s been very beneficial for me – particularly listening in to the zoom meetings across the different chapters – this helps me to keep across what is happening across the country. Also, the connections with people in the Society. In WA, certainly Rick Cummings has been a great mentor, and Jess Dart has been very influential. I am now taking over delivering courses to the International Public Policy Association (IPPA) in WA and another benefit is having the code of conduct and other documents to assist evaluators.

How can the Society best position itself to still be relevant in the future?

I don’t know whether I have any new ideas, but I do think that the Society needs to keep promoting the importance of evidence for good decision making. I guess my question is once we have promoted them, what do we need to do to keep them alive in the eyes of government. We got IPPA WA to promote evaluation via Public Service Evaluation Award. But has it changed the attitudes of government departments towards evaluation? I was also concerned recently when I looked at a Commonwealth government tender on providing a theory of change – their documentation on Theory of Change was very muddled and confusing. So, there is room to improve their guidance and understanding of concepts. As there is now a good relationship between The Australian Centre for Evaluation (ACE) and the AES, there may be room to expand the relationship to improve evaluation knowledge and methods within government. Also, there are several other organisations where the AES could consider a link – for example, social impact and behavioural economics to name a few.

What are your thoughts on being a Fellow? Are there any areas where you feel you would like to contribute?

I think the experience of nominating was very beneficial in terms of making you reflect on where you have been. It is also important to have your peers recognise you, and I felt truly honoured. At this stage, my contribution would be in training through IPPA as well as my role as a mentor in the AES.

Have you got any advice that you would give to an emerging evaluator?

My advice to an emerging evaluator would be to try many different things, and to try different experiences. Find out what you love and then build on that strength. When I think about me, at the beginning of my career in evaluation, I shadowed Jess, and because I wanted to go overseas, I did some volunteer work with World Vision. This experience gave me frameworks and stories of my own, as well as a network which gave me some good connections. Find those people who are willing to give advice or to work on projects. Build skills – do volunteer opportunities, you will make connections. All evaluators have different skills – mix with them and find what drives you.

Any other comments?

My passion has been the international space, and I would say that getting into this space has been wonderful. I have had the good fortune to travel extensively, and to be involved in a great number of different projects –this has given me a wonderful life!