Abstract
Evaluation is essential for assessing whether policy and program outcomes align with their intended goals. A key critique of Western evaluation approaches is their failure to incorporate Indigenous ways of knowing, being and doing. Developing an Indigenist evaluation concept can enhance evaluation quality by recognising Indigenous perspectives. This study draws on Indigenous Lifeworlds as a theoretical basis to explore the idea of ‘collective capability’, asserting Indigenous societies operate at the ‘collective’ level over the individual and focus on ‘capability’ as a way of realising potential through collective effort. We conducted 20 in-depth interviews with Aboriginal and Torres Strait Islander Knowledge holders involved in evaluation to identify attributes of collective capability. Through manifest and latent content analysis, we defined collective capability and its key components. Our definition describes it as people coming together in a relational, collaborative environment to pursue a common goal, where cultural values are prioritised, knowledge is shared and the process is as important as the outcome. The two main elements of collective capability are relationality and knowledge sharing, each with several sub-elements. The next phase of the research will focus on integrating these elements into the context of Indigenist evaluation, developing criteria and processes for their operationalisation.
Keywords
A reform is underway to improve Indigenous evaluation practices, to better understand how best to design evaluations by centring Indigenous knowledge and worldviews. Available evidence highlights evaluations are philosophically and methodologically misaligned with Indigenous ways of knowing, doing and being.
For the first time, we have key Indigenous concepts characterised in the context of Indigenist evaluation, to contribute to evaluation methodology and inform practice. Collective capability has been defined and can inform evaluation principles and frameworks.What We Already Know
The Original Contribution the Article Makes to Theory and/or Practice
Background
Evaluation practices tend to be dominated by neo-liberal values and Western-framed epistemologies and methods, which are individualistic, competitive and outcomes-focused (Baum et al., 2016; Masters-Awatere & Nikora, 2017). Consequently, evaluation design and methodologies are often predetermined and formulated by the funders, extractive and positioned from settler colonial perspectives (Finlay et al., 2023; Taylor, 2003). There is a realisation that Western neo-liberal approaches are not working for evaluations of Indigenous programs, policies and services because they exclude Indigenous knowledge, values, merit and perspectives (Maddox et al., 2021; Masters-Awatere & Nikora, 2017; McCausland, 2019).
Internationally, there have been advances in Indigenous-led evaluation practice, guided by methodologies, frameworks and principles that are aligned with Indigenous Lifeworlds (Chandna et al., 2019; Gaotlhobogwe et al., 2018; Maddox et al., 2021; Masters-Awatere, 2002; Waapalaneexkweew (Nicole Bowman) & Dodge-Francis, 2018; Wehipeihana, 2019) – a concept which recognises peoples’ living realities and shared understandings of the social and cultural conditions related to Indigenous ways of knowing, of being and of doing (Martin & Mirraboopa, 2003; Peacock & Prehn, 2021; Wilson, 2020; Zubrzycki et al., 2017). For Indigenous people in an Anglo-colonised country like Australia, there are two unique sets of intersubjectivities to the Indigenous Lifeworld: first, as Indigenous people with their own distinct culture and knowledge, ways of understanding the social and natural environment and collective sense of caring for others; and second, as colonised peoples living within a dominant culture imbued with colonial society institutions and structures (Walter, 2023).
In Australia, an Indigenous-led evaluation practice that is ‘by, for, with and as’ Aboriginal and Torres Strait Islander people is yet to be established. There has been a paucity of theorising and resourcing for Aboriginal and Torres Strait Islander people and communities to (re)design evaluation processes beyond engagement (Kelaher et al., 2018; Maddox et al., 2021; McCausland, 2019; Vine et al., 2023; Williams, 2018). Too often, evaluations are conducted in communities and organisations without meaningful involvement of Aboriginal and Torres Strait Islander people, and without consideration of the broader social and historical context. Current evaluation practice remains consultative, lacking genuine partnerships and relationship-building with Aboriginal and Torres Strait Islander communities (McCausland, 2019; Rogers et al., 2017; Rossingh & Yunupingu, 2016).
Progress to support Indigenous evaluation has resulted in frameworks and cultural protocols to guide Indigenous approaches to evaluation, support engagement processes and Indigenous capability in evaluations (Grey et al., 2018; Kelaher et al., 2018; Rogers et al., 2017; Williams, 2018). These are, however, often positioned in cross-cultural settings whereby ‘co-design’ is how Indigenous perspectives and knowledge are integrated in evaluation practice that are predominantly led by non-Indigenous evaluators. There is little to no Indigenous leadership or governance structures embedded in processes or planning of evaluations (Kelaher et al., 2018; Maher et al., 2021; Vine et al., 2023; Williams, 2018). Epistemic injustices continue to perpetuate dominant non-Indigenous worldviews and practices, and Indigenous sovereignty, grounded in Indigenous knowledge systems and practices, remains largely undervalued, despite being essential to evaluation as a tool for self-determination (Dudgeon & Bray, 2023). The absence of Indigenous epistemologies and Indigenous concepts in evaluation methodologies, engagement, design and methods leads to evaluations that ‘narrowly measure predetermined outcomes achieved through a linear cause–effect intervention’ (Lokuge et al., 2017; Patton, 2006), misaligning them with Aboriginal and Torres Strait Islander priorities and worldviews (Dudgeon & Bray, 2023; Maddox et al., 2021; Maher et al., 2021; Williams, 2018). These misaligned practices show evaluations serve funders and commissioners, not Aboriginal and Torres Strait Islander communities (Finlay et al., 2021, 2023; Stewart & Jarvie, 2015).
Australia’s Indigenous Evaluation Strategy (IES) is a government framework highlighting the inclusion of Aboriginal and Torres Strait Islander perspectives and knowledge systems as central to evaluation practice in the public sector (Productivity Commission, 2020b). The IES was developed in response to the lack of high-quality evaluations of Indigenous policies and programs, which have been ineffective in reporting evidence-based policy efforts such as Closing the Gap. A 2012 Productivity Commission roundtable highlighted both the scarcity of high-quality evaluations and the limited uptake of findings by government agencies, while a 2016 report by the Centre for Independent Studies found that only 8% of Indigenous programs are evaluated and published (Hudson et al., 2017). Further, despite evidence from evaluations being generated, the uptake and translation to policy is poor, misguided and there is little improvement in outcomes for Aboriginal and Torres Strait Islander people (Kelaher et al., 2018; Productivity Commission, 2020c; Stewart & Jarvie, 2015). To address these gaps, the IES recommends strengthening evaluation and cultural capability within the Australian Public Service, to ensure that commissioning processes, policy design and program development are better aligned with Aboriginal and Torres Strait Islander worldviews and knowledge systems (Finlay et al., 2021; Productivity Commission, 2020b). A companion guide was also developed offering practical advice to Australian Government agencies on evaluating policies and programs affecting Aboriginal and Torres Strait Islander peoples. The guide outlines stages of evaluation, including design, planning, conduct, reporting and use of findings, and provides guidance on embedding evaluation into policy design, selecting programs to evaluate and promoting a culture of evaluation (Productivity Commission, 2020a). Additionally, the guide recommends establishing Indigenous governance and leadership structures in evaluation planning and design, ensuring that evidence generated through evaluations to be more aligned with Aboriginal and Torres Strait Islander peoples’ perspectives and aspirations of health and well-being (Productivity Commission, 2020a; 2020b).
Elements of Indigenist Evaluation
Context, collectivism and capabilities are important elements in Indigenist evaluation, which asserts Indigenous leadership and decision-making, and privileges Indigenous Lifeworlds and knowledge (Maher et al., 2021; Rigney, 2006; Walter, 2023). Māori, for example, assert that evaluations of Indigenous programs are ‘by Indigenous, for Indigenous, with Indigenous and as Indigenous’ (Kerr, 2012; Wehipeihana, 2019). Māori epistemologies are central to Kaupapa Māori evaluation, with relational elements of Māori sovereignty, knowledge, and cultural practices contributing to the growth of Māori evaluators. To advance Indigenist evaluation in Australia, we can draw lessons from Kaupapa Māori evaluation. Efforts should focus on embedding foundational elements such as Indigenous leadership, strengthening the capability of Indigenous peoples to engage in evaluation design and processes and developing strategies that enable Indigenous-led approaches (Cargo et al., 2019; Maher et al., 2021).
Towards ‘Collective Capability’
Building an Indigenist evaluation practice through the concept of collective capability is one way to embed an Indigenous-led approach in evaluation and would dismantle the standard Western neo-liberal evaluation paradigm. We propose that embedding Indigenous epistemologies in standard evaluation practice leads to the recognition and valuing of Aboriginal and Torres Strait Islander lifeworlds, knowledge systems, and experiences. In addition, evaluation design and planning should at a minimum include Indigenous leadership and governance (Kelaher et al., 2018; Maddox et al., 2021; Vine et al., 2023). If adopted, an assets-oriented evaluation practice that has better alignment with Aboriginal and Torres Strait Islander priorities and the context in which programs and services are delivered would occur (Rossingh & Yunupingu, 2016; Williams, 2018). For example, the Ngaa-bi-nya framework, developed from an Indigenous standpoint, offers a practical approach to evaluating health, social and justice programs by emphasising community-defined success factors within their historical, political and social contexts (Williams, 2018).
Further, we argue that a collective capability concept is a way to improve evaluation methodological quality and establish a robust evidence-based evaluation practice for Indigenous programs, policies and services in Australia. Through a collective capability concept, we can establish an assets-based framework to inform and strengthen Indigenist evaluation practice.
The aim of this article is to develop a definition of collective capability by identifying its key elements in the context of Indigenous evaluation in Australia.
Methodology
Study Context
This study centres Aboriginal and Torres Strait Islander people, voices, experiences and leadership through each stage of the research. Study participants are recognised as Knowledge holders with expertise as Aboriginal and Torres Strait Islander people, including holders of Indigenous knowledges, lifeworlds, experiences and professional insights (The University of Melbourne, 2017). Leadership contributions included the Aboriginal researchers within the research team (BM, JG, RL) who have a wide range of skills, expertise and knowledge in Indigenist research, Indigenous evaluation, service delivery and policy.
Theoretical Framing
The Indigenous Lifeworld provides the theoretical grounding for this qualitative study. It recognises that Indigenous Knowledge holders’ lived realities and shared understandings of their social world are integral to giving meaning to collective capability in Indigenist evaluation. The Indigenous Lifeworld includes the intertwined intersubjectivities of Indigenous peoplehood and of Indigenous colonised that make up the duality of the Indigenous Lifeworld (Walter, 2023). Indigenous cultures are highly contextual; both context and relationships matter (Martin and Mirraboopa, 2003; Rosile et al., 2018; Wright et al., 2021). Relationality and collectivism are part of the fabric of Indigenous societies (Tynan, 2021). Both intersect with kinship (family immediate or extended), intra tribal (totemic), tribal (nation) and extant (historical) intertribal relations; they also support Aboriginal and Torres Strait Islander people to participate in decision-making, identifying solutions that are relevant for their priorities and enhancing their lives (Leßmann, 2022; Miller, 2017; Yap & Yu, 2016). Indigenous Lifeworlds frame the research in Indigenous ways of knowing, ways of being and ways of doing within the social, cultural and physical world in which Indigenous people exist in contemporary society (Martin & Mirraboopa, 2003).
Methods
Study Design
Two methods were used in this qualitative study. This included yarning with Aboriginal and Torres Strait Islander Knowledge holders and sense-making sessions with independent Aboriginal researchers.
Yarning is a two-way sharing approach for developing rapport and establishing a culturally safe setting between the researcher and Knowledge holders (Bessarab & Ng’andu, 2010; Kovach, 2010; Maher et al., 2021). The lead author conducted sessions using Zoom video teleconferencing software.
During yarning sessions, Knowledge holders were asked to describe the words ‘collective’ and ‘capability’ as individual terms. This was intended to set the scene for the broader discussion about the composite term ‘collective capability’, which they were then asked to describe alternative words, terms and phrases they would use to describe their interpretation of collective capability. Yarning duration ranged from 30 minutes to 90 minutes (mean 61 minutes).
Sense-making sessions to determine the rigour and credibility of the data (Urquhart et al., 2025) occurred with independent Aboriginal researchers to discuss and make sense of the lead researcher interpretations of the content analysis from the yarning and included researcher reflexive notes (Prion & Adamson, 2014). This included discussion and sense-making for the preliminary definition of collective capability the lead author had developed.
Participant Recruitment
We used chain referral (snowball) to recruit eligible Aboriginal and Torres Strait Islander Knowledge holders from across Australia. We purposefully selected Knowledge holders with rich expertise in Indigenous evaluation, starting with those known to the research team (Champion et al., 2008; Patton, 2014). Interviewed participants were invited to suggest further experts from their broader networks. Knowledge holders recruited included people with knowledge and experience in evaluating Indigenous programs and evaluation users, including policymakers, managers, administrators, commissioners of evaluation and evaluation practitioners.
Data Analysis
MAXQDA, a software, was used to organise, store and analyse the yarning data (VERBI Software, 2022). Data analysis comprised two phases – manifest content analysis and latent content analysis.
Manifest content analysis involves examining ‘what was said’, looking for the appearance of words or terms commonly used and being as close as possible to the text (Bengtsson, 2016; Hsieh & Shannon, 2005; Kleinheksel et al., 2020). Latent content analysis refers to the aspects and process of interpretation and meaning-making of text (Graneheim et al., 2017). We used a method described by Bengtsson modified to include a sense-making component with Aboriginal people (Bengtsson, 2016). We started with analysing the manifest content to establish key terms for collective capability. Latent content analysis followed to make meaning and interpret the manifest content, looking at the context surrounding the manifest content. The process by which we undertook the manifest and latent content analysis is shown in Figure 1. Content analysis process (adapted from Bengtsson, 2016)
Manifest content analysis was applied first to identify synonyms and antonyms for the term collective capability. This was done by asking Knowledge holders to describe the concepts ‘collective’ and ‘capability’ separately. They were then invited to describe the composite concept ‘collective capability’. We used the following process to look at ‘what was being said’. Stage 1 Decontextualisation and Stage 2 Recontextualisation: In the initial stage of the analysis, the lead researcher (BM) and two Indigenous co-authors (JG and RL) reviewed a sample of transcripts to identify key terms and words (meaning units) that Knowledge holders used to describe ‘collective’, ‘capability’ and ‘collective capability’ and labelled these with a code. The lead researcher then explored the remaining data using the same approach to label the data. Stage 3 Categorisation: Coded manifest content terms were then grouped into categories and sub-categories. Stage 4 Compilation: From the key terms, synonyms and antonyms were established and appropriate quotations were used to check that the lead researcher was staying close to the original text and context.
In the next stage, we conducted latent content analysis to identify the deeper and often hidden meanings in the text. For each category or theme, the lead researcher chose appropriate meaning units presented in the running text as quotations. We used the following process to find the meaning for themes and sub-themes. Stage 1 Decontextualisation: The lead researcher familiarised themselves with the data and read through the transcribed text to obtain the sense of the whole before creating meaning units. Meaning units were then coded. Stage 2 Recontextualisation: After the meaning units were identified and coded, the lead researcher ensured all aspects of the content have been covered in relation to the aim and then the original text was re-read alongside the final list of meaning units. Stage 3 Categorisation: Meaning units were then divided into themes and sub-themes. Stage 4 Compilation: For each theme and sub-theme, the lead researcher selected appropriate meaning units presented in the running text as quotations, interpreted these and assigned a meaning. Stage 5 Sense-making (validity and reliability): The different terms that emerged from the manifest content analysis were presented to a group of Indigenous researchers, along with the interpretation of these terms (latent content). A working definition was also presented to the group to refine the definition through a series of sense-making discussions. The feedback from these discussions enabled the lead researcher to formulate the definition and to clarify and confirm the final themes and sub-themes for the latent content.
This process provided the meaning (and definition) from the manifest content analysis key terms by the lead author interpreting what Knowledge holders had said.
Ethics
The study was conducted with approval from the Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS) Human Research Ethics Committee (no. EO239-20210114) and the Australian National University Human Research Ethics Committee.
Knowledge holders received detailed information about the research purpose, data use and confidentiality measures. They reviewed and edited their interview transcripts, chose whether to be informed of research outputs and were assured anonymity through randomly assigned numbering in place of names.
Results
Our results reflect the breadth of Indigenous evaluation expertise. Knowledge holders clearly identified the contributions of Indigenous-led approaches to evaluation, as shown in the following statements: ‘In terms of Indigenous evaluation, it’s evaluating from an Indigenous lens, and an Indigenous standpoint. Within that it’s also considering cultural protocols, practices, and being respectful of the way you go about evaluating. Indigenous evaluation, I suppose the other main aspect is the reciprocity and allowing community ownership. It’s their data, it’s their stories, and more often than not Indigenous evaluation doesn’t necessarily sit in Indigenous hands’ (KH15 [{R/MA} O/IO/SP/RI/NFP]). ‘I think that it’s more that it has a relational component that, because it should have a community development approach, meaning coming to evaluation, using community development skills like informing people, having a wide range of stakeholders, building trust, identifying who locally is best placed to contribute and be staff, possibly. Indigenous evaluation is respectfully developmental in its approach and then, it’s based on principles’ (KH10 [{R/EE/IE/CM} O/IO/SP/IA/UC/NFP/GA]).
Current evaluations offer limited benefit to Aboriginal and Torres Strait Islander people. Decolonising evidence practices and embedding Indigenous methodologies are essential, yet challenging, as reflected in the following statements: ‘There’s often a discourse around various hierarchies of evidence and, seemingly, the higher up you go the more objective it is or the necessity of objectivity in evaluation to render it having merit and value. I think those are things that those indigenous evaluations, using indigenous methodologies that we might employ in various ways, we can actually lift the lid on that. There’s still a lot that keeps us out of that evaluation or that we have to fight for to provide a different perspective on evaluation’ (KH1 [{R/EE/PP/MA} O/IO/IA/RI/GA]). ‘I think what comes to mind for me when I reflect on evaluation and specifically indigenous evaluation, is for a long time we’ve had a government or a state and federal government who have been the steward and the administrator and the designer of policy and program that’s supposed to benefit community, but we all know that the approach the government take is not grounded in meaningful relationship and meaningful partnership with community’ (KH8 [{R/EE/PP} O/CC]).
Knowledge holders then identified key terms for collective capability as synonyms to establish core attributes and antonyms to confirm what was not collective capability. Additionally, the deeper meaning of the key terms informed the definition for collective capability.
Knowledge Holder Characteristics
The Knowledge holders included policymakers, community researchers, evaluators, consultants, community development practitioners, Aboriginal Health Workers, academics and Chief Executive Officers of Indigenous organisations.
Knowledge holder characteristics
aPercentages do not sum to 100% as response was ‘select all that apply’.
Manifest Content Analysis
Knowledge holders initially spoke about the term ‘collective’ as a shared vision approach, involving community and people coming together, collaborating for a common goal and agreed good, and the power in this. Examples of statements from Knowledge holders included the following: ‘a different group of people coming together, people with different views coming together’ (KH12 [{R/EE/MA/PP} O/IA/RI/IA/SP]). ‘it’s [collective] about people coming together for a common and agreed good’ (KH14 [{R/EE/IE/MA/CM} O/SP/IA/UC/GA]). ‘There’s power in collectiveness’ (KH6 [{R/IE/CM/PP/MA} O/IA/RI/GA]). ‘Collective is a shared vision approach’ (KH3 [{R/PP/MA} O/IO/IA]).
They then yarned about ‘capability’ as having the ability to do something, having skills to achieve something and being capable. Additionally, this type of capability was in the context of cultural capability, relationships and knowledge sharing. Examples of statements from Knowledge holders included the following: ‘Whereas we talk about capability, we talk about relationships, insights, lived experience, cultural knowledge, wisdom’ (KH9 [{R/MA/EE} O/IO/RI/GA]). ‘Capabilities it’s about knowledge and knowledge-sharing’ (KH5 [{R/EE/CM/MA} O/IA/RI]). ‘More so knowledge seems to resonate a lot more than capability’ (KH15 [{R/MA} O/IO/SP/RI/NFP]).
The manifest content analysis revealed five antonyms (terms by negation) describing collective capability. The five terms were ‘not neat’, ‘not linear’, not predictable’, ‘not individualistic’ and ‘not by non-Indigenous people’. These terms were exemplified by statements from knowledge holders including: ‘You got to take those small steps. I think that collective capability that you race to get there, it’s not about the race’ (KH3 [{R/PP/MA} O/IO/IA]). ‘There is something magic there that makes it bigger and better and it’s messy. It’s not neat and predictable or linear or able to be captured. It takes relationships with people to be able to understand what’s going on in that collective space’ (KH7 [{R/EE/PP} O/IA/UC]). ‘Collective capability isn’t having six white folks around a table. For me, that’s not collective capability’ (KH9 [{R/MA/EE} O/IO/RI/GA]). ‘They’re [non-Indigenous people] trying to get a bigger slice of the pie than they should be getting because that means they’re not really on board with the collective then, are they? They’re more interested in themselves’ (KH12 [{R/EE/MA/PP} O/IA/RI/IA/SP]).
Three synonyms (terms by example) describing collective capability emerged: ‘collaboration’, ‘building relationships’ and ‘knowledge sharing’.
‘Collaboration’ was exemplified by statements from Knowledge holders including: ‘You got to take small steps. I think that collective capability that you race to get there, it’s not about the race. It’s about coming together and working together as a goal’ (KH19 [{R/IE/CM/PP} O/IO/SP/IA/UC]). ‘If I were to explain collective capability, it’s just about getting everyone on the same journey and giving, helping people identify what they need and trying to meet those needs in order to be on a journey together’ KH10 ({R/EE/IE/CM} O/IO/SP/IA/UC/NFP/GA). ‘I’m just having a bit of a debate in my own head about whether it’s collaborative and cooperative. Maybe mostly, mostly collaborative, which provides a positive intent’ (KH1 [{R/EE/PP/MA} O/IO/IA/RI/GA]).
Collaboration involves people coming together to work together for a purpose. Working together and coming together were used interchangeably by Knowledge holders to describe collective capability and in essence are features of collaboration. Collaboration also involves being on the same journey and supporting people on that journey.
‘Relationships’ were exemplified by statements from Knowledge holders including: ‘I just really think it comes back to our way, you know, everything’s relationship based for our mob. And relationships take time to build, and trust takes time to build, and you know, the only way that I got some insights into some of those experiences was really being intuitive about things’ (KH11 [{R/CM/PP} O/IO/GA]). ‘Everything’s about relationships. And that can only occur when we’ve got genuine trust with people and we unpack a system’ KH20 ({R/EE/CM/PP/MA} O/IO/SP/IA/UC/RI/GA). ‘Having clarity and agreement about how decisions are made’ (KH18 [{R/IE/EE/CM/PP/MA} O/IO/SP/IA/UC/RI/GA). ‘When I think of collective capability it’s by definition, I think, relational’ (KH1 [{R/EE/PP/MA} O/IO/IA/RI/GA]).
Relationships tie together ways of working in practice. Knowledge holders expressed the important role that relationships play in Indigenist evaluation, and that time and care are required to build trusting and genuine relationships. For many of the Knowledge holders, they spoke about coming together for a purpose or goal, to achieve this collectively, and that this was dependent on building trusting and genuine relationships. Collective capability also included bringing everyone along on the journey.
‘Knowledge sharing’ was exemplified by statements from Knowledge holders including: ‘I think about knowledge holders as collective, then I think about how that knowledge is then passed down and taught, so I think of a collection of knowledge teachers’ (KH9 [{R/MA/EE} O/IO/RI/GA]). ‘you can see yourself and you can see your culture and you can see your knowledge in a valued place within that structure. Definitely that gives a sense of empowerment’ (KH 18 [{R/IE/EE/CM/PP/MA} O/IO/SP/IA/UC/RI/GA]). ‘collective capability where we’re sharing each others’ knowledge to improve in things. I think part of collective capability is that ability to learn, and develop, and grow, and get better’ (KH 3 [{R/PP/MA} O/IO/IA]).
Ways of knowing involve knowledge being shared that includes cultural values and reciprocity. Two-way exchange of skills and expertise that are shared are also features of knowledge sharing. Knowledge holders described seeing themselves, their culture and their knowledge in a valued place. It was also identified that sharing knowledge helps to make improvements on what exists, and to learn and grow to get better are features of collective capability.
Latent Content Analysis
The latent content analysis revealed two higher-order themes that explained the meaning of ‘collective capability’: ‘relationality’ and ‘knowledge exchange’.
Relationality means applying relational practice for collective capability to be realised. Relationality includes people, place and purpose, incorporating the combination of features such as working together, coming together and collaborating that includes values and skills equally. Relationality instils values based relationships to inform the purpose of ways of working through collaborations and working together sustainably. The action and intention of establishing and building sustainable relationships is reliant on trust and time, it is not a process that is rushed or transactional. Knowledge holders identified that relationships are fundamental to collaborations and that relationships require the building of trust and time.
Being guided by relationality ensures that working relationships are coming from a place of genuine intent, humility and fosters a supportive working environment and network. This meaning is exemplified by the following statement: ‘What I have seen in evaluation that has been done collectively is there is a really strong energy for people to be involved because there is that common cause. There is incredible flexibility and allowances made for everybody within the collective because of that shared understanding and that shared story. There is really strong relationships and relationships keep everybody strong. Relationships protect us, they’re protective factors. They’re all things that are part of collective capability that you don’t get with just putting people with skills together in a group’ (KH7 [{R/EE/PP} O/IA/UC]).
Knowledge exchange was interpreted as a two-way exchange of knowledge through sharing and learning in an ongoing way that is built and strengthened through sharing, teaching and learning in an ongoing way and therefore reflects reciprocity. Sharing knowledge through experience-based approaches helps build the capability of the collective and contributes to a sense of empowerment. Knowledge, as its own entity, is considered an asset that is culturally valued and informs the fluidity and continuation of culture. Further, it does not move in one direction and there is a principle of valuing the knowledge being shared. ‘it’s the knowledge sharing but not just in a way where that’s all going in one direction and potentially being misused or abused or ignored but rather there’s a principle of valuing all of the knowledge that’s being brought and sharing, it elicits a lot of sense of generosity and vulnerability within a safe space that’s created for that to happen because there is a shared agreement around value’ (KH 18 [{R/IE/EE/CM/PP/MA} O/IO/SP/IA/UC/RI/GA]).
Defining Collective Capability
At first, the composite term ‘collective capability’ appeared unfamiliar to Knowledge holders. With further prompting through the yarning process, Knowledge holders conceived terms (manifest) and definitions (latent) via ‘terms and definition by negation’ (characterised as antonyms) and the ‘terms and definition by example’ (characterised as synonyms). We found that collective capability in the context of Indigenist evaluation is embodied through the concepts of ‘relationality’ and ‘knowledge sharing’.
Therefore, the definition of collective capability is ‘people coming together relationally to work toward a common goal in a collaborative learning environment, where cultural values are prioritised, knowledge is shared, and the process is equally important as achieving the goal’.
Discussion
This the first study that defines collective capability and its constituent elements in Indigenist evaluation in Australia ….by Indigenous, for Indigenous, with Indigenous. Guided by Indigenous Lifeworld theory, the definition aligns with Indigenous ways of knowing, ways of being and ways of doing and adds to the momentum of other Indigenous-led evaluation guidance (Martin & Mirraboopa, 2003; Vine et al., 2023; Walter, 2023; Williams, 2018; Wright et al., 2021).
Relationality and knowledge sharing stand in stark contrast to existing evaluation processes and designs that are often determined by what fits best with the funding cycles (McCausland, 2019). In many current evaluation frameworks, there is an absence of cultural values guiding evaluation methodologies to evaluate Indigenous programs. The current orthodoxy in evaluation aligns with Knowledge holders’ expressed views of what collective capability is not, that is, a neo-liberal approach to evaluation where evaluations are neat, predictable, individualistic and driven by non-Indigenous people.
Our findings reiterate the inherent Aboriginal and Torres Strait Islander leadership and governance to establish the values, environment and culture for Indigenist evaluation to occur. For example, it is essential for leadership structures to embed meaningful knowledge exchange, inclusivity, respect and the building of trust with Aboriginal and Torres Strait Islander communities and organisations. Process elements that are important to Indigenist evaluation involve leadership by Aboriginal and Torres Strait Islander people and communities ensuring time and trust accounted for in building relationships and were identified and reinforced by Knowledge holders in this research (Grey et al., 2018; Rogers et al., 2017; Vine et al., 2023; Wright et al., 2021). Relationships are also important to collective decision-making. Establishing Indigenous decision-making (governance) mechanisms, including in evaluation processes, ensures that Aboriginal and Torres Strait Islander priorities and values, and therefore trust and respect, are informing the design and planning for evaluations and may lead to evaluations that are meaningful and of benefit to Aboriginal and Torres Strait Islander people (Cargo et al., 2019; Maddox et al., 2021; Maher et al., 2021).
Knowledge sharing as an important element of collective capability encompasses valuing two-way exchange of knowledge and skills and may assist with how evaluations embed Indigenous knowledge systems and support a collective capability approach. An important strategy to this principle is reflexivity, the skill and intentional practice of deep listening and critical self-reflection. Together, reciprocity and reflexivity inform a respectful practice that results in cohesiveness and connectedness of the collective (Liwanag & Rhule, 2021; Nicholls, 2009; Russell-Mundine, 2012).
Through the process of knowledge exchange you would see an increase in learning, knowledge and skills transfer, and embedding ‘learning by doing’ approaches to Indigenist evaluations with Aboriginal and Torres Strait Islander communities, organisations and evaluation teams. Knowledge supports capability by creating learning environments that enhance knowledge exchange, skills transfer, reflexivity and reciprocity. Centring Indigenous knowledges, practices and worldviews in evaluation methodology aligns with local context in which Indigenous programs and services are situated (McCausland, 2019; Vine et al., 2023; Williams, 2018). Collective capability as embodied through ‘relationality’ and ‘knowledge sharing’ in Indigenist evaluation reflects an outcome in and of itself, as a self-determined practice for evaluation marked by a collaboration achieving its goals. The concept also highlights an intermediate process leading to a self-determined Indigenous evaluation practice and is congruent with empowerment in that it is conceptualised as both a process and outcome (Wallerstein, 2006).
Processes that uphold the elements of collective capability may lead to a self-determined practice for evaluation, as an outcome, and ostensibly lead to longer-term benefits for Aboriginal and Torres Strait Islander people and communities (Wallerstein, 2006). Achieving a collective capability practice for evaluation will require building a new generation of evaluators and evaluative thinkers who can critically self-reflect and build harmonious working relationships to support and direct the planning and commissioning of evaluations that reflect ‘by, with, for and as’ Aboriginal and Torres Strait Islander people (Grey et al., 2018; Rogers et al., 2017).
Strengths and Limitations
The study draws on an Indigenist research framework to privilege Aboriginal and Torres Strait Islander ‘ways of knowing, ways of doing and ways of being’. Its strength has been the Aboriginal and Torres Strait Islander leadership and involvement in all the research stages. The development and defining of the concept ‘collective capability’ grounds itself in Aboriginal and Torres Strait Islander Lifeworlds and therefore resonates with the population.
Collective capability offers a transformative approach to evaluation practice in Australia. Drawing on a deeper analytical analysis using latent content analysis explains the key terms and determines how collective capability emerges in practice. Establishing a definition to guide evaluation practice provides clarity of the meaning of collective capability and a theoretical foundation grounded in Indigenous Lifeworlds.
A potential limitation of this study is that several researchers involved in the sense-making of findings process did not have direct experience in evaluation; therefore, the translation of findings to an evaluation context may not have been fully understood to inform practice. Counterbalancing that potential deficit however is that those same people were potential end-users and beneficiaries of evaluation practice and their expertise in terms of lived experience should not be dismissed. They are well versed in Indigenous methodologies and methods being applied to research, and this experience can be related to the philosophical nuances of the research in this study.
Implications for Future Research and Policy
This study establishes the key foundations of collective capability through relationality and knowledge sharing by providing a definition and key elements to progress Indigenous-led evaluation practice in Australia.
Innovations and concepts developed as Aboriginal and Torres Strait Islander, however, come with warnings about co-opting and theft of those ideas and concepts by non-Indigenous people and systems to appropriate as their own (Lovett et al., 2020; Walter & Carroll, 2020; Whittaker, 2019). The introduction of Indigenist ideas and concepts within colonial structures can result in Aboriginal and Torres Strait Islander people being pitted against each other, referred to as ‘crabs in a bucket’ pulling one another down as a result of cultural values and integrity lacking within these systems and structures (Bailey, 2020; Clark & Augoustinos, 2015; Gooda, 2012). Aboriginal and Torres Strait Islander leadership and governance are essential in ensuring that the use of concepts, definitions and terms, as determined by Aboriginal and Torres Strait Islander peoples, are upheld in their original intent and purposes (Prehn & Walter, 2023).
Developing tools to apply and assess collective capability in the context of ongoing or completed evaluations will be the subject of the next phase of this research. These tools can then be used to build further Indigenist evaluation principles and ultimately an Indigenist evaluation framework. The definition, that is, results of this article, can contribute to existing evaluation principles and frameworks. Developing tools will strengthen evaluation practice by ensuring that evaluations have a way of monitoring collective capability, for example, checklist given to funders for Request for Quotation, community has a checklist to ensure that evaluators are adhering to what this means for community.
The further development of integrating these foundational elements for Indigenist evaluation requires broader engagement from the community, academic and public sectors. Building and supporting an Indigenist evaluation workforce will be essential to the planning, design and implementation of evaluations and the translation of evaluation findings to benefit communities as well as inform policy. This will likely include practical application in Indigenist evaluation, governance and leadership training and resources and materials integrating relationality and knowledge exchange as Indigenous concepts to consolidate an Indigenist evaluation methodology.
Conclusion
Collective capability through relationality and knowledge sharing is a potentially transformative Aboriginal and Torres Strait Islander developed concept, grounded in the Indigenous Lifeworld, if practice is implemented with integrity.
Footnotes
Ethical Considerations
This study has been approved by the Australian Institute for Aboriginal and Torres Strait Islander Studies research ethics committee, EO239-20210114.
Author Contributors
BM conceived the idea for the manuscript. BM conceptualised the study method and then had input from all authors. BM produced the initial draft of the manuscript. All authors were involved in drafting the manuscript. All authors approved the final version of the article and take responsibility for its content.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: RL was supported by a National Health and Medical Research Council (NHMRC Fellowship 1122273).
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.