Abstract
CGIAR consists of a network of international publicly funded agricultural research for development institutes. Over five decades it has worked to increase food abundance, reduce hunger and poverty rates, and lower the geographical footprint of agriculture in lower- and middle-income countries. CGIAR's first formalised research program on climate change was set up in 2009. Here we report on an analysis of 300 outcomes generated over the lifetime of this program, which ran until 2021. Outcomes were characterised in relation to the climate objective, geography, thematic scope, and contribution to global goals. More than half of the outcomes analysed were related to policies for agriculture development under climate change. Twenty-six percent of outcomes related to climate, information and financial services, and 22 percent were related to organisational programming. Most outcomes analysed were at an early stage of maturity, focusing on design and planning stages of policies, strategies, and investments. Fewer than five percent of outcomes had evidence of impact at scale. Outcomes were facilitated by a wide network of partners and contributed to more than ten Sustainable Development Goals. The results of the analysis show the value of outcome-oriented science in being able to harness diversity, balance strategy with opportunism, plan flexibly, work across multiple vulnerability contexts, and allocate resources towards outcomes. These elements have played a significant role in fostering change across contexts, in adjusting research to emerging needs and context changes, in creating conditions for spillovers, and in ensuring global relevance. To transform the food system, the research for development agenda needs to be bolder. It will require more outcomes of various types, achieved through diverse partnerships, spanning a diversity of geographies, vulnerable contexts, and priorities. Significant and intentional investments in strengthening monitoring, evaluation, reporting and learning capacity will be required to further realize the potential for outcome-oriented research.
Introduction
The need to transform our food systems is clear, if we are to achieve the Sustainable Development Goals (SDGs) by 2030 and keep up with the global climate targets (Webb et al., 2020). Several recent reports expand on the transformation agenda and on actions needed to keep us within key planetary boundaries in the face of climate change and other global challenges (GCA, 2019; GLOPAN, 2020; HLPE, 2020; Pharo et al., 2019; Rockefeller Foundation, 2021; Steiner et al., 2020). The costs of the transformation are estimated at more than US$1.3 trillion per year (Steiner et al., 2020), a budget more than twice as high as current global climate finance flows (Climate Policy Initiative, 2021). This begs for drastic changes in existing delivery mechanisms, from finance, to policy, to research and development, and partnership models.
Research can play a key role in food systems transformation. Public and private investors need robust and timely evidence on what to prioritize, how, for whom, and at what costs/benefits. They require reliable data to set bold, credible, and practical targets and to translate ambitions agendas into actions. Technologies and innovations need to be rigorously co-designed, tested, monitored, and evaluated to ensure a just transition for all. Decades of participatory action research have left an important legacy of learnings about science-driven approaches to catalyse change (Nowak et al., In press). Science, technology and innovation can create impact when designed with context (Oliver and Cairney, 2019); when stakeholders are strategically engaged throughout the research process, from design, co-production, to dissemination (Dinesh et al., 2018; Kristjanson et al., 2014; Norström et al., 2020; Oliver and Cairney, 2019); when research questions respond to intermittent openings of “policy windows” or opportunities for informing new agendas (Dunn and Laing, 2017; Rose et al., 2020); when findings and solutions are reliable, available, accessible, and effectively communicated to users (i.e., rigorous methods, short time span between research and publication, non-technical language, open access publications, etc.) (Cvitanovic et al., 2015; Dunn and Laing, 2017; Khoury et al., 2019); or when researchers become part of influential networks that facilitate uptake and scaling of findings (Ball and Exley, 2010). The principles listed are not exhaustive; they have proven effective in bringing returns to the society (Alston et al., 2020; Pardey et al., 2016) and will be critical elements for implementing transformation agendas moving forward (Hall and Dijkman, 2019).
CGIAR is a network of international publicly funded agricultural research for development institutes, established in 1971. Over five decades it has worked to increase food abundance, reduce rates of hunger and poverty, and lower the geographical footprint of agriculture in many lower- and middle-income countries, returning ten times the costs as benefits (Alston et al., 2020) and making important contributions to the global agenda on SDGs. CGIAR's first formalised research program on the challenges of climate change, the research program on Climate Change, Agriculture and Food Security (CCAFS), was set up in 2009 and ran to the end of 2021. The program invested some US$ 350 million in action research involving all the international agricultural research centres of CGIAR and integrating thematic work across multiple global, regional, and local partners. Pro-poor climate change adaptation and mitigation practices, technologies, and policies were worked on to enhance food systems, adaptive capacity, and rural livelihoods. CCAFS aimed at benefiting the rural populations; it provided diagnosis and analysis to guide cost-effective investments, the inclusion of agriculture in climate change policies, and the inclusion of climate issues in agricultural policies, from the subnational to the global level (Vermeulen et al., 2012). CCAFS worked for more than a decade with hundreds of partners on a food system transformation agenda, showing how research can make a big difference relatively quickly (Meadu et al., In Press). This was accomplished in part through maintaining a strong focus on outcomes, especially behavioural changes in people, and implementing activities following the ‘three thirds’ principle: a third of the effort spent on working with next-users to build relationships and define their needs, a third spent on the research itself, and a third spent on enhancing next-users’ capacity to take up research outputs (Vermeulen and Campbell, 2015).
Early on, CCAFS put in place a relatively sophisticated monitoring and evaluation system, which evolved through time. This allowed program staff to monitor progress towards a set of targets and to record outcomes throughout the life of the program in a relatively consistent manner. In this paper, we use this information for two purposes:
To distil typologies of outcomes and impact pathways enabled by CCAFS research and critically reflect on its contribution to increased productivity for food and nutrition security, increased climate adaptation and greenhouse gas mitigation efforts worldwide; and To highlight key learnings for the design of outcome-oriented agriculture research for development (AR4D) programs of the CGIAR and beyond.
In the next section, we outline the methods and data used. Following this, we present some of the characteristics of the outcomes achieved. Here, we discuss the strengths and weaknesses of CCAFS approaches to outcome-oriented work. We conclude by highlighting the necessity of agricultural research for development addressing the enablers of change as well as developing and testing technical solutions: informing policy change, guiding investment, and implementing the novel partnerships that are necessary for enabling impact at scale.
Outcome harvest
We used the outcome harvest approach to collect information on outcomes achieved by CCAFS throughout the years. Outcome harvests are evaluation approaches used for identifying and analyzing types of changes occurring from one or more interventions (Wilson-Grau and Britt, 2017). The approach is typically used in assessing programs with extensive timeframes, geographical spaces, and thematic areas, and represents a viable, cost-effective alternative to more robust, yet resource-intensive impact evaluations. In view of the outcome orientation of CGIAR and its constituent programs such as CCAFS, we judged this to be an appropriate method of analysis to help us critically reflect on achievements and lessons learned.
We screened documents with narratives of project outcomes submitted by CCAFS personnel as part of annual reporting processes between 2011 and 2020. Here we define an outcome as a change in knowledge, skills, attitudes and/or relationships, manifested as a change in behavior, to which research outputs and results (i.e., publications, methodologies, datasets, etc.) and activities have contributed (Szilagyi et al., 2020). Outcomes were included from all four CCAFS's flagship programs, namely Priorities and Policies for Climate Smart Agriculture (FP1), Climate Smart Technologies and Practices (FP2), Low Emissions Development (FP3), and Climate services and safety nets (FP4). The outcome documents describe the activities conducted, the related research outputs, partners, as well as quantified progress to targets (where available), such as number of beneficiaries, of research outputs produced, of area covered, and innovations developed. The reports also provide evidence on the contribution of the research efforts to the outcome, reported in the form of narratives and supporting documents. The structure and content of these outcome reports changed with time, with more detail and more rigorous documentation of outcomes observed in the period 2017–2020.
We assessed the eligibility of outcome reports for analysis in two stages (Figure 1). First, we assessed the level of information provided. Reports were excluded if they provided inadequate information describing the outcome and its associated outputs and activities. Second, we evaluated the outcome narrative and excluded those reports that omitted any documented effect (such as the publication of a scientific article, a progress report, dissemination of a methodology, a training course) and that were duplicated (i.e., submitted twice for the same year and project and with the same level of detail). Our aim was to collect information describing what changed (the outcome), where (the geographical context and decision-making level), when (the year), for whom (the next users of the outcome), to what extent (stage or maturity level of the outcome), and how (which activities, outputs, partnerships contributed to the change). In addition, we aimed to collect insights into the relevance of CCAFS outcomes to broader institutional and global-level targets. These include system-level targets of the CGIAR, most of which are mapped on to the SDGs (e.g., reduced poverty, improved food and nutrition security for health, and improved natural resources and ecosystems services). A visual summary of the protocol with data sources, selection, exclusion criteria and information extracted is shown in Figure 1.
Overview of outcome harvest process.
The outcomes harvested from the reports were categorized to enable a systematic analysis of the qualitative information extracted. To this end, we created several outcome typologies, which are detailed in Table 1. Typologies refer to: the climate objective covered by the outcome, which is linked to the three pillars of climate-smart agriculture; the scope of the outcome, which gives an indication of the type of work carried out and the next user(s) of the outcome; the maturity level, which approximates the level of development of the outcome, from design to testing/piloting, and scaling; the activity, which is linked to the CCAFS ‘three-third’ principles and offers insights into the outcome delivery model used; and the contributing partner, which highlights the partnership strategy used to deliver on outcomes. We drew on the knowledge and reflections of program staff and other personnel to review and validate the typologies and the findings, as well as to enrich the results, by substantiating the outcome claims reported (see next section).
Description of outcome typologies and example from the CCAFS outcome database. Sources of typologies: (CCAFS, 2016; CGIAR, 2021; Dinesh et al., 2018)
Results and discussion
In the following paragraphs we present and discuss findings from the outcome harvest. These are organized in six themes that describe key characteristics of the outcomes observed and of their impact pathways, namely: harnessing diversity, balancing strategy with opportunism, planning flexibly, working across vulnerability contexts, and resourcing outcomes for global relevance.
Harnessing diversity
The research programs of CGIAR have produced a wide range of high-quality research outputs over the last decade. Apart from producing high-quality publications (Rünzel et al., 2021), these research programs have also worked successfully to unlock change in food systems. An overview of the outcomes produced by CCAFS is shown in Figure 2, in relation to the country associated with the reported outcome, the climate objective being addressed (adaptation to climate change, increases in productivity, and mitigation of climate change), the scope of the outcome (policy, program, services, global, farm), and the maturity level (1, 2, 3). A substantial proportion of the outcomes reported were associated with multiple classes of the typology – more than one country and multiple climate objectives and/or outcome types. Several outcomes were reported more than once across years as the maturity level increased through time. For such reasons, the numbers in Figure 2 sum to more than the total number of outcomes analysed (300).
CCAFS outcomes span different geographies (at global and national levels), climate objectives, scopes, and maturity levels. Note: Numbers represent frequencies with which the outcome types were reported in relation to the geography (global or country). In some cases, the total number of an outcome type (e.g., adaptation) is higher than the total number of outcomes reported, because of double (i.e., same outcome occurring in different countries) or triple counting (i.e., one outcome addressing all three CSA pillars).
The outcomes produced are highly diverse. They span 57 countries, with a large proportion of outcomes concentrated in CCAFS's five target geographies: Southeast Asia, South Asia, East Africa, West Africa, and Central America / northern South America. Despite this concentration, some outcomes (spillovers) were also achieved in other regions that were not a focus of effort, including southern Africa and southern South America. Many outcomes were not linked to a specific country, but had a wide geographical scope (regional, continental or global). Outcomes were achieved across the three objectives: 86% were associated with adaptations to climate change, 52% with increases in productivity for food and nutrition security, and 50% with mitigation (See Table 1 for examples). Half of the outcomes were associated with more than one objective, especially targeting adaptation and productivity. On outcome types, 52% were policy- and investment-related, 26% were associated with climate information and financial services, 22% with organisational plans and programming, 15% with observable changes at the farm level, and 9% each with global processes. Almost a quarter of the documents analysed (24%) reported two outcome types, typically combining policy and programming or policy and services. Most outcomes (55%) were reported at maturity level 1, with 41% at maturity level 2 and 3% at maturity level 3.
These observations highlight the broad operating mandate of CCAFS, which in turn reflected the high-level theory of change that guided the program's operation within four action areas: (1) Working with partners, especially implementing partners and local organizations, to build field-based evidence; (2) Working with partners, especially climate risk management service providers, to understand how to strengthen institutions and services through better use of climate information; (3) Working with partners, particularly policy partners, to understand what works for coordinated policy and governance; and (4) Working with partners, particularly the large agencies and companies driving implementation, to understand what works for investment to reach scale (CCAFS, 2016).
Balancing strategy with opportunism
Figure 2 highlights a key learning with respect to the balance needed in an AR4D portfolio. To help bring about food system transformation, outcomes with different time horizons and effort levels are required. Short-term gains, as indicated by outcomes at maturity level 1, are key; they provide the foundations, experiences and learnings necessary for building impacts at scale. For AR4D projects that operate in three-year funding cycles, short-term gains are vital in keeping funders on board. But more difficult work is also critical, which may require longer periods of time to ensure benefits to end-users such as farmers. This mix of relatively low-hanging outcome opportunities and much more resource-intensive outcome work is reflected in Figure 2. That nearly half of all outcomes related to policies and investment may seem surprising, but the period when CCAFS was operating coincided with (and may indeed have been partially responsible for) the rise in prominence given to climate-smart agriculture (CSA) (Lipper et al., 2018; Steenwerth et al., 2014), particularly in many countries of sub-Saharan Africa and Asia. There were thus many opportunities for policy and investment outcomes while CSA interventions were being designed and implemented. At the same time, this also highlights the fundamental shift in focus of the CGIAR towards an outcome orientation and the necessity of addressing the enablers of change.
Some outcomes, and the beneficial development impacts that may ensue, can take considerable time to mature; only three percent of the outcomes were at level 3 during a twelve-year program. Moreover, only 15% of outcomes were at the farm level. Other evaluation of the work of CCAFS and partners have highlighted the often slow progress made in the wider implementation of climate-resilient practices at the farm level (Niles et al., 2021; Thornton et al., 2018). Taken together, these points suggest the need for relatively long funding cycles if AR4D is to achieve impacts at scale, both on farm and policy-wise. They also highlight the need to establish low-cost monitoring mechanisms to track outcomes over time, beyond the life of the project or program. This applies both to policy processes, where the time lags between policy design and implementation may be very long, and to adaptation measures on the ground, which may have short-term benefits but may be maladaptive in the long run.
Planning flexibly
Several factors drive the need for outcome planning to be flexible. One is making the most of opportunities for achieving potential outcomes as and when they arise, often with short windows for action, as discussed above. On other occasions, changing circumstances may require revisiting the impact pathway that was designed at the start of the project. There are many examples of such changes in the outcomes reported. Two examples follow and both related to what may be termed a “missing middle” in the CCAFS theory of change as originally envisaged: the need to strengthen the links and flow of information between local farmers’ needs and contexts on the one hand, and policymaking for climate action (adaptation and mitigation) on the other. The first example relates to policy and investment outcomes, for which there was no general recipe for success (Nowak et al., 2021). Such outcomes were generally informed by a mix of participatory, stakeholder-focused activities and science-led outputs. These ranged from robust climate analyses to user-centred decision-support tools for priority setting, to capacity building and awareness raising. As our findings revealed, more than 70% of the outcomes were achieved through combining evidence generation, with stakeholder engagement (planned or ad-hoc) and science communication and outreach. In West Africa, the disconnect between national policy makers and local food system actors on the ground, and the difficulty of getting local voices heard in national policy debates, was identified as a key challenge. To bridge this missing middle, science-policy dialogue platforms were set up in five countries of the region. This (unforeseen) mechanism required a great deal of time, skill and resources to set up, but proved crucial to achieving several policy and investment outcomes across the region, from improved delivery of climate services, to increased finance for climate-smart agriculture, or adoption of on-farm climate-smart management techniques (Zougmoré et al., 2019).
The second example of a missing middle refers to the Participatory Integrated Climate Services for Agriculture (PICSA), an approach to support and empower farmers in their decision-making processes for their own contexts (Clarkson et al., 2022). The success of PICSA in Rwanda was dependent on new, bespoke partnerships that were not anticipated in project design; partnerships which linked information dissemination, via radio listening clubs throughout the country, with effective use, through training “farmer promoters”, and thence to well-documented positive livelihood outcomes for more than 100,000 Rwandan farmers. This missing middle here refers to bridging the gap between production of knowledge and farmers’ being able to use it, facilitating changes in behaviour that which in turn unlock higher-level outcomes leading to positive development impacts (Nowak et al., 2021). These are just two examples from the outcome database that reaffirm the importance of periodically and critically reviewing theories of change and their embedded assumptions, as a key strategy for adaptive management and learning, so that programs can effectively and rapidly respond to, and align with, emerging needs and changes in context, to maximize impact.
Working across vulnerability contexts
We explored CCAFS’ work across vulnerability contexts, considering CGIAR's mandate to work for rural, vulnerable people in lower- and middle-income countries. Figure 3 shows the number of CCAFS outcomes achieved per country between 2011 and 2020 plotted against the ND-GAIN country index averaged for 2011–2020. ND-GAIN summarizes a country's vulnerability to climate change and other global challenges along with its readiness to improve its resilience (Chen et al., 2015). A low ND-GAIN score suggests high climate vulnerability and low readiness to build resilience. Although it has been criticised as being somewhat arbitrary, the index is comparable with other indicators related to climate change and development (Miola and Simonet, 2014). Figure 3 suggests an unclear relationship between the ND-GAIN index and the number of outcomes achieved; fewer outcomes were observed in countries with lower (<30) and higher index (>55), while a higher concentration of outcomes occurred in countries with medium index scores (35–50). Colombia stands out as the country with the greatest number of outcomes achieved and a medium-high ND-GAIN (NW quadrant of Figure 3). Kenya and India follow closely in the number of outcomes, yet with lower ND-GAIN scores (SE quadrant), suggesting a broad portfolio of high-impact, vulnerability-focused work. The NW quadrant illustrates spillover effects in more than 17 countries (medium-high ND-GAIN, few outcomes), while the SW quadrant includes the countries where most CCAFS work has been carried out.
CCAFS outcome-oriented work targeted vulnerable contexts to different extents. Based on ND-GAIN (2011–2020 average) and total outcomes achieved by CCAFS (2011–2020). Circles with flags represent country short names, based on ISO3.
The “ideal” nature of the relationship in Figure 3 is not easy to articulate. On the one hand, it might be imagined that countries with a higher ND-GAIN index would have fewer outcomes. On the other, even countries with a relatively high ND-GAIN index (61 for Chile, 55 for China, 54 for Uruguay and 53 for Costa Rica) may have quite high levels of vulnerability coupled with low levels of resilience readiness compared with most high-income countries with index scores above 70. Moreover, national measures of vulnerability and readiness may hide considerable sub-national variability and may not necessarily equate with ease of achieving outcomes. Short-term gains may more easily be achieved in familiar environments, which would explain the large number of outcomes achieved in countries such as Colombia, Kenya and India, home to large CGIAR centres and long-established partnership networks. Nevertheless, CCAFS outcomes were often (though not always) targeting highly vulnerable situations.
Resourcing outcomes for global relevance
Outcomes in the database were evaluated in very general terms with respect to the resources used and contributions to development impacts. Figure 4 illustrates the contribution of CCAFS outcomes to the SDGs (4a) and linkages between CCAFS AR4D budgets and SDG targets, based on number of outcomes reported and their major link to one of the SDGs (4b). Because CGIAR claims contribution to key SDGs through its system-level targets, we considered it pertinent to explore the extent to which CCAFS, as the main climate-oriented programme of CGIAR, has aligned its outcome-oriented work to these global efforts. Three caveats should be made about these data. First, we did not have details on the actual costs of these activities; this is not surprising, given that over the life of CCAFS there were more than 400 projects. Second, the information on outcomes presented does not indicate the magnitude of their reach, such as number of beneficiaries reached; one outcome is no less significant than ten outcomes, if it reaches meaningful amounts of beneficiaries or covers large geographical or agro-ecological areas. Third, the number of outcomes may be only a poor predictor of the importance of the CCAFS work associated with each SDG and SDG target. Reporting contributions to SDGs and related targets was not consistent during the lifespan of CCAFS, and some valuable linkages between CCAFS's work and SDGs may have been missed. Nevertheless, the findings in Figure 4 are important in three ways.
SDGs targeted by CCAFS outcomes (a, top) and CCAFS AR4D budgets mapped against SDG targets, based on the outcomes reported (b, bottom). Notes: Figure 4a: Numbers represent frequency of SDGs reported in relation to a CCAFS outcome. SDGs 15, 5, 8, 3, and 4 had less than 10 outcomes mapped, and were not included in the figure. Figure 4b: CCAFS outcomes are mapped to SDG targets. Each outcome (and hence SDG target) may have received budgets from one one or more activity lines. SDG targets refer to: SDG 1.4: access to economic resources, basic servives and to productive resources and assets; SDG 2.1: access to safe, nutritious and sufficent food; SDG 2.3: double agricultural productivity and incomes; SDG 2.4: sustainable food production systems and resilient agricultural practices that ensure productivity, environmental sustainability and adaptation to climate change and variability; SDG 12.2: sustainable, efficient management of natural resources; SDG 13.2: national policies, strategies and plans taking into consideration climate change; SDG 13.b: capacity for climate change-related planning and management in low-income economies; SDG 15.3: combat desertification, restore degraded lands and soils; SDG 17.16: multi-stakeholder partnerships to exchange knowledge, expertise, technology, and financial resources for achieving the SDGs. Source: adapted from (UNSTATS, 2022).
First, Figure 4.a suggests significant alignment between CCAFS's R4D agenda and the global Agenda on Sustainable Development. In total, CCAFS outcomes contributed to 14 out of the 17 SDGs, to different extents. As expected, most outcomes were reported in relation to action to combat climate change (SDG 13), followed by partnerships for the goals (SDG 17), ending poverty (SDG 1), ending hunger and achieving food and nutrition security (SDG 2), and responsible consumption (SDG 12). SDGs related to the sustainable use of terrestrial ecosystems (SDG 15), gender equality (SDG 5), decent work and economic growth (SDG 8), health and well-being (SDG 3) were less prominent in outcome reports, each receiving less than ten mentions. These findings illustrate the global relevance of CCAFS's mandate and work, particularly in the context of supporting climate action in the agricultural sector. It also highlights some critical gaps that need to be addressed more thoroughly in a food system transformation agenda, which places increased emphasis on gender empowerment, health outcomes, and formal education for sustainable development.
Second, results indicate no discernible relationship between size of budget allocation and the frequency of CCAFS outcomes achieved and SDG targets contributed to (Figure 4.b). We found no golden rule or ideal amount of finance required to produce an outcome or to contribute to a globally relevant target. As our findings highlight, AR4D can produce different types of outcomes, at different maturity levels, in different geographies, and within very complex socio-economic and institutional environments; and each of these individual cases comes with a case-specific price tag. In addition, the value for money of an investment can be significantly underrated. Steep budgets are often required for setting up project infrastructure and processes that may eventually help to unlock outcomes. This is especially the case when working in new environments, on new topics (such as CSA was in 2010), or with new partners who require significant engagement and trust building. However, as illustrated previously, outcomes can be slow to appear, and without any mechanisms to track their development (or evolution) beyond the project lifespan, they may pass unobserved.
Third, Figure 4 invites reflection on the delivery model of outcome-driven research, particularly in the case of complex AR4D programs. While finance plays the fundamental role of the engine that sets the mechanism in motion, research stakeholders are the passengers that give direction and perspective to the process. Across its projects and programs and over the years, CCAFS has sought to engage with a large diversity of partners, as a key strategy to build a bridge between knowledge and action and to drive impact. Our analysis revealed that, in 47% of the cases studies, outcomes were delivered in partnership with governmental institutions, 46% in partnership with non-CGIAR research partners, 39% with development organizations, 17% with other CGIAR research centres, 14% with financial institutions, 12% with private sector actors, 9% with bilateral donor governments, 7% with community-based organizations, and 3% with national agricultural research and extension systems (NARES). More than half of the outcomes reported involved more than two partner types (typically government and academia/research). There has been an evident under-representation of private sector and grassroot organizations, which are critical for creating enabling environments for scaling and for building value for money.
Future perspectives
Our analysis indicated that more than half of the outcomes analyzed were related to the development or modification of national and local policies for agriculture development under climate change. A quarter of the outcomes related to climate, information and financial services, and the remainder were related to organizational programming. Although most outcomes were at an early stage of maturity, a few demonstrated evidence of impact at scale. Our results highlight the value of outcome-oriented science in harnessing diversity, balancing strategy with opportunism, allowing for flexible planning, working across multiple vulnerability contexts, and providing a basis for allocating resources for impact.
A wide range of outcomes will be needed to achieve transformation of the food system. This diversity is important not only in relation to the types of outcomes targeted (i.e., science-informed policy targets and plans, financial and information services, changes in farmer livelihoods, etc.), but also with regards to the level of maturity reached by those outcomes. Changes in behaviour achieved at scale are critical for transformation. However, design of new plans, policies, changes in attitudes and knowledge, and livelihood improvements in pilot contexts are important in their own way, as they provide the foundations, experiences, and learnings necessary for bringing change to the next level. This approach can help to provide a healthy balance between achieving outcomes at scale with more strategic, low-hanging outcome opportunities, which are highly valuable in sourcing additional funds and providing the necessary groundwork for scaling.
Unlocking the richness of outcomes will increasingly involve new and more varied partnerships; these would complement traditional partners that in the past have proven fundamental for delivering, disseminating, and using research for achieving outcomes on the ground. Many of the outcomes highlighted here have demonstrated the benefits of engaging new stakeholders in the research development and dissemination process. The partnerships and alliances forged not only helped to co-design the research agenda and to increase its visibility, but also facilitated its use in concrete actions in policies, programs, investments, and on-farm actions. Wider and more diverse webs of partnerships can be challenging to set up and manage, but they are likely to be necessary for an ambitious agenda for transformation.
Significant investment in strengthening monitoring, evaluation, reporting, and learning capacity will be needed to further realize the potential for outcome-oriented research. Providing compelling evidence of outcomes requires adequate tools, time, skills, and generous financial resources, which need to increase proportionally with project ambitions. For large-scale programs or institutions, a common monitoring and evaluation system is fundamental to ensure coherent and consistent reporting and to enable adaptive management and learning through periodic revision of theories of change. At CCAFS, a bespoke online system was developed only during its second phase, in response to the increasing complexity of projects being developed (i.e., multiple and interlinked research activities, partners, outputs, and outcomes). Its late set up hampered our ability to map the evolution of outcomes and innovations over time; consequently, important learnings about these evolutions were likely overlooked in this analysis.
To be useful and usable, sophisticated tracking systems require adequate incentives, particularly when reporting frequency is demanding. At CCAFS, budgets for project continuation or scaling were decided partly based on outcomes reported. This provided staff with incentives to be more results-oriented but also to allocate efforts towards periodic monitoring and evaluation activities. In addition, investments in strengthening capacities should not be underestimated. While reviewing the reports we noticed highly variable quality of outcome evidence; this made it challenging to draw definite conclusions about linkages between outcomes reported, targets set and achieved, and budgets allocated. While evidence quality is typically linked with time and financial resources to conduct rigorous impact assessment, it is also influenced by people's skills and knowledge to report on findings. Building human capacity to track outcomes and to periodically revise theories of change will be critical for improving the quality of reporting and the effectiveness of AR4D.
Without implicating them in any way, we thank Valerie Nelson and John Morton for originally suggesting a synthetic review of program outcomes. The CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) was carried out with support from the CGIAR Trust Fund and through bilateral funding agreements (see https://ccafs.cgiar.org/donors).
Footnotes
Acknowledgements
We also thank the Climate Action Research Theme, the Accelerating Impacts of CGIAR Climate Research for Africa (AICCRA) project, and the Performance, Innovation and Strategic Analysis (PISA) for Impact Program of the Alliance of Bioversity International and CIAT for supporting the authors' time to contribute.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for the publication of this article was made available through the Performance, Innovation and Strategic Analysis for Impact Program of the Alliance of Bioversity International and CIAT.
