Participatory research at scale: A comparative analysis of four approaches to large-scale agricultural technology testing with farmers

Abstract

Tailoring agricultural technology options to the diverse conditions of smallholder farmers requires innovative approaches for testing these technologies with farmers across varied contexts, while incorporating their feedback into learning and decision-making processes. This study compares four such approaches: the Farmer Field School on Participatory Plant Breeding (FFS-PPB), Farmer Research Network (FRN), Crowdsourcing–Triadic comparisons of technologies (Tricot), and adapted Mother–Baby Trial (MBT) as implemented by four concrete projects. The objectives are to provide detailed descriptions of these approaches and their project-specific implementations, identify and analyze common aspects and differences, and derive insights to guide future farmer-inclusive projects aiming at contextual scaling of agricultural technologies. A literature review, key informant interviews, and a systematic content analysis were conducted for the analysis. Common features include cascade training models, simple farmer-managed experiments, and the use of digital tools for data collection. Major differences lie in the extent of farmer–researcher collaboration and decision-making, as well as how technology option-by-context interactions are addressed. The FRN, FFS-PPB, and adapted MBT approaches involve farmers in decision-making throughout most stages of research, including co-learning cycles that adapt the research design and technology options to farmers’ needs. Although these approaches require more training and expertise, they increase the likelihood of achieving relevant results that farmers can implement in practice. In contrast, more standardized approaches like the Crowdsourcing–Tricot streamline the implementation, data management and analysis of large-scale trials, but have limitations in capturing the underlying reasons for farmers’ preferences. Among the studied approaches, the FRN as implemented by the Women's Fields project in Niger is particularly effective in identifying which options best suit specific farming contexts.

Introduction

Conventional agricultural research typically involves technologies developed and tested by researchers on-station, which are then disseminated broadly through extension services (Ronner et al., 2021: 117; Sinclair and Coe, 2019: 3). This approach often leads to blanket recommendations that fail to address the diverse and marginal contexts of smallholder farmers (Nelson et al., 2019: 125–126; Ronner et al., 2021: 117). In response, new Agricultural Research and Development (ARD) approaches emerged in the early 1990s, such as ‘Farmer First’ (Chambers et al., 1989) and ‘Farming Systems Research’ (Collinson, 2000), which consider farmers as research partners. The focus shifted to transferring principles and methods rather than technologies and practices. New methodologies, such as Participatory Technology Development and Participatory Learning and Action, were developed to involve farmers in the research process (Christinck and Kaufmann, 2018: 174–175). Notable examples include the Participatory Plant Breeding programs of ICARDA and ICRISAT West Africa (Ceccarelli and Grando, 1999; Weltzien et al., 2006; Weltzien et al., 2020). The core idea of participatory research is that collaboration between farmers and researchers, leveraging their distinct knowledge and skills, produces better outcomes than working independently (Hoffmann et al., 2007: 355). Farmers contribute contextual knowledge, while researchers offer scientific insights and broader contextual understanding (Christinck and Kaufmann, 2018: 180; Hoffmann et al., 2007: 361). Participatory research implies intentional and focused interaction between researchers and participants, leading to changes in research priorities and methods, decision-making processes and power relations (van de GEVEL et al., 2020). Collaboration among farmers and researchers was thus recognized as essential for orienting applied agricultural research towards farmers’ needs and opportunities in their specific contexts.

The learning about the diversity of smallholder farmers’ needs and preferences led to the notion of providing ‘a basket of options’ to farmers, rather than the widespread ‘package of practices’ (Haussmann et al., 2020: 327). The idea is to offer a range or basket of diverse options, and enabling the farmers to choose the options that suit their individual situations and needs (Ronner et al., 2021: 116–117). This concept is in the same line of thinking as the option-by-context (OxC) interaction approach which aims to better understand which options work best in which contexts (Sinclair and Coe, 2019: 3). This notion of “contexts” includes environmental, crop management, economic, and social conditions (Nelson et al., 2019: 128). Understanding the OxC interactions helps move away from blanket recommendations, enabling farmers to make informed decisions about the most appropriate options for their specific needs (Haussmann et al., 2020: 326–327). The OxC concept extends the classical breeders’ approach to analyze genotype-by-environment (GxE) interactions. The “Options” do not only include genotypes (G), but also other technologies. The “Context” does not only include environmental variables (E), but also socio-economic and cross-cutting variables like gender.

Participatory research has successfully developed innovations tailored to specific communities or farmer types (Douthwaite et al., 2009), but these may not spread to other communities as many factors influence the suitability of agricultural innovations (Sinclair, 2017: 49). Considering the diversity of farmers’ contexts requires assessing a variety of options across a wider range of contexts (Nelson et al., 2019: 132). Gathering context-specific information efficiently calls for innovative methods to integrate farmers’ feedback into the learning process (Ronner et al., 2021: 121). Participatory research at scale is not just about reaching large number of farmers, but refining options and information through co-learning cycles (Descheemaeker et al., 2019; Nelson et al., 2019: 132).

The theoretical framework for scaling varies across disciplines. Scholars in systemic social innovation explored three types of scaling: scaling up, scaling out, and scaling deep. Achieving systems change will likely require a combination of these three scaling types (Moore et al., 2015). In the context of this paper, scaling participation up refers to enhancing the engagement of research institutions and decision-makers with participatory research. Scaling participation out involves enabling more farmers to experiment with new options and provide feedback to researchers. These efforts aim to positively influence the perceptions of both researchers and farmers about the value of participatory research, thereby achieving scaling deep. Our study focuses on large-scale testing approaches with farmers (mainly scaling out), while the Scaling Readiness approach provides methods and tools for organizations to assess the maturity of innovations for scaling and identify actions to accelerate scaling (Sartas et al., 2020).

Co-learning, as defined by Farnworth and Jiggins (2003), involves the mutual exchange of knowledge between farmers and outsiders, leading to new understanding and collaborative decision-making on action plans. Prior studies have described collaborative learning cycles through a four-stage framework in the context of farming and food systems (Restrepo Rodríguez et al., 2014); a seven-step method for implementing co-learning process in transdisciplinary agricultural research (Christinck and Kaufmann, 2018); and an analysis of the collaborative advantages of the long-term farmer-plant breeder collaboration in West Africa (Christinck et al., 2020). These studies depict the co-learning cycle as an iterative process where learnings gained through action and exchange lead to the modification of the original strategy or solution. Key outcomes of co-learning processes include the empowerment of participants in terms of agency and problem-solving capacity (Restrepo Rodríguez et al., 2014: 48), and the strengthening of farmers’ capacities in conducting variety evaluation and seed activities, including sustaining farmers’ seed cooperatives (Christinck et al., 2020: 192). Concrete outcomes in the West African case were the development, registration, and seed production and marketing of best-suited varieties to farmers’ production constraints and uses.

New approaches for large-scale technology testing have been developed and implemented with farmers in a range of contexts for several years, to facilitate learning and understanding of OxC interactions at scale. To provide guidance for projects aiming at contextual scaling of diverse technologies, the present study aims to analyze four approaches for large-scale technology testing being implemented by four different projects:

Farmer Field School on Participatory Plant Breeding (FFS-PPB) approach—Sowing Diversity = Harvesting Security (SD = HS) project, Zambia.

Farmer Research Network (FRN) approach—Women's Fields project, Niger.

Crowdsourcing-Triadic comparisons of technologies (Tricot) approach—Seeds for Needs (S4N) project, Ethiopia.

Adapted Mother–Baby Trial (MBT) approach—Seeds for Zambian Incomes and Livelihoods (SeZIL) project, Zambia.

Specific objectives of the present study are to describe these four approaches and their project-specific implementations, to identify and analyze common aspects and differences, and to derive conclusions that can guide future projects aiming at testing agricultural technologies with large numbers of smallholder farmers.

Methods

The first author conducted a systematic literature review in the Web of Science Core Collection database, with simultaneous searches in CAB Abstracts and AGRICOLA. The literature review focused on understanding the principles and methodologies of the four selected approaches. In addition, she consulted the websites and knowledge bases of the four selected projects. This allowed to gather information on the project context, goals, and to supplement the literature review with grey literature related to the approaches. Next, she identified together with the project teams one expert per project to serve as key informant and co-author of the study. They were chosen based on their in-depth knowledge and direct involvement with the respective projects. The first author then conducted semi-structured online interviews with the four key informants to gather detailed, first-hand information on how the approaches were implemented in practice by each project. The main topics covered in the interviews included: project background, design and implementation of large-scale on-farm trials, data collection methods and tools, results sharing and follow-up decisions based on learnings, as well as roles and responsibilities of the different partners involved. Following these interviews, the expert key informants also shared additional grey literature, such as project reports, to supplement the information gathered from the literature review and interviews. Finally, follow-up online interviews were conducted as needed to clarify information or address any new questions that arose during the analysis phase of the study. The use of multiple data sources, including the literature review, website consultations and expert interviews, enabled data triangulation and helped ensure the validity and reliability of the findings.

The analysis of the most common aspects and differences was performed using a content analysis (Bryman, 2012). A matrix was created using the information documented for the description part of each project. This included project-related details such as the project goal, and large-scale approach-related parameters such as the diversity of tested options, the large-scale trial design, implementation and management, the training approach, the data collection method and tools, the data analysis, the results and knowledge sharing. The matrix helped to identify common aspects and differences, which were then analyzed in depth.

The impact aspect of the different projects, namely the adoption rate of the tested options, was not included in the analysis since it would go beyond the scope of the present study.

Results

Description of the large-scale testing approaches and their application in the selected projects

FFS-PPB approach in the SD = HS project in Zambia

The Farmer Field School approach was developed by the Food and Agriculture Organization of the United Nations in the 1990s, initially to train farmers on integrated pest management in rice farming (Visser et al., 2020: 219). FFS is a group-based adult education approach that builds technical and leadership skills through practical experiments and exercises (Smolders, 2006; Visser et al., 2018). Farmers acquire problem-solving skills through observation, analysis and decision-making during FFS activities (Visser et al., 2018). The core principle is mutual learning, where farmers decide on the learning agenda and learn from each other in groups, with the guidance of the facilitator (Visser et al., 2020: 219). The FFS process strengthens social cohesion and inclusion (Charatsari et al., 2020: 1150). In recent decades, the FFS approach has been applied to genetic diversity management within farming communities, referred to as FFS-PPB (Smolders, 2006). In FFS-PPB, farmers are trained in three methods: (a) Participatory Variety Selection (PVS) for testing and evaluating crop varieties; (b) Participatory Variety Enhancement (PVE) for improving local varieties; and (c) Participatory Variety Development (PVD) for developing new varieties (Visser et al., 2018).

Oxfam Novib launched the SD = HS project in Zambia in 2019, implemented by the Community Technology Development Trust (CTDT) and the Zambia Alliance for Agroecology and Biodiversity (Nangamba and Ngenda, 2023). The project uses the FFS-PPB approach to enhance smallholder farmers’ capacity to access, use and improve plant genetic resources for food and nutrition security and climate change adaptation (Sowing Diversity = Harvesting Security, 2023). Project partners include the Zambia Agricultural Research Institute (ZARI), the Department of Agriculture—Agricultural Advisory Services, and Zambian universities (Nangamba and Ngenda, 2023). By 2023, the SD = HS project in Zambia consisted of 73 FFS-PPB involving 2122 farmers across four districts in Lusaka and South Province. The outcomes of their PVD, PVE, and PVS activities have so far resulted in one new sorghum variety, five restored local maize varieties, and eight varieties of various crops selected in the PVS trials.

Figure 1 outlines the FFS-PPB approach as implemented in the SD = HS project in Zambia, depicting the PVS activities, as the study focuses on technology testing with farmers. The FFS starts with the training of Master Trainers, who then train community-selected farmers as facilitators in FFS methods and moderation (Nangamba Luo, 2023). Facilitators guide the FFS participants throughout the learning process. Together with community leaders, facilitators organize a Community Mobilization meeting to introduce the project and select volunteer and motivated farmers. Each FFS comprises 25–30 farmers, typically organized into gender-segregated subgroups of five to enhance interaction during learning activities. The next step is the diagnostic stage, where FFS farmers identify the major constraints in cultivating their crops and varieties. Based on the problem analysis, the FFS selects one crop, decides on the breeding objectives (two to five most desired traits), the selection method(s), and develops the FFS curriculum. The FFS then searches for suitable germplasm among the community and research partners. The trial implementation phase starts with the site selection, land preparation, and hands-on training for planting the FFS study plots. The weekly or biweekly learning sessions take place mostly in the study plots. A typical FFS session includes: (a) opening and discussion of the planned activities, (b) agro-ecosystem analysis (AESA) where each subgroup observes and records data on plant growth, environment, and traits assessment, and (c) reporting of the results in plenary and joint decision-making on crop management in the trial. The session can also include a special topic on organizational, technical, or social aspects related to the learning activity of the day. The observations from the AESA analysis are documented on the AESA record sheets, this includes morphological, physiological and agronomic data, trial-based information, and environmental data (e.g. weather conditions, pest, and/or disease incidence). At maturity stage, one FFS session is dedicated to the final evaluation of the varieties by the FFS farmers. This is followed by a Farmers’ Field Day to share the progress and learnings with their community. Alongside the data collected by the farmers throughout the season, the facilitators enter farmer-based information, the final ranking of the varieties and the evaluation of the FFS curriculum into the data collection Kobo App. These data are centralized in the “Kobo” platform and managed by CTDT and Oxfam for monitoring and evaluation (M&E) and advocacy purposes. At the FFS level, the season finishes with an evaluation of the FFS functioning and curriculum. All operational partners and facilitators meet at the national evaluation workshop to review the FFS season and plan the next one.

Figure 1.

Illustration of the Farmer Field School on Participatory Plant Breeding approach as implemented in the Sowing Diversity = Harvesting Security project in Zambia.

After two to three PVS trial seasons, the FFS participants decide on the varieties for seed production. This is done locally by the Farmer Seed Enterprises. Additionally, the project finances Community Seed Banks to conserve and make the genetic diversity available at community level.

FRN approach in the Women's Fields project in Niger

The Farmer Research Network is a principle-based approach developed in the context of the Global Collaboration for Resilient Food Systems (CRFS) program of the McKnight Foundation, formerly the Collaborative Crop Research Program. The concept of FRN aims to enable inclusive farmer participation in agroecological research, emphasizing farmers’ learning and agency (Richardson et al., 2022). The three core principles are: (a) farmers who represent the social and biophysical diversity of their communities participate in the whole research process, (b) the research is rigorous, useful, practical and beneficial to the farmers, and (c) the network fosters collaboration and opportunities for learning and knowledge sharing (Haussmann et al., 2020: 316). A FRN consists of farmer groups working together with research and development organizations to facilitate access to technical, institutional, and financial support, while also networking to share local and global knowledge (Richardson et al., 2022: 248). The FRN approach seeks to transform ARD, by considering farmers as research partners (as opposed to passive beneficiaries) and shifting from one-size-fits-all solutions to the concept of basket of options (Haussmann et al., 2020).

The Farmer Federation FUMA Gaskiya in Niger, which currently has over 21,000 members (more than 50% women), launched the “Women's Fields” project in 2012 with CRFS support to tackle issues of low soil fertility and time constraints faced by its women farmers (Collaborative Crop Research Program, 2018; Moussa et al., 2021). Key partners include the National Institute for Agronomic Research, Niger; the International Crops Research Institute for the Semi-Arid Tropics (ICRISAT); the Universities of Maradi, Niger, and Hohenheim, Germany; the non-governmental organization RAIL-Niger; and the MOORIBEN Farmer Union Maddaben of Falwel. The project focuses on intensifying pearl millet-based production systems using low-cost agroecological intensification (AEI) options in Niger's Maradi, Tillabery, and Dosso regions (Haussmann et al., 2020: 324; Moussa, 2024: 1). Initially starting with on-farm testing involving 150 women farmers, the project expanded to reach over 5000 farmers by 2023.

FUMA Gaskiya and its research partners co-developed low-cost AEI options. By 2017, six AEI options were available for testing at large-scale and dissemination: (a) fertilization with sanitized human urine—called locally “Oga” (Moussa et al., 2021); (b) fertilization with compost; (c) partial weeding; (d) seedballs with wood ash; (e) seedballs with NPK fertilizer (Nwankwo et al., 2019); and (f) combinations of individual options (Moussa, 2024). Diversification options, such as new pearl millet varieties and pearl millet-cowpea intercropping systems, were progressively added to the basket of options. Following the “opposite pyramid approach” (Herrmann et al., 2013), on-farm trials were initially researcher-led and done in small plots, but over time trial responsibility was shifted to farmers who tested in larger plots.

Figure 2 summarizes the main steps of the FRN approach as implemented in the Women's Fields project. The planning of large-scale trials begins during the off-season with a call for expressions of interest, during which farmers express their interest in testing one or a combination of two AEI options on pearl millet (Moussa, 2023, 2024). The project team then prepares the trial protocols, record sheets, inputs, and materials for the large-scale trials. The data manager trains the technicians and local animators on data recording with the FRN App developed by FUMA Gaskiya. The technicians train the local animators on manufacturing the options (e.g. Oga, seedballs, compost) and on trial implementation. Both local animators and relay farmers are responsible for training the farmer testers in conducting trials. Each farmer tester runs a simple trial to compare the performance of the chosen option(s) with their usual practice. The farmers are responsible to plant, manage, and harvest the trial, while the local animators collect trial management and performance data using the FRN App, along with rainfall data at the village level. Relay farmers assist the local animators in this task. At crop maturity, one to two best trials per option and per village are selected for a participatory evaluation. The technicians record the farmers’ preference index as well as their reasons for choosing or rejecting the evaluated options in the FRN App. All collected data are centralized and managed in the FUMA data management platform.

Figure 2.

Illustration of the Farmer Research Network approach as implemented in the Women's Fields project in Niger.

At the end of the trial season, the project team does the data cleaning with the technicians and local animators, and then performs the analyses. An end-of-season meeting is organized to share and discuss the trial results with the team, technicians and local animators (who act as farmer representatives). The farmers’ preferences are used to plan the next trial season. The farmers can visualize the results of the large-scale trials on the tablet/smartphone of their local animators. The large-scale trial results are also shared with the community and stakeholders through Open Field Days, exchange visits and radio programs, and in publications.

Crowdsourcing–Tricot approach in the S4N project in Ethiopia

The Crowdsourcing–Tricot approach was developed by Bioversity International ¹ within the framework of the S4N program to provide crop genetic diversity for climate adaptation (CGIAR, 2023). It was designed for testing crop varieties with a large number of individual farmers in different environments to provide specific recommendations for addressing climatic stresses (van Etten et al., 2019). This study uses the term “Crowdsourcing–Tricot” to clarify that the approach to large-scale variety testing integrates both the crowdsourcing concept and the tricot format of experiments. The approach also involves a feedback mechanism using a ranking method (van Etten et al., 2019: 281). The principle of crowdsourcing is to involve large groups of volunteers to contribute individually to scientific tasks, such as conducting experiments and/or gathering observations. The tasks are assigned and data are collected through digital tools (Steinke et al., 2017: 2). This approach is used in different disciplines, such as in ecology to monitor biodiversity across wide geographical areas (Magurran et al., 2010) or in agricultural research to monitor pest and diseases as the PlantVillage initiative (van de GEVEL et al., 2020). Triadic comparison of technologies (tricot) is a balanced incomplete randomized block design, where each farmer tests a different combination of three varieties (or other technology options) (van Etten et al., 2020).

The S4N project in Ethiopia started in 2010 to harness the genetic diversity of durum wheat (Triticum durum Desf.) conserved ex situ for climate adaptation in marginal environments (Fadda et al., 2020: 3). The main project partners are the Ministry of Agriculture—Bureau of Agriculture and Rural Development, the Ethiopian Biodiversity Institute, the Ethiopian Institute of Agricultural Research, Tigray Agricultural Research Institute, Amhara Regional Agricultural Research Institute (ARARI), and Mekelle University (Gebrehawaryat Kidane, 2023). The program operates in the Tigray, Amhara, and Oromia regions and has recently extended to the Southern Nations, Nationalities, and Peoples’ Region. Since the start of the project, two durum wheat varieties have been released in 2017 (Fadda et al., 2020: 5), and new durum wheat varieties might soon be registered (Gebrehawaryat Kidane, 2023). Currently, over 35,000 farmers are cultivating about 30 superior varieties in the project regions (Alliance of Bioversity International and CIAT, 2023).

The first phase of the project was the characterization of 373 landraces of durum wheat conserved in the Ethiopian National Gene Bank and 27 improved varieties, during the 2011 and 2012 cropping seasons (Fadda et al., 2020: 3). This screening was conducted on-station in the Tigray and Amhara regions (Gebrehawaryat Kidane, 2023). Men and women wheat farmers visited these trials to evaluate phenological traits, spike morphology, and the overall preference. From those accessions, 20 landraces and one improved variety as check were selected to start the large-scale variety testing using the Crowdsourcing–Tricot approach (second phase). The tricot experiments started in 2013 in Amhara and Tigray regions, with 200 farmers in each region. The number of farmers increased steadily as the approach was replicated to other crops and regions over time. The “One to five approach” was used for scaling out the variety testing, each farmer tester enrolls five new farmers for the following trial season.

Figure 3 provides an overview of the Crowdsourcing–Tricot approach as implemented in the S4N project in Ethiopia, with ARARI as implementing partner. First ARARI selects the varieties (between 10 and 30 options) for testing, and multiplies the quantity of seeds needed for the trials. The project team trains the Development Agents (DAs) from the Bureau of Agriculture on the tricot trial and data collection. Then the DAs select the farmers randomly from the targeted districts and villages. ARARI designs the tricot trials using the open-source ClimMob software and prepares the seed packages containing a combination of three varieties. The initial training is organized before the start of the cropping season. The selected farmers learn about the trial objectives, tricot methodology, good cropping practices, and receive their trial packages. The training includes explanations on trial planting and ranking of the varieties for the traits predetermined based on focus group discussions. The DAs appoint chairpersons among the farmer testers (one chairperson = 10 farmers) for helping farmers at trial planting and for the trial follow-up. Each individual farmer tester implements the tricot trial on his/her land. He/she manages the trial either under his/her usual cropping practices or the good crop management practices learned during the initial training. Each farmer tester ranks the varieties in the presence of the enumerator, who enters the farmer's responses into the Open Data Kit (ODK) data collection App. Agronomic data such as plant height, flowering time, and yield-related data are additionally collected by each enumerator in 15 trials. The use of remote sensors ease the recording of meteorological data in representative locations. Data analysis is usually done by the Bioversity International team using the data analysis packages available in the ClimMob software. The result reports are generated automatically using ClimMob and sent to the Bureau of Agriculture to provide feedback to the farmers during the final workshop. Each farmer receives a personalized result sheet summarizing their rankings for each trait and the “overall preference ranking” across all farmers. The tricot trial ranking data are combined with mostly environmental data for the GxE analysis.

Figure 3.

Illustration of the Crowdsourcing–Triadic comparisons of technologies (Tricot) approach as implemented in the Seeds for Needs project in Ethiopia.

The project is also active in strengthening informal seed systems through Community Seed Banks, fostering access to seeds at the village level. Farmers’ members of the Community Seed Banks are trained in the production and storage of good quality seed (Gebrehawaryat Kidane, 2023).

Adapted MBT approach in the SeZIL project in Zambia

The Mother-Baby Trial approach is an on-farm participatory method to introduce and test a range of technology options suited to a heterogeneous context (Rusike et al., 2005). It consists of researcher-designed replicated trials, either on-station or on-farm, to test a complete set of technologies (mother trials—MTs), wherein a subset of these technologies is tested in farmer-managed trials (baby trials—BTs) (Snapp, 2002). In some cases, BT farmers can select the technology options to test, while in others, scientists assign them as part of the trial design (Atlin et al., 2002). The design can vary, but usually data are collected by researchers in the MTs, while farmers’ preferences are collected by project staff in the BTs (Rusike et al., 2005).

The term “adapted MBT approach” is used in this study because the first author developed an approach based on the MBT design to meet the SeZIL project's need of reaching 1000 farmers in a variety testing network. This approach combines the MBT design with principles of PVS and participatory methods for technology evaluation with farmers (Ashby, 1990). In the adapted MBT approach, both MTs and BTs are managed by farmers under their usual cropping practices. Farmers receive training and are responsible for collecting data on simplified protocols translated into the local language. The MT and BT farmers score performance-related traits and the overall appreciation of the variety. Yield data are collected in the MTs, while the yield is scored in the BTs.

The SeZIL project, led by KWS SAAT SE & Co. KGaA, was launched in Zambia in 2021. It aims to increase incomes and livelihood of smallholder farmers in Zambia through improved access to diverse crop varieties, the hands-on training needed to evaluate them, and long-term improvements to their ability to produce those that best fit their needs (KWS SAAT SE & Co. KGaA, 2024). This 3-year pilot initiative is a collaboration between Good Nature Agro (GNA), a Zambian for-profit social seed company, and the ZARI. The target region is Kasenengwa District in Eastern Province, where most GNA smallholder farmer-outgrowers are operating. The project started with four different crops: maize (staple and commercial), sorghum (climate-resilient), common beans (balanced diet), and sunflower (staple and commercial). Soybean as cash crop was added from the second year. At the project end, a total of over 1000 female and male smallholder farmers participated in the MBTs, with a total of 38 maize, 26 sorghum, 33 common bean, 38 soybean, and 15 sunflower varieties tested in the MTs.

The project started with the preselection of 1000 GNA outgrowers to establish the testing network. The strategy involved the selection of 48 GNA elite farmers—called Private Extension Agents (PEAs)—to conduct the MTs (12 PEAs per crop), and selecting 40 GNA outgrowers under each PEA for the BTs. An initial meeting introduced the SeZIL project to the preselected MT farmers, explained the trial objectives and design, and confirmed their participation. These MT farmers then helped finalize the BT farmer selection. The project team, along with the MT farmers and some BT farmers, developed the training curriculum and validated the data to be collected. Varieties for testing were selected based on important traits identified in farmer group discussions.

Figure 4 illustrates the adapted MBT approach implemented in the SeZIL project. Before the trial season, MT and BT farmers are trained on site selection, trial demarcation, planting and data collection, with separate training sessions for MT and BT trials. In the first BT season, farmers could choose four varieties for testing, while from the second season, all BT farmers tested the same set of varieties. Mid-season refresher hands-on training on trait scoring and data recording was provided separately for MT and BT farmers. During the first two MT seasons, key trial activities, like planting and harvesting, were conducted by farmers with staff support. From the third season, MT farmers handled these tasks independently with minimal supervision. The BT farmers were trained in harvesting their trials and managed this activity themselves, with assistance from the chairpersons when needed. To facilitate data recording by the farmers, the trial books were simplified and translated into the local language, with a color-coded scoring method for the BTs. At physiological maturity, 20 to 40 BT farmers participated in MT variety evaluations, voting and discussing varieties. The SeZIL team then analyzed and presented the yield and preference results in simple graphs for MT farmers to review and select varieties for the next season. The MT farmers then shared their results with their BT farmers. The findings were also shared with the GNA and KWS maize breeding programs in Zambia.

Figure 4.

Illustration of the adapted Mother-Baby Trial approach as implemented in the Seeds for Zambian Incomes and Livelihoods project in Zambia.

Based on the lessons learned, the BT approach was iteratively adapted. During the first trial season, the project team had to train the BT farmers, even though this task was initially assigned to the MT farmers. The issue arose from the heavy workload faced by the MT farmers. Key changes in the second trial season included (a) appointing and training selected BT farmers (chairpersons) as trainers; (b) assigning the same subset of five to eight varieties to all farmers (balanced design); (c) using a cascade training approach (project staff trained the chairpersons who trained their fellow BT farmers); (d) reducing the number of assessed traits to five; and (e) collecting only qualitative data and less data on crop management. The MT approach remained mostly unchanged, except for a reduction in the number of MTs to select those with different conditions identified in the OxC analysis of the first season. Some initial seed production activities began, aligning with KWS's long-term goal of producing stable and adapted varieties suited to farmers’ conditions and market demand.

Similarities and differences among the large-scale testing approaches

Before going into the detailed analysis, it is important to underline that our results (Table 1) may not reflect the way other FRNs and FFSs operate. Both approaches are principle-based, meaning their application in practice is context specific. For example, in the FRN Women's Fields project, the farmer federation set the research objectives, whereas in other FRNs, these might be negotiated with farmers based on the expertise of collaborating researchers (Richardson et al., 2022: 254). Moreover, not all FRNs conduct large-scale experiments. While this study focuses on large-scale agricultural technology testing, FRNs and FFSs are not only limited to technology development or testing. They also apply various research and development activities with farmers or other partners. FRNs engage in a broad range of research, such as providing agroecological scientific evidence for policy advocacy, developing landscape management models, or co-designing agro-sylvo-pastoral systems (CRFS, 2024).

Table 1.

Main differences between the four studied approaches to large-scale agricultural technology testing in the selected projects.

Analyzed aspects	FFS-PPB approach/SD = HS project Zambia	FRN approach/Women's Fields project Niger	Crowdsourcing–Tricot approach/S4N project Ethiopia	Adapted MBT approach/SeZIL project Zambia
Decision-making on objectives	Farmers set the breeding objectives.	The farmer federation formulated the objectives. (FRN-P1: co-creation of research).	Researchers set the objectives.	Project partners set the objectives with consultation of groups of farmers.
Decision-making on research design	Oxfam-designed trials. Oxfam recommends the data to collect (standardized record sheets).	Farmer–researcher co-design research, co-decision on the data to collect. (FRN-P1).	Researcher-designed trials. Researchers decide on the traits to assess based on focus group discussion with farmers.	Researcher-designed trials, discussed and validated with farmers. Co-decision on the data to collect.
Diversity of the tested options	PVS trials: 5 varieties + local check. Varieties chosen based on the breeding objectives.	10 AEI options (basket growing over time). One option or combination of two options per tester + farmers’ check.	10 to 30 varieties, with three varieties per tester. Every 3–4 years new material is introduced.	7 to 23 varieties (MTs) and 3 to 7 varieties (BTs) depending on the crop + farmer's check.
OxC interaction analysis	Each FFS develops or selects locally adapted varieties that fit to its specific context.	Options are designed for the contexts of FUMA members. Farmer typology database.	Mostly environmental data are used for OxC analysis. Gender, location, age are cross-cutting factors.	Gender, age, crop management and environmental data are used for OxC analysis.
Co-learning	Farmer-to-farmer, farmer-breeder/ extension officer interaction.	Farmer-to-farmer and farmer–researcher co-learning at different levels. (FRN-P3).	Limited interaction in the tricot trials. Farmer–researcher interaction in on-station trials.	Farmer-to-farmer and farmer–researcher co-learning through various activities.

AEI: agro-ecological intensification; FFS-PPB: Farmer Field School on Participatory Plant Breeding; FRN: Farmer Research Network; FRN-P: FRN-Principle; MBT: Mother–Baby Trial; MTs: mother trials; BTs: baby trials; SD=HS: Sowing Diversity = Harvesting Security; S4N: Seeds for Needs; SeZIL: Seeds for Zambian Incomes and Livelihoods.

Note: Italicized results represent general features or principles of the approach; nonitalic results are project-specific.

An additional aspect is the focus on crop varieties in three of the selected projects, unlike the FRN project. As mentioned earlier, principle-based approaches can be applied across a broad range of research topics. For approaches such as Crowdsourcing–Tricot, van Etten et al. (2019) suggested its application beyond crop varieties to fertilizers or crop management products, among others. While we did not find concrete examples in the literature, these might be unpublished or not yet widely adopted. A recent publication highlights the adaptation of the Tricot methodology to evaluate consumer preferences (Olaosebikan et al., 2024). Examples in the literature for the MBT approach include applications in soil fertility management (Rusike et al., 2005) and PPB (Atlin et al., 2002; De Groote et al., 2002).

Large-scale agricultural technology testing approaches aim to tailor solutions to specific contexts. To achieve this, the studied projects share common practices to enable the testing of diverse options with a large number of farmers. These include the use of a cascade training model, where the project trains lead farmers who then train their fellow farmers (Conan et al., 2013) or the Training of Trainers, where Master trainers train facilitators on the FFS methodology (Visser et al., 2020). Another common practice is the use or intention to use digital tools to facilitate data collection and, to some extent, for sharing results with farmers. The challenge is that most farmers have a low literacy level, necessitating the training of lead farmers or enumerators for data recording in the large-scale trials. All studied projects tend to reduce the complexity of the on-farm experiments to facilitate their implementation at scale.

The main differences among the approaches and projects refer to who decides about objectives, and research design, the diversity of the tested options, the extent to which OxC interaction analysis is taken into account, and the extent of co-learning (Table 1). The results that represent general features or principles of the approach are written in italics in Table 1, while the nonitalic results are specific to the selected project.

Decision-making on the objectives

Decision-making in setting the research objectives varies from being largely farmer-driven to predominantly researcher-driven.

In the FFS-PPB approach, each FFS defines its breeding objectives based on the farmers’ assessment of production constraints and current crop and variety diversity. The farmer federation FUMA Gaskiya (FRN Women's Fields) formulated the research objectives and co-developed the AEI options with research partners. In this case, being organized into cooperatives might offer an advantage for identifying common problems among members, compared to projects collaborating with individual farmers. Consultative participation was used in the adapted MBT approach. Research priorities were first identified by the project partners and then co-validated with GNA farmers. The choice of the crops was based on GNA and KWS strategies, as well as farmers’ interest in diversifying with high-demand crops. Farmers’ interest in climate-resilient crops grew after observing the potential of sorghum under drought conditions in their trials.

All three approaches strive to follow principles of Farmer Participatory Research, involving farmers actively in most stages of the research process. Co-creating the research agenda ensures that the research addresses farmers’ real problems and delivers relevant results. In the Crowdsourcing–Tricot approach, researchers decide on the research objectives.

Decision-making on the research design

Decision-making on the research design varies significantly across the four approaches. In the Crowdsourcing–Tricot and the FFS-PPB approaches, scientists design the research. Conversely, the FRN and adapted MBT approaches actively involve farmers in both the research design and decisions about data collection. For instance, in the Women's Fields project, farmers choose the number and combination of options to test, including the plot size. Farmers and researchers also agree on who collect what data. While this flexible approach supports farmer-relevant research—one of the FRN principles—and helps building farmer–researcher relationships, it can lead to more demanding data analysis due to the unbalanced design. An additional example from the SeZIL project is that farmers’ feedback and lessons learned led to a simplification of the large-scale BT protocols, thereby improving data quality and robustness.

In the Crowdsourcing–Tricot approach, the strategy is to standardize the experimental design to digitally support the entire process, from designing the trials to data management and result reporting. This means that farmers cannot choose the varieties to test; instead, the varieties are assigned according to the incomplete balanced design. While the ranking format facilitates data recording by the farmers and data analysis, it limits farmers’ choices in their responses. For instance, two different varieties can be equally important, just serving different purposes or production objectives. From a farmers’ perspective they would be valued equally, but the format imposes the ranking of the best and the worst. In the FFS-PPB approach, the plot design and data collection are predefined in the Oxfam Facilitators’ Field Guide. The AESA analysis involves extensive data collection by farmers, which led the SD = HS project to limit the number of varieties tested in the PVS (Table 1).

Diversity of the tested options

The projects S4N in Ethiopia and SeZIL in Zambia offer a higher number of options for testing (Table 1). The number of options each farmer can test in the large-scale trials ranges from two in the Women's Fields up to eight in the SeZIL project (BTs). Although the number of options per farmer is limited, activities such as participatory variety evaluations and Open Field Days can enhance farmers’ exposure to all tested options, while fostering experience and knowledge sharing. Additionally, it is important to include a clearly diverse range of options, so that different types of farmers have a chance to identify new opportunities for a specific context.

The collaboration with national and international research institutes is essential to access agricultural technologies, innovations, and scientific knowledge. For example, the SD = HS project sources most genetic material from ZARI and the University of Zambia and targets to expand the collaboration with international research institutes to access even more diversity (Nangamba Luo, 2022). Both scientific and local knowledge are necessary for the development or adaptation of relevant agricultural technologies.

Option-by-Context interactions

The Women's Fields project (FRN) uses a multi-dimensional approach to better understand OxC interactions. The AEI options were first tested in researcher-designed on-farm trials to assess their performance under specific agro-ecological and socioeconomic conditions. These trials confirmed that sanitized human urine “Oga” outperformed the control (farmers’ fertilization practices) under a wide range of field conditions (Moussa et al., 2021). Specific adaptations were found; for example, in a year with normal rainfall pattern, “Oga” combined with organic matter significantly improved yield compared to the control on the poorest soil type (locally named Jampali). These validated options were then proposed to a larger number of farmers for testing in their fields, under their own management. The participatory evaluations of the options conducted in the farmer-managed trials help to refine and reshape the basket of options. Additionally, the Women's Fields project created a farmer typology database to better understand which options are adapted to which farm types. The SeZIL project (adapted MBT) tested the varieties in different contexts to better understand GxC interactions. The MT results provided information on genotype-by-farmer and genotype-by-year interaction patterns for yield and for farmers’ preferences, as well as the reasons for these variety preferences. While the BTs provided insights on the variations of farmers’ preferences. In contrast, the Crowdsourcing–Tricot approach uses mostly environmental factors to assess their effects on the yield and overall preference ranking. Based on these results, S4N Ethiopia provided recommendations to farmer seed producer groups on the wide or specific adaptation of the varieties to environmental conditions. Bioversity International is very much advanced with robust statistical models and tools for the analysis of ranking data and GxE interactions of the tricot trials.

The FFS-PPB approach differs from these strategies, as it does not aim for context-specific recommendations across FFSs; rather, each FFS develops or selects locally adapted varieties fitting to the specific context and need of each FFS established in its community.

Knowledge sharing and co-learning

The FFS, FRN, and adapted MBT approaches implement participatory research principles, valuing local knowledge and fostering mutual learning between farmers and researchers. In the FFS-PPB approach, knowledge sharing and co-learning among farmers are inherent to the FFS learning methods. Exchanges also occur between the FFS and its community, as well as among project partners, including FFS facilitators, during the annual national workshop. One challenge mentioned during the interview was the insufficient number of breeders in the country, which limits the FFS farmers’ access to such knowledge and diversity of germplasm.

Similar to the FFS-PPB approach, co-creation, and knowledge sharing is a FRN principle. The Women's Fields project illustrates this well, with knowledge being shared at multiple levels: (a) Within the project: co-learning occurs throughout the research cycle. Nonexhaustive examples include the co-creation of the AEI options, result sharing via the FRN App and Open Field Days; (b) Between FRN projects: joint activities are organized between FRN projects; (c) At the CRFS West Africa Community of Practice level: project teams (including researchers, farmer representatives, and students) meet annually to share knowledge and experiences. The adapted MBT approach similarly includes activities to strengthen exchange among farmers and farmer–researchers, such as seasonal meetings, farmer group trainings, participatory variety evaluations and end-of-season workshops. The first author (i.e. researcher) participated in most of these events, thereby gaining a better understanding of farmers’ situations and slowly building mutual understanding.

In the Crowdsourcing–Tricot approach, activities, or events that foster exchange and mutual learning are limited. The approach extensively uses digital tools for gathering and sharing data. The S4N project in Ethiopia reported that the trial results are sent via email to the Bureau of Agriculture, who should present them to the farmers. However, there are instances where the partner may consider its responsibilities fulfilled once the trials are harvested, leading to results not being systematically shared with farmers (Gebrehawaryat Kidane, 2023).

Discussion

The analysis of the four large-scale technology testing approaches demonstrated both commonalities and differences among them. Common aspects include simple farmer-led experiments; forward cascadal training and information flows to reach large number of farmers; backward cascadal information flows to transfer data back to the project team; and (at least partial) use of digital data collection tools. Key differences include the extent of farmer–researcher collaboration and co-learning process, as well as the way of addressing OxC interactions. We discuss these aspects herein.

Cascadal information flows and training

All four technology testing approaches use cascading training and information flows of varying magnitude to reach the scale. This dissemination model involves training a few individuals who then train a larger number of farmer participants. The FFS-PPB approach uses a “training of trainers” model to train facilitators on moderation skills, as they are responsible to guide the FFS in the learning process. In the other approaches, the intermediaries (e.g. lead farmers) are primarily trained in conducting experiments and data collection. Their roles can vary; for example, in the SeZIL project the chairpersons are responsible to train and supervise their group of farmers (Figure 4), while the local animators in the Women's Fields project perform similar task but also record data and share results with the farmers (Figure 2). When farmer testers collect data themselves (SeZIL project), the method and record sheet must be highly simplified (e.g. images to illustrate traits, and color codes to score them). Cascade training enables low-literate farmers running trials as the trainers can support them in data collection. Overall, this training model facilitates scalability in a cost-effective way, while also enabling farmers gaining new leadership skills as trainers (VoxDev, 2020).

Digital data collection tools

The use of digital data collection tools can improve data quality by eliminating errors associated with the process of digitizing paper-based records and speeding up data availability for analysis. However, digital data collection requires intensive training of literate farmers, as well as the availability of smartphones and connectivity for data synchronization. Consequently, not every individual farmer tester can record data. Three studied projects reported their strategy of selecting literate farmers, such as local animators (Women's Fields Niger) or facilitators (SD = HS Zambia), and training them in recording data with digital tools (Gebrehawaryat Kidane, 2023; Moussa, 2023; Nangamba Luo, 2023). The initial plan of S4N Ethiopia was for each individual farmer to collect data on their smartphones, but this did not work out (unlike in India and Nicaragua). Instead, the project decided to introduce trained enumerators who visit the farmers at specific times to encode their rankings into the ODK Collect App. This also allows collecting additional “standard” agronomic data, such as plant height and yield measurements (Gebrehawaryat Kidane, 2024: 3). While this enables to capture a broader set of data, it reduces farmers’ responsibility and capacity building in data recording.

Simplifying the type of data to be collected can facilitate digital recording, but it comes with trade-offs. For example, ranking facilitates digital data collection, reduces errors compared to rating (Quirós et al., 2024: 2), and is easier to explain to low-literate farmers (Steinke et al., 2017: 2–3). However, it is less informative than rating. The strategy of the Women's Fields project is to gather only essential data (see details in section “FRN approach in the Women’s Fields project in Niger”) needed by both farmers and researchers to assess the performance and acceptability of the options. These are collected by well-trained, literate local animators and relay farmers.

Collaborative learning process and adaptive action

The FRN, FFS-PPB, and adapted MBT approaches have strong components of collaborative learning processes. Our results show that in the adapted MBT and FRN approaches, farmers are involved in decision-making at most stages of the research process. For example, the farmers can choose the number and combinations of options for testing in the FRN Women's Fields project. Another example is the adaptation of the BT design in the SeZIL project based on farmers’ feedback after the first trial season. The decision of assigning the same varieties to all BT farmers (balanced design) from the second trial season led to fewer mistakes at planting and improved data quality. While co-designing research takes into account farmers’ capacities and perspectives, it can also add complexity to data analysis (Richardson et al., 2022: 254). Moussa et al. (2021) also reported the challenge of unbalanced datasets resulting into lower statistical power. Beyond these specific examples, our results show that the FRN, FFS-PPB, and adapted MBT approaches encompass the main elements of collaborative learning as conceptualized by Restrepo Rodríguez et al. (2014: 38). This study suggests that a collaborative learning process includes four phases: (a) establishing collaboration, (b) dialogue, (c) discovery, and (d) application of new knowledge. The first phase (a) is the identification of the partners and establishment of the collaboration between farmers, researchers and other stakeholders. It consists mainly in agreeing on shared goals and approach, and defining the rules of operation. The dialogue process (b) is the combination of knowledge, perspectives of the different actors to get a joint understanding of the problems and constraints. The discovery phase (c) is about filling the knowledge gaps, for example through practical experiments, to come up with suggestions for new or refined processes or activities; and (d) is the phase where the new practices are consolidated into broader activities, such as pilot-scale activities (Christinck and Kaufmann, 2018; Restrepo Rodríguez et al., 2014: 44). Restrepo Rodríguez et al. (2014: 48) found as main outcomes the strengthening of social capital (e.g. reinforcing networks) and human capital (e.g. capacity building), and enhancing problem-solving capacity of the participants. The Women's Field project reported the adaptation of the technology options by the farmers. The “Oga” option was initially introduced as fertilizer for pearl millet, but farmers now also use it to moisten compost instead of water (Moussa, 2023). Co-learning processes, where the learning capacities are strengthened through interactions with others, increase the likelihood of achieving relevant results for farmers and can be implemented in practice (Christinck et al., 2020: 179).

In the Crowdsourcing–Tricot approach, the collaborative learning process is not a primary focus. Instead, the approach emphasizes streamlining experimental design, cost-effective on-farm trials, and data collection fitting to statistical ranking models (Coto et al., 2019; Quirós et al., 2024: 2; van Etten et al., 2019). Quirós et al. (2024: 5) report that data standardization facilitates the generation of standard result outputs for different audiences. The approach is designed to rapidly identify and disseminate climate-resilient varieties at scale (Fadda et al., 2020: 4). However, this method has limitations in fostering farmer–researchers collaboration and capturing the underlying reasons for farmers’ preferences. These are important for guiding the breeding program and selecting a range of promising options for testing to address farmers’ constraints and needs.

Option-by-Context interactions

Overall, the understanding of OxC is important for researchers, farmers, and development organizations. Researchers aim for context-specific recommendations, while farmers, facing diverse and changing conditions within their farms, need to access the information on which options perform best under which conditions, and for which production objectives. This highlights the need to move away from “one-size-fits-all” solutions, as effective agricultural technologies must be tailored to local contexts and needs, to enhance sustainability.

Our study underlines the complexity of understanding OxC interactions in large-scale farmer-managed trials. The first challenge is the need of large datasets on agro-ecological and farmers’ socioeconomic conditions. For approaches like Crowdsourcing–Tricot, data collection to characterize the biophysical context is less challenging because climatic data are gathered using remote sensors. However, the non-inclusion of socioeconomic factors, such as access to labor force and markets, production objectives or farm types and size, may result in recommendations of options that do not fit the situation of some groups of farmers. Descheemaeker et al. (2019: 171) emphasize that farmers’ decision-making and the performance of the options are influenced not only by ecological but also by socioeconomic and agronomic factors.

The Women's Fields project adopted a strategy similar to that recommended by Nelson et al. (2019: 137) to prioritize contextual factors based on the farmers’ trial objectives and researchers’ hypotheses about OxC interaction. In addition, the farmer typology database, owned by the farmer federation FUMA Gaskiya, is used to better understand OxC interactions and design future research activities. Descheemaeker et al. (2019: 173) specify that farm typologies can assist in understanding and classifying the diversity amongst farms. The dynamic and complex changes in farmers’ socioecological contexts, however, present challenges for OxC analysis and underscore the need to foster farmer learning and their ability to choose what best fits their context, as key goals of large-scale agricultural technology testing with farmers.

The second challenge concerns the analysis of OxC interactions. An unbalanced design reduces the ability to detect significant effects of contextual factors on yield and farmers’ preferences, or other variables. Descheemaeker et al. (2019: 186) underline the value of on-farm trials to better understand farmers’ constraints and identify best-fit options or refining the initial set of options, but also raised the challenge of explaining the variability in yield from on-farm trials. Although statistical models are used to unravel the complexity of wide-ranging explanatory factors, a significant proportion of the variability usually remains unexplained. It is thus important to involve farmers, and their detailed knowledge about their fields, and practices in the analysis of trial results and to develop analysis tools that can be used for discussions with farmers.

Conclusions

If the primary focus of the large-scale testing with farmers is to assess agricultural technology options across diverse environments and provide recommendations on adaptation to specific climatic conditions, the Crowdsourcing–Tricot approach seems to be the most effective. Its extensive use of the ClimMob software digitally supports all steps of the experimentation cycle and provides a standardized database. However, this approach is limited in assessing the suitability of the new technologies in relation to the socioeconomic conditions of farmers. If the aim is to address the diversity of smallholder farmers’ needs and preferences in an inclusive manner, where learnings gained in the large-scale testing activities serve to iteratively adapt the research approach and technology options to farmers’ realities, the FRN (Women's Fields Niger) or adapted MBT approaches (SeZIL Zambia), with close farmer–researcher interaction, are recommended. These approaches, however, require more intensive training and transdisciplinary expertise within the project team. If the main goal is to strengthen farmers’ capacity to identify or develop crop varieties suited to specific conditions at the FFS community level, improve farmers’ leadership skills and enhance social cohesion, the FFS-PPB approach is certainly the most appropriate.

Each approach has its strengths and weaknesses, and can adopt elements from the others to become more effective. The FRN and adapted MBT approaches, for example, could adapt digital tools from the Crowdsourcing–Tricot approach for specific aspects of their work. The FRN Women's Fields project offers valuable elements, such as co-developing technology options based on identified constraints among Farmer Federation members and using information from their farmer typology database to better understand OxC interactions.

Large-scale farmer-led experimentation with new agricultural technologies can adapt these technologies and validate their robustness in specific contexts. It can also help to understand specific adaptation patterns and enable farmers to choose what fits best into their specific farming context. No matter which approach is being used, large-scale testing must be accompanied by efforts to make the tested options available, accessible and affordable for the target groups of farmers, in order to reach the desired impact.