Leveraging an Implementation– Research Partnership to Improve Effectiveness of Nutrition-Sensitive Programs at the World Food Programme

Abstract

Background:

Nutrition-sensitive programs can accelerate progress in addressing malnutrition. However, evidence gaps exist related to their effectiveness and how to optimize program design and implementation.

Objective:

We present the process the International Food Policy Research Institute and the World Food Programme (WFP) used to develop nutrition-sensitive program guidance and plans for improving program effectiveness and contributing to the evidence base through rigorous evaluations.

Methods:

A 5-step process, using principles of design thinking (a systematic, iterative analytical approach to problem solving), was used to develop, test, and refine WFP’s nutrition-sensitive guidance. The guidance focuses on improving nutrition outcomes for nutritionally vulnerable groups across the life cycle: women and children in the first 1000 days, preschoolers, schoolchildren, and adolescents.

Results:

Through iterative consultations, we created WFP’s nutrition-sensitive guidance that includes harmonized theories of change across WFP’s programs; 7 opportunities to enhance the programs’ nutrition-sensitivity; and mapping of these opportunities to WFP programs and key evidence gaps. This guidance has been rolled out to WFP’s offices worldwide to support improved nutrition outcomes. Finally, several evaluation designs have been proposed to fill identified evidence gaps.

Conclusions:

By leveraging our implementation–research partnership, we expect that WFP’s programs will be more effective and cost effective for improving nutrition. This can be assessed through coupling newly designed nutrition-sensitive programs with rigorous evaluations. Evaluation results will be used to refine WFP’s nutrition-sensitive guidance and improve their programs globally. This guidance, and creation process, could be useful for others interested in designing nutrition-sensitive programs and increasing program effectiveness for nutrition.

Introduction

To reduce malnutrition globally, investments in nutrition-sensitive programming and comprehensive evaluations are urgently needed. The 2013 Lancet Series on Maternal and Child Nutrition estimated that scaling up the recommended package of nutrition-specific interventions, for which there was rigorous evidence available, to 90% population coverage in the 34 countries with the highest malnutrition burden would only reduce child stunting by 20%. ¹ This highlights the need for other types of interventions and programs, in other sectors, to address the complex challenges of reducing malnutrition. These should be designed to address one or more of the immediate (eg, food, health, and care) and underlying (eg, food security, environmental conditions, and caregiving resources) determinants of nutrition and/or the enabling environment for nutrition (eg, knowledge and evidence, politics and governance, etc.) for nutrition as laid out in the guiding framework for the aforementioned Lancet Series. ²

Nutrition-sensitive programs seek to address these challenges by setting clear nutrition goals and combining inputs from multiple sectors to simultaneously address multiple drivers of malnutrition. Well-designed, targeted, and implemented nutrition-sensitive programs that integrate or colocate nutrition and health components with elements from other sectors (agriculture, social protection, or water, sanitation and hygiene [WASH]), and that are gender focused, will likely lead to larger impacts than programs that only address immediate determinants of nutrition problems (eg, food, health, and care). Likewise, impact on nutrition can be achieved through using nutrition-sensitive programs as platforms to improve the delivery, coverage, and scale of nutrition-specific interventions. ^1,3

Although some evidence of the effectiveness of nutrition-sensitive programs for different malnutrition problems across the life cycle exists, ^{4 -7} more rigorous evidence on what types of nutrition-sensitive programs work, how they work, and how cost-effective they are for improving a range of nutrition outcomes across the life cycle (stunting, wasting, micronutrient deficiencies, and overweight/obesity) is urgently needed. ³ The creation of this evidence necessitates careful attention to program design and implementation coupled with rigorous comprehensive evaluations. ^3,8

Close collaborations between program implementers and evaluators are key to expanding the evidence base for nutrition-sensitive programs. ^{8 -10} To be successful, collaborations need to start early and maintain ongoing communication for optimal alignment of priorities, expectations, and time frames of program implementers and evaluators. ⁹ In addition, multiyear funding as well as high-level negotiation and endorsement are needed, especially early on, to build commitment and momentum throughout the involved organizations. Implementation–research collaborations can be highly beneficial to all involved and can lead to more effective and cost-effective programs and high-quality evidence that can contribute to the global evidence base.

In this article, we discuss the systematic 5-step process the World Food Programme (WFP) and the International Food Policy Research Institute (IFPRI) used to develop WFP’s nutrition-sensitive program guidance. We then present some of the key results that emerged from this process, including the development of (1) requirements for WFP programs to be considered nutrition sensitive; (2) 7 nutrition-sensitive opportunities; (3) potential pathways showing nutrition-sensitive opportunities in WFP programs; and (4) key evidence gaps. We conclude the article by discussing how WFP envisions using this process and associated results in the future to systematically collect and map evidence of different opportunities to nutrition impacts which can further inform the design/redesign of nutrition-sensitive programs both within WFP and globally. This initiative is unique in the high-level commitment by both organizations in mobilizing the staff and other resources to support the evolution and implementation of this collaboration, the systematic approach used to look across a variety of programs to identify ways to improve their nutrition sensitivity, and the ongoing collaboration to identity opportunities to test the hypotheses set forth in WFP’s nutrition-sensitive guidance. The model presented in this article can be used by other institutions that are committed to achieving nutrition impact at scale by enhancing the nutrition sensitivity of their programs.

Methods

The approach used by WFP and IFPRI has been informed by design thinking and consists of 5 steps: identify a need or problem, iterate on possible solutions, apply solutions, test solutions, and analyze and share results. ¹¹ These steps can be envisioned as a circular process in which the results at the last phase feed into the identification of new needs or problems, refinement of the solutions, or development of new solutions (Figure 1A). This process is unique in that it applies systematic thinking across WFP’s programs and the platforms that are used to deliver them. This begins from the initial steps of the process as described below and continues iteratively, which is not typically done when planning individual programs.

Figure 1.

A and B, Five steps in the design thinking process and the implementation–research partnership on making World Food Program (WFP)’s programs nutrition-sensitive.

WFP and IFPRI used an adapted version of this process to develop WFP’s nutrition-sensitive guidance and to provide technical assistance to WFP country offices as described herein. ^12,13 In this adapted process, the 5 steps are recognition of the need to make WFP’s programs more nutrition sensitive; iteration between WFP program implementers and IFPRI researchers to create key inputs for the nutrition-sensitive guidance; roll out of the guidance to WFP’s regional and country offices and design and/or redesign of nutrition-sensitive programs; development and implementation of program evaluations; and analysis, dissemination, and utilization of results from the program evaluations (Figure 1B).

This article focuses on the first 3 steps of this 5-step process, as the ongoing partnership process is currently at step 4. Through this process, it is expected that WFP’s guidance will be revised, refined, and expanded as new evidence emerges.

Step 1—Identify Need: Make WFP’s Programs Nutrition-Sensitive

Ambitious targets for nutrition were set by the Agenda 2030 for Sustainable Development and the United Nations Decade of Action on Nutrition, which were reiterated by WFP’s own Strategic Plan (2017-2021) ¹⁴ and the new Nutrition Policy (2017). For WFP to achieve its full potential to eliminate malnutrition in all its forms, it was clear that action would be needed across the organization. WFP incorporated nutrition-sensitive programming as a priority in its Corporate Results Framework (2017-2021) ¹⁵ and identified 5 key program areas for potential improvement: general food assistance (GFA), school feeding (SF), Food Assistance for Assets (FFA), Smallholder Agriculture Market Support (SAMS), and social protection (SP; Table 1). These programs represent a majority of WFP’s portfolio as well as the most consolidated concepts across WFP offices. They target poor and nutritionally vulnerable households and address critical immediate and underlying determinants of nutrition as well as the enabling environment.

Table 1.

World Food Programme’s 5 Program Areas for Nutrition-Sensitive Programming.

Program Areas	Description
General Food Assistance (GFA)	GFA programs aim to protect the food security, nutrition, and livelihoods of vulnerable households during and following emergencies and in fragile and conflict-affected settings. To accomplish this, GFA programs provide unconditional transfers to food-insecure households in times of crisis and recovery. These transfers can be in the form of in-kind, cash-based, or commodity voucher support
Food Assistance for Assets (FFA)	FFA programs provide a direct food or cash-based transfer to meet identified nutrient consumption gaps of the most vulnerable over the short term. FFA programs also work to build natural and physical assets of food-insecure and vulnerable households and communities, using their own labor, that are designed to reduce food insecurity through reducing risks associated with disasters, strengthening livelihoods, and building resilience over the longer term
Social Protection (SP)	SP programs are carried out with the purpose of supporting national and local capacity to build social protection systems. SP engages in the delivery of safety net programs to improve food security and nutrition for the most vulnerable and in technical assistance in the development of national safety net programs. These programs include a variety of activities that can protect vulnerable populations and prevent a worsening of their situation, or may also promote improved livelihood outcomes, such as food or cash transfers, community asset creation, delivery of basic social services, and school meals
School Feeding (SF)	SF programs are a complex type of safety net/social protection intervention that aims at ensuring children have access to education, health, and nutrition. SF programming targets food-insecure households, especially households with vulnerable children (eg, orphans) and children in emergency and protracted crisis situations. SF programs can take a variety of forms, including cooked meals prepared at school, in the community or delivered from centralized kitchens; high-energy biscuits or nutritious snacks provided in school; or family food rations (THRs), conditional on attending school
Smallholder Agriculture Market Support (SAMS)	SAMS initiatives are a type of WFP program connecting smallholder farmers to markets, giving them an opportunity to grow their businesses and improve their lives and those of their entire communities. SAMS builds on WFP expertise in pro-smallholder procurement, ensuring that WFP food purchases contribute to forging stronger economies. Under SAMS, stable demand from WFP and partners provides smallholder farmers with an incentive to invest in expanding their production activities.

Abbreviations: FFA, Food Assistance For Assets; SAMS, Smallholder Agriculture Market Support; SF, school feeding; SP, Social Protection; WFP, World Food Program; THR, Take Home Ration.

As an early step in initiating nutrition-sensitive programming at WFP, an internal WFP Nutrition-Sensitive Working Group at Headquarters was formed to facilitate dialogue between WFP’s nutrition staff and these other program and relevant operational areas (eg, supply chain and cash transfers unit), raise awareness of the need for action, build trust and confidence, and establish mutual responsibility for nutrition-sensitive programming across the organization. Next, WFP conducted a landscape analysis, which found that many of its country offices were already incorporating nutrition actions in these program areas; however, more guidance was needed on the types of interventions that should be prioritized for inclusion and for systematic integration of nutrition across programs. Thus, the need to create nutrition-sensitive guidance for WFP’s programs was recognized as an essential basis for designing and implementing effective nutrition-sensitive programs. To create this guidance, partnering with IFPRI was a priority for WFP due to IFPRI’s wealth of experience working with program implementers on designing and evaluating nutrition-sensitive programs across a range of sectors.

IFPRI, likewise, noted the demand for evidence-based programming and understood the associated urgency for building the evidence-base for nutrition-sensitive programming. Recent reviews of the effectiveness of agriculture and social protection programs highlight the lack of rigorous evidence of the impact of these programs on nutrition outcomes. ^3,8,16 A consistent and strong recommendation provided in these reviews is the need for rigorous, theory-based impact evaluations that will generate a solid body of evidence on what works and what does not work to improve nutrition; what are the pathways of impact; what is the cost and cost-effectiveness of achieving these improvements; and how to measure them. ⁸ The opportunity for leveraging an implementation–research partnership to systematically design and test more nutrition-sensitive programs to fill key evidence gaps was recognized by both organizations.

Step 2—Iterate on Possible Solutions: Create WFP’s Nutrition-Sensitive Program Guidance Through Iterative Consultations

Together WFP and IFPRI launched the partnership with an inception workshop in June 2016. The workshop laid the foundation for the partnership and was attended by WFP Headquarters, regional, and country staff and IFPRI researchers representing different areas of expertise (eg, economics, nutrition, and program evaluation). Presence of senior management from each organization emphasized the strategic importance of this endeavor. During this launch workshop, WFP program implementers representing the range of activities (eg, GFA, SF, SAMS, FFA, and SP) presented their current program models and some key success stories, and IFPRI researchers shared experiences and findings from previous research conducted through research–program partnerships. Small group sessions were held to identify perceived barriers and opportunities for nutrition-sensitive programming at WFP.

Following the inception workshop, a set of consultations with WFP’s Nutrition-Sensitive Working Group; program practitioners in the areas of GFA, SF, SAMS, FFA, and SP; and IFPRI were undertaken to further understand how these 5 program areas were conceptualized, what activities are implemented, what impacts are expected, and to identify nutrition-sensitive opportunities for each program area. To achieve this, preexisting theories of change (ToC) for each program area and other background documents were reviewed and individual program workshops were held. At each program workshop, the program-specific ToC was discussed and revised using a harmonized structure across programs, aimed at linking program activities with common outcomes relevant for nutrition. During this process, opportunities for enhancing nutrition impacts, requirements for doing so, and entry points for these opportunities were also identified, prioritized, and mapped through specific impact pathways. This prioritization process assessed opportunities in terms of their likelihood of impact, in light of the evidence currently available, as well as their implementation feasibility. Parallel to this work, WFP and IFPRI staff working on the nutrition-sensitive partnership identified key evidence gaps and associated research questions that could be addressed through evaluations.

This series of consultations resulted in a report led by IFPRI which was to inform the development of WFP’s nutrition-sensitive guidance. ¹² A follow-up workshop with program teams was held in December 2016, where a summary of the report, and how its content was intended to be used, was presented. After several iterations by WFP and IFPRI staff, the nutrition-sensitive guidance, geared toward use by WFP regional and country offices, was released in Interim form in early 2017. ¹³

Step 3—Apply Solutions: Roll Out Program Guidance and Redesign Programs to be More Nutrition Sensitive

The roll out of WFP’s nutrition-sensitive guidance across WFP began in 2017. The guidance and an accompanying set of training materials that WFP Nutrition developed specifically for field level were presented and used by WFP Nutrition Division staff at regional and country-level meetings to help country and regional office staff design nutrition-sensitive programs themselves. In addition, WFP and IFPRI staff participated in joint consultations, including on-site training sessions with the country offices in Sri Lanka and Honduras in 2017 and in Sri Lanka and Zimbabwe in 2018 and held program-specific webinars available to regional and country staff. These consultations were a mini replication of the iterative workshops conducted at WFP Headquarters in 2016. Program staff who participated considered the malnutrition burden among different vulnerable populations, the drivers of these problems, and the possible objectives the country office could work toward, given its program platforms and national priorities. Selected programs were then discussed in detail, and country- and program-specific ToCs, adapted and contextualized from the general program ToCs in the guidance, were developed to guide the codesign of nutrition-sensitive programs or the retrofitting of existing programs to be nutrition sensitive.

Steps 4 and 5—Design and Implement Program Evaluations and Analyze, Disseminate, and Use the Results

Recognizing the need to assess the effectiveness and cost-effectiveness of WFP’s newly designed nutrition-sensitive programs, moving forward, the next steps in the process focus on consolidating the approach and rigorously evaluating these programs and analyzing, disseminating, and using the evidence generated. WFP program implementers and IFPRI are working together throughout the program evaluation period to help ensure that a program is designed in a way that it can be rigorously evaluated; the program and evaluation priorities, expectations, and timelines are aligned; and data are used to improve ongoing and future programming. ⁸ Developing evaluations that are scientifically rigorous, feasible to implement, and able to answer program-relevant questions requires early and continuous communication between program implementers and researchers. To achieve this, WFP and IFPRI are working closely together, conducting in-country workshops and virtual consultations to codevelop proposals for program evaluations to test different types of nutrition-sensitive opportunities across program areas and regions.

Results

In this section, we describe some of the key results that have emerged by leveraging the WFP–IFPRI implementation–research partnership and following the process described above.

Key Components Developed for WFP’s Nutrition-Sensitive Guidance

A report on how to make WFP’s programs nutrition sensitive was developed through the series of workshops held between WFP and IFPRI, program and literature reviews, and iterations on draft reports. ¹² WFP used this report along with other external resources and internal feedback to create their nutrition-sensitive guidance. ¹³ Key outputs from the iterative consultations are presented below.

Requirements for nutrition-sensitive programming

WFP determined that it would be essential to have a list of 5 nutrition-sensitive requirements to clarify what would qualify a program as nutrition sensitive (Table 2). Some of these requirements (eg, including a nutrition objective and specific nutrition actions) are encompassed in the definition of nutrition sensitive set forth in the 2013 Lancet Series on Maternal and Child Nutrition. ³

Table 2.

Requirements for WFP Programs to be Considered Nutrition Sensitive.

Mandatory Requirements Inclusion of nutrition objectives, outcomes, and indicators Program design based on a comprehensive gender and nutrition situation analysis that focuses on key nutritionally vulnerable groups across the life cycle Inclusion of deliberate nutrition actions and/or provision of appropriate delivery platforms for achieving nutrition objectives Inclusion of actions to address gender inequality and lack of nutrition knowledge Alignment with national nutrition plans where they exist

Highly Recommended Requirements Programs should be based on a clearly defined “program impact pathways” including inputs, activities, outputs, and impacts Programs should identify enabling and constraining factors along the pathways and include actions to address them

Abbreviation: WFP, World Food Program.

Design of common ToCs for each of the included programs

Based on the review of the existing ToCs, a harmonized structure was created and applied across the 5 programs and shared in individual consultations with each program area. During these consultations, each group iterated on their program-specific ToC and the common structure that had been applied. This process led to the development of ToCs for each of the 5 programs that adhered to the harmonized framework and a consensus that this framework was workable for each program. The ToC for the GFA program is provided in Figure 2 as an example. These ToCs are intended to serve as a general example that country program staff can adapt for their context-specific program. In addition, these ToCs are intended to be living documents, since programs inevitably change. Using the framework, WFP and IFPRI also identified a common set of indicators that could be used for assessing processes, outputs, outcomes, and impacts to allow for comparisons across contexts and different types of programs and emphasizing the importance of defining a nutrition objective that is relevant, attainable, and measurable within the context and time frame of a program.

Figure 2.

General food assistance (Emergency Context) program: Nutrition-sensitive theory of change. Gray boxes identify underlying determinants of nutrition that this type of program is not currently working toward achieving. These underlying determinants may be addressed through other types of World Food Program (WFP) programs. Yellow boxes identify outcomes or impacts related to reducing households’ and people’s vulnerability to future shocks and strengthening their resilience. Solid arrows indicate links that currently exist within the program and/or for which there is some evidence that the link works as indicated. Dashed arrows indicate links that are dependent on the specific activities chosen or for which there is currently limited evidence as to how well the links work as indicated.

Nutrition-sensitive opportunities

Through the consultations between WFP and IFPRI to develop WFP’s nutrition-sensitive guidance, the following 7 opportunities were identified for making the 5 programs more nutrition sensitive:

Targeting nutritionally vulnerable groups

Ensuring adequate quantity and nutritional quality of transfers

Adding or colocating nutritionally relevant complementary activities (eg, distribution and promotion of adequate use of specialized nutritious foods [SNFs]; strengthening and promotion of preventive health services, social and behavior change communication [SBCC] and WASH services, etc)

Making transfers conditional upon actions that would improve nutrition outcomes

Creating nutritionally relevant community and household assets and ensuring adequate scale, quantity, and quality of these assets

Aligning programs with national nutrition action plans and strategies, and advocating for nutrition, and

Applying gender and protection lenses in program design (eg, targeting, selection of interventions, measurement, etc)

It was hypothesized that these opportunities would work in an additive or synergistic way such that the joint potential of including 2 or more opportunities would be greater than that of any 1 opportunity alone or the sum of the combined impacts, respectively. To illustrate how using these opportunities can make WFP’s programs more nutrition sensitive, we created a series of ToCs for WFP’s GFA program that first highlight the standalone opportunity of ensuring the adequate quantity and quality of transfers and then the combination of the 3 main opportunities (highlighted in italics above) that taken together are hypothesized to lead to more and larger impacts on nutrition and nutrition-related outcomes (Figures 3 and 4). Among the 7 nutrition-sensitive opportunities identified above, the 3 main opportunities that are most broadly relevant to WFP’s programs (in italics above) are further elaborated on here.

Figure 3.

Stand-alone potential of improving the size or quality of the transfer in a General Food Assistance (GFA) program to achieve nutrition impact. Teal boxes identify the entry point for making the program more nutrition sensitive by ensuring the adequacy of the transfer and the pathway through which this change is expected to contribute to nutrition impacts. Green boxes identify an entry point for making the program more nutrition sensitive by adding additional program activities and the pathway through which this change is expected to contribute to nutrition outcomes. Red boxes identify the entry point for making the program more nutrition sensitive by adding additional nutritionally vulnerable groups.

Figure 4.

Joint potential of targeting nutritionally vulnerable groups, optimizing the transfer and including nutrition-relevant activities to achieve nutrition impacts. Teal boxes identify the entry point for making the program more nutrition sensitive by ensuring the adequacy of the transfer and the pathway through which this change is expected to contribute to nutrition impacts. Green boxes identify an entry point for making the program more nutrition sensitive by adding additional program activities and the pathway through which this change is expected to contribute to nutrition outcomes. Red boxes identify the entry point for making the program more nutrition sensitive by adding additional nutritionally vulnerable groups.

Targeting nutritionally vulnerable groups

Improving program targeting is a prime opportunity to make WFP’s programs more nutrition sensitive and can take 1 of 2 forms, either targeting exclusively nutritionally vulnerable groups (defined in the guidance as women and children in the first 1000 days, preschool children, school-aged children, and adolescents) or adding a nutritionally vulnerable group in addition to those already targeted by a given program. Nutritionally vulnerable groups can be targeted to receive the whole program, an additional “top-up” transfer (in the case of a transfer program), an SNF, or other complementary program activities that address barriers to better nutrition.

To implement this nutrition-sensitive opportunity, a robust gender (ie, consideration of gender dynamics, gender equality, and women’s empowerment within a context and how these may influence programming) and nutrition context analysis is a key entry point. It is recommended that country offices maintain updated nutrition data on the vulnerable groups within the geographic areas in which they work, including on nutritional status (eg, anthropometric and micronutrient measures) and the immediate and underlying determinants of malnutrition. Country offices can use these data to better understand the nutritional needs of vulnerable groups within their context and can target them with tailored programs.

Ensuring adequate quantity and nutritional quality of transfers (whether cash based or in kind)

Ensuring the adequacy of all types of transfers is another opportunity for WFP programs to become more nutrition sensitive and to optimize their potential for nutrition impact. ⁶ The entry point for this opportunity is also at the context analysis stage. Diagnostics that provide data on the nutritional status of the target population, as well as information on the immediate and underlying determinants of malnutrition, are essential, especially information that characterizes the diet and nutrient gaps of vulnerable groups and populations. As WFP is taking a life cycle approach, the needs of the specific population group(s) being targeted are critically important when determining the transfer, whether in kind or cash based. In addition, dietary preferences of the target population or intrahousehold dynamics related to food consumption may also inform adjustments. Data should also be collected on systems-level factors that may affect the utilization of the transfers, such as the availability and prices of nutrient-dense foods and other aspects of the local food environment.

One tool that could be used to consolidate this information is the Three-Pronged Approach, which includes various planning tools: an integrated context analysis, seasonal livelihood programming (SLP), and community-based participatory planning. ¹⁷ For example, SLP is a planning tool for governments and coordinating partners to identify appropriate short- and long-term interventions for building resilience (by combining seasonal, livelihood, gender, and other program aspects) and to provide a framework to align ongoing efforts at national, subnational, and local levels. As another example, WFP has recently worked with national governments and other partners to carry out country-level Fill the Nutrient Gap (FNG) analyses incorporating cost of the diet modeling. ¹⁸ This process combines secondary data analysis, stakeholder working group discussions, and modeling to identify key barriers to improved nutrient intake. Further intervention modeling is then conducted to explore options for addressing these barriers and providing associated recommendations. The FNG can serve as a key input for informing both WFP’s nutrition-sensitive and nutrition-specific interventions.

Once the nutrition goals and target groups have been specified, other factors should also be considered to determine the optimal transfer modality and nutrient gaps to be filled, including food safety, costs, and related supply chain logistics. While these are direct concerns when WFP is delivering transfers in the form of in-kind food, they are also increasingly relevant for cash-based transfers, where WFP works with its retail partners to improve supply chain efficiencies and ensure food quality.

The timing of the transfer is also important. For example, providing school meals as breakfast early in the day can increase attendance and enhance attention and concentration during lesson time. ^19,20 Seasonal considerations for transfers may also be important. Transfers can mitigate the heightened risk of food insecurity during the lean season. Finally, the duration of the transfer, in addition to its regularity and predictability, is essential for optimizing program impacts. Recent evidence from the evaluation of an in-kind transfer (micronutrient fortified corn–soy blend) program combined with a nutrition, health, and hygiene SBCC strategy and support to preventive health services and encouragement to attend requisite visits demonstrated the importance of receiving the transfer throughout the full 1000-day window for reducing child stunting. ²¹

Adding or colocating nutritionally relevant complementary activities

To further improve the potential of WFP programs to achieve nutrition impacts, they can be linked to other nutrition-relevant activities that can be either new nutrition-relevant components added to the existing program (integration) or standalone programs implemented by partners (colocation). These can include activities targeted toward improving nutrition, health and hygiene knowledge (eg, SBCC strategy) or toward improving access to and utilization of health services, among others. Integrating or colocating WFP programs with nutrition-relevant activities can improve the potential of these programs to achieve nutrition impact by addressing the multiple immediate and underlying determinants of malnutrition, including household, community, and systems-level constraints.

The entry point for this nutrition-relevant opportunity is once again at the context analysis stage. Data on the nutrition and health issues of the targeted population and on the immediate and underlying causes of malnutrition should be collected where they do not already exist. Additionally, communication with implementation partners would be necessary to map out what programs and interventions are already being delivered in targeted areas and to which target groups. This information will allow for the selection of appropriate program activities to address the identified causes of malnutrition and the design of tailored interventions adapted to local contexts. In settings with limited WFP capacity or expertise, colocation of programs with partners working in nutrition, health and hygiene, or other sectors, may enable WFP and its partners to better capitalize on their respective strengths and available resources and achieve synergies in outcomes and impacts. However, it should be noted that colocation can be difficult to achieve as partners’ time frames vary and new partnerships require time to establish.

Mapping of nutrition-sensitive opportunities to programs

These opportunities have the potential to improve the nutrition sensitivity of the selected WFP programs; however, the feasibility of their inclusion and potential for impact on nutrition outcomes vary across programs, contexts, and needs. In hypothesizing how the nutrition-sensitive opportunities could map to the 5 programs, WFP and IFPRI ranked the potential feasibility of including each nutrition-sensitive opportunity in each program, identified potential nutrition objectives that could be addressed if the opportunity was included, and what underlying or immediate determinants of nutrition would be addressed to achieve the nutrition objectives and associated outcomes. An example of these rankings for WFP’s GFA program is provided in Table 3. Rather than attempting to be precise or exhaustive, this matrix is meant to serve as a starting point for program implementers to guide their decisions about different nutrition-sensitive opportunities that could be included in their selected programs.

Table 3.

Seven Opportunities to Make WFP’s GFA Programs Nutrition Sensitive and Possible Nutrition Objectives.

Opportunity	Nutrition Objective	Basic, Immediate, or Underlying Determinants Addressed	Potential for Impact	Feasibility of Inclusion
1. Target nutritionally vulnerable groups	– Increase diet diversity – Improve nutritional status	– Access to resources – Empowerment and equity	High	High
2. Ensure adequate quantity and nutrition quality of transfers	– Increase diet diversity – Improve nutritional status	– Access to resources – Access to nutrient-rich products – Empowerment and equity	Medium to high	High
3. Add or colocating nutritionally relevant complementary activities	– Increase diet diversity – Reduce morbidity – Improve nutritional status	– Health, hygiene, nutrition knowledge/adoption of practices – Access to nutrient-rich products	Medium to high	Medium to high
4. Make transfers conditional upon actions that would improve nutrition outcomes	N/A	N/A	N/A	N/A
5. Create nutritionally relevant community and household assets and ensure adequate scale, quantity, and quality of these assets	N/A	N/A	N/A	N/A
6. Align programs with national nutrition action plans and strategies, and advocate for nutrition	– Increase diet diversity – Reduce morbidity – Improve nutritional status	– Enabling environment	Medium	Medium
7. Apply gender and protection lenses in program design	– Increase diet diversity	– Access to resources – Empowerment and equity	High	Medium to high

Abbreviations: GFA, General Food Assistance; N/A, not applicable; WFP, World Food Program.

Key evidence gaps and research priorities

Across the 5 program areas, key evidence gaps related to nutrition-sensitive programming relevant for WFP’s programs were identified. Accordingly, several high-level key research questions that can address the identified evidence gaps are suggested here. Their associated evidence gaps are described further below.

What is the optimal size, composition, timing, and frequency of transfers for different nutritionally vulnerable groups in different contexts?

How can WFP’s program platforms be used to reach nutritionally vulnerable groups and provide nutritionally relevant interventions, and what is the impact on nutrition outcomes of doing so?

What are the benefits of including additional nutrition-relevant interventions (through integration or colocation) on nutritional outcomes for nutritionally vulnerable groups?

How does making WFP’s programs more nutrition-sensitive impact the enabling environment and underlying determinants of nutrition, including food environment outcomes?

What are the potential pathways of impact on nutrition, and how can these pathways be better leveraged or further optimized to improve nutrition outcomes?

What is the optimal size, composition, timing, and frequency of transfers for different nutritionally vulnerable groups in different contexts?

Although WFP has been providing in-kind transfers for many years in various contexts through different programs, there is still much to learn about the optimal size, composition, duration, and timing of in-kind transfers. As WFP continues to diversify transfer modalities, more evidence is needed on the relative benefits of different modalities and combinations of transfers in different contexts with the specific goals of achieving nutrition outcomes. For example, evidence related to transfer programs reducing poverty is fairly well established, yet evidence of their impacts on nutrition outcomes is less conclusive. ⁶

To contribute to nutrition impacts, transfers must be designed to address existing nutrient gaps for the target population and be delivered in a timely manner for a sufficient duration. An evaluation of FFA programs from 2002 to 2011 found limited impacts on food security, possibly due to transfers of inadequate size or delivered in a untimely manner. ²² These limited impacts on food security outcomes suggest that impacts on nutrition would be unlikely. More deliberate consideration of nutrition objectives in the design of transfers is a clear opportunity for increasing WFP’s potential to achieve nutrition impacts, especially since transfers are utilized in almost all of its program areas. For example, if additional “top-up” transfers for nutritionally vulnerable household members are included in the program design, the potential for nutrition impacts could be further increased. However, rigorous evidence of the effectiveness and cost-effectiveness of including this type of nutrition-sensitive opportunity is necessary to support additional investments and programming shifts.

How can WFP’s program platforms be used to reach nutritionally vulnerable groups and provide nutritionally relevant interventions and what is the impact on nutrition outcomes of doing so

Another evidence gap relates to how these 5 WFP programs can be used as platforms to reach nutritionally vulnerable groups. The SF programs are particularly well positioned to reach several nutritionally vulnerable groups (eg, school-age children and adolescents) through both school meals and other nutrition-relevant activities. ^3,23 School feeding programs could potentially be linked with other health and nutrition services, including SBCC related to reproductive health and nutrition messaging, for example. In addition, SF programs could design complementary strategies to reach other nutritionally vulnerable household members such as out-of-school adolescents, preschoolers, or women and children in the first 1000 days. Another type of program that could potentially be used to reach older adolescents who are out of school is WFP’s FFA program. If targeted appropriately, FFA programs can provide older adolescents (beyond school age) with livelihood strengthening opportunities and support that they may not otherwise receive. To date, evidence of the effectiveness of reaching nutritionally vulnerable groups, outside the direct program beneficiaries, through these platforms is limited.

What are the benefits of including additional nutrition-relevant interventions through integration or colocation on nutritional outcomes for nutritionally vulnerable groups

To address the nutritional issues of vulnerable groups, interventions from across a variety of sectors are needed. ³ However, the combination of interventions needed and how to best optimize their joint potential for impacts are largely unknown. One example of an effective multisectoral program comes from the series of evaluations of a gender- and nutrition-sensitive homestead food production program in Burkina Faso that included an agricultural production intervention designed to increase production of nutrient-rich fruits, vegetables, and animal source foods coupled with a nutrition, health, and hygiene SBCC strategy. The first program evaluation found that the program was effective in improving maternal and child nutrition outcomes and maternal empowerment. ^4,5 A subsequent evaluation assessed impacts after additional nutrition-sensitive program changes were implemented. Results showed that some of the program benefits identified in the first study were larger when combined with a WASH intervention and greatest when WASH was combined with the provision of a daily lipid-based nutrient supplement to children aged 6 to 24 months. ²⁴ Other examples from nutrition-sensitive in-kind transfer and cash-transfer programs highlight the importance of including SBCC for achieving nutrition impacts. ^25,26

How does making WFP’s programs more nutrition-sensitive impact the enabling environment and underlying determinants of nutrition, including food environment outcomes

Another key knowledge gap, particularly relevant to FFA and SAMS programs, is their impacts on the enabling environment and some of the underlying determinants of nutrition such as access to basic social, health, and hygiene infrastructure (through FFA) and the availability and affordability of nutrient-rich foods at local markets (through SAMS). Evidence is limited on the effects of these types of programs on health and nutrition outcomes and their underlying determinants. ²² For example, results are mixed on the effect of WASH interventions on nutrition and health outcomes. ^{24,27 -29} Measurement of changes in food environments related to program interventions in terms of availability and affordability of nutrient-rich foods in markets in settings where these programs exist is also an understudied area in general.

What are the potential pathways of impact on nutrition, and how can these pathways be better leveraged or further optimized to improve nutrition outcomes?

Finally, across the identified WFP programs and similar programs, there has been little evaluation of the pathways of impact, including assessment of factors such as coverage, adherence, and quality of interventions, which limits the understanding of how these types of programs work to achieve impact and how they can be further improved for nutritional impacts. Evaluating program impact pathways entails measurement of diet or other nutrition outcomes in combination with measurement of more intermediate outcomes, program outputs, processes, and inputs. In recent years, the importance of implementation research has emerged, ³⁰ and some efficacy and effectiveness evaluations are including this type of research in their evaluation portfolios. ^9,31,32 WFP has also begun to use this approach and has an example from a recent pilot study in Ethiopia. ³³ The evaluation of newly designed nutrition-sensitive WFP programs should include rigorous process evaluations to assess implementation, operational, or utilization challenges that could jeopardize their impact. These could be implemented either on their own or in the context of impact evaluations.

Roll out of the Guidance to WFP’s Country Offices

In 2017, WFP began to roll out the Nutrition Sensitive Programming Guidance. Reception of the guidance by country offices varied depending on the level of understanding of what nutrition sensitive means for each country context. The strong endorsement of nutrition-sensitive approaches by the 2017 Nutrition Policy, the Executive Director and the Executive Board in 2017 prompted many country offices to engage with the concepts from the beginning. WFP’s Nutrition Division spent much of the first year working on conceptual clarity and supporting country offices to select appropriate nutrition objectives. To date, many programs have embraced “improving diets” as a guiding objective, while others target-specific issues of “thinness” or “anemia.” Joint visits to country offices by WFP and IFPRI were viewed as helpful to country office staff in utilizing a nutrition-sensitive framework in program design. For example, a WFP staff member from the Sri Lanka country office said, “(the) FFA joint mission outcomes have been very helpful to understand the deeper food insecurity issues in the country and how they link to the nutrition challenges and nutrition-sensitive programming.”

Next Steps

Building on the existing partnership, WFP and IFPRI are planning comprehensive evaluations to assess the impact of the newly designed nutrition-sensitive programs (compared to a control group)—and/or the added benefit of making a program more nutrition sensitive (compared to a standard program model)—on nutrition outcomes for the target population. Associated process evaluations will provide understanding of how impacts are achieved and will be used to identify barriers and facilitators in program implementation and utilization. Finally, cost studies combined with the impact evaluations will be used to estimate cost-effectiveness, when possible. Cost studies will be important for identifying the additional cost of making WFP’s programs more nutrition sensitive. By coupling cost studies with rigorous impact evaluations, these additional costs can be weighed against the nutrition impacts to assess cost-effectiveness of these approaches for achieving nutrition objectives. A recent cost study of multisectoral nutrition programs in Burundi and Guatemala has illustrated this point. In that study, the most expensive versions of the programs were also the most effective for achieving the programs’ nutrition objectives. ³⁴

Evaluating nutrition-sensitive programming is especially difficult due to various factors including long and complex pathways before impacts can be achieved; program complexity due to multiple goals, program components, and outcomes and impacts; the perception from program implementers that evaluations are audits or assessments of their individual performance; and often short-term funding cycles that do not allow full achievement of goals, quality implementation, and impacts before evaluations are completed. ^8,35 Despite these challenges, operational research, and in turn program effectiveness, can be greatly enhanced by having researchers and program implementers collaborate to build a research and learning agenda and then jointly carry out the program implementation and evaluation work focusing on their respective strengths. For program evaluations to be useful to program implementers and the wider scientific community, it is essential that program staff and researchers work together from the beginning. The importance of early and frequent engagements between program implementers and researchers was highlighted in the Alive and Thrive programs in Bangladesh, Ethiopia, and Vietnam. ^9,10 In many cases, including key government stakeholders from the beginning is important to ensure that the results will be useful within a country context.

Stakeholders should work together to identify research aims that help address key questions related to program effectiveness, cost-effectiveness, or to implementation and/or utilization challenges, which are deemed useful by the program implementers and fill key evidence gaps for the wider scientific community. It is essential to identify study designs that are rigorous, feasible, and allow enough time for impact on selected outcomes to be observed. The importance of a shared vision for evaluation priorities and feasible and rigorous study designs were also highlighted in the Alive and Thrive projects in Bangladesh, Ethiopia, and Vietnam. ¹⁰ Especially important for success is that each partner recognizes the priorities, time frames, constraints, and limitations of the other partners. ^8,9 From the program and government sides, research understanding and “buy-in” are necessary to achieve the agreed upon objectives. From the researchers’ side, understanding programmatic realities and limitations is important. Finally, plans for dissemination of the results should be established early on to help facilitate translation and uptake of the research findings. Integration of lessons learned in knowledge management strategies aimed at advising on future nutrition-sensitive programming and course correction of existing programs, as well as support to national policy-making, is an important step in the process. An example of embedding process evaluation in an impact evaluation and using the close collaboration between implementers and evaluators to course correct for increased program effectiveness has recently been described for a homestead food production program in Burkina Faso. The authors describe how barriers to optimal program implementation and uptake were identified through the process evaluation, shared with program implementers, and used to improve ongoing program implementation. ³² These plans should include presentations and publications of key results in appropriate fora and journals as well as translation of the research into action programmatically and potentially at the policy level.

Moving forward, the evidence generated from the evaluation of WFP’s nutrition-sensitive programs will be used and disseminated to further refine the nutrition-sensitive guidance, to improve the effectiveness and cost-effectiveness of WFP’s nutrition-sensitive programs, and to contribute to the global evidence base on nutrition-sensitive programming. Hopefully, funding will be adequate to use a systematic approach to generate evidence of the benefits of the identified nutrition-sensitive opportunities to address an array of nutrition issues relevant for nutritionally vulnerable groups across program areas and different regions. If collected and analyzed in a systematic manner, this evidence could be summarized in a matrix form for easy reference to inform future programmatic approaches.

Conclusion

The approach and implementation–research partnership presented in this article can be used by other institutions dedicated to improving nutrition outcomes at scale. The approach could be replicated by other institutions to systematically assess how to make their programs more nutrition sensitive. The implementation–research partnership can be especially useful for critically thinking through feasibility of using different nutrition-sensitive opportunities across a range of programs and contexts, generating hypotheses regarding the expected outcomes of implementing these opportunities across programs, contexts, and population groups based on existing evidence and designing and testing the implementation, effectiveness, and cost-effectiveness of new nutrition-sensitive program models.

The early stage of WFP’s and IFPRI’s partnership consisted of an iterative consultation phase used to create WFP’s nutrition-sensitive program guidance. This guidance laid out the hypothetical program ToCs for each target program using a common framework and identified opportunities for making each type of program more nutrition sensitive. This presented the broad realm of possible interventions and identified priorities based on the existing literature. Recent evidence from Ecuador and the Dominican Republic suggests that making WFP’s programs nutrition sensitive can lead to positive nutrition impacts such as increasing dietary diversity ³⁶ and reducing anemia. ³⁷ However, large gaps remain in programmatic understanding of what types of nutrition-sensitive components are feasible to implement effectively and cost effectively, with which programs, in which contexts, for which nutritionally vulnerable groups to achieve which nutrition objectives.

IFPRI has extensive experience working with program implementers in designing and carrying out rigorous comprehensive evaluations of nutrition-sensitive programs. WFP, on the other hand, is well positioned to implement nutrition-sensitive programs and achieve impacts at scale as it currently reaches 80 million beneficiaries with direct program support on an annual basis. By making a variety of program areas nutrition sensitive—including social safety nets, resilience-building initiatives, and humanitarian response—WFP provides an array of potential platforms for reaching population groups across the life cycle and throughout the humanitarian-development nexus. Additionally, WFP supports national governments by strengthening their capacities to design and implement effective nutrition programs beyond WFP’s own programs. WFP’s broad platforms with wide reach across contexts offers a unique opportunity to conduct evaluations that are not possible with smaller organizations. Together, WFP and IFPRI aim to leverage this unique combination of the 2 organizations’ expertise to generate results that will enhance the knowledge of stakeholders from diverse sectors and regions while also building local capacity where the research and programs are implemented. Coupling WFP’s nutrition-sensitive programs with well-designed, rigorous comprehensive evaluations has the potential to increase WFP’s program effectiveness while contributing to the global evidence base of what works, how, and to what extent in nutrition-sensitive programs across a range of sectors, target groups, and contexts.