The Costs of Implementing Vaccination With the RTS,S Malaria Vaccine in Five Sub-Saharan African Countries

Abstract

Background. The World Health Organization has recommended pilot implementation of a candidate vaccine against malaria (RTS,S/AS01) in selected sub-Saharan African countries. This exploratory study aimed to estimate the costs of implementing RTS,S in Burkina Faso, Ghana, Kenya, Mozambique, and Tanzania. Methods. Key informants of the expanded program on immunization at all levels in each country were interviewed on the resources required for implementing RTS,S for routine vaccination. Unit prices were derived from the same sources or from international price lists. Incremental costs in 2015 US dollars were aggregated per fully vaccinated child (FVC). It was assumed the four vaccine doses were either all delivered at health facilities or the fourth dose was delivered in an outreach setting. Results. The costs per FVC ranged from US$25 (Burkina Faso) to US$37 (Kenya) assuming a vaccine price of US$5 per dose. Across countries, recurrent costs represented the largest share dominated by vaccines (including wastage) and supply costs. Non-recurrent costs varied substantially across countries, mainly because of differences in needs for hiring personnel, in wages, in cold-room space, and equipment. Recent vaccine introductions in the countries may have had an impact on resource availability for a new vaccine implementation. Delivering the fourth dose in outreach settings raised the costs, mostly fuel, per FVC by less than US$1 regardless of the country. Conclusions. This study provides relevant information for donors and decision makers about the cost of implementing RTS,S. Variations within and across countries are important and the unknown future price per dose and wastage rate for this candidate vaccine adds substantially to the uncertainty about the actual costs of implementation.

Keywords

costs of implementing vaccination malaria RTS,S vaccine sub-Saharan Africa

Introduction

Despite an estimated 60% decrease of malaria deaths worldwide since 2000, the latest data from the World Health Organization (WHO) indicate that about 438,000 individuals still died of malaria in 2015.¹ More than 90% of these deaths occurred in sub-Saharan Africa, primarily in children younger than 5 years of age. Almost all malaria-related deaths are caused by the Plasmodium falciparum parasite transmitted by female Anopheles mosquitos.¹

Infection by P. falciparum may lead to morbidity varying from nonspecific mild febrile illness to severe or life-threatening disease with anemia or circulatory shock, respiratory distress, and coma. The clinical picture may change rapidly (within 24 hours) from a seemingly mild episode to life-threatening organ dysfunction, particularly in young children and in adults with poor or no preexisting immunity.¹ As a result, the management of malaria puts substantial pressure on the health systems of sub-Saharan African countries.²

Intensive efforts to develop a vaccine to prevent clinical malaria in young children have taken place over the past decades with more than 30 candidate vaccines in different stages of evaluation.³ So far, only the RTS,S candidate vaccine has completed a Phase III clinical trial program and received a positive assessment by the European Medicines Agency.⁴ This vaccine is under consideration for routine use in endemic countries with a schedule potentially starting at 5 months of age. The WHO has recommended a pilot implementation of the RTS,S candidate vaccine in selected sub-Saharan African countries with moderate-to-high malaria transmission intensity, to better understand the feasibility of implementing a four-dose vaccination schedule requiring additional immunization contacts, to assess the effectiveness of the vaccine in real life settings, and to further document the safety profile of the vaccine.^1,5

The Phase III study of the RTS,S candidate vaccine with follow-up ranging up to 4 years showed the vaccine’s rapidly waning immunity and moderate efficacy in preventing clinical malaria even in the age group of children achieving the highest degree of immunity, those aged 5 to 17 months and receiving three doses followed by a fourth dose about 18 months after the third.⁶ Nevertheless, modeling studies have suggested potential for a substantial impact of the vaccine on the public health burden of malaria and a likely favorable cost-effectiveness results, although dependent on the vaccine price, when vaccination is implemented on top of other preventive measures such as insecticide-treated bed nets or seasonal malaria chemoprevention in areas with parasite prevalence above 5%.^1,7,8 Furthermore, resistance that is developing in mosquitoes to commonly used insecticides and of the parasite to many of the available antimalaria drugs (including emerging artemisinin resistance in Southeast Asia) threaten to compromise the effectiveness of those preventive measures.¹

Ultimately, decisions about implementation of malaria vaccination must be based on a comprehensive assessment of the public health burden of the disease, the economic value and coverage of vaccination compared with other preventive interventions, health priorities, and the health care system’s capacity to deliver the vaccine in its optimal schedule.

The aim of the present study was to provide detailed information about the incremental costs of adding malaria vaccination to the Expanded Program on Immunization (EPI) based on micro-costing with resource utilization data derived from interviews with key informants of the health care system at the country, regional, district, and health facility levels. The study was carried out in five sub-Saharan African countries—Burkina Faso, Ghana, Kenya, Mozambique, and Tanzania—which may be considered as a fair cross-section of sub-Saharan African countries in terms of geography, malaria endemicity, and economic development.

Methods

The main outcome of the cost analysis for introducing the RTS,S candidate vaccine is the incremental cost of vaccination per fully vaccinated child (FVC) in the short term. We take the short-term perspective as this is likely to be more appropriate for a vaccine that has received a positive opinion from the European Medicine Agency and is already in pilot implementation phase in three countries (specifically Kenya, Ghana, and Malawi; two of these countries are part of this study). Furthermore, if recommended, the new vaccine would be added to an existing and functioning program, the EPI, a consolidated health system platform. The practical consequence of taking a short-term perspective is that only resources that need to be acquired would contribute to the incremental cost as opposed to spare resources. The main classification we applied to costs is between variable and fixed. The variable or recurrent costs depend on the actual number of vaccine doses delivered, whereas the fixed or non-recurrent costs include the costs of introducing the vaccination program and establishing a certain delivery capacity, within which the costs do not depend on the number of doses delivered.

Following the microeconomic theory of production, both in the short-term and within the maximum production capacity, the incremental cost is mainly influenced by the variable rather than by the fixed cost. The contribution of the fixed cost to the incremental cost is positive when vaccine delivery requires an expansion of resources beyond existing spare capacities of the current vaccination programs. No fixed cost is included if spare capacities cover or exceed the needs. The following formula shows the aggregation of the fixed cost components (C_fix) described in more detail later in the text.

C_{fix} = \sum_{i = 1}^{n} p_{i} * (q_{i} - e_{i}) if q_{i} > e_{i}

For each component to be costed i = 1, . . . , n, the unit cost p_i is multiplied by difference between the quantity needed q_i and the existing spare capacity e_i. We collected information directly on $(q_{i} - e_{i})$ when this difference was positive and not on the single terms of the subtraction between $q_{i}$ and $e_{i}$ .

Vaccination Schedule

Following the design and results of the Phase III trial of the RTS,S candidate vaccine, costs were modeled based on administration of the recommended four doses of vaccine. It was assumed that the first three doses could be administered during existing routine EPI visits (e.g., for delivery of vitamin A supplement, growth monitoring, and measles vaccination). The administration of the fourth dose 18 months after the third dose was assumed to require an additional visit beyond the routine EPI window.

Data Collection

A questionnaire was prepared with the contribution of all researchers. Several versions were prepared until reaching an agreement on all the questions for capturing country characteristics while maintaining comparability across country. The questionnaire was piloted in a few health facilities before starting the study and modified according to the inputs from the pilot phase. In order to reflect the heterogeneity of the incidence and burden of malaria, several study sites for interviews with key informants were selected in different areas of each country covering both rural and urban areas and accommodating regions of different endemicity. Exploring areas with different malaria endemicity can provide important indications as endemicity levels can be good indicators of socioeconomic and health system characteristics, with highly endemic areas being the poorest and with the fewest spare capacities. The informants were expert representatives of the Ministry of Health and EPI representatives having responsibilities at the central, regional/provincial, district, and health facility levels. A table with the list of districts and regions included for each country is provided in the appendix (Table A1).

The interviews were, as far as possible, carried out as face-to-face encounters at the informants’ workplace supplemented by telephone contacts when a direct meeting was not possible. The data collection process was guided by the study questionnaire administered by a research team in each country; informants received an inventory of the information required prior to the interview to allow them to prepare the data collection in advance. The study was approved by the national institutional review boards of each country, and interviews were only performed after informants had signed an informed consent form.

In addition to point estimates of the quantity of each resource item required, the informants were also asked to provide unit prices for each item. Informants were also required to provide a plausible minimum and maximum value allowing the researchers to include parameters uncertainty in their estimates. In case unit prices were not obtained from the informants of a country, unit prices from various international organizations were applied (Table 1). The main results are expressed in terms of the total costs per FVC reported in 2015 US dollars (US$) with purchasing power parities used for currency conversions from the local currency to US$.⁹

Table 1

Unit Prices for Certain Items and Assumed Useful Life for Capital Goods (US Dollars, 2015)^a

Cost Item	Burkina Faso	Ghana	Kenya	Mozambique	Tanzania	Source
Exchange rate (local currency v. US$)–Avg, min, max	0.0017, 0.0016, 0.0019	0.2755, 0.2387, 0.3106	0.0112, 0.0109, 0.0115	0.0319, 0.0297, 0.0333	0.0006, 0.0005, 0.0007	Oanda, historic exchange rates
Gasoline (per L)	1.18	0.98	1.15	1.27	1.13	World Bank, World Development Indicators²⁹
Cold room (per m²)	295	648	954	409	614	Market price
Storage volume (per m³ monthly US$)	0.89	9.70	9.54	0.99	0.020	WHO Choice
Average annual salaries
Health facility	3474	3534	2925	2554	5759	Our survey
District/region	—	2602	5315	2724	6924	Our survey
National	2238	6409	12,828	3634	8662	Our survey
	Unit Price		Life Years
Life years, cold room	—		13.6			WHO Choice
Car	17,621		7.3			WHO Choice¹⁰
Motorbike	1418.9		7.1			WHO Choice¹⁰
Cold box	144.96		5.7			WHO Choice¹⁰
Refrigerator	620.61		8.4			WHO Choice¹⁰
Shelf, cupboard	220.75		7.9			WHO Choice¹⁰
Syringes (unit)	Average: 0.0873 (Min: 0.0399; Max: 0.243)					UNICEF Supply Catalogue³⁰
Safety box	0.00402 (per syringe)					UNICEF Supply Catalogue³⁰
Air freight	17.68 (per m³)					UNICEF Supply Catalogue³⁰

It is important to note that these are the average salaries for the professionals identified in the survey and not general average salaries. The low level of the salary at the national level in Burkina Faso reflects the fact that, for example, it was identified that a low-level administrative profile was needed. Similarly, low-level health care workers were needed at health facilities in Kenya.

Perspective and Scope

Costing was performed from the health care system perspective taking into consideration the expected costs associated with implementation of a malaria vaccination program. Cost estimates also included components that could be covered by external funding, for example, nondomestic funding sources such as GAVI (Global Alliance for Vaccines and Immunization) and other aid agencies, in particular the vaccine acquisition costs. The study did not distinguish between economic and financial costs. Most of estimated costs may express a potential financial disburse for the national health system, such as the purchase of the vaccines as well as all the incremental resources needed. Certain costs are likely to be nonfinancial opportunity costs, such as the extra time required from workers already operating for the EPI without hiring extra worker.

Cost Components

The main items in the cost estimations included the purchase prices of the vaccine and vaccine supplies (syringes, cotton, alcohol, and safety boxes), wastage, cold chain storage/distribution, administration of the vaccine, management, training, and social mobilization. These elements are further detailed below.

Purchases

The purchase costs for the vaccine included freight to the country and wastage. Wastage was a composite variable, including the following: 1) vaccine loss along the line of cold chain supply/distribution/storage due, for example, to inadequate control of the temperature range and damage; 2) vaccine wastage during reconstitution; 3) vaccine wastage in dose administration (e.g., if both doses of a reconstituted vial were not administered within the allowed time window). The informants were asked to use the EPI vaccine most similar to the malaria candidate vaccine (e.g., other needle-based vaccines requiring reconstruction, such as yellow fever vaccine when included in routine EPI of endemic countries) for estimating this loss/waste and to provide separate estimates for each of the components mentioned. The total wastage was calculated on the assumption that the informants reported their estimate of the average total waste from the central storehouse to the point of vaccination delivery.

Costs were calculated based on different assumptions for the price per vaccine dose: US$2, US$5, and US$10. The detailed cost estimates are only presented for the mid-price of US$5, for simplicity and as other cost items than vaccine purchase do not depend on the assumed vaccine price.

Distribution (Cold Chain)

For the estimation of cold space requirements and the costs of vaccine delivery, the costing scenario was based on an injectable vaccine with the physical characteristics of the RTS,S candidate vaccine: a lyophilized vaccine to be injected after reconstitution with a liquid adjuvant, each requiring cold storage at 2°C to 8°C. The two-vial package including the vaccine and adjuvant had a volume of 9.7 cm³ and contained two vaccine doses after reconstitution.

Distribution included transportation and cold chain costs. Informants were asked to indicate the frequency of distribution of vaccines from the central distribution point to the peripheral levels and how this frequency was determined. Transportation included recurrent costs (mainly fuel, driver costs, and maintenance) from the central distribution point to subsequent storage points and from these to the peripheral health facilities. The main components of cold chain costs related to cold boxes, vaccine carriers, space required for cold storage of the malaria vaccine, and the amount of spare capacity available.

Storage (Cold Chain)

Storage consisted of capital costs of cold rooms and associated recurrent costs to power them. Costs were split between the central, regional, and local (peripheral) levels. The unit prices were derived from the replacement value of a cold room annualized over its assumed length of useful life (WHO-Choice; see Table 1).¹⁰ The quantity was the estimated share of cold room volume and surface required for storage of the vaccine doses at the different levels depending on the average storage period at each level. To determine the economic costs, the informants were asked to evaluate the current level of capacity utilization and the possible need for additional capacity to accommodate for the malaria vaccination implementation.

Management

Implementation of malaria vaccination will require additional management resources at all levels for monitoring, evaluation, and quality control. Informants were asked to estimate the amount of time dedicated to one single type of vaccine required for the managers involved and the anticipated need for hiring additional managers. The cost related to these additional resources were conservatively fully allocated to RTS,S.

Vaccine Administration

At the health facility level, the administration of the vaccine involved various types of resources such as consumables (vaccine supplies such as syringes, safety boxes, disinfectants, cotton pads, and recording tools), personnel, and capital costs for waste management. Personnel time was assessed as the estimated time required for preparation and administration of one vaccine dose. The costing of personnel time was based on estimates of the extent to which the vaccine would be administered by different categories of personnel (physicians, nurses, other health care assistants). The time needed for administering a vaccine with similar characteristics to the RTS,S malaria vaccine candidate was self-reported by the interviewees. Self-reported durations were averaged per vaccine type across health facilities and then averaged across health facilities within each country.

Two different scenarios were considered: 1) administration of all doses of the vaccine at the health facility and 2) administration of the first three doses at the health facility and the fourth in an existing outreach setting. Compared to administration at the health facility, the outreach setting required additional resources for transportation, cold chain distribution, and storage space where the vaccine administration would take place. Outreach vaccination was quite common in the selected country areas, except for the areas in Tanzania, where this mode of vaccination is rarely practiced, and then only considered as a part of community sensitization. As such, the cost calculation for Tanzania was limited to the scenario with delivery of all four doses at the health facility.

Training

For each training session organized for the introduction of the malaria vaccine, the estimated costs comprised the renting of space, daily allowance for the trainees, remuneration of the trainers, accommodation, food, and traveling costs for all. Informants were asked to base their estimates on experiences from the most recent addition of vaccines to the EPI.

Social Mobilization

Costs associated with social mobilization campaigns included transportation, per diems for people contributing to the campaign and the costs of materials prepared and used in the campaign (such as T-shirts, leaflets, radio/TV communications). As for training, the informants were asked to base their assessment on the experience acquired during the mobilization campaign for recent additions of vaccines to the EPI, noting if these were organized centrally or locally, who participated in these activities, and over what period of time.

Allocation of Non-Recurrent (Fixed) Costs per FVC

For the calculation of the cost per FVC, non-recurrent (fixed) costs exceeding the existing spare capacities (C_fix) at each level (national, regional, district, health facility) were allocated among all the children potentially vaccinated (= the target group) during the next N years. C_fix at each level were sensitization, training, supply chain, and personnel. Vaccine-specific training and social mobilization were assumed to have an effect lasting for 10 years.¹¹ Therefore, sensitization and training costs were divided by the projected number of children in the target group over 10 years. The number of years considered per type of resource other than training and social mobilization depended on its assumed years of useful life (Table 1).¹⁰ Therefore, costs from supply chain items and personnel were divided by 1 year of projected number of target children as such costs were annualized based on the years of useful life (the former) or on the yearly salary (the latter).

At the national level, the number of potentially vaccinated children was estimated based on the United Nations’ demographic projections (medium variant forecast).¹² The number of births per year in a country was diminished by half of the corresponding infant mortality rate under the assumption that half of the infants who died during the first year of life would not receive any malaria vaccine dose.

At the regional and district levels, the number of children potentially vaccinated was based on the most recent national census data,^13–17 with the total population of all ages updated to 2015 based on the country-specific population growth rate. The number of births for every region/district included in the study was estimated assuming that the proportion of births relative to the total population and the infant mortality rate at the region and district level were the same as at the national level.

At the health facility level, the number of potentially vaccinated children was based on the catchment population of the health facility assuming that the size of the birth cohort each year remains the same from 2015 to 2025. This information was collected as part of the questionnaire. Information on the estimated number of children to be vaccinated is reported in the appendix (Table A3).

The outcome of this analysis was the incremental fixed cost per child in the target group (= per child potentially vaccinated).

Calculation of the Recurrent (Variable) Costs per FVC

Recurrent costs were related to personnel time for vaccine administration, supplies, transportation, vaccine purchase, and wastage. In each country, all variable cost components were calculated as the average of reported figures across the health facilities included in the study.

Importantly, personnel cost at the health facility level was included even in the case that no additional staff was expected to be hired in order to account for the additional pressure on the EPI staff and to highlight the potential investment needs for the malaria vaccine introduction. Staff time employed in vaccine administration depends on the number of doses being administered; thus, it was considered as a variable cost.

Calculation of the Incremental Costs per FVC

For each country, the marginal cost per FVC was given by the sum of the fixed cost per child in the target group and the variable cost associated with four doses each of these children should receive. All levels (national, regional/province, district, and health facility) were aggregated in this estimate.

As a summary:

Costfvc = Fixed cost per child + Variable cost for 4 doses

\begin{matrix} Fixed cost per child \\ = \frac{FC at national level}{N estimated target children at national level} \\ + \frac{FC at regional / province level}{N estimated target children at regional / province level} \\ + \frac{FC at district level}{N estimated target children at district level} + 0.8 \\ * \frac{FC at health facility level}{N estimated target children in the HF catchment population} \end{matrix}

Due to the multilevel nature of the data and the likelihood of correlated costs across the various levels, the calculation was based on the assumption that only 80% of the non-recurrent costs incurred at the health facility level contributed to the total costs. Therefore, a “between-levels adjustment” equal to a 20% reduction in fixed costs at the health facility level was applied. In other words, the estimated total fixed cost per FVC is not the sum of the average fixed costs across levels (national, subnational, and health facility levels) because there were potential risks of “double counting” when summing up fixed costs across all levels. Double counting may occur more specifically at the health facility level. For example, several health facilities from the same district may report the need for a new motorbike for vaccination outreach but this new resource would actually be shared across these facilities and managed directly at the district level. The between-levels adjustment therefore represents potential economies of scope.

\begin{matrix} Variable costs per FVC \\ = [Personnel cost for administration of 1 dose \\ + Air freight per dose + Vaccine cost per dose \\ + (Vaccine wastage at national level \\ + Vaccine wastage at regional level \\ + Vaccine wastage at the district level \\ + Vaccine wastage at the HF level) \\ / (number of levels)] * 4 doses \end{matrix}

Personnel cost of administering one dose represents the opportunity cost of time of already employed EPI personnel at the facility level, which is likely to be nonfinancial considering that that the extra work is performed by current workers and no extra worker is hired.

Airfreight costs, expressed per m³ (Table 1), were divided by the volume of each RTS,S dose. Vaccine wastage was valued at the dose price. The average wastage rate across levels was taken in order to avoid double counting.

In the scenario with the fourth dose delivered in an outreach manner, the personnel and transportation costs associated were added; the latter consisting of personnel time to reach the communities and fuel required.

The total incremental costs per FVC were calculated by summing up the fixed costs at each level allocated to each potentially vaccinated child and adding the variable costs including the costs of the vaccine doses and supplies. All the fixed costs were discounted at 3% per annum.

In addition, each cost was estimated with an average, a minimum, and a maximum value. This was possible as interviewees were asked to report a range of plausible value for each resource needed.

Importantly, this study is exploratory of the cost of introduction of RTS,S in malaria endemic countries of sub-Saharan Africa. Our sample of countries, provinces/districts, and health facilities has been determined with the aim of representing as much as possible variations over a number of dimensions (level and characteristics of endemicity, wealth, and level of urbanization).

Results

A total of 59 interviews were carried out over the first semester of 2015. Table A1 reports where interviews were undertaken. Table A2 reports a summary of reported resources needed at the national level and the average of the resources reported as needed at lower levels.

A breakdown of the incremental cost estimates per FVC assuming a vaccine price per dose of US$5 is presented in Table 2. The lowest costs per FVC were found for Burkina Faso with all four doses delivered at health facilities. The highest costs per FVC were found for Kenya, 46% higher than those in Burkina Faso. Only the mean incremental values are presented in the table because for all the cost items except those related to the vaccine (including wastage), the range between minimum and maximum was less than US$1.

Table 2

Distribution by Activity and Items of Incremental Economic Cost per Fully Vaccinated Child (US Dollars, 2015)^a

Cost Items	Burkina Faso		Ghana		Kenya		Mozambique		Tanzania		SD Among US$ Cost Items	CV US$ Cost Items	Ranking of Variation Based on CV (1 = Highest Variation)
	US$	% of Subtotal	US$	% of Subtotal	US$	% of Subtotal	US$	% of Subtotal	US$	% of Subtotal	SD Among US$ Cost Items	CV US$ Cost Items	Ranking of Variation Based on CV (1 = Highest Variation)
Non-recurrent costs
Cold room space/equipment	0.28	14.5	0.38	23.3	0.91	4.9	0.51	14.7	2.52	53.1	0.93	1.21	3
Training	0.15	8.1	0.51	31.3	0.62	3.3	0.15	4.2	0.27	5.7	0.21	0.75	5
Social mobilization	0.14	7.1	0.30	18.6	0.23	0.91	0.10	2.9	0.00	< 0.1	0.12	0.90	4
Human resource	1.73	89.4	0.60	37.1	14.14	75.8	3.13	90.7	3.08	64.9	5.47	1.45	1
Adjustment between levels	0.36	18.9	0.17	10.3	2.75	14.7	0.43	12.5	1.12	23.7
Subtotal	1.93	100	1.62	100	18.65	100	3.46	100	4.74	100
Recurrent costs, vaccine-related
Vaccine	20.00	89.4	20.00	85.4	20.00	88.8	20.00	92.3	20.00	88.2	0	0	—
Vaccine wastage	2.36	10.6	3.43	14.6	2.53	11.2	1.70	7.7	2.68	11.8	0.62	0.29	9
Air freight	0.00	—	0.00	—	0.00	—	0.00	—	0.00	—	0	0	—
Subtotal	22.36	100	23.43	100	22.53	100	21.70	100	22.68	100
Recurrent costs, Scenario 1^b
Labor	0.04	6.1	0.10	12.7	0.08	11.9	0.08	8.1	0.19	17.6	0.06	0.68	6
Supplies	0.61	93.9	0.69	87.3	0.60	88.1	0.97	91.9	0.90	82.4	0.17	0.27	10
Subtotal	0.64	100	0.79	100	0.68	100	1.05	100	1.09	100	0.21	0.29	8
Recurrent costs, Scenario 2^c
Labor	0.16	19.3	0.32	22.1	0.19	23.9	0.29	21.2	—	—	0.08	0.32	7
Supplies	0.61	7.9	0.69	46.9	0.60	75	0.97	70.6	—	—	0.17	0.24	11
Fuel	0.07	72.8	0.47	31.7	0.01	1.1	0.11	8.2	—	—	0.21	1.26	2
Subtotal	0.83	100	1.48	100	0.80	100	1.37	100	—	—
Total costs, Scenario 1	24.93		25.84		36.35		26.21		28.51
Total costs, Scenario 2	25.12		26.53		36.48		26.53		—

CV, coefficient of variation is given by SD/mean across countries for each cost item: it is also called stan; SD, standard deviation.

Adjustment between levels, that is, it was assumed that only 80% of the calculated costs for the health facilities were part of the actual total costs, so the amount indicated in the adjustment row is deducted from the other rows to arrive at total costs. Vaccine price per dose is assumed at US$5.

Scenario 1: Four doses administered at health facilities.

Scenario 2: Three doses administered at health facilities, 1 as outreach. Scenario 2 was not calculated for Tanzania as vaccinations are not administered in outreach settings.

The non-recurrent costs related to the introduction of the new vaccine made up a highly variable proportion of the marginal costs per FVC, ranging from 7% in Ghana to 37% in Kenya. In absolute terms, the non-recurrent costs were eight times higher in Kenya than in Ghana. At least in part, this discrepancy was due to the far higher salaries at the national level in Kenya than in Ghana and the fact that each health facility in Kenya reported a need to hire more personnel whereas the facilities in Ghana basically did not report any need for additional personnel. In total, the non-recurrent human resources costs related to implementation were 23 times higher in Kenya than in Ghana.

The recurrent costs were entirely dominated by the vaccine costs (including wastage), which were very similar across the countries and made up 93% to 97%. The estimated wastage rates were similar in Burkina Faso, Kenya, and Tanzania (10% to 11%), whereas the wastage rates were higher in Ghana (14.1%) and lower in Mozambique (7.3%).

Overall, the cost item presenting the highest variation across country was, among non-recurrent costs, human resources, with the highest standard deviation (SD; 5.47) and coefficient of variation (CV; 1.45). Among non-recurrent costs, the second cost item presenting the highest variation across country was cold room space/equipment (CV = 1.21). A high level of variation (CV = 1.26) was in fuel costs, item included in recurrent costs, outreach scenario.

In all the four countries with existing outreach vaccination settings, the costs of delivering all four doses in the health facility were lower than the costs of delivering the first three doses in the health facility and the last dose in the outreach setting, but the difference was marginal, ranging from US$0.11 to US$0.69 per FVC. Table A5 in the appendix provides additional details on outreach travel time reported.

Table 3 presents the spread of the mean total costs per FVC when the price of a vaccine dose is varied. Appendix Table A4 shows the detailed cost items, when all costs are presented at their minimum and maximum estimate. Additional information on time estimated for vaccine reconstitution and administration is provided in Appendix Table A5.

Table 3

Economic Costs (in 2015 US Dollars) per Fully Vaccinated Child Varying the Vaccine Price Between US$2, US$5, and US$10^a

Vaccine Price	Burkina Faso	Ghana	Kenya	Mozambique	Tanzania
US$2	11.52	11.78	22.83	13.19	14.90
US$5	24.93	25.84	36.35	26.21	28.51
US$10	43.40	47.36	57.23	43.15	47.67

Assuming all four doses administered at health facilities.

The distribution of the non-recurrent costs of introducing the new vaccine across the various administrative levels varied quite considerably between countries with regard to the district and health facility levels, whereas the costs at the national and regional levels were low in all the countries, both in an absolute and relative sense (Table 4). In Ghana and Mozambique, the proportion of non-recurrent costs incurred at the district level was much higher than in any of the other three countries.

Table 4

Non-Recurrent Incremental Economic Costs per Fully Vaccinated Child at Different Levels of New Vaccine Implementation (2015 US Dollars)

Level	Burkina Faso		Ghana		Kenya		Mozambique		Tanzania
	US$	%	US$	%	US$	%	US$	%	US$	%
National	0.12	6.1	0.08	5.2	0.04	0.3	0.04	1.3	0.02	0.4
Regional	0.05	2.5	0.17	10.4			0.01	0.2	0.07	1.5
District	0.30	15.7	0.70	43.3	2.11	16.4	1.68	48.6	0.17	3.4
Facility (adjusted)^a	1.47	75.7	0.67	41.1	10.99	83.6	1.73	50.0	4.50	94.6
Total	1.94	100	1.62	100	13.13	100	3.45	100	4.75	100

Discussion

The estimated costs per FVC were rather similar across the countries with Kenya as an outlier because of very high introduction costs mainly due to the need to hire additional personnel and relatively high salaries compared with the other countries. Comparatively high introduction costs were also responsible for Tanzania’s position with costs per FVC in-between Kenya and the other three countries. Recurrent costs were similar across countries and in all countries delivering the fourth dose in an existing outreach setting was only slightly costlier than delivering all four doses in health facilities. Non-recurrent cost variations across countries mirrored the different situations in terms of resources needed on top of available resources. Some countries were fully using their capacities when, for example, after a recent and efficient vaccine introduction, whereas other countries have large spare resources. For example, all the countries studied had recently introduced a Rotavirus vaccine. However, the variation in the introduction year likely affected our estimates (Ghana and Tanzania introduced it in 2012, Burkina Faso in 2013, Kenya in 2014, and lastly Mozambique in 2015).¹⁸ Therefore, our estimates would reflect, to a certain extent, the vaccine management in different settings as well as the recent decisions on introducing additional vaccines (or not) to the EPI.

We employed an incremental cost analysis in the present study that includes the additional fixed costs based on new resources needed beyond the spare capacity in the health system. We took this approach as we consider that the information obtained is easily interpretable and most relevant for decision makers in resource allocation decisions. Additionally, in the short term, incremental costs are given by variable rather than by fixed production inputs within the existing maximum production capacity.¹⁹ Within the context of our study, this means that, where no additional fixed cost is incurred (e.g., no new fridges are needed), the incremental cost of the malaria vaccine is given by the variable cost of the vaccine only (e.g., by the cost of the vials). On the contrary, when fixed costs are incurred (e.g., a new fridge is needed) these contribute to the marginal cost and are, therefore, factored in our estimates.

We defined our costs as incremental. However, we could have interpreted costs as marginal by applying the idea that a new vaccination program represents “the next logical batch of output.”²⁰ As our study involved the resources needed for an additional vaccination program added to EPI and not for an additional quantity of output produced, our estimated costs are more incremental rather than marginal. However, as the new vaccination program relies on an already existing platform of production of the same type of service, vaccination, in the short term, fixed costs still play a role only if these go beyond the maximum current EPI delivery capacity; variable costs, instead, always play a role.

We did not report the cost per dose of vaccine administered, because the non-recurrent costs per FVC were calculated by dividing the total non-recurrent costs by the total number of potentially vaccinated children expected over the time horizon of the calculation for the associated resource. In practice, complete vaccination coverage may probably not be attained, particularly during the first years following implementation and the cost per dose administered depends on the assumed vaccination coverage. For comparison with other studies, a simple approximation may be applied by deducting the vaccine purchase cost of US$20 for four doses from the estimated costs per FVC and dividing by four. The approximate costs for administering one dose obtained by this calculation ranged between US$1.28 in Burkina Faso and US$4.12 in Kenya (considering the possibility of the dose delivered in an outreach manner and including wastage) or between US$0.64 and US$3.45 with delivery at the health facility and excluding wastage. We compared the recurrent costs per dose delivered with the delivery costs estimated in other studies on new vaccines (HPV vaccine specifically).²¹ Our delivery cost per dose ranged from US$0.16 in Burkina Faso to US$0.27 in Mozambique and from US$0.2 in Kenya to US$0.37 in Ghana, considering health facility or outreach delivery, respectively. Our estimates are similar to the estimates of the cited study although higher for health facility–based administration and lower for the outreach administration (0.09 and 0.57 in 2014 US$).²¹ The comparison is difficult for the different approach used (full cost v. incremental in our case) and for the different sample included: Botwright et al²¹ included 12 countries across continents. In addition, Botwright et al. measured costs as part of implementation (demonstration) projects. Their costs will be more easily compared to those currently measured during implementation of RTS,S in selected countries.²²

In another recent study, Galactionova and colleagues estimated the costs of implementing the malaria vaccine in six sub-Saharan countries based on secondary, routinely collected, and publicly available data.²³ Four of the countries examined were the same as in our study and assuming a vaccine price of US$5 per dose, their estimated total economic costs per FVC with four doses delivered by the same schedule as used in our study ranged from US$36.25 in Burkina Faso to US$44.49 in Kenya. Despite the different approaches to data collection, the estimated total costs per FVC were rather similar to ours but their estimates of the introduction costs were much lower, in particular for Kenya with US$1.58 in financial costs per surviving infant. There is, however, a major difference between that study and ours, because Galactionova et al. estimated the financial costs under an assumption of 100% spare capacity to accommodate the vaccine. However, as it is recognized that some countries may need to invest in scaling-up across a range of service inputs, assumptions for capacity scale-up are presented. While the financial costs are based on the assumption of 100% spare capacity, the economic costs are implicitly evaluated under the assumption of no spare capacity. In this study, only the estimated resources beyond the existing capacities are included (incremental costs). Another difference was that they estimated that vaccines and supplies made up 84% of the recurrent economic costs, whereas our estimates for these were in the range 93% to 97%. This difference may be due to lower recurrent costs per FVC in our study related to our assumption of full coverage as explained in the following paragraph. A further difference is the exclusion in our study of postintroduction monitoring costs, including those for vaccine safety. These costs were estimated in Galactionova et al. as 2% to 13% of the annual economic cost (depending on the country). The exclusion of these costs is certainly a limitation in our study, although we do not expect this to have a strong impact on the final cost due to the implementation piloting of RTS,S currently in place in Ghana, Kenya, and Malawi, where the safety profile of the candidate vaccine is under evaluation.²⁴

A previous study estimated the cost of introduction of a malaria vaccine (not specific to RTS,S characteristics at that time) through the EPI in Tanzania.²⁵ That study assumed three vaccine doses and reported both average (including spare capacities) and marginal costs. Expressed in 2005 US$, their cost estimate for the marginal FVC was just below US$30 at a vaccine price of US$10 per dose. Our estimate for Tanzania at US$10 per dose is just below US$48 (four doses, 2015 US$). As usual, comparisons are difficult to make between cost studies. However, we can speculate that given that the activities considered for cost estimates are very similar, the estimates from the two studies are not too different: Hutton et al. estimated that the cost for four doses at US$10 would be approximately US$42.5 in 2015 US$.¹⁷

When comparing our cost estimates with those of other relevant studies, we could not directly validate our results against some with other approaches such as, for example, the content of comprehensive multi-year plan (cMYP) for immunization. This is due to the fact that cMYPs have different perspectives from our study: cMYPs take a macroeconomic (national) and financial perspective; our study takes the microeconomic perspective and our estimates are economic, not nationally representative and incremental. However, cMYPs may have influenced the data we collected and the replies of our interviewees, particularly at the national level, where informants took much information from vaccination documents including the cMYP.

For the calculation of the total costs per FVC, we assumed complete coverage of the malaria vaccination for the anticipated birth cohorts as an estimate of the number of children the non-recurrent introduction costs should be divided by. In other words, we divided the non-recurrent (fixed) costs among all the projected number of children that theoretically could be vaccinated. This is a logical assumption given that the setup of a new vaccination program should guarantee the service to the entire eligible population. Therefore, we consider that the fixed incremental costs should be divided among the whole projected target population instead of just a part of it as the aim should be to cover the whole eligible population. Given the weight of the vaccine purchase in the non-recurrent costs, assuming lower and probably more realistic coverage rates of 60% to 70% would only marginally change the introduction costs per FVC. However, as all the children receiving between one and three vaccine doses would incur costs for the vaccines, but not be accounted for in the denominator (total number of FVCs), the lower coverage could have a considerable impact on the cost per FVC.

The wastage rate is generally an essential factor that may vary greatly, partly depending on the vaccine presentation. Around the year 2000, wastage rates ranging from 3% to 51% were reported,²⁶ and they may not have improved by much since then, as more recent data indicate rates of 5% for rotavirus and pneumococcal vaccines but 40% for measles vaccine.²⁷ Our estimates ranging from 8% to 17% are relatively low but realistic. Before the introduction of the vaccination in routine practice this rate cannot be assessed.

Compared with using secondary data, primary data collection for facility-based costing data collection has the potential to better capture the costs of vaccination through more detailed analysis of the resources involved. However, cost studies at the facility level are costly to undertake and it is important to consider the balance between feasibility and generalizability, as there are also data indicating that within-country differences in costs at the facility level may be substantial. In Uganda, for example, facility-level costs were found to vary by a factor of 60.²⁷

We aimed to investigate a sample of health facilities in different settings in each country but the results can only be interpreted as indicative for other settings. Our study was exploratory in character, and to obtain more definitive results much larger samples should be investigated as recommended by a costing guide prepared for the Gates Foundation.²⁸ This is a limitation of this study. Also, while we present, for convenience, our results as “per country,” in reality our results reflect an average across “more local” situations: for example, we did not include outreach costs for Tanzania as no outreach is done in the settings we evaluated, which may not necessarily apply to the whole country.

Using interviews and questionnaires for data collection and not actual time-and-motion studies or other methods for measurement of resources needed may lead to bias due to strategic answers by those informants intending to favor implementation of the new vaccine.²⁶ The need for increasing the capacity of the cold chain may have been underestimated for this reason in order to improve the chances of having the new vaccine implemented in the EPI. In the event that this type of bias occurred in our study, both the introduction costs and the recurrent costs have been underestimated because the wastage would become higher in practice than anticipated. Arguably, an opposite bias would also be possible, if respondents inflate the requirement of additional resources in order to obtain a higher vaccine introduction grant or more resources allocated to different levels. Again, it is a question of balancing feasibility and the much higher costs of collecting possibly more accurate data. Whether and to what extent such hypothetical bias occurred in our study is difficult to judge or estimate.

This study showed that even though vaccine purchase including wastage make up about 95% of the recurrent costs per FVC, other incremental economic costs associated with implementation of a new vaccine like the RTS,S candidate vaccine are not negligible. Our costing approach aimed at estimating the incremental economic costs associated with the implementation of the RTS,S candidate vaccine allowing for spare capacity to be used without adding to the costs as would be the case if estimating the full opportunity costs of implementation. Excluding the vaccine purchase, the incremental economic costs estimated this way amounted to between US$4.93 and US$16.35 per FVC dependent on the amount of spare capacity and wage levels of the countries investigated. These costs are substantial (between 25% and 85% of the vaccine purchase cost at an intermediate vaccine price of US$5 per dose) and should be taken into account in implementation decisions.

Supplemental Material

Focus_On_Patient_REVISED.rjf_online_supp – Supplemental material for The Costs of Implementing Vaccination With the RTS,S Malaria Vaccine in Five Sub-Saharan African Countries

Supplemental material, Focus_On_Patient_REVISED.rjf_online_supp for The Costs of Implementing Vaccination With the RTS,S Malaria Vaccine in Five Sub-Saharan African Countries by Elisa Sicuri, Fadima Yaya Bocoum, Justice Nonvignon, Sergi Alonso, Bakar Fakih, George Bonsu, Simon Kariuki, Oscar Leeuwenkamp, Khatia Munguambe, Mwifadhi Mrisho, Vincent Were and Christophe Sauboin in MDM Policy & Practice

Supplemental Material

Manuscript_Supplemental_Appendix.rjf_online_supp – Supplemental material for The Costs of Implementing Vaccination With the RTS,S Malaria Vaccine in Five Sub-Saharan African Countries

Supplemental material, Manuscript_Supplemental_Appendix.rjf_online_supp for The Costs of Implementing Vaccination With the RTS,S Malaria Vaccine in Five Sub-Saharan African Countries by Elisa Sicuri, Fadima Yaya Bocoum, Justice Nonvignon, Sergi Alonso, Bakar Fakih, George Bonsu, Simon Kariuki, Oscar Leeuwenkamp, Khatia Munguambe, Mwifadhi Mrisho, Vincent Were and Christophe Sauboin in MDM Policy & Practice

Supplemental Material

Manuscript_Supplemental_STROBE_checklist.rjf_online_supp – Supplemental material for The Costs of Implementing Vaccination With the RTS,S Malaria Vaccine in Five Sub-Saharan African Countries

Supplemental material, Manuscript_Supplemental_STROBE_checklist.rjf_online_supp for The Costs of Implementing Vaccination With the RTS,S Malaria Vaccine in Five Sub-Saharan African Countries by Elisa Sicuri, Fadima Yaya Bocoum, Justice Nonvignon, Sergi Alonso, Bakar Fakih, George Bonsu, Simon Kariuki, Oscar Leeuwenkamp, Khatia Munguambe, Mwifadhi Mrisho, Vincent Were and Christophe Sauboin in MDM Policy & Practice

Footnotes

Acknowledgements

The authors are very grateful to the health systems personnel who agreed to participate in the study. The authors would like to thank Business & Decision Life Sciences platform for editorial assistance and manuscript coordination, on behalf of GSK. Fabien Debailleul coordinated the manuscript development and provided editorial support. Niels Neymark, independent scientific writer, provided writing support, on behalf of GSK.

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: S. Alonso, V. O. Were, S. Kariuki, M. Mrisho, G. Bonsu, B. Fakih, F. Yaya Bocoum, J. Nonvignon, and E. Sicuri have nothing to disclose. C. Sauboin is an employee of the GSK group of companies and hold shares in this company. O. Leeuwenkamp reports consulting fees paid to his institution by the GSK group of companies until May 2015. His wife was employed by the GSK group of companies at the time of the study and holds shares in this company.

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: GlaxoSmithKline Biologicals S.A. (Rixensart, Belgium) funded this study (GSK Study Identifier: HO-14-14725) and all costs related to the development of the related publications.

Authors’ Note

Previous presentations of data: Sicuri E, Alonso S, Fakih B, Bonsu G, Kariuki S, Leeuwenkamp O, Mrisho M, Nonvignon J, Were V, Bocoum FY, Sauboin C. Cost of implementation of malaria vaccination programmes in five sub-Saharan African countries (Burkina Faso, Kenya, Ghana, Mozambique and Tanzania). African Health Economics and Policy Association, 4th Conference. 2016.

Author Contributions

All authors comply with the ICMJE criteria for authorship. E. Sicuri, F. Y. Bocoum, J. Nonvignon, S. Alonso, B. Fakih, S. Kariuki, O. Leeuwenkamp, K. Munguambe, M. Mrisho, V. Were, and C. Sauboin were involved in the conception and/or the design of the study. E. Sicuri, F. Y. Bocoum, J. Nonvignon, S. Alonso, B. Fakih, S. Kariuki, G. Bonsu, O. Leeuwenkamp, M. Mrisho, and V. Were participated in the collection or generation of the study data. E. Sicuri, F. Y. Bocoum, J. Nonvignon, S. Alonso, B. Fakih, S. Kariuki, G. Bonsu, O. Leeuwenkamp, K. Munguambe, M. Mrisho, V. Were, and C. Sauboin conducted the study. E. Sicuri, F. Y. Bocoum, S. Alonso, B. Fakih, S. Kariuki, O. Leeuwenkamp, M. Mrisho, V. Were, and C. Sauboin contributed to the analysis tools. E. Sicuri, F. Y. Bocoum, J. Nonvignon, S. Alonso, B. Fakih, S. Kariuki, G. Bonsu, O. Leeuwenkamp, K. Munguambe, M. Mrisho, V. Were, and C. Sauboin were involved in the analyses and/or the interpretation of the data. All authors read and approved the final manuscript.

ORCID iD

Christophe Sauboin

Supplemental Material

Supplementary material for this article is available on the Medical Decision Making Policy & Practice website at .

References

World Health Organization. Malaria vaccine: WHO position paper—January 2016. Wkly Epidemiol Rec. 2016;91(4):33–52.

Lee

Olsen

Koehlmoos

, et al. A cross-sectional study of malaria endemicity and health system readiness to deliver services in Kenya, Namibia and Senegal. Health Policy Plan. 2017;32(Suppl. 3):iii75–iii87.

World Health Organization. Tables of malaria vaccine projects globally. “The Rainbow Tables” [cited December 5, 2019]. Available from: www.who.int/immunization/research/development/Rainbow_tables.

European Medicines Agency. First malaria vaccine receives positive scientific opinion from EMA [cited December 5, 2019]. Available from: http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2015/07/news_detail_002376.jsp&mid=WC0b01ac058004d5c1.

Adepoju

RTS,S malaria vaccine pilots in three African countries. Lancet. 2019;393(10182):1685.

RTS

Clinical Trial Partnership. Efficacy and safety of RTS,S/AS01 malaria vaccine with or without a booster dose in infants and children in Africa: final results of a phase 3, individually randomized controlled trial. Lancet. 2015;386(9988):31–45.

Penny

Galactionova

Tarantino

Tanner

Smith

TA.

The public health impact of malaria vaccine RTS,S in malaria endemic Africa: country-specific predictions using 18 month follow-up Phase III data and simulation models. BMC Med. 2015;13:170.

Penny

Verity

Bever

, et al. Public health impact and cost-effectiveness of the RST,S/AS01 malaria vaccine: a systematic comparison of predictions from four mathematical models. Lancet. 2016;387(10016):367–75.

World Bank. PPP conversion factor GDP (LCU per international $) [cited May 15, 2017]. Available from: http://data.worldbank.org/indicator/PA.NUS.PPP.

10.

World Health Organization. Cost effectiveness and strategic planning (WHO-CHOICE) [cited May 15, 2017]. Available from: http://www.who.int/choice/en/.

11.

Manzi

Schellenberg

Hamisi

, et al. Intermittent preventive treatment for malaria and anaemia control in Tanzanian infants: the development and implementation of a public health strategy. Trans R Soc Trop Med Hyg. 2009;103(1):79–86.

12.

United Nations Population Division. World Population Prospects: 2015 Revision [DVD]. New York: United Nations Population Division; 2015.

13.

Burkina Faso. Annuaire Statistique 2014 [cited December 5, 2019]. Available from: http://www.cns.bf/IMG/pdf/annuaire_2014_du_ms.pdf.

14.

Ghana Statistical Service. 2010

Population and Housing Census final results [cited

December 5, 2019]. Available from: http://statsghana.gov.gh/gsscommunity/adm_program/modules/downloads/get_file.php?file_id=27.

15.

Kenya. Available from: https://www.knbs.or.ke/?wpdmpro=population-distribution-by-sex-number-of-households-area-and-density-by-county-and-district.

16.

Mozambique National Institute of Statistics. [cited December 5, 2019] Available from: http://www.ine.gov.mz/operacoes-estatisticas/censos/censo-2007/rgph-2007

17.

International Household Survey Network. Tanzania Population and Housing Census2012. [cited December 5, 2019] Available from: http://catalog.ihsn.org/index.php/catalog/4618.

18.

Mwenda

Burke

Shaba

, et al. Implementation of rotavirus surveillance and vaccine introduction—World Health Organization African Region, 2007-2016. MMWR Morb Mortal Wkly. 2017;66(43):1192–6.

19.

Varian

HR.

Intermediate Microeconomics: A Modern Approach. 8th ed. New York: W. W. Norton; 2010.

20.

Drummond

Sculpher

Torrance

O’Brien

Stoddart

GL.

Methods for the Economic Evaluation of Health Care Programmes. 3rd ed. Oxford: Oxford University Press; 2005.

21.

Botwright

Holroyd

Nanda

, et al. Experiences of operational costs of HPV vaccine delivery strategies in GAVI-supported demonstration projects. PLoS One. 2017;12(10):e0182663.

22.

Framework for Policy Decision on RTS,S/AS01 Working Group; WHO Secretariat. Malaria Vaccine Implementation Programme (MVIP) [cited October 22, 2019]. Available from: https://www.who.int/immunization/sage/meetings/2019/april/1_Session_7_Framework_for_Policy_Decision_on_RTSS-AS01_-_MALARIA_VACCINE_(for_print).pdf?ua=1.

23.

Galactionova

Bertram

Lauer

Tediosi

Costing RTS,S introduction in Burkina Faso, Ghana, Kenya, Senegal, Tanzania and Uganda: a generalizable approach drawing on publicly available data. Vaccine. 2015;33(48):6710–8.

24.

van den Berg

Ogutu

Sewankambo

Biller-Andorno

Tanner

RTS,S malaria vaccine pilot studies: addressing the human realities in large-scale clinical trials. Trials. 2019;20(1):316.

25.

Hutton

Tediosi

The costs of introducing a malaria vaccine through the expanded program on immunization in Tanzania. Am J Trop Med Hyg. 2006;75(2 Suppl.):119–30.

26.

Edmunds

Dejene

Mekonnen

Haile

Alemnu

Nokes

The cost of integrating hepatitis B virus vaccine into national immunization programmes: a case study from Addis Ababa. Health Policy Plan. 2000;15(4):408–16.

27.

Griffiths

Bozzani

Chansa

, et al. Costs of introducing pneumococcal, rotavirus and a second dose of measles vaccine into the Zambian immunization programme: are expansions sustainable. Vaccine. 2016;34(35):4213–20.

28.

Brenzel

Common Approach for the Costing and Financing of Routine Immunization and New Vaccine Introduction Costs (NUVI). Working paper. Seattle: Bill and Melinda Gates Foundation; 2013.

29.

World Bank. World Development Indicators 2017. Washington: World Bank; 2017.

30.

UNICEF Supply Catalogue. Available from: https://supply.unicef.org/all-materials.html.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.42 MB

1.00 MB

0.10 MB