Do Optimization Models for Humanitarian Operations Need a Paradigm Shift?

Abstract

Optimization approaches for planning and routing of humanitarian field operations have been studied intensively. Yet, their adoption in practice remains scant. This opinion paper argues that effectiveness increase realized by such approaches can be marginal due to triviality of planning problems, external constraints, and information losses. Cost increases, on the other hand, can be substantial. These include costs of implementation and use, data gathering, and mismatches with organizational cultures. Though such costs are a key concern for humanitarian organizations, OR/MS studies typically consider effectiveness measures only. We argue a paradigm shift towards cost‐effectiveness maximization and increasing the strength of the presented evidence is needed and discuss corresponding future research needs.

Keywords

planning routing humanitarian logistics decision support cost‐effectiveness analysis

Introduction

Decision support software has substantially transformed private sector logistics. This has not happened yet in humanitarian logistics. Given the substantial funds spent on humanitarian operations, this is rather surprising. Logistics efforts play a key role in delivering disaster relief and development services and transportation is, after salaries, the largest cost category for international humanitarian organizations (IHOs) (Pedraza‐Martinez et al. 2011).

Gustavsson (2003) suggests three internal hurdles that have kept IHOs from realizing these apparent gains: (1) lack of logistics expertise, (2) undervaluation of IT systems, and (3) difficulties in securing the necessary funding. We add a fourth proposition:

Advanced planning and routing systems are often not cost‐effective in comparison to simpler systems and other innovations competing for limited resources.

To state this proposition more precisely, let effectiveness be defined as the extent to which an IHO’s operations decrease harm, suffering, health burden, distress, or inconvenience caused by humanitarian crises (cf. Holguín‐Veras et al. 2012), which we jointly refer to as disutility. Cost‐effectiveness is the extent to which a course of action is effective compared to its costs. In our context, such “course of action” refers to investing in a planning system or some other innovation instead. Cost‐effectiveness is commonly quantified through the incremental cost‐effectiveness ratio—the cost increase when choosing one course of action instead of the other divided by the difference in their effectiveness. Using this ratio, our proposition could be read as (cf. Russell et al. 1996):

The incremental cost‐effectiveness ratio for advanced planning and routing systems exceeds the maximum amount of money IHOs are willing to pay for one additional unit of effectiveness.

This opinion paper both substantiates our proposition and discusses implications for future research needs. It draws upon more than a decade of experience gained through working on fleet management issues with multiple IHOs, grey and academic literature on humanitarian fleet management, in‐depth interviews with eight logistics experts from a representative set of IHOs and three consultants in decision support systems who have been involved with applications for the humanitarian sector, and results from an extensive modeling study. Results and methods are described in detail in an online companion paper (see De Vries and Van Wassenhove 2017).

Cost‐Effectiveness of Planning Systems

To avoid confusion on terminology, we define planning of humanitarian operations as deciding on the (approximate) timing of delivery of goods and services to beneficiaries. Routing is concerned with determining the actual routes to be taken by mobile units. For ease of exposition, we refer to the latter as vehicles. A planning system defines the process and methods used to take planning and routing decisions. We further distinguish planning systems that are (1) centralized, decentralized, or hybrid, and (2) impact‐based or proxy metric‐based. In a centralized system, a person, team, or IT system recommends or makes decisions, whereas local staff or a driver makes these decisions in a decentralized system. Hybrid systems combine the two structures, for example by making centralized decisions or suggestions on the time‐window of a delivery and allowing local staff to determine the routing. When planning and routing are optimized in terms of effectiveness—that is, they minimize disutility to beneficiaries—we call them impact‐based planning and routing. Systems that optimize with respect to other metrics like travel times or priority levels are referred to as proxy metric‐based. Advanced planning and routing refers to mathematically optimized planning and routing based on detailed information on travel times and requested aid deliveries.

Planning system characteristics affect costs and the amount of delay in aid delivery. Delay, in turn, typically induces a certain amount of disutility (Gralla et al. 2014, Holguín‐Veras et al. 2016). Disutility tends to increase exponentially with delay, particularly in a disaster relief setting where it is frequently called deprivation cost (Holguín‐Veras et al. 2013, Holguín‐Veras et al. 2016). For example, one day without drinking water may be bearable but five days can be lethal. Effectiveness can be measured as the expected average disutility per aid request. Cost‐effectiveness of a planning system is therefore a function of: (1) costs of the system, (2) delay in fulfilling aid requests, as determined by the system, and (3) the relationship between delay and disutility.

A planning system’s cost‐effectiveness is not only determined by the planning system itself. The same planning system may be highly cost‐effective in one context, organization, or disaster and highly cost‐ineffective in another. Our companion paper provides a holistic framework of cost‐effectiveness determinants and interactions among them. Next, we highlight a few.

Factors affecting the importance of routing optimization. Advanced routing systems have traditionally flourished in applications where (1) travel times are long, (2) decision space is large, and (3) high‐quality solutions are hard to find. Our study reveals that these conditions rarely hold in humanitarian contexts. For example, the decision space is often rather confined due to the small number of destinations per trip (often just one), various types of vehicle assignment constraints, security issues, time‐windows of specific appointments, and sparsity of road networks.

Factors affecting the importance of prioritization. Prioritization becomes important when (1) resources are too scarce to immediately serve incoming aid requests, (2) some requests are more urgent than others, and (3) differences in urgency can be adequately identified. The extent to which these conditions hold is highly context‐specific and strongly impacts the cost‐effectiveness of various planning systems. For example, decentralized systems can result in a lack of coordination (cf. Pedraza‐Martinez and Van Wassenhove 2012, Stapleton et al. 2009, UNHCR 2006) and hence suboptimal prioritization.

Operational uncertainty. Real‐time information systems are virtually absent in the humanitarian context and much local information is not captured, stored, and shared. Operational uncertainty therefore induces information gaps at the central level—e.g., on dynamic issues like security, weather and road conditions, and demand mobility—and hence jeopardizes the effectiveness of centralized planning systems.

Organizational culture. The fit between planning system and organizational culture and values highly determines cost‐effectiveness. For example, systems involving a dispatcher or algorithms that tell field staff what to do and where to go may cause frustrations and discrepancies with perceived needs. Similarly, systems involving black box optimization may lack the transparency to generate trust. More generally, the system determines autonomy and bureaucracy. Studies among social workers show that these are major determinants of job satisfaction, burnout rates, and staff turnover (Arches 1991, Kim and Stoner 2008), each of which may clearly affect both costs and effectiveness.

Planning system costs. A planning system may require an IT solution, possibly including costly vehicle routing software, expensive support and maintenance services, and a planner or dispatcher. Moreover, implementing (rolling out) such system requires substantial training and consumes scarce human resources and budgets. Planning may also require time‐consuming activities like data gathering, information exchange (planner vs. staff and drivers), and urgency assessments.

Implications

These observations have at least four major implications for research and practice. While these may not come as a surprise, they are particularly relevant in the humanitarian context and raise questions about the practical relevance of advanced planning and routing tools presented in a large number of academic publications. First, cost‐effectiveness of planning systems is highly context‐specific in general, making generalizability of results a key concern for research in this area. This also suggests humanitarian organizations may need to reconsider their common practice of globally implementing standard IT systems in highly diverging operational contexts.

Second, effectiveness increase realized by advanced systems can be marginal—due to triviality of planning problems, information gaps, and external constraints—whereas cost increases induced by the planning system may be substantial. Our modeling results convey an even more extreme message. We estimated effectiveness for an advanced centralized planning system incorporating both urgency levels of aid requests and travel times and a basic decentralized planning system considering travel times only. We did so based on data for a “typical program” from one of the organizations involved in this study. Figure 1 depicts the results. For each context considered, information gaps at a central level render advanced systems substantially less effective. This shows that more advanced systems can be both less effective and (presumably) more costly.

Figure 1

Disutility Increase (%) When Implementing Advanced, Data‐Intensive Central Planning Instead of Basic Decentralized Planning. The Highlighted Point Represents A “Typical Program” From One of the Organizations Involved in Our Study. Operational Uncertainty Refers to Variance in Travel Times Due to Weather, Security Issues, and Road Conditions. Assessment Quality Refers to the Probability the Urgency of an Aid Request is Correctly Identified

Third, advanced planning systems are often not cost‐effective compared to other innovations. For our “typical program” case the effectiveness increase due to optimized routing and prioritization is small compared to that of tackling managerial issues such as reducing delays in submitting aid requests and optimizing car pooling (i.e., removing organizational and operational constraints).

Finally, optimizing cost‐effectiveness of the planning system is rather different from optimizing planning decisions (i.e., maximizing effectiveness). In particular, pursuing the first makes optimal planning criteria and optimal planning hierarchy highly context‐specific, as we posit with the help of Figures 2 and 4. The remainder of this section discusses these propositions, which were tested to the extent possible through extensive numerical experiments (see the companion paper for more details). They were specifically based on (and apply to) the typical humanitarian context where the number of destinations per trip is small, road networks are rather sparse, and disutility increases convexly with delay in demand fulfillment.

Figure 2

Suggested Context‐Specific Optimal Planning Criteria

Optimal planning criteria. Using richer objective functions may lead to more effective decisions but can also be more expensive due to software requirements, training, and data gathering. Whether this is beneficial strongly depends on the travel burden and observable variation in urgency levels among aid requests. The larger the travel delay compared to the time on site, the larger the importance of incorporating routing efficiency in the objective function. Similarly, the larger the variations in urgency levels and the better these urgency levels are assessed, the larger the importance of incorporating prioritization.

Consider the case when travel times are small and the need for prioritization is large, that is, the lower right quadrant of Figure 2. This context may apply to urgent situations like sudden‐onset disasters and mass casualty events. Here, routing will have little effect on field delays, so a heuristic that focuses on prioritization solely (i.e., priority level‐based planning) will yield close to maximum effectiveness. Incremental effectiveness of more advanced systems will be small or even negative while cost increases may be substantial. Responders indeed often utilize heuristic prioritization rules in such contexts (Frykberg 2005, Gralla et al. 2016, Griekspoor and Collins 2001).

Similarly, in the upper left quadrant, where variations in urgency levels are very small or cannot be observed, there is little a priori need for prioritization. Effectiveness of minimizing travel times (i.e., travel time‐based planning) will be close to the maximum, so that incremental effectiveness of more advanced systems will be small. One example is the planning of mobile sleeping sickness screening teams in the DRC. Annual meetings presently yield a list of sites to be screened, and the specific sequencing of the visits is largely based on travel times (De Vries et al. 2019). Another example is the program considered in our companion paper, for which travel times are substantial and differences in urgency moderate. As shown in Figure 1, travel time‐based planning works comparatively well in this context.

In the lower left quadrant, where travel times and observable variations in urgency levels are small, routing and prioritization will have little impact on effectiveness. A simple heuristic planning policy, e.g. assigning vehicles to requests in the order of requisition, will yield close to maximum effectiveness. This reflects current practice for several of the development programs covered by our interviews. Here, incremental effectiveness of more advanced methods will be too small to justify the investment.

In the upper right quadrant, adequate prioritization and vehicle routing can have a substantial impact. Incremental effectiveness of impact‐based planning over systems using simpler objective functions therefore could be large enough to justify the corresponding cost increase. We do not claim this is always the case. Proxy metric‐based planning may also be comparatively effective under such circumstances (Gralla and Goentzel 2018). Though several academics have proposed models and methods that apply impact‐based planning (see, e.g., Pérez‐Rodríguez and Holguín‐Veras 2015), we know of no real‐life applications.

Optimal planning hierarchy. Using our model, we estimated effectiveness of centralized, decentralized, and hybrid planning systems for 100 parameter settings. Specifically, we varied three contextual factors: (1) uncertainty about road networks and travel times, which determines the size of the information gap centralized systems encounter, (2) urgency levels, which determine the need for prioritization and hence for centralized decision making, and (3) the travel burden, which determines the need for routing optimization. Figure 3 depicts the results.

Figure 3

Contexts in Which Centralized (white), Hybrid (light gray), and Decentralized (dark gray) Systems Maximize Effectiveness for the “Typical Program” Considered

Merging these results with the premise that centralized systems are more expensive, we suggest the context‐specific optimal planning hierarchy depicted in Figure 4. Centralized systems are only cost‐effective compared to others when information gaps are small and prioritization is important, that is, in the lower right quadrant of Figure 4. This may well represent the context of emergency medical service provisioning in high income countries, where centralized planning systems are indeed common (Andersson and Värbrand 2007).

Figure 4

Suggested Context‐Specific Optimal Planning Hierarchy

When uncertainty is high and prioritization important, hybrid systems are to be preferred. This may well reflect disaster relief settings (cf. Holguín‐Veras et al. 2012). Gralla and Goentzel (2018) show that decision makers in such setting indeed make planning decisions in a hybrid manner by incorporating priority levels of destinations and relief items while making decisions locally. By exploiting local knowledge, hybrid systems can be both more effective (as in our numerical study) and less expensive than a centralized one. Since hybrid systems facilitate incorporating priorities, effectiveness increase can be large enough to make them cost‐effective compared to decentralized systems.

Decentralized systems are cost‐effective compared to more centralized systems when the context fits the upper left quadrant, that is, when prioritization is relatively unimportant and uncertainty is large. This typically occurs in development assistance settings (Holguín‐Veras et al. 2012). For example, NGO Marie Stopes International uses decentralized planning for its mobile family planning teams (Marie Stopes International 2018). Family planning is not subject to high urgency, and decentralized planning safeguards staff’s professional freedom and exploits their local knowledge.

Finally, differences in effectiveness will be minor when both uncertainty and the need for prioritization are small. Here, each system has access to accurate travel time information and yields near‐optimal decisions by minimizing travel delays only. This suggests that the cheapest system, likely being the decentralized one, will be most cost‐effective.

The Way Forward

As evidence‐based decision making is gaining traction in the humanitarian sector (cf. ALNAP 2017, EvidenceAid 2017, The Humanitarian Evidence Program 2017), the most important preconditions for impactful OR/MS research in this field seem to be that (1) the research questions or propositions our community seeks to investigate are those for which the humanitarian sector seeks stronger evidence and (2) the OR/MS study actually contributes to a stronger evidence base. The case of vehicle planning shows that there are steps to be taken on both.

Relevance: shifting from an effectiveness paradigm to a cost‐effectiveness paradigm. Cost‐effectiveness is a key concern for humanitarian organizations (Beck 2006, Knox‐Clarke and Darcy 2014), whereas OR/MS studies typically consider effectiveness measures such as travel times, delay, and coverage levels (see De la Torre et al. 2012, Najafi et al. 2013, Ortuño et al. 2013, Özdamar and Ertem 2015, for overview articles). As a consequence, there is a tendency toward developing optimization approaches involving a high level of decision centralization and/or requiring large quantities of data. As our 2 × 2 diagrams suggest, simple heuristics and approaches involving limited centralization are often more cost‐effective. Very little work is happening in these areas. To direct future research toward the most relevant quadrants of Figures 2 and 4, we propose five basic questions to be asked before developing a solution approach.

What constraints do humanitarian contexts, humanitarian principles, and organizational culture put on planning systems?

Context determines data availability, data quality, and which actors have access to what information. Context also determines the time available to make decisions. Culture and principles like transparency determine acceptability of decision structures and decision support methods. Each of these lead to more or less fixed constraints on planning systems.

What types of costs come with different planning systems?

As argued, planning systems can types of costs (actual and opportunity costs). To get a sense of what systems might be cost‐effective, answering this question is key.

Can planning be effectively done through simple heuristics or decision rules?

Examples in literature show that exploring this question can pay off. Gralla and Goentzel (2018) and Knott (1988) note the limited implementability of optimization methods in humanitarian contexts. Building upon current planning practices, they propose simple but effective decision rules for humanitarian transportation planning. De Vries et al. (2019) analyze the planning problem for mobile disease surveillance teams in the DRC. Though this problem is extremely complex, simple planning rules were shown to be near‐optimal. Similarly, Bartholdi et al. (1983) developed an effective heuristic for charity Meals on Wheels which is “so simple that a computer is not even required.”

Can planning be effectively done at a decentralized level?

That this is possible has been shown by Gralla and Goentzel (2018), who present effective decision rules that can be utilized at a decentralized level. Similarly, De Vries et al. (2019) propose near‐optimal planning rules where prioritization is done at a central level and routing at a local level.

Can planning be effectively done through off‐the‐shelf methods?

Off‐the‐shelf solutions tend to be cheaper than dedicated ones (Pollock et al. 2003), might work relatively well in certain humanitarian contexts, and hence may be cost‐effective. Analyzing usefulness of standard planning systems in well‐defined humanitarian contexts can therefore be very beneficial.

Rigor: increasing the strength of the presented evidence. Our interviews and literature review did not reveal any evidence of implementation of advanced planning and routing methods in the humanitarian sector. The only available evidence on their (cost‐)effectiveness therefore comes from modeling studies. Such studies essentially estimate or proxy this on the basis of mechanism‐based reasoning (Van de Klundert 2016) for which the resulting evidence is perceived to be comparatively weak (Howick et al. 2010,2011). To really build a stronger evidence base, implementation, evaluation, and refinement will be key. We therefore strongly advocate future research following a design science approach, involving multiple reflective or design cycles (cf. Hevner 2007, Van Aken 2004).

Improving external validity of results forms a second avenue for building stronger evidence. As argued, diversity of humanitarian contexts makes cost‐effectiveness of planning and routing methods highly context‐specific. What works well in one context might be far from optimal in another. Urgency levels and (observable) variation therein, travel times, number of destinations per trip, and road network density are among the determinants. Adequate assessment of the role of context is therefore key to providing humanitarians with nuanced managerial insights. Analysis of one case study, as often seen in OR/MS studies, is generally not enough. A review of the humanitarian logistics literature by Leiras et al. (2014) found that only 23 of 160 analytical papers included a case study at all, indicating that substantial progress is still to be made.

Final remarks. “Rigorous” is defined as “accurate and exact” (Cambridge Dictionary 2017). Part of our field tends to adopt a rather literal interpretation of this definition, focusing on acquiring a deep mathematical understanding of stylized problems (Fisher 2007, Tang 2015). This also appears to happen in research on optimization models for humanitarian operations. History has shown that this trend tends to increase the gap between theory and practice (Corbett and Van Wassenhove 1993). As there are many big problems out there for which our field has solutions to offer, this would be a missed opportunity. We therefore propose three action items:

Rigor: Use internal validity of models and external validity of results as primary measures of rigor.

Relevance: Assess propositions for which practitioners seek stronger evidence, especially those related to cost‐effectiveness.

Bridging Rigor & Relevance: Engage with practitioners, implement, evaluate, and refine.

We believe the third proposed action serves as an important tool for reaching the first and second. It enables building sensitivity to the propositions practice seeks to assess, the specifics of problems, validity of assumptions, and the variety of contexts for which generalizability needs to be analyzed. A healthy dose of practice‐based research has always been a secret behind relevance of our discipline, and enables us to provide cost‐effective and evidence‐based solutions to a heavily resource‐constrained sector involving human suffering.

References

ALNAP . 2017. The quality and use of evidence in humanitarian action. Available at http://www.alnap.org/what-we-do/evidence (accessed date Setpember 18, 2017).

Andersson

Värbrand

. 2007. Decision support tools for ambulance dispatch and relocation. J. Oper. Res. Soc. 58(2): 195–201.

Arches

1991. Social structure, burnout, and job satisfaction. Soc. Work. 36(3): 202–206.

Bartholdi

J. J.

Platzman

L. K.

Collins

R. L.

Warden

W. H.

. 1983. A minimal technology routing system for Meals on Wheels. Interfaces. 13(3): 1–8.

Beck

2006. Evaluating humanitarian action using the OECD‐DAC criteria. London, Overseas Development Institute.

Cambridge Dictionary . 2017. Cambridge dictionaries online. Cambridge University Press, Cambridge, UK. Available at https://dictionary.cambridge.org (accessed date September 18, 2017).

Corbett

C. J.

Van Wassenhove

L. N.

. 1993. The natural drift: What happened to operations research? Oper. Res. 41(4): 625–640.

De la Torre

L. E.

Dolinskaya

I. S.

Smilowitz

K. R.

. 2012. Disaster relief routing: Integrating research and practice. Socio‐Econ. Plan. Sci. 46(1): 88–97.

De Vries

Van Wassenhove

L. N.

. 2017. Evidence‐based vehicle planning for humanitarian field operations. INSEAD Working Paper.

10.

De Vries

Wagelmans

A.,

Van de Klundert

. 2019. Optimizing population screening for infectious diseases. (Unpublished manuscript)

11.

EvidenceAid . 2017. Who we are. Available at http://www.evidenceaid.org/who-we-are (accessed date September 18, 2017).

12.

Fisher

2007. Strengthening the empirical base of operations management. Manuf. Serv. Oper. Manag. 9(4): 368–382.

13.

Frykberg

2005. Triage: principles and practice. Scand. J. Surg. 94(4): 272–278.

14.

Gralla

Goentzel

. 2018. Humanitarian transportation planning: Evaluation of practice‐based heuristics and recommendations for improvement. Eur. J. Oper. Res. 269(2): 436–450.

15.

Gralla

Goentzel

Fine

. 2014. Assessing trade‐offs among multiple objectives for humanitarian aid delivery using expert preferences. Prod. Oper. Manag. 23(6): 978–989.

16.

Gralla

Goentzel

Fine

. 2016. Problem formulation and solution mechanisms: A behavioral study of humanitarian transportation planning. Prod. Oper. Manag. 25(1): 22–35.

17.

Griekspoor

Collins

. 2001. Raising standards in emergency relief: How useful are Sphere minimum standards for humanitarian assistance? BMJ. 323(7315): 740.

18.

Gustavsson

2003. Humanitarian logistics: Context and challenges. Forced Migrat. Rev. 18(6): 6–8.

19.

Hevner

A. R.

2007. A three cycle view of design science research. Scand. J. Inf. Syst. 19(2): 4.

20.

Holguín‐Veras

Jaller

Van Wassenhove

L. N.

Pérez

Wachtendorf

. 2012. On the unique features of post‐disaster humanitarian logistics. J. Oper. Manag. 30(7): 494–506.

21.

Holguín‐Veras

Pérez

Jaller

Van Wassenhove

L. N.

Aros‐Vera

. 2013. On the appropriate objective function for post‐disaster humanitarian logistics models. J. Oper. Manag. 31(5): 262–280.

22.

Holguín‐Veras

Amaya‐Leal

Cantillo

Van Wassenhove

L. N.

Aros‐Vera

Jaller

. 2016. Econometric estimation of deprivation cost functions: A contingent valuation experiment. J. Oper. Manag. 45, 44–56.

23.

Howick

Glasziou

Aronson

J. K.

. 2010. Evidence‐based mechanistic reasoning. J. R. Soc. Med. 103(11): 433–411.

24.

Howick

Chalmers

Glasziou

Greenhalgh

Heneghan

Liberati

Moschetti

Phillips

Thornton

2011. The 2011 Oxford CEBM levels of evidence (introductory document). Technical report. Oxford Centre for Evidence‐Based Medicine, Oxford.

25.

Kim

Stoner

. 2008. Burnout and turnover intention among social workers: Effects of role stress, job autonomy and social support. Admin. Social Work 32(3): 5–25.

26.

Knott

1988. Vehicle scheduling for emergency relief management: A knowledge‐based approach. Disasters. 12(4): 285–293.

27.

Knox‐Clarke

Darcy

. 2014. Insufficient evidence? The quality and use of evidence in humanitarian action. Technical report.

28.

Leiras

de Brito

Jr. Queiroz Peres

Rejane Bertazzo

Tsugunobu Yoshida Yoshizaki

. 2014. Literature review of humanitarian logistics research: Trends and challenges. J. Human. Log. Supply Chain Manag. 4(1): 95–130.

29.

Marie Stopes International . 2018. Driven by data: Improving lives and extending choice through insight and action. Available at https://mariestopes.org/resources (accessed date June 21, 2019).

30.

Najafi

Eshghi

Dullaert

. 2013. A multi‐objective robust optimization model for logistics planning in the earthquake response phase. Transport. Res. E. Log. Transport. Rev. 49(1): 217–249.

31.

Ortuño

Cristóbal

Ferrer

Martín‐Campo

Muñoz

Tirado

Vitoriano

. 2013. Decision aid models and systems for humanitarian logistics. A survey. Decision aid models for disaster management and emergencies. Springer, Berlin, 17–44.

32.

Özdamar

Ertem

M. A.

. 2015. Models, solutions and enabling technologies in humanitarian logistics. Eur. J. Oper. Res. 244(1): 55–65.

33.

Pedraza‐Martinez

A. J.

Van Wassenhove

L. N.

. 2012. Transportation and vehicle fleet management in humanitarian logistics: Challenges for future research. EURO J. Transport. Log. 1(1‐2): 185–196.

34.

Pedraza‐Martinez

A. J.

Stapleton

Van Wassenhove

L. N.

. 2011. Field vehicle fleet management in humanitarian operations: A case‐based approach. J. Oper. Manag. 29(5): 404–421.

35.

Pérez‐Rodríguez

Holguín‐Veras

. 2015. Inventory‐allocation distribution models for postdisaster humanitarian logistics with explicit consideration of deprivation costs. Transport. Sci. 50(4): 1261–1285.

36.

Pollock

Williams

Procter

. 2003. Fitting standard software packages to nonstandard organizations: The biography of an enterprise‐wide system. Technol. Anal. Strateg. Manag. 15(3): 317–332.

37.

Russell

L. B.

Gold

M. R.

Siegel

J. E.

Daniels

Weinstein

M. C.

. 1996. The role of cost‐effectiveness analysis in health and medicine. J. Am. Med. Assoc. 276(14): 1172–1177.

38.

Stapleton

Pedraza Martinez

Van Wassenhove

L. N.

2009. Last mile vehicle supply chain in the international federation of red cross and red crescent societies. INSEAD Working paper.

39.

Tang

C. S.

2015. Making OM research more relevant: “Why?” and “How?”. Manuf. Serv. Oper. Manag. 18(2): 178–183.

40.

The Humanitarian Evidence Program . 2017. The Humanitarian Evidence Program. Available at http://fic.tufts.edu/researchitem/the-humanitarian-evidence-program (accessed date September 18, 2017).

41.

UNHCR . 2006. Evaluation of the utilization and management of UNHCR's light vehicle fleet. Technical Report.

42.

Van Aken

J. E.

2004. Management research based on the paradigm of the design sciences: The quest for field‐tested and grounded technological rules. J. Manag. Stud. 41(2): 219–246.

43.

Van de Klundert

2016. Healthcare analytics: Big data, little evidence. Optimization challenges in complex, networked and risky systems. INFORMS, Catonsville, MD, 307–328.