Data, analytical techniques and collaboration between researchers and practitioners in humanitarian health supply chains: a challenging but necessary way forward Open Access

The third United Nations Sustainable Development Goal (SDG 3) is to “ensure healthy lives and promote well-being for all at all ages” (UN, 2021). Despite improvements in health before 2020, the rate of progress has been insufficient to meet most SDG targets (UN, 2020), notably in sub-Saharan Africa and Southern Asia. Sub-Saharan Africa has the highest mortality rate among under-fives. Together with Southern Asia, it accounts for 86% of maternal deaths worldwide. The COVID-19 pandemic has made reaching the SDGs more difficult by throwing progress off track in many health domains. Border closures have resulted in potential vaccine shortages in dozens of low- and middle-income countries (LMICs). Disruption to global supply chains can lead to shortages of contraceptives, and the shortage of health professionals has been exacerbated, setting off alarms about preparedness for health emergencies (UN, 2020).

The focus of this paper is on humanitarian health supply chains. In this paper, we define humanitarian health supply chains encompassing health care and medical programs in both development (e.g. childhood immunization, family planning, HIV prevention, diagnosis and treatment in LMICs) and relief operations (epidemic outbreaks such as Ebola, COVID-19, plague and measles). Important differentiating factors between humanitarian health supply chains and regular health supply chains often include the presence of multiple stakeholders or the fragmentation of humanitarian supply chains.

New techniques for data capturing, processing and analytics, such as big data, blockchain technology and artificial intelligence (AI), are increasingly put forward as potential “game changers” in the humanitarian field (Dubey et al., 2020; Swaminathan, 2018; Rodríguez-Espíndola et al., 2020; Tang, 2021). Although these have proven useful in commercial settings, their application to humanitarian health settings warrants caution, as the supply chain here is fundamentally different from its commercial counterparts (Pedraza‐Martinez and Van Wassenhove, 2016). Holguín-Veras et al. (2012) identify several differentiating features in terms of the:

• objectives pursued;

• origins of commodity flows;

• decision-making structure;

• knowledge of demand;

• state of the social networks;

• state of the supporting systems; and

• periodicity and volume of logistic activity.

Tang (2021) highlights three characteristics of economic development programs that differ from traditional operations research/operations management (OR/OM) problems focused on profit maximization: interdependent goals, complementary efforts and contextual challenges. Another distinguishing feature between commercial and humanitarian supply chains is that the former tends to be proactive, whereas the latter, especially in relief settings, is reactive (Dubey et al., 2021).

The ultimate goal of (new techniques of) data capture and analysis in humanitarian settings is to help practitioners in humanitarian health supply chains solve real-life problems. To achieve this goal, collaboration between academics and researchers is indispensable. This collaboration is needed both for obtaining the necessary input data and for eventually implementing suggested interventions in practice (e.g. by creating rigorous implementation research frameworks to gauge the impact of these interventions; Maric et al., 2021).

The interactions between data, analytical techniques and collaboration between academics and practitioners in impactful research in humanitarian health environments are legion, forming a complex system of interconnected loops. Only by understanding these loops, it is possible to see why data analytics may not be a “silver bullet”, how it depends on collaborations and ask what can be done about it. The simplified causal loop diagram (CLD) presented in this paper clearly shows these interconnections. A CLD is a system dynamics (SD) tool used to visualize how different variables in a system are interrelated (Sterman, 2000). Based on the CLD and a combination of research experience, interactions with practitioners throughout projects and academic literature, we discuss the challenges inherent to humanitarian health settings as a basis for developing recommendations to steer future research in a direction where collaborations and analytics can foster sustainable solutions, resulting in both impactful practical recommendations and relevant research output.

The main contribution of our work is taking a systems approach to the problem. Other papers have already discussed several difficulties and challenges related to data and analytics in humanitarian operations (de Vries and Van Wassenhove, 2020; Pedraza‐Martinez and Van Wassenhove, 2016). Our work distinguishes itself from those papers by bringing the interface and interactions between data, analytical techniques and collaborative research together in a CLD, thereby showing how everything is interconnected. Those connections and loops have not been explicitly explained in literature. The CLD helps explain why interventions often do not yield the intended improvements and highlights that a change in one of the connections will affect the entire system in the long run. A proper understanding of these connections can lead to the escaping from what is often stated as a “deadlock”. We indicate how this can be accomplished through an exemplary case and by suggesting future research directions. Our paper aims to improve the understanding of the complex system encompassing both data analytics and collaborations among researchers and practitioners in humanitarian health supply chains.

The remainder of this paper is organized as follows: Section 2 presents three real-life examples of experiences gained by the authors from research projects – both successes and failures, to provide the reader with contextual information and insights. Section 3 describes the different factors between data, analytical techniques and collaborative research. We represent the interactions between them in a CLD and highlight the challenges inherent to humanitarian health supply chains. In Section 4, we identify ways to increase the impact of data analytics and collaborations in fostering sustainable solutions, illustrated by a case study (Section 4.1), and suggest directions for further research (Section 4.2). Section 5 concludes with a summary of the main insights.

In this section, we present three cases from our experiences in the field of humanitarian health OR/OM: childhood immunization, family planning and supply chain resilience in health emergencies. Each case focuses on a different part of the data capturing, data analytics and collaboration triptych. The childhood immunization case shows the difficulties typically encountered during data capturing; obtaining reliable and relevant context-related data is often imperative but takes time and requires collaboration between researchers and practitioners. The family planning case shows that simply having data is not enough; to be useable, the data must be cleaned and one needs to understand the context in which they were collected. The supply chain resilience case highlights that constructing a model that can encompass different locations and different organizations – for generalizability – is time-consuming and beyond the scope of most academic studies.

Despite being one of the most successful and cost-effective health interventions, immunization presents substantial challenges in LMICs (De Boeck et al., 2020; Decouttere et al., 2021b).

First, reported immunization coverage rates are often inaccurate. While the ratio between the number of vaccinated children and the total child population is an internationally used metric (Stashko et al., 2019), in practice, errors in the numerator and denominator frequently occur. For the numerator, registration of the vaccines administered is challenging in low-resource settings. Even more problematic is the denominator, which takes the population estimate (often based on historical census data and projections) for the catchment area as the basis which may differ substantially from the actual number (Stashko et al., 2019). Factors that “blur” the actual number of children to be vaccinated in a catchment area include freedom of health-care choice, migration, home-births and demographic evolution. Related to the denominator errors is the so-called “aggregation fallacy” – that is, while national immunization coverage may look promising, subnational differences exist – which explains why reported/estimated coverage rates can exceed 100%, unrealistic year-to-year fluctuations are reported, and disease outbreaks can occur in areas with high reported coverage (Stashko et al., 2019).

A second challenge relates to the tendency to focus on short-term solutions. For example, when a measles outbreak occurs, mass vaccination campaigns are put in place. When immunization status is hard to establish because of lack of well-functioning immunization registries, every child in the targeted age group is vaccinated in the campaign regardless of whether they already received a vaccine. This again leads to immunization coverage rates that exceed 100%. Mass vaccination campaigns absorb immunization resources and remove the urgency from correcting deficiencies in the routine immunization system that caused the local under-immunization, preventing the identification of sustainable interventions.

Through an ongoing collaboration with the University of Rwanda, several authors of this paper conducted community-level research, stakeholder workshops and field visits to analyze the immunization system in Rwanda. A substantial portion of the data used for operational planning in the vaccine supply chain are paper-based or entered on Excel sheets, which is also the case in Rwanda. As a result, data are scattered and of variable quality, making it necessary to use multiple sources to obtain the necessary input, including reports from WHO and UNICEF and primary data collected by local researchers. For the latter, it was necessary to apply for ethical clearance, a lengthy and difficult process.

For the research (Decouttere et al., 2021a, 2021b), we first drew a CLD, which was adapted through several iterations with stakeholders during field visits, stakeholder workshops and discussions with local research staff. After the CLD was constructed, an analytical SD model was developed, again requiring multiple iterations with local researchers. These researchers reconnected through regular follow-up interviews with the stakeholders in the communities to clarify unexpected data input and model output. During the COVID-19 pandemic, we continued the collaboration through online meetings thanks to the longstanding relationship with and involvement of local researchers.

Data collection (based on stakeholder engagement) during this research was time-consuming and slowed the delivery of research output. It was, nevertheless, valuable (and even imperative), as it allowed to:

• identify relevant problems;

• make an overview of the available and useful data and data sources;

• identify required additional data and construct a plan to collect these data (including applying for ethical clearance); and

• build a long-lasting relationship with local research partners.

For many years, a few of the authors have collaborated with a mid-size non-governmental organization (NGO) dealing with family-planning services (de Vries et al., 2021; Alban et al., 2022) that operates some 500 mobile outreach teams in 30 African countries. The teams generally visit villages after a marketing campaign, but their visits are irregular and client numbers vary widely. Some villages have large numbers of clients even though they are visited infrequently; others have small client numbers but are visited every month. Clearly, there are resource allocation problems here, such as assigning villages to outreach teams and determining visit frequencies. How marketing efforts influence client numbers and whether villages are saturated or constitute a source of new clients remains unclear.

The organization systematically captures considerable data regarding clients visiting the mobile outreach teams (through software that allows them to record client information), which itself is rare in the humanitarian health field. However, the data are “dirty” – names and reasons for the visit as well as information about arrival and departure of the mobile outreach team are often poorly recorded, village names are entered in different ways and geographical coordinates are imprecise. In short, it is difficult to use the data for analysis without cleaning them and making linkages, for example, to characteristics of the village in terms of population, poverty, marketing efforts and client numbers. It is important to note that this NGO invests substantial efforts in data gathering, making it a clear leader in the sector. Hence, this signals that the challenges for many other organizations are even much larger than described above.

While donors to this NGO insist that priority be given to the young and the poor, very limited information linking villages with such groups is captured. Under increasing pressure from donors, staff put a lot of time and effort into collecting data and making reports to comply with donors' reporting requirements, but these are not always useful for improving internal efficiency or for research purposes. They tend to serve auditing purposes instead of being geared toward learning and performance improvement. Moreover, auditing and reporting take away resources that could be invested in developing sustainable solutions.

The organization is extremely cautious about suggesting – let alone imposing – rules on the mobile outreach teams, which makes it difficult to change key operating variables (e.g. visit frequencies) and to determine the impact of better-designed visit allocation rules. Given a reluctance to intervene at the local level, they are hesitant to test and implement recommendations from research insights.

This type of situation is common in development settings and makes systemic improvements hard for several reasons. First, there is little reliable information (i.e. not enough solid and useful data) to do a proper analysis of the drivers of performance. Second, organizations often do not have the competences or resources to do this type of analysis in-house. Third, they do not see how this can be changed and, thus, expend little effort collecting and analyzing data to improve operations. Also, they do not have the authority to run pilots or change routines in the field, where operations are typically decentralized and obtaining good information is difficult.

Using information from a large humanitarian organization regarding the impact of likely scenarios of disasters on its operations, several authors of this paper developed an SD model to optimize the disaster preparedness of the organization (Stumpf et al., 2022, Rustemeier, 2017). After the research and development phase, a broker consulting organization tested and implemented the model in the humanitarian context. The goal of the broker was to apply the model to different humanitarian organizations and disaster responses to be able to generalize the findings for a larger part of the sector. The model so far has been used for both food and non-food items. For example, a study was conducted on the response capacities of a humanitarian organization in the event of a possible cholera outbreak by studying the availability of oral rehydration solutions and normal saline for the management and treatment of cholera.

When a humanitarian organization approaches the broker organization with an interest in using the SD model, a one-day workshop is held to develop the disaster scenario. This secures model contextualization based on operational parameters, assures the relevance of the analysis for the humanitarian organization and greatly enhances the sense of ownership by field office staff. It avoids the lack of stakeholder engagement that often hampers projects where the benefits are not mutually understood or communicated. For a period of one to three weeks, the humanitarian organization is supported with data collection. Data requirements include supply chain information during both current operations and the disaster scenario selected from multiple external stakeholders (e.g. suppliers and transportation companies). Different data sources may be combined (e.g. storage and transport technical reports, historical data and extensive qualitative discussions with staff). All assumptions are verified through discussion with the organization's supply chain and program departments. The results of the model are outlined in a practitioner report and presented to the humanitarian organization's leader and senior management team during a final workshop.

Although the field study is necessary to understand the challenges and context, it is time-consuming. The data may not be in the correct format or may not even have been collected. Hence, the engagement of the humanitarian organization's employees is vital to obtain reliable estimations. This is particularly challenging when the practitioners need to imagine a “fictitious” disaster; they can find it hard to accept that their assumptions must be reliable and to recognize that, even if the SD model strives to represent the organization's operations in the most realistic way possible, it cannot represent every detail of the real supply chain. The context is even more complex when the scenario includes a disease outbreak in LMICs with limited financial resources, as the local health system is overwhelmed and does not function properly. An epidemic can worsen if natural disasters occur simultaneously, creating additional challenges regarding data availability and reliability. Clearly, data analytics for effective decision-making is difficult to achieve. Rolling out a model in different locations and for different organizations to allow for its generalizability is time-consuming and typically lies outside the scope of academic studies.

Drawing upon our collective experience in humanitarian OR/OM, we start by describing the different factors between data, analytical techniques and collaboration for impactful research and identifying the interactions and loops between these factors (Sections 3.1.1–3.1.3). They emphasize the complex interrelations captured in a CLD (Section 3.1.4).

In many instances, an underlying phenomenon initially does not create an acute problem (e.g. local under-immunization does not result in an outbreak of disease). Indeed, decision-makers may not be aware of the problem or feel any urgency to invest in solutions. Awareness suddenly increases when a serious problem occurs (e.g. a measles outbreak), resulting in a perceived urgency to change. As the apparent problems are often life-threatening, resources are focused on finding a “quick fix” (immediate solution) to improve the as-is situation. Decisions are often made at short notice and under time pressure, resulting in myopic solutions that relieve the symptoms of the underlying problem(s) but not their root cause(s). Consequently, these myopic solutions are rarely close to long-term sustainable solutions. It is only a matter of time before the system re-enters an acute state with pressing problems, repeating the cycle.

Given the resource-limited setting (in terms of for example money, people, assets and time), a tradeoff exists between investing in myopic quick-fixes in response to an acute problem and sustainable solutions whose benefits only become apparent in the long term, consistent with the “shifting the burden” systems archetype described in Meadows (2008). To find heuristic solutions that can also be implemented during an acute state and that are close or nearly close to long-term optimal solutions, considerable resources need to be invested. Given the tradeoff mentioned above, these resources are often used for implementing myopic solutions because the system is constantly re-entering acute states characterized by pressing problems.

Decision-makers need to be aware of the problem if they are to start investing in sustainable solutions and developing research that helps to define a solution. They need the right data, that is, to monitor the relevant metrics that identify the problem, based on research techniques and validated with real-life data of satisfactory quality. Being able to make sense of data requires a knowledge of the context in which they are collected (which is often linked to the reporting of metrics) and may require multiple iterations with people in the field. To ensure high-quality data, decision-makers must be aware of the value they represent and, therefore, allocate sufficient resources. We have seen some organizations deliberating on what the relevant metrics are, yet few succeed. Logistics in the humanitarian sector are often neither considered a priority at leadership level (at best, it is considered a support function) nor is there a donor appetite for it.

The use of analytical techniques does not necessarily lead to the identification of sustainable solutions. Several steps are necessary for this to happen. First, analytical techniques must address a real-life problem (as opposed to finding a problem that fits the technique). Engaging with people in the field is a better way to direct research efforts to relevant problems, increase understanding of the local context and establish a relationship with practitioners and stakeholders. This incentivizes (further) collaboration between humanitarian organizations and researchers and increases the willingness of stakeholders to engage in research projects and primary data collection. When analytical techniques are coupled with real-life, high-quality data, embedded in the local context and targeted at solving relevant problems, it increases the ability to deliver impactful research output. When this output enables researchers to both solve real-life problems encountered by practitioners and publish in academic journals, it increases the incentive to develop analytical techniques.

Going one step further, the development of analytical techniques that are firmly linked with practice is needed to close the gap – going back to practitioners with solutions that integrate several problem-solving iterations – and ultimately leads to valid research insights and the identification of sustainable solutions. When changes to practice are based on research insights that actually resolve or reduce the problems at hand, it again increases the incentive for practitioners to collaborate with researchers.

Clearly, many factors intervene in the relationship between data, analytical techniques, collaboration between researchers and practitioners and impactful, implementation-oriented research. Moreover, multiple interactions and causal loops between these different factors lead to a complex system. We use a systems approach to represent the dynamic complexity of the interactions discussed in Sections 3.1.1–3.1.3.

Figure 1 presents the resulting CLD with feedback loops that are either reinforcing (R) loops or balancing (B) loops. In a reinforcing loop, an initial deviation in a variable in the loop will lead to a further deviation of that variable in the same direction. For example, an initial increase in loop R1 of “identification of sustainable solutions” will increase “ability to deliver relevant research output”, which will increase “analytical techniques development”. This, in turn, will increase “analytical techniques implementation”, and by closing the reinforcing loop, this will lead to a further increase in the initially increased variable “identification of sustainable solutions”. In contrast, a balancing feedback loop will convert an initial deviation in a variable to a deviation in the opposite direction. For instance, in loop B1, a higher “perceived urgency to change” will increase “investment in myopic solution”, which will decrease “situation awareness” because the problem symptoms have been fixed. Consequently, the “perceived urgency to change” will be reduced and the initial “investment in myopic solution” as well.

Figure 1 explains two main insights derived from the three case studies. First, it reveals the undermining effect (R4) of quick fix solutions (B1) on resolving problems through sustainable country development (B2) in which we recognize the “shifting the burden” systems archetype (Meadows, 2008). Second, it clarifies the synergistic effect between “fostering sustainable solutions” (R1) where analytical techniques are developed and implemented and “implementation success” (R2) which further incentivizes research collaboration between practitioners and researchers. This, in turn, enables stakeholder engagement and primary data collection, increasing the ability to deliver impactful research output and further contributing to “analytical techniques development”. Conversely, a low ability to deliver relevant research output will ultimately lead to few possibilities to identify sustainable solutions that can be used in practice. This, in turn, further decreases the ability to deliver relevant research output, creating a negatively reinforcing loop. Moreover, several of the feedback loops are closely interacting with each other. Therefore, a change in one of the variables or connections will ultimately affect the entire system.

If we consider the ultimate goal of the humanitarian system, then only the balancing loop B2 “supporting country development” is involved in actually reducing the unresolved problems. In contrast, when one invests in myopic solutions following the “quick fix” reinforcing loop, the research opportunity is lost, as the analytical processes that would enable to solve the cause of the problems are not implemented.

In this section, we review major challenges that hamper the positively reinforcing connections in the CLD in Figure 1 when applied to humanitarian health operations. To identify these challenges, we draw upon a combination of our own research experience, interactions with practitioners throughout projects and academic literature.

We searched the databases Web of Science and Scopus using the following search terms: (data analytics) AND ([humanitarian operations] OR [development operations] OR [global health operations]). We then selected articles that describe challenges related to data, analytical techniques and collaboration between academics and practitioners in humanitarian health environments. It is not our aim to provide an exhaustive overview of papers using data analytics for humanitarian health operations – for this, interested readers can consult, for example, Akter and Wamba (2019), Ali et al. (2016), Gupta et al. (2019), Maric et al. (2021) and Sharma and Joshi (2019). Practitioner input was obtained from:

• one of the coauthors who is a practitioner; and

• interactions with practitioners from humanitarian, medical and supply chain organizations throughout different research projects.

Using the CLD, we conclude that unless a problem is identified and unless relevant and reliable data are coupled with knowledge about the context, data analytics have limited use in these settings because the proposed solutions will neither model a real-life problem nor solve it.

Gonçalves (2011) quantifies the tradeoff for humanitarian organizations between providing relief/recovery after a disaster and building capacity. Deploying substantial resources in the form of disaster relief shows clear benefits in the short term, making this a compelling strategy but inevitably results in fewer resources available for capacity building. If the humanitarian organization is in the “quick fix” loop (Figure 1), then it is too busy with life-saving and health activities to care about data collection – data are not a priority (Kunz, 2019).

Organizational values (e.g. capacity to ferment positive change in reaction to observed or experienced flaws) have an impact on the consideration of the tradeoff between short-term myopic and long-term sustainable solutions (Dennehy et al., 2021). The idea still prevails in the field that humanitarian organizations should focus their energy and resources on delivering medical care to people in need rather than developing IT applications (Falagara Sigala et al., 2020). Donors often consider the latter as overheads which should be minimized. Nearly every humanitarian organization performs an Emergency Response Evaluation after a severe crisis and poorly engineered data systems are frequently pinpointed as failures, yet investment in systems and apps does not materialize, as it is not their core business.

Most commercial companies have a large database. In humanitarian health settings, data are seldom properly collected, stored or shared, and information systems are lacking (de Vries and Van Wassenhove, 2020; Pedraza‐Martinez and Van Wassenhove, 2016). People working in the field still rely heavily on paper-based lists and Excel sheets to manage operations (Falagara Sigala et al., 2020). Field data are characterized by noise, dirt and missing elements, leaving the researcher to make sense of it using a combination of contextual knowledge and modern data analytics techniques (Besiou and Van Wassenhove, 2020).

Without context, data often do not make sense. A practitioner told us:

For data collection, researchers need to work with people on the ground, establish partnerships with local agents, obtain ethical clearance and explain to local partners the benefits from the intervention. Multiple iterations are often required to understand the context and correctly interpret the data. This takes a considerable amount of time and effort from researchers (i.e. beyond that typically committed in the course of a young academic's career) and local partners. During the data collection phase, there is neither output in terms of publication in scientific journals nor any guarantee that these efforts will eventually result in a published paper. This makes it a “risky” investment and reduces the appetite to engage in this type of research (Kohrt et al., 2019; Standley et al., 2022). Consequently, some researchers tend to work on problems using hypothetical data or flock to the scarce cases where data are readily available, letting techniques or methodology dominate research relevance and impact.

Data collection is often not standardized nor integrated across different platforms, agencies, sectors and partners (Nair, 2022). When collecting data, it is important to avoid duplication of effort (e.g. different research institutions asking for the same data in a slightly different format or granularity), thereby wasting local partners' time and creating frustration. People in the field are often asked for data collection or reporting without being told what the data are for and how they are going to be used. Consequently, field practitioners might find it hard to appreciate the purpose of data, despite the drive toward digitalization and technology in the humanitarian sector. In addition, local partners often focus on gathering data required by donors for reporting purposes, which are not always useful for research purposes (e.g. national versus subnational immunization rates). Different donors have different reporting requirements (Falagara Sigala et al., 2020). As resources are limited, this pulls people away from collecting relevant data for research.

In other instances, there may be a data overload – but not in the right format or difficult to access, for example, because an Enterprise Resource Planning (ERP) system is bought and installed without training people in understanding the methodology, context or procedure (Falagara Sigala et al., 2020). The myopic view referred to in Section 3.1 means organizations neither design connected applications nor share data, even though they receive money from the same donors. Current data storage and IT systems used by organizations are diverse, siloed and offer limited scope for collaboration (Akter and Wamba, 2019; Dubey et al., 2020). Sometimes, the required data are not directly available but could be extracted from “open sources” like geographical data and social media (Warnier et al., 2020). It is possible for a data overload to exist where comprehensive, cross-functional and accurate data are missing, resulting in a fragmented and volatile information landscape (Comes et al., 2020; Griffith et al., 2019; Prasad et al., 2018).

Difficulty of access may be because of different priorities (Section 3.2.1) or because multiple stakeholders need to grant access. Hence, it is important to consider data ownership, governance and security in humanitarian health settings. While NGOs and global donor agencies operate data systems on behalf of governments, they do not own the data; ownership remains with the national governments of LMICs or with the beneficiary. As such, the approach to data and analytics must be cognizant of (and embedded in) the data governance framework of any given application. A local NGO, an international NGO, a donor and the government may all have data, but data-sharing rules may not have been agreed upon. In addition, ethical clearance procedures can take considerable time.

Nowadays, humanitarian organizations are more aware of the importance of supply chain management and increasingly adopt analytical and technological innovations. As academic interest in these settings grows, efforts have proliferated to adapt techniques developed for the commercial context to humanitarian operations (Besiou and Van Wassenhove, 2020; Holguín-Veras et al., 2012). However, humanitarian health settings are very different, as COVID-19 has shown, and examples of a mismatch between the real-life challenges and the academic literature abound. Lemmens et al. (2016) found that general supply chain network models were unable to cope with the complexities of a vaccine supply chain. De Vries and Van Wassenhove (2020) note that although routing optimization software is used extensively for private sector logistics, this has not happened in humanitarian logistics. The conditions in which advanced routing systems perform well rarely hold in humanitarian settings.

Model outcomes need to be understood and linked to reality, which has proven challenging. A gap often exists between an optimal solution generated by a model and what is feasible in practice. There may be cultural hurdles to the acceptance and usefulness of analytical techniques: local staff might not trust “black box” optimization (de Vries and Van Wassenhove, 2020). Consequently, pure OR or AI may have little value for some humanitarian health issues in LMICs. Unlike the commercial sector, limited budgets and a challenging environment (e.g. unreliable internet connections, power outages and limited access to upgrades) limit the use of IT solutions that require costly software (Besiou and Van Wassenhove, 2020; Falagara Sigala et al., 2020; Griffith et al., 2019; Sharma and Joshi, 2019). Decision support tools need to be cost-effective, although measures of cost-effectiveness may vary greatly among countries (de Vries and Van Wassenhove, 2020).

Besiou and Van Wassenhove (2020) found that papers published in special issues (i.e. of POM, JOM and EJOR) on humanitarian operations focused on traditional OR/OM problems with little consideration for the specific context and on so-called “CNN disasters” that attract media attention in developed regions (perhaps because of easier access to data). Tang (2021) provides several examples showing that a development program is more likely to be sustainable if the underlying constraints and context are taken into consideration.

Successful application of data analytics in humanitarian health settings is highly dependent on the link with practice. Researchers must first acquire a deep understanding of the problem, underlying constraints and the context before diving into analytical modeling or else risk focusing “purely on developing mathematical methods to solve stylized problems (puzzles) without reference to any real problem situation”, creating a vicious circle “where some academics know more and more about less and less and become disconnected from the real world” (Dyson et al., 2021).

Humanitarian health operations are complex and involve a multifaceted relationship between data, analytical techniques, collaboration and impactful research. A proper understanding of the connections and loops in the CLD (Figure 1) can lead to the escaping of what is often stated as a “deadlock”. In this section, we show how this can be accomplished through an exemplary case (Section 4.1), and we use the CLD (Figure 1) to develop recommendations to steer future research in directions where analytics are useful (Section 4.2).

Attempts to use software systems to digitize stock and flow data for vaccine supply chains in LMICs have met with limited success. Among the reasons for this are the following:

• System design is often focused on administrative reporting and central planner needs (Ramanujapuram and Malemarpuram, 2020).

• Too much too soon. Systems attempt to digitize all parts of the supply chain information flow in one go. This increases cognitive complexity and data entry effort, leading to low adoption and poor data quality.

• End-user-friendly design is based on perceptions of end-users in developed countries instead of including actual end-users in LMICs.

The Electronic Vaccine Intelligence Network (eVIN), a software tool developed by Logistimo, captures vaccine stock, in-bound receipts, orders and temperature recordings. Following a pilot implementation in one state in India, it now operates almost nationwide (Ministry of Health and Family Welfare, Government of India, 2018). The implementation of eVIN has led to an increase in systematic data capture and record updating, fewer stock-outs of vaccines and a reduction in doses wasted/discarded. In addition, the captured data by eVIN created numerous analytical studies on the factors that lead to stock-outs and ways to redesign the replenishment process. This case study demonstrates that systematic data collection and rigorous analytics are feasible in humanitarian health supply chains in LMICs if carefully designed and applied.

Several factors contributed to the success of eVIN:

• Developing a bottom-up design that aligns with the top-down supply chain planning hierarchy.

• Starting simple and progressively advancing the complexity of what gets captured in the system. This allows the user to adapt and select the level of system complexity according to their degree of familiarity and ease with the system.

• Using a ubiquitously accessible medium/device for data collection.

• Creating partnerships with academia to carry out analytics.

It takes time and considerable effort to understand real problems in humanitarian health settings (Besiou and Van Wassenhove, 2020), establish relationships with practitioners (Kunz et al., 2017) and interact with them iteratively throughout a research project. Nevertheless, we are convinced it is worth the effort for analytics to have a positive impact on current practice. Researchers can significantly contribute to relevant solutions for humanitarian decision makers. Based on the CLD (Figure 1), we point out several directions for future research to activate the positively reinforcing loops in the CLD.

Research is needed on the reasons behind the frequent occurrence of noise and imprecise elements in field data such that improvement efforts are targeted toward effective incentives or changes. Comes et al. (2020) indicate the need for experimental and empirical research to assess and quantify the perceived priority of information to understand information acquisition in different contexts. This kind of research would be insightful in both humanitarian development and relief settings. In addition, future research could investigate the application of statistical techniques to deal with missing or noisy data in humanitarian health supply chains, building models using sparse data and developing modelling techniques targeting insights rather than purely numerical results.

Alignment of data and reporting requirements between different donors might result in improved data quality and accessibility, while reducing the burden on local staff. Researchers could investigate the impact of donor focus on data collection efforts and investigate how a shift in donor focus might improve data quality and accessibility.

Another way to improve accessibility and quality of data is through the collaboration in data collection, construction of shared databases and model platforms (Nair, 2022). To set up a shared database that is useful for all parties involved, there is a need for coordination between different organizations and within the organization. This results in the need of a neutral body collecting and processing these data and making them freely available to all in an easy-to-use format. Previous research has explored the design principles that need to be considered in ERP systems for humanitarian organizations in a disaster relief context (Falagara Sigala et al., 2020). Future research could investigate similar coordination efforts for data sharing both within and between humanitarian organizations and research institutions. Building shared databases raises many interesting research questions: who is going to collect the data, who pays for it and how is it governed?

Lack of trust among supply chain partners is a major issue because collaboration requires sharing of sensitive data and information (Dubey et al., 2020; Nair, 2022). Tang (2021) emphasizes the challenge to monitor and respond to conditions in LMICs when information is not available or verifiable. Blockchain technology is increasingly put forward to collect information and facilitate data sharing in a safe and transparent way in humanitarian settings (Dubey et al., 2020; Rodríguez-Espíndola et al., 2020; Tang, 2021). It might be interesting to investigate why organizations are not using blockchain technology and how they differ from those that successfully apply this technology on a regular basis.

Qualitative research is underused and, to some extent, undervalued in traditional OR/OM journals, as a view exists that research not involving quantitative modeling should not be classified as OR/OM (Dyson et al., 2021). However, the field of humanitarian OR/OM requires both quantitative and qualitative methods, as humanitarian health settings are very complex and have strong behavioral aspects (Besiou and Van Wassenhove, 2015). “Even for hard modelling, the exploration with stakeholders, the structuring of the problem, the development of diagrammatic representations, and the choice of methodologies are crucial stages that deserve the explicit systematic exploration of problem structuring rather than ad hoc approaches that are not open to scrutiny and may well lack rigor” (Dyson et al., 2021). Similarly, in health system design, qualitative and quantitative methodologies need to be combined to reach implementable results from analytical research efforts (Decouttere et al., 2016). External validity of qualitative research can be assured by including a range of data sources such as interviews, observations, documents and artifacts (Comes et al., 2020; Prasad et al., 2018).

Qualitative research has high value for both the individual researchers and the research community as a whole. It increases understanding of the context and problems on the ground and is more transparent for stakeholders. When combined with quantitative modeling, qualitative research can help identify the research questions to be answered through analytical modeling, define the required data and, as a result, collect the right data to investigate the problem at hand (Gralla et al., 2014). It also provides interesting avenues for future quantitative research, especially problems and settings that have not yet been extensively researched. The field of humanitarian health operations offers many interesting and socially relevant research opportunities.

Solely applying typical OR/OM methodologies might not be sufficient to result in impactful research that takes account of long-term effects and the unintended consequences of symptomatic solutions. The need for interdisciplinary research becomes apparent. This includes the use of data analytics methodologies beyond typical OR/OM methodologies to obtain and interpret relevant data such as spatial and geophysical modeling or using machine learning techniques to assess and verify data quality, to estimate vulnerability to disasters (e.g. landslides and flooding) and to produce high-resolution maps of health indicators (Bosco et al., 2017; Kunz, 2019).

AI and big data are increasingly put forward as potential “game changers” in the humanitarian field (Swaminathan, 2018; Rodríguez-Espíndola et al., 2020; Tang, 2021). Most publications focus on the benefits of big data (Sharma and Joshi, 2019). However, it is not a given that AI and big data always contribute to humanitarian health operations.

Research on perceptions of AI and big data and the feasibility of applying them in resource-constrained environments offers insights on the potential hurdles of implementing these techniques in humanitarian settings. Research is needed to investigate the performance of easy-to-implement heuristics compared to optimal solutions, as there is often a gap between them. This includes research on the usefulness of AI in humanitarian health settings:

• Does AI significantly outperform current practice or heuristics that are easy to implement?

• If so, can we derive heuristics from AI solutions that are intuitive and easy to implement in practice?

The importance of easy-to-implement heuristics should not be underestimated, as decision-makers are known to follow intuitive rules to make quick decisions in stressful conditions (Comes et al., 2020).

Researchers should be aware of potential biases when relying on big data. Trustworthiness of data sources may be questionable, particularly when big data is generated on social media platforms (Ali et al., 2016; Kunz, 2019). Information obtained through big data technologies and practices is partial and linked to the geographical and social context of the people who produce it (Sharma and Joshi, 2019). An important limitation of using big data comes from the “digital divide” between those who have access to and those with limited or non-existent access to connectivity and digital technologies (Kunz, 2019; Sharma and Joshi, 2019). This is especially relevant for humanitarian health operations, where connectivity may be limited in areas hit by a disaster and where a large share of the population in LMICs still does not have internet access. A portion of the intended beneficiaries, thus, risk becoming “invisible” in the humanitarian space and potentially overlooked (Sharma and Joshi, 2019). This raises important concerns around fairness and equity. Future research could investigate the consequences of the “digital divide” as increasing emphasis is put on data analytics.

Stakeholder engagement throughout an entire research project is necessary to make the research useful and impactful. To support the work of humanitarian health organizations with data analytics, it is imperative that academic research has a clear link with practice. Spending time with practitioners enables researchers to understand their reality, concerns and constraints (Pedraza‐Martinez and Van Wassenhove, 2016). This again underscores the importance of gaining insights on the ground and obtaining data first-hand through field studies (Besiou and Van Wassenhove, 2020; Kunz et al., 2017). It implies using both qualitative (interviews and case studies) and quantitative (optimization and simulation) methods and including multiple stakeholders (Besiou and Van Wassenhove, 2015; Dyson, 2021; Van Wassenhove, 2019), as well as data from governments and organizations. To close the loop, we must go back to the stakeholders with solutions that integrate several problem-solving iterations (Besiou and Van Wassenhove, 2020; Kunz et al., 2017; Van Wassenhove, 2019; Van Wassenhove, 2022).

This will take a considerable amount of time and effort from both researchers (i.e. a timeframe that goes beyond typical commitments in young academics' careers) and their local partners. To increase stakeholder engagement during research projects, we must provide researchers with the right incentives (i.e. universities' evaluation criteria and journals' definition of academic contribution) and work with donors to provide organizations with the right incentives (i.e. funding) (Kohrt et al., 2019; Standley et al., 2022). For example, Nature recently presented a new framework that aims to improve inclusion and ethics in global research collaborations to avoid and prevent “helicopter research” (Nature, 2022). The latter occurs “when researchers from high-income settings, or who are otherwise privileged, conduct studies in lower-income settings or with groups who are historically marginalized, with little or no involvement from those communities or local researchers in the conceptualization, design, conduct or publication of the research” (Nature, 2022).

How can research papers go beyond findings to, for example, enable advocacy, marketing, implementation and fundraising to make solutions happen? To make a compelling case to drive better research and get solutions implemented, the definition of research “contribution” needs to be more inclusive, explaining the benefit to humanity beyond knowledge advancement. The involvement of broker consulting organizations can help to build a bridge between academics and practitioners. These organizations can apply and implement the model developed by academics in different locations and tailor it to different humanitarian organizations.

Humanitarian health supply chains provide health care and medical aid to the most vulnerable in development and disaster relief settings alike. Researchers in this field have the potential to contribute to relevant solutions for humanitarian decision makers. However, a gap between practitioner challenges and academic literature remains. Although new techniques for data capturing and processing can potentially improve data analytics in the future, larger data sets and quantification alone will not provide a “silver bullet” for all problems in humanitarian health settings.

This paper has revealed the dynamic mechanisms behind data, analytical techniques, research collaboration and impactful research in the field of humanitarian health supply chains based on a systems approach. The resulting CLD shows the interconnections and feedback loops between the system elements. This allows to explain observed behavior and challenges from the three presented cases. Based on the CLD, we show that humanitarian operations involve complex systems and doing research in this field presents abundant challenges:

• Humanitarian organizations often focus on myopic solutions because of competing priorities.

• Obtaining reliable and relevant context-related data is often imperative but takes time and requires collaboration between researchers and practitioners.

• An overload of data exists that are neither in the right format nor are easily accessible.

• Currently applied analytical techniques fail to grasp the complexity of humanitarian health operations.

• Researchers do not always work on real-life problems.

Nevertheless, researchers can significantly contribute to relevant solutions for humanitarian decision-makers. Humanitarian health supply chains are, above all, a rewarding and impactful environment in which to do research. Using the developed CLD, we provide several directions for future research that we believe will increase the impact of data analytics in fostering sustainable solutions:

• research on improving the accessibility and quality of data;

• qualitative research;

• interdisciplinary research;

• research on the use and usefulness of data analytics in humanitarian health settings; and

• stakeholder engagement.

Note that increasing the impact of data analytics in fostering sustainable solutions will not entirely overcome the gap between practitioner challenges and academic literature. Practitioners sparsely publish in academic journals for a variety of reasons, including different incentive structures to academics. These are not directly addressed in the CLD presented in this paper but could remain challenging even after other challenges are dealt with.

We hope this paper will convince researchers and practitioners alike of the value and contribution of collaborations between researchers and humanitarian organizations, resulting in research projects and partnerships that provide both impactful practical solutions and relevant research output.

Funding: K. De Boeck is funded by a PhD fellowship from the Research Foundation – Flanders (project number 1113720N).