Performance measurement in humanitarian logistics: a customer-oriented approach

Humanitarian logistics (HL) involves various parties including governments, military, non-governmental and commercial organisations that differ widely in size, aims, structure and knowledge (Van Wassenhove, 2006; Argollo Da Costa et al., 2012). Donors and beneficiaries are both customers in humanitarian supply chains (HSCs); the suppliers are both donors and actual paid suppliers (Oloruntoba and Gray, 2009; Charles et al., 2010). This is a more complex structure than in a commercial context, where suppliers are being paid and customers pay for the goods and services they receive. Without a clear goal from a single key stakeholder group, such as the bottom line in commercial operations, finding the strategic position necessary for the development of supply chain management (SCM) is challenging. Performance measurement (PM) in particular becomes difficult, as various stakeholder groups define performance in different ways. However, seeing the growth in demand for HL, it is essential to enable organisational learning based on PM.

Due to the importance of stakeholders in HL, this paper will employ an approach based on stakeholder theory (ST) in order to develop a PM framework. Using the stakeholder salience framework by Mitchell et al. (1997), this paper adopts a systematic literature review to determine the salience of stakeholder groups of the logistics departments or personnel within humanitarian organisations. Stakeholders’ definitions of performance, the parts of the supply chain that are visible to them and other characteristics drawn from the literature review form the foundation of the development of a PM framework. This paper contributes to current research on PM in HL by adding a stakeholder perspective. A research gap is addressed by combining aspects of stakeholder management with PM. This can help practitioners recognise which elements of performance are important to their most salient stakeholder groups.

Section 2 provides an introduction to ST and PM in HL. The research design is discussed in Section 3. The results of the systematic literature review are presented in Section 4. Section 5 develops and discusses a PM framework based on the views of the stakeholder groups with the highest salience. Section 6 provides a summary and conclusion.

ST stems from literature on corporate social responsibility, organisation theory, strategic planning and systems theory (Freeman, 2010). ST states that each organisation has relationships with groups that either affect its decisions or are affected by them, the stakeholders. A stakeholder is “any group or individual who can affect or is affected by the achievement of the organisation’s objectives” (Freeman, 2010). Attention is being paid to stakeholders not just because that will, in turn maximise shareholder profit, but because of their own intrinsic value (Phillips et al., 2003). ST is concerned with the way stakeholder relationships influence the processes and outcomes, both for the organisation and the stakeholders (Jones and Wicks, 1999). It is important to note that all the stakeholders’ claims are considered to be of value (Donaldson and Preston, 1995). This is opposed to a strict shareholder view, where the financial considerations of one stakeholder group that expects high dividends are the primary concern, and other views are not considered.

While ST is popular, it is often unclear who and what really counts, as there is no priority ranking which would make the theory more practical for managers and academics. One way to classify stakeholders is to categorise them into primary and secondary ones, with the former having an official or contractual relationship with the organisation, whereas the latter only have a moral, informal relationship (Carroll, 1993). According to Mitchell et al. (1997), an alternative way to classify stakeholders is according to three attributes: their power, the legitimacy of their claim, and its urgency (Figure 1). A stakeholder that possesses more of these attributes simultaneously is said to be more salient. The authors also outline ways to interact with the eight different stakeholder types that can be identified in this manner. Balancing the demands of different stakeholder groups is an important role of the HSC manager, as it is of any other manager (Schilling, 2000; Oloruntoba and Gray, 2009). However, it is important to note, that the salience of a particular stakeholder group is not always the same. Stakeholder salience also depends on the issue under investigation and the reasons for applying ST (Phillips et al., 2003). Therefore, the typology developed in this paper cannot provide a general theory on stakeholder salience in regards to the HL department or personnel. The attributes will vary depending on the specific disaster, programme and organisation. The literature-based approach provides indications, but empirical data is needed to adapt it to different situations.

PM in HL is largely based on the same principles that apply to commercial logistics (Van Der Laan et al., 2009). However, there are some key differences. Most importantly, non-profit organisations do not have the financial bottom line as their primary performance measure (Kaplan, 2001; Speckbacher, 2003). There is no single key stakeholder group that insists on the profitable operation of the business as the shareholders do in commercial entities. PM in HL is very complex because several stakeholder groups with varying definitions of performance are involved. The effectiveness of a mission and financial sustainability are the two bottom lines in HL as the ultimate aim is to ease suffering and avoid death (Beamon and Balcik, 2008; Moore and Taylor, 2011).

The source of revenue in commercial supply chains are customers who receive an economic benefit from the transaction (Beamon and Balcik, 2008). An organisation’s performance depends on meeting the demands of its customers (Cuthbertson and Piotrowicz, 2011). However, even in a commercial environment, the influence logistics performance has on customers is difficult to measure (Fawcett and Cooper, 1998). In HL, the revenue sources are not associated with the customers who receive an economic benefit, leading to a more complicated stakeholder environment for PM. To advance the understanding of this structure and its influence on PM, this paper integrates ST with PM.

A wealth of logistics and SCM PM literature exists and is reviewed in various papers (e.g. Beamon, 1999; Shepherd and Gunter, 2005; Griffis et al., 2007; Akyuz and Erkan, 2010). There are various approaches to PM that account for more than purely financial performance indicators. The Supply Chain Operations Reference (SCOR) model follows a standard management process spanning all interactions from the supplier’s supplier to the customer’s customers (plan, source, make, deliver, return) and was developed specifically to facilitate effective communication across organisational boundaries in supply chains (Supply Chain Council, 2014). Beamon (1999) proposes a three-part PM framework based on resource performance metrics (inventory holding cost, resource utilisation, etc.), output performance metrics (sales, on-time deliveries, manufacturing lead time, etc.) and flexibility metrics (highest possible output, shortest possible delivery lead time, etc.). The balanced scorecard (BSC) takes into account a financial perspective, as well as a customer perspective, a learning and growth perspective and an internal business processes perspective (Kaplan and Norton, 1992). It has been applied in supply chain contexts (Brewer and Speh, 2000; Bhagwat and Sharma, 2007). In addition, the BSC has also been applied in a non-profit context (Kaplan, 2001). Several studies of PM in HL have also taken it as a starting point (e.g. Schulz and Heigh, 2009; Van Der Laan et al., 2009; De Leeuw, 2010). The BSC is used as a basis for the framework developed here because of its versatility, the previous work that indicates applicability in the context of HL, the relative ease of use, which will be important in the operational environment of HL, as well as its ability to balance various components that contribute to performance. This latter factor is of particular importance in a study that emphasises the differences between different stakeholder groups.

The BSC is used widely, but not without criticism. For example, companies might also wish to include social and environmental factors (Epstein and Wisner, 2001). This would also be of particular importance in a humanitarian context. Another important topic in commercial SCs has been the need to implement measures that go beyond one company to reflect the growing importance of long-term strategic supply chain partnerships (Atkinson et al., 1997; Epstein and Wisner, 2001). Given the complex network of stakeholders in HSCs, this would be a prominent feature, particularly with regards to services. Therefore, developing a PM system for HL will not only benefit humanitarian organisations, but lessons from this research work might also be transferrable to other SCs.

A systematic literature review has been selected as a suitable approach to connect various streams of academic research and to extract the criteria of stakeholder salience from the existing body of literature. A systematic or structured review has its origins in medicine where it is used to amalgamate evidence from several studies in order to provide advice to practitioners based on an exhaustive review of available evidence (Bryman, 2012). A number of HL studies is in existence, and can be used to draw conclusions regarding stakeholder salience, if surveyed methodically in a systematic literature review. In social research, this methodology is used because the results it generates are more unbiased and comprehensive than those of the traditional narrative review (Tranfield et al., 2003). This paper aims to develop a theoretical framework that can then be validated empirically. This theoretical grounding is particularly useful, as HL still has a relatively narrow theoretical foundation (Richey, 2009).

Millar (2004) recommends defining the purpose of the study, as well as the criteria for the selection of papers, before conducting the review. The purpose of this study is to determine the salience of groups that are stakeholders of logistics departments of humanitarian organisations, particularly regarding PM in that environment. To be included in the review, studies must be accessible through one of three database packages, contain the search terms “HSC”, “HL”, “relief logistics” or “relief supply chain”, and must have been published in an academic journal. To capture the breadth of this relatively young field, no further restrictions, for example regarding research methodology or the time period in which the studies were published, were used.

A literature search was conducted using three database packages, Science Direct Freedom Collection, Emerald EMX 150 and EBSCO Business Source Premier. The results were filtered to only include academic journal papers. This initial search yielded 365 papers. In a next step, the titles, key words and abstracts were read and papers that were found to be about research areas other than HL were eliminated from the sample. This reduced the number of papers to 189. After a cross-check between the results from the three databases, duplicates were removed, yielding a final number of 150 papers for further analysis (Table I).

Within the identified literature a search for stakeholders of HL departments or personnel was conducted. The stakeholder groups were established mainly based on six groups discussed by Kovács and Spens (2007). However, it has to be noted that Kovács and Spens (2007) take a network approach, which identifies actors. Not all stakeholders are actors. For example, the beneficiaries are stakeholders because they are affected by HL, but lack the purchasing power that would make them actors (Heaslip et al., 2012). Therefore, beneficiaries were added as a seventh stakeholder group. In addition, volunteers, field staff and headquarters were included to represent internal stakeholders, bringing the number of stakeholder groups to a total of ten. These terms have been chosen due to their prominence in the analysed papers; however it can be difficult to differentiate between them. The media, although not mentioned in Kovács and Spens (2007), play a vital role in raising awareness and donations, as well as sometimes hindering operations (Van Wassenhove, 2006) and were included as the final stakeholder group. Implementing partners that are used by humanitarian organisations to execute their programmes, were not included as a separate group, as there is a wide variance within that group, that cannot be captured adequately in a limited number of search terms as was necessary for this research. These eleven stakeholder groups present broad categories, which might have further subgroups, such as military being split into national troops, UN peacekeepers and insurgents.

Each of the 150 papers was then scanned for any mention of each of the eleven stakeholder groups. Search terms were truncated when necessary, for example, “beneficiar” to find “beneficiary” and “beneficiaries”. Substitute search terms were also used, for example “press”, “journalist” and “news” instead of “media”. The results were only counted when reading the relevant passages revealed that they were indeed discussing this group as being affected by or affecting HL. Numeric results give an insight into the prevalence of each group in the literature and may enable some conclusions about the importance attributed to them. The qualitative part of the study involved content analysis to capture the attributes assigned to each stakeholder group. Based on these attributes, the groups were sorted into the stakeholder types described in the aforementioned framework (Mitchell et al., 1997).

The 150 papers have been published in 39 different journals, with 18 each published in the International Journal of Physical Distribution & Logistics Management, and the Journal of Humanitarian Logistics and Supply Chain Management (Table II). The oldest paper dates back to 1996 and 93 of the papers have been published after 2010 (Table III). Each paper included between one and eleven stakeholder groups, with 69 per cent of papers discussing five or less. This can be seen to indicate that many papers were highly specialised. A full summary of the papers and the stakeholder groups identified in each can be found in Table AI.

Figure 2 depicts the percentage of papers that discussed each stakeholder group. Beneficiaries were clearly the most prominent group, appearing in 79 per cent of the papers. The ones that did not make any mention of them were usually highly quantitative studies, for example, on vehicle routing. Unsurprisingly for publications on supply chain topics, suppliers featured in 60 per cent. Another group to appear in over half of the publications were governments that provide the context for HL, often as donors, as well as contacts in the disaster area. Due to their importance as the second customer group, donors were represented in 49 per cent of papers. Among the internal stakeholder groups, field staff were by far the most prominent. They were most commonly discussed as part of specific logistical issues that occurred locally in the disaster area. Although only 28 per cent of papers discussed the role of headquarters, they tended to feature prominently when they were mentioned. Even less notice (20 per cent) was given to volunteers. This might be explained by the unclear dividing lines between staff and volunteers in many NGOs. Other NGOs were discussed as partners, as well as competitors. The different roles they can play made this stakeholder group difficult to capture in the content analysis. In total, 40 per cent of papers featured them, ranging from a prominent discussion of potential partnerships, to casual mentions of the competitive environment. Only 29 per cent of the sample discussed the role of the military, which can be attributed to the reluctance of many NGOs to work with the military (Tomasini and Van Wassenhove, 2009; Heaslip et al., 2012). It also shows HL as an area quite separate from military logistics, despite certain similarities between the two. A third of the papers mention the media as a stakeholder group, often linked to their influence on donors. Logistics providers are mentioned in only 30 per cent of papers, which could be down to the lack of close relationships in an area that often organises transport on an ad hoc basis (Tomasini and Van Wassenhove, 2009; Balcik et al., 2010).

The Table AII links the 11 stakeholder groups to the previously discussed stakeholder types and associated attributes of power, legitimacy and urgency based on the way they were described in the literature. A selection of quotes from papers within the sample was used to reflect those descriptions. Based on these descriptions, the attributes of power, urgency and legitimacy were assigned to the groups. This process resulted in a stakeholder typology according to Mitchell et al. (1997), presented in Table IV.

There is at least one example of each stakeholder type. The definitive stakeholders are the donors and the headquarters of an organisation. Both possess power, legitimacy and urgency. However, beneficiaries as the other key customer group, as well as field staff as a crucial internal stakeholder group, are of similar importance, even though they do not possess the power to enforce their claims, making them dependent stakeholders. The split between internal and external stakeholders was very visible in the amount of literature devoted to them. While internal stakeholders are covered less frequently, they are still essential for the topic of PM. They will be the ones implementing a PM system, collecting data and utilising it. Based on this stakeholder typology, a stakeholder focused PM framework can now be developed.

The stakeholder typology developed based on this systematic literature review can be applied in various contexts. Here, it is used in the context of PM. There have been several reviews of PM in HL (see e.g. Beamon and Balcik, 2008; Van Der Laan et al., 2009; Abidi and Klumpp, 2013). As pointed out by Abidi et al. (2013), definitions of success in HL differ between stakeholder groups. This makes the application of our stakeholder typology to PM in HL a particularly promising area of research. While previous studies on PM in HL consider the importance of stakeholders, they do not include an in-depth stakeholder analysis.

The importance of donors and beneficiaries as definitive and dependant stakeholders was clearly shown above. Previous research indicates that the customer perspective in a BSC for HL has to include both donors and beneficiaries to provide any reasonable means for PM (De Leeuw, 2010). However, the importance of the customer is not limited to this perspective in HL. The internal business process perspective also involves the customer groups as they provide important inputs, as well as receiving outputs. Such bi-directionality is typical of supply chains that include a strong service element (Sampson, 2000). Without the co-creation of services by customers, the service supply chain cannot function. The learning and growth perspective should also be customer-oriented, as customer relationship management is among the key skills for logisticians in this area (Kovács and Tatham, 2010; Kovács et al., 2012). Finally, the financial perspective lacks the key measurement of the bottom line in comparison to for-profit organisations. Therefore it becomes more ambiguous. Humanitarian organisations tend to have a variety of different approaches to financial management (De Leeuw, 2010). However, the close interaction with donors features prominently and it is noted that finances are managed tightly to satisfy donor scrutiny (Schulz and Heigh, 2009; De Leeuw, 2010). Therefore, at least one of the key customer groups features in all four of the BSC’s perspectives. Although Kaplan (2001) recognises the importance of customers in non-profit organisations and restructured the BSC accordingly, this deep penetration of a customer-centric view is new.

Based on the literature review of PM in HL, taking a stakeholder perspective will mean more than just splitting the BSC’s customer perspective into two, to satisfy both donors and beneficiaries. A true focus on customers will have to go even further. In order to achieve that, a humanitarian organisation has to be aware of the way supply chains look from the customers’ point of view (Maull et al., 2012).

The donors see performance mainly through media coverage on the overall disaster response, which may or may not feature a particular organisation. They are likely to get information about many different humanitarian organisations and will base their donation behaviour on it (Oloruntoba and Gray, 2009). PM should therefore be done in terms of what the donor or the potential donor can see and is informed about, in order to ensure maximum donations in a highly competitive market. This could include outcome measures that show the improved situation of the beneficiaries, or input/output measures that capture the efficiency of the supply chain. However, it has to be kept in mind that a large part of the supply chain is invisible to the donors. In addition, particularly donations from governments, which are the key source of income for many humanitarian organisations, are tied to wider political agendas (Van Wassenhove, 2006; Day et al., 2012). Thus, donations might not be influenced by performance. In practice, an analysis of key donor groups has to take place before applying the PM framework presented below. It might be necessary to differentiate between donor motivations. As this will depend on the specific humanitarian organisation, supply chain or mission under investigation, such differentiation is beyond the scope of this theoretical paper.

Beneficiaries will generally have little awareness of or interest in the wider workings of the supply chain. They can in many cases compare the outputs of several HSCs; however their sources of information can be unreliable and biased, for example, due to a perceived association of a humanitarian organisation with a military force. Their PM will be based on criteria such as the speed of the initial response, or the responsiveness to urgent needs.

Kaplan (2001) puts the organisation’s mission at the top of his strategy map for non-profit organisations. Recognising the difficulties that many non-profit organisations have in clearly defining their mission, the proposed framework mirrors the BSC strategy map for for-profit organisations that states “Improve Shareholder Value” as its primary aim, noting that value arises from the interplay of various, mostly intangible, assets (Kaplan and Norton, 2000). Here, it is replaced by “Improve Customer Value” which is then split into donor value and beneficiary value. Conventionally, customer value is seen as the sum of all benefits customers receive in return for their payment (Slater, 1997). In the case of non-profit organisations, this applies to donors who expect to receive benefits in return for their investment. However, the benefits they expect are related to the satisfaction of achieving good in a socio-political context related to the organisation’s mission (Moore, 2000). Beneficiaries are the customer group that gains tangible benefits from the performance of HL.

Figure 3 presents the proposed framework. The examples provided are based on case studies carried out by De Leeuw (2010). We introduce categorisation into two different groups for whom value is being created, giving the framework a customer-centric foundation. The findings are separated in three columns. The first column presents aspects of donor value, the second column displays factors that are relevant for both customer groups, and the third column gives parts of beneficiary value.

The first level supporting the value creation for donors and beneficiaries is the customer perspective. This is to be expected given the previously discussed importance of the customer in the supply chain. This level contains aspects of HL that each customer group values the most. For the donors these are aspects that can be expected from customers in commercial supply chains as well, namely an efficient and effective supply chain that delivers maximum output for the customers’ investment. As with for-profit organisations, sustainability is a growing concern. The donors are more concerned with the process of HL. Both customer groups are concerned with a reliable performance of HL. For the beneficiaries, the dependable delivery of humanitarian aid is a foundation for rebuilding lives. Beneficiaries are more concerned with tangible outputs of HL. Appropriate items with an acceptable quality, delivered consistently, and in a timely manner are essential for this customer group. While these factors tie into the donors’ requirement for efficiency and effectiveness, they are not identical. All of these aspects, both process-oriented and output-oriented should be key measures of performance as it is relevant to customers.

The next level is the internal process perspective. This is important because the measures relevant to the customers can only be achieved by efficient and effective internal processes. For both customer groups, constant improvement and the possibility to contact the humanitarian organisation are essential, even though motivations might differ. There are also elements that each customer group values individually. Social, cultural and environmental awareness is more important to beneficiaries, as the humanitarian organisation interacts directly with their daily lives through HL. The beneficiaries might be remote from the organisation both geographically and culturally, making social and cultural awareness paramount to satisfactory HL performance. Donors are more concerned with performance reporting the acquisition and retention of donors, and particularly with receiving information about internal processes, showcasing the use of their investment.

The learning and growth perspective ensures a sustainable creation of customer value for donors and beneficiaries. It supports the internal processes by developing and improving them. The beneficiaries’ focus here will be more towards the local issues in the affected area, while donors are more concerned with the bigger picture. However, both groups champion continuity. Taking into account the high levels of personnel turnover, this is a key challenge for HL. Any sustainable creation of customer value hinges on organisational learning and knowledge management practices. This should be a primary focus of PM in the learning and growth perspective. Despite a focus on continuity, it has to be taken into account that circumstances in affected areas change quickly and therefore, learning will also have to occur in the short term. This is of great importance to beneficiaries as their needs change over time, particularly when moving from immediate relief to long-term recovery.

Finally, the financial perspective is essential as a foundation for all the other perspectives and therefore also for the creation and improvement of customer value. Both customers are concerned with it, as any HL activities need financial backing, so this perspective ties directly into the previously discussed aspects of sustainability and continuity. However, the donors who provide the money are much more directly affected by this perspective. They need a level of transparency, as well as appropriate budgeting in order to monitor the value they can expect to gain from their investment, whether financial or in-kind, in the humanitarian organisation. While the customer value created in HL is ultimately not financial unlike in commercial supply chains, the financial perspective is still vital to donors. Particularly governmental donations may be tied to reporting of financial performance.

Through a systematic review of academic literature on HL, the attributes ascribed to 11 different stakeholder groups of the logistics department/ personnel of humanitarian organisations have been determined. Based on their power, the legitimacy and the urgency of their claim with particular regard to PM in HL, the stakeholder groups were assigned to seven stakeholder types in a typology developed by Mitchell et al. (1997). This revealed that donors and beneficiaries are the two external stakeholder groups with the highest salience.

Based on these findings, a customer-centric approach to PM has been applied. This builds on research on service supply chains that also depend on high customer involvement. A framework has been proposed for PM in HL. The BSC that has previously been used in commercial supply chain PM, as well as in humanitarian contexts, forms its foundation.

The stakeholder perspective taken in the development of this framework has led to customers no longer being confined to just one perspective in the BSC. Examples have been given of how measures in each of the four perspectives are relevant to one or both of the customer groups. Instead of putting the shareholder value on top of the strategy map, it was replaced by customer value in this framework. The customer perspective is the first layer that is helping ensure customer value for beneficiaries and donors. It is supported by the internal process perspective. The learning and growth perspective ensures a sustainable advantage. Finally, the financial perspective, while being relegated to the last rung compared to its prime position in a commercial environment, is still essential in providing the means needed for any humanitarian operation.

This study has several limitations. Regarding the systematic literature review, the number of reviewed papers is relatively small, many do not contain empirical evidence and they often build upon each other. Therefore, this paper can only provide a guideline, and the stakeholder typology presented here is to be followed up with empirical data in later research. The proposed framework for PM in HL is yet to be tested empirically too. So far, it has only been based on the existing literature.

The proposed stakeholder framework can provide insights for managers in HL regarding the different influence each stakeholder group has. In relating HL issues to a wider body of literature on stakeholder management, challenges in HL could be handled through various well-published approaches that have been discussed regarding stakeholders in the business community, as well as in non-profit organisations in general. One application of the typology is shown in the discussion of a PM framework for HL. This can provide insights for practitioners regarding the diverging opinions of the key stakeholder groups concerning performance. The PM framework is of interest to HL managers as a tool to help balance the concerns of the two key stakeholder groups. The four perspectives in the framework give insights into priorities and preferences regarding performance. Furthermore, the PM framework can have a policy impact, as it highlights the differences between aspects that are of importance to donors and beneficiaries. This has the potential to be reflected in humanitarian aid policies.

This study offers a foundation for future research to be carried out on both stakeholders in HL and PM. Further research should aim to empirically validate or refine the framework that was developed in this paper. The HL stakeholder typology can also be applied to issues other than PM in situations where the salience of stakeholder groups plays an important role. In the future, it would be desirable to gain insights from HL practitioners, as well as humanitarian organisations, to test and tailor the framework for specific organisations, programmes or disasters. With regards to the PM framework developed through this research, there is a need for empirical work and adaptation of the framework to realistic scenarios at specific humanitarian organisations or for particular missions. This is essential to further refine the framework, particularly regarding the aspects of performance that are of particular importance to specific groups of donors, particularly governmental to stakeholders. Such work could lead to a wider policy impact.

Table AI 

Table AII