Evaluating protection, inclusiveness and dignity in humanitarian aid: a systematic literature review Open Access

Humanitarian aid involves providing assistance – whether in-kind, cash-based or a combination of both – to individuals and communities affected by natural disasters, conflicts or other emergencies to alleviate suffering and meet their basic needs (Gutjahr and Nolz, 2016). The growing humanitarian needs and the increasing gap between appeals and funding have resulted in several challenges in prioritizing and delivering assistance (Daar et al., 2018). Thus, it is crucial to evaluate the impact of aid carefully as it informs decision-making (e.g. in resource allocation and the prioritization of interventions) and provides evidence for replication and scaling (Jayadi, 2024; Doocy and Tappis, 2017). For instance, evaluations of the 2016 flood response in Northern Bangladesh revealed unexpected efficacy outcomes. Some households in severe need received insufficient aid, while others received assistance that did not address their primary needs, leading them to perceive the aid as ineffective. Based on these findings, involved aid organizations and their partners, such as ActionAid Bangladesh, changed the intervention design for future programs (Alam, 2017).

Moreover, the number of funders requesting rigorous impact evaluations has increased considerably in recent years (Worden and Saez, 2021). They are eager to promote accountability and gain a better understanding of the needs of disaster-affected communities. However, progress in the field is slow (Heyse et al., 2021). Evidence shows that many humanitarian organizations still struggle to identify evaluation methods that are accessible, time-efficient, cost-effective and capable of providing concrete insights for future program improvements. For instance, the World Food Program (WFP) implemented its impact evaluation strategy in 2019 to shift from short-term evaluation series to thematic portfolios of randomized evaluations that ask similar questions across multiple locations (WFP, 2019). According to the strategy document, such assessments are intended to provide essential insights to humanitarian organizations and their grantees about which relief modalities are more effective in addressing problems identified when targeting nutrition in a specific context. Nevertheless, to the best of author's knowledge, very few details about the work are publicly accessible and best practices have not been widely shared within the sector (as of January, 2025).

One main problem with current evaluation frameworks in humanitarian aid is that they focus on how external aid structures enter a disaster-affected community rather than on empowering and organizing the community itself (Bealt and Mansouri, 2018). While evaluations should assess whether interventions are empowering – aligned with the needs, motivations and abilities of beneficiaries as key stakeholders (Van Wassenhove, 2006) – current frameworks prioritize aspects like cost-effectiveness instead. Empowerment refers to enabling individuals or communities to take control of their lives, make informed choices and exercise their rights (Bealt and Mansouri, 2018). Humanitarians and aid policymakers often evaluate only whether assistance reaches its targets with the least deficiencies and duplications (Abidi et al., 2014; Banomyong et al., 2019), thus neglecting its impact on local communities.

The problem raises the critical question of whether humanitarian aid interventions can be evaluated based on local realities. Such a local perspective means going beyond merely assessing the appropriateness and success of past humanitarian responses. Instead, it should provide transparent insights into how interventions supported the recovery of crisis-affected regions, how experiences differed across various subgroups in the target area and how future programs could be improved. At the heart of these questions lies the mechanism of evaluation – the indicators. To create aid systems that are globally consistent (adhering to humanitarian principles, programmatic standards and accountability standards) (García Castillo, 2021; Hilhorst et al., 2021) and locally relevant (Roepstorff, 2020), there is a need to incorporate localized indicators into humanitarian evaluation frameworks.

Localized indicators support tailoring aid to the context of targeted communities. More specifically, field reports [e.g. Mosel and Holloway (2019)] assert that careful consideration of social aspects such as protection, inclusiveness and dignity (PID) could ensure that humanitarians do not inadvertently harm or exacerbate the vulnerabilities of the very populations they aim to assist. Academic literature also stresses the importance of considering social aspects (e.g. dignity), notably with the increasing use of emergent technologies in aid (Baharmand et al., 2021). However, research about PID indicators is scarce and fragmented. The evidence has also rarely been documented in the field. Jeong and Trako (2022) review several impact evaluation reports and note that, for instance, gender outcomes, particularly program impacts on gender-based violence, were some of the least-explored outcomes. In other words, there is a lack of understanding in research and practice about what localized indicators are available and where to find relevant ones for PID.

This study aims to explore the following two research questions (RQs):

This paper systematically reviews the academic literature and humanitarian reports published between 2014 and 2024 in mainstream operations and supply chain management journals, popular interdisciplinary journals and a common humanitarian information-sharing platform (cf. Section 3). The objective of the study is to identify indicators that address the complex socio-cultural and environmental intricacies of regions receiving aid and actors involved in delivering operations. Such localized indicators are necessary for rigorous assessment of the impacts of aid.

The rest of the paper is organized as follows: Section 2 provides background about the evaluation, indicators and PID and explains the research gap. Section 3 describes the methodology for conducting the systematic review. Section 4 covers the descriptive results of the literature review. Research gaps and directions for future work are discussed in Section 5. The paper concludes in Section 6, where the limitations of the study are also reported.

The evaluation of humanitarian aid is crucial for several reasons. First, evaluating humanitarian operations allows for assessing the effectiveness and efficiency of the assistance provided. It helps determine whether the aid meets the needs of the target beneficiaries (Moore et al., 2003). By evaluating the impact of aid, humanitarians can identify areas for improvement and make necessary adjustments to current and future interventions (Jayadi, 2024).

Moreover, evaluation ensures accountability and transparency in humanitarian aid (van Voorst et al., 2022). It helps identify gaps or shortcomings in the delivery of assistance and holds organizations accountable for their actions (VanRooyen, 2013). This accountability is essential for maintaining the trust and confidence of donors, governments, and target populations (Patel and Wild, 2018).

Furthermore, evaluation enables learning and knowledge sharing within the humanitarian sector. By transparent and systematic assessment of the outcomes and impacts of interventions, humanitarians can identify and document best practices and lessons learned (Rossignoli et al., 2017). This knowledge can then be shared with other organizations to increase the impact of aid globally (Doocy and Tappis, 2017).

An indicator is a qualitative or quantitative means that provides insight into a broader concept or phenomenon (Veenhoven, 2002). Although indicators in aid are typically derived from multiple measures or data sources, they may not cover all aspects (Goulart et al., 2021). However, literature asserts that current indicators in evaluation frameworks support policymakers to ensure that aid is globally consistent, i.e. interventions adhere to humanitarian principles, programmatic standards and accountability standards (Broaddus-Shea et al., 2019).

The four core humanitarian principles of humanity, neutrality, impartiality and independence ensure that assistance is provided to all people in need. Programmatic standards offer specific guidance on how to deliver humanitarian aid effectively in different sectors, such as food security, health and water and sanitation (García Castillo, 2021). Accountability standards cover transparency, participation and complaint mechanisms, ensuring that humanitarian organizations are accountable to the people they serve (Hilhorst et al., 2021).

Based on the abovementioned principles and standards, some indicators for local involvement in interventions have been introduced in the field (mainly through the accountability standard), e.g. coordination among local and international organizations (Broaddus-Shea et al., 2019; García Castillo, 2021), alignment of aid with national and local procedures, delivery of assistance through local systems and local logistics performance (Roepstorff, 2020).

Focusing on PID ensures that aid is provided in a way that respects the rights and well-being of affected populations and promotes their empowerment. Protection involves safeguarding individuals’ safety, security and rights in crises. Protection ensures that while they aim at saving lives and alleviating suffering, humanitarians also provide physical protection from harm, such as violence or exploitation and ensure access to essential services and resources (Young and Maxwell, 2013). By prioritizing protection, aid seeks to mitigate the negative impacts of crises and promote the well-being of affected populations (Horst and Nur, 2016).

Inclusiveness means ensuring that aid reaches all individuals in need, regardless of their background or circumstances (Asongu and Odhiambo, 2019). Inclusive humanitarian aid recognizes the rights and needs of marginalized and vulnerable groups, such as refugees, internally displaced persons and those living in poverty. It aims to address the specific challenges faced by these groups and promote their participation and representation in decision-making (Young, 2023). By being inclusive, humanitarian aid can effectively address the diverse needs of affected populations and promote social justice (Asongu and Odhiambo, 2019).

Dignity emphasizes the inherent worth and rights of every individual (Allahi et al., 2020). Humanitarian aid should uphold the dignity of beneficiaries and respect their autonomy and choices. By prioritizing dignity, humanitarian aid seeks to restore a sense of agency and self-worth to individuals affected by crises (Affolter and Allaf, 2013).

This study follows the systematic literature review approach outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines (Moher et al., 2010; Page and Moher, 2017) as the primary methodology for collecting and reviewing the articles. The PRISMA guidelines have been used widely in systematic reviews, including those published in the humanitarian literature (Jayadi, 2024; Anjomshoae et al., 2022). Similar to Jayadi (2024), relevant reports from humanitarian interventions were also gathered and added to the article base for review.

The two wellknown databases of Scopus and Web of Science were used to collect research articles. These two databases cover several journals that publish papers about humanitarian aid and related topics (Rojas Trejos et al., 2023). Only peer-reviewed journal articles written in English were considered and the publication period ranged from 2014 through April 2024. This timeline specifically captures papers that discuss cash-based and in-kind assistance.

Using Boolean operators (and, or), this review followed specifically Anjomshoae et al.’s (2022) approach to develop the following combinations of keywords for the search query in both databases: (“humanitarian*” OR “in-kind aid” OR “cash-based assistance” OR “cash and voucher assistance”) AND (“evaluation*” OR “performance*” OR “measure*” OR “assess*” OR “monitor*” OR “Theory of Change” OR “Logical Framework” OR “dignity” OR “protection” OR “inclusive*”) AND (“indicator*” OR “metric*” OR “KPI*”). The two terms “Theory of Change” and “Logical Framework” were incorporated to capture humanitarian technical reports, as Anjomshoae et al. (2022) note that these keywords are often used in field reports about performance measurement and evaluation. The keyword “local*” was not included in the search query to avoid excluding the papers that do not necessarily express local-related matters in their text. Searching titles, abstracts and keywords resulted in 486 papers. Then the list of journals was limited to the following: International Journal of Logistics Management (IJLM), International Journal of Logistics: Research and Applications (IJL-RA), International Journal of Operations and Production Management (IJOPM), International Journal of Physical Distribution and Logistics Management (IJPDLM), Journal of Business Logistics (JBL), Journal of Operations Management (JOM), Supply Chain Management: An International Journal (SCM-IJ), Supply Chain Management (SCM), Transportation Journal (TJ), Transportation Research Part E (TR-E), Journal of Humanitarian Logistics and Supply Chain Management (JHLSCM), Production and Operations Management (POM), International Journal of Disaster Risk Reduction (IJDRR), European Journal of Operational Research (EJOR), Socio Economic Planning Sciences, Computers and Industrial Engineering (CIE), Production Planning and Control (PPC), World Development, International Journal of Production Research (IJPR), Journal of Development Economics, International Journal of Production Economics (IJPE), Disasters, International Review of the Red Cross, Annals of Operations Research (ANOR), Journal of International Development, International Journal of Humanitarian Action, IEEE Transactions on Engineering Management, and Computers in Industry (CII). Multiple surveys in the humanitarian literature [e.g. Jayadi (2024)] assert that topics related to humanitarian operations in different contexts are widely covered in the abovementioned journals. The search resulted in 44 articles after refining the initial list of papers with respect to the selected journals. Then, the abstract of these papers were examined and papers that did not focus on the evaluation or performance measurement of humanitarian aid were excluded. The exclusion criterion resulted in 28 papers that were kept for full text analysis. Eight other papers were also added using snowballing (Minoiu and Reddy, 2010; Neanidis, 2012; Crothers, 2014; Horst and Nur, 2016; Roepstorff, 2020; Thravalou et al., 2021; Arduin and Saïdi-Kabeche, 2022; Midgley, 2023).

For field reports and white papers published by humanitarian organizations, the Jayadi’s (2024) approach was followed. The ReliefWeb website–which offers an open database with over 4000 sources (e.g. reports, infographics and training programs) published by different humanitarian actors–was used to collect reports. The reports were retrieved using the keyword “indicator”. The keyword was selected after several rounds of searching using different keywords (e.g. “evaluation” and “assessment”). It was observed that by searching the dabase using “indicator” as the keyword and then filtering the results for “Evaluation and lessons learned” (as content format) more relevant and comprehensive reports could be collected compared to only searching for “evaluation”, “assessment” and “monitor”. Other filters were as follows: the content language was set to “English” published by “International organizations”, “Non-governmental organizations” or “Red Cross/Red Crescent Movement” over the past decade (January 2014 to April 2024). The query resulted in 61 documents. After removing duplications (executive summaries of complete reports) and situational updates, 39 reports were collected for further analysis.

To begin the analysis, the final sample consisted of 75 papers and reports published between 2014 and 2024 (cf. Supplementary material). The steps instructed by Anjomshoae et al. (2022) were followed to analyze each document (research article or report) to identify its main evaluation theme(s). This step was conducted for all documents in the selection. Articles with a similar theme were grouped until a comprehensive list of evaluation themes could be prepared with no unclassified articles. To maintain consistency throughout this process, the categorization was compared to commonly used classifications in the literature [e.g. Anjomshoae et al. (2022) and Jayadi (2024)]. The categories were refined until no inconsistency with existing classifications could be found as far as possible. For instance, Anjomshoae et al. (2022) and Jayadi (2024) do not consider any social-related measure in their categorization. Next, the 75 documents (cf. Supplementary material) were divided into three categories (cf. Table 1) based on the expertise of the author in humanitarian operations.

This section presents the literature review findings in three subsections following a top-down approach. Using the defined search query (cf. Section 3), articles that discuss evaluation in humanitarian aid (in general) were collected and analyzed. Respective results are presented in Subsection 4.1. Then, Subsection 4.2 provides more detailed findings about the state of research on indicators in humanitarian aid. Finally, Subsection 4.3 describes the state of research on indicators for PID. The rationale for following the top-down approach was twofold. First, it provided quality assurance as some papers may not explicitly discuss PID or local realities. Second, the approach could enable a broader discussion on the state of research on evaluating humanitarian aid, thereby providing more opportunities for identifying future research avenues.

Figure 1 provides a visual representation of the different evaluation frameworks and their key components that were found in the review. This hierarchical structure helps to understand the relationships between different approaches and their main elements. The frameworks are discussed in the following.

The Balanced Scorecard has been used widely for humanitarian contexts to provide a comprehensive view of organizational performance [e.g. Anjomshoae et al. (2017) and Agarwal et al. (2022)]. The approach incorporates financial and nonfinancial indicators across four perspectives: financial, customer (beneficiaries), internal processes and learning and growth (Anjomshoae et al., 2019; Wagner et al., 2021). The framework helps humanitarian organizations to align their operations with strategic objectives, although its application in humanitarian settings often requires modifications to address the unique challenges these organizations face (Wagner et al., 2021).

Another widely used framework is the Logical Framework Approach (LFA). Due to its straightforward structure, LFA allows for easy communication among stakeholders (Ediae et al., 2024). It also emphasizes a clear articulation of objectives, activities and expected outcomes. It was also observed in the reviewed papers that some scholars refer to results-based management (RBM) as their guiding approach for evaluation. RBM focuses on achieving specific results and impacts and encourages using indicators to measure progress toward predefined goals, thus fostering accountability (Mazuryk and Shyrokova, 2024). Moreover, the literature introduces various Performance Measurement Systems (PMS) developed specifically for humanitarian logistics and supply chain management, focusing on operational performance during disaster response and recovery phases. These systems often categorize indicators into qualitative and quantitative measures, addressing the efficiency of operations and aid delivery (Santarelli et al., 2015).

Several articles propose using the well-established Analytical Hierarchy Process to prioritize indicators in humanitarian evaluation frameworks based on qualitative and quantitative assessments. According to Anjomshoae et al. (2019), the method is particularly useful in complex decision-making scenarios where multiple criteria must be considered. It enables humanitarians to derive an overall performance score from various indicators.

Mainstream research on indicators for humanitarian aid comes from the operations management discipline focusing on humanitarian supply chains. This literature has focused narrowly on supporting humanitarian decision-makers by optimizing short-term indicators like cost-effectiveness and demand coverage in the distribution of relief materials (Beamon and Balcik, 2008; Kovács and Tatham, 2009; De Vries and Van Wassenhove, 2020; Abidi et al., 2014; Banomyong et al., 2019; Abidi et al., 2020; Cardoso et al., 2023). Frequently used indicators were response time (e.g. average time taken from the onset of a disaster to the arrival of the first supplies), cost efficiency, needs satisfaction (e.g. demand coverage and reliability of information) and operational effectiveness (e.g. percentage of aid delivered ontime, the accuracy of supply inventories, level of coordination among SC actors). Some composite indicators have also been proposed to evaluate the overall performance of humanitarian operations. These indicators aggregate multiple performance dimensions, such as response time, beneficiary satisfaction and resource utilization, into a single score (Cardoso et al., 2023; Wagner et al., 2021). This approach facilitates a more holistic evaluation of humanitarian operations (Abidi et al., 2020).

These studies apply restricted definitions of performance metrics (e.g. cost-effectiveness or efficiency) without challenging underlying aid assumptions and priorities (Cardoso et al., 2023). For instance, indicators such as cost-effectiveness are inadequate to depict or forecast impacts on long-term needs. These indicators lack meaningful participation and engagement of the affected communities (Rossignoli et al., 2017).

Moreover, the majority of existing research prioritizes quantitative measures and outcomes that can be easily assessed, reflecting a positivist approach that values objectivity and generalizability. The mathematical methodologies that use such measures to support humanitarians rely on top-down optimization and static designs based on rigid assumptions (Gutjahr and Nolz, 2016; De Vries and Van Wassenhove, 2020). As such, current decision support systems based on quantitative models in humanitarian research have significant limitations in adapting to local contexts as they overlook the complex and context-specific nature of humanitarian interventions (Rodríguez-Espíndola et al., 2023). For instance, a lack of comprehensive frameworks for evaluating local aspects, such as gender equality and women’s empowerment in humanitarian settings, is a critical gap (Jia and Williamson, 2019; Reinsberg and Swedlund, 2023). Overlooking the deep-seated sociocultural, political and historical contexts of regions receiving aid leads to inefficiencies and occasional mismatches between the assistance provided and the actual needs of the targeted community (Daar et al., 2018; Reinsberg and Swedlund, 2023).

Some studies suggest using indicators from development contexts to evaluate aid effectiveness [e.g. Jia and Williamson (2019)]. One commonly suggested indicator for aid coming from development studies [e.g. Headey (2008) and Walther (2020)] is economic growth. The development literature argues that there is a positive long-run relationship between development projects and economic growth, indicating that increasing such projects can have a sizable impact on long-term economic development (Minoiu and Reddy, 2010). Another commonly suggested indicator is human welfare, measured by the Human Development Index and infant mortality rates. Research asserts that development projects contribute to higher human development and lower infant mortality, particularly in countries below the median welfare distribution (Neanidis, 2012). However, development and humanitarian aid have critical differences; for example, development projects do not necessarily imply interdependence and coordination among several organizations, as in humanitarian aid (De Cordier, 2009). Thus, indicators for development cannot be directly applied to aid as they fail to capture the complexity of aid.

Critical perspectives and indigenous concepts can be crucial to transforming embedded power relations and priorities in humanitarian aid (Crothers, 2014). Some studies highlight the need for humanitarian and international development fields to consider the interventions of black and Indigenous theory [e.g. Pallister-Wilkins (2021)]. By incorporating these perspectives, it becomes possible to challenge the effectiveness of traditional aid approaches and consider the desires and aspirations of local grassroots social movements. The shift can help to transform embedded power relations and prioritize the needs of marginalized communities (Pallister-Wilkins, 2022; Midgley, 2023). Moreover, the literature stresses that understanding the characteristics, determinants and perspectives of experienced medical humanitarians is essential for enhancing humanitarian efforts [e.g. van Voorst et al. (2022)]. By eliciting relevant information from participants’ perspectives, including their philosophical and ideological underpinnings, it becomes possible to reinforce collective motives and efforts (Pallister-Wilkins et al., 2023). The approach can improve coordination, collaboration and accountability within the sector, but they are not referred to in the literature for developing localized indicators.

Table 2 synthesizes the abovementioned findings into three main research focus areas, highlighting the indicators used, the limitations identified and the key references for each area.

Although a burgeoning body of research argues for localized humanitarian strategies, proposed localization alternatives retain problematic binaries between international and local actors (Roepstorff, 2020; Frennesson et al., 2021; Frennesson et al., 2022; Lucatello and Gómez, 2022). Most academic discussions around humanitarian aid reflect predominantly Western perspectives, overlooking the voices and perspectives of the affected communities (Roepstorff, 2020). Although critical management theories, such as the Stakeholders Management theory, illuminate issues of power and marginalization in humanitarian assistance (Oloruntoba et al., 2019; Prakash et al., 2020), they do not support how to address them (Lee, 2008). According to the literature, various stakeholders affect humanitarian aid, including beneficiaries, humanitarian organizations, governments, nongovernmental organizations, volunteers and the private sector (Kovács and Spens, 2009). Collaboration and coordination among these stakeholders are essential in effectively managing and distributing humanitarian aid [e.g. Xu et al. (2021)]. To realize the inclusion of diverse stakeholders, Freeman’s grid of power and interest in the Stakeholders Management theory (Freeman et al., 2018) has been used widely in the literature. However, according to Freeman’s grid, aid beneficiaries should only be kept informed because they belong to the stakeholders’ category of high interest – low power. This classification does not prioritize beneficiaries as the center of humanitarian aid (Eggert, 2018; Rahmani, 2012).

The state of research on protection in the humanitarian literature is multidisciplinary. The literature emphasizes integrating protection concerns into humanitarian programming and policies (Francis and Maguire, 2016). It suggests tools and guidelines, such as The Sphere Project, to support protection in humanitarian assistance (Cetinoglu and Yilmaz, 2021). Research also highlights the ethical challenges and uncertainties associated with using information and communication technologies (ICTs) in humanitarian aid (Hunt et al., 2016), specifically concerning privacy, security and inequalities. Ethical considerations, such as avoiding harm, protecting relationships and addressing expectations, are crucial in the design, application and evaluation of ICTs in humanitarian contexts (Hunt et al., 2016; Horst and Nur, 2016).

Moreover, research suggests empathy can shape humanitarian protection work, particularly in negotiations with affected populations. It also calls for a relational approach to engagement in frontline humanitarian negotiations [e.g. Sutton and Paddon Rhoads (2022)]. Evidence from the field supports that understanding and responding to the needs, concerns and emotions of individuals affected by crises can contribute to more effective protection measures (Sutton and Paddon Rhoads, 2022). However, how protection can be evaluated in aid has remained unanswered.

Research about inclusiveness in humanitarian literature is scarce and varied. Some studies have focused on the impact of inclusiveness in refugee settlements and highlighted the importance of international coordination and generous humanitarian assistance packages in creating peaceful conditions in refugee camps (Moore et al., 2003; Guo et al., 2020). Other research has explored the role of sympathy, perceived threat and perceived contribution in shaping the provision of humanitarian assistance and acceptance of permanent settlements for asylum seekers (Thravalou et al., 2021).

Furthermore, research has examined the intersection of inclusiveness with advancing the rights and inclusion of persons with disabilities in humanitarian action (Mojtahedi et al., 2023). Regarding aid types, studies find that aid for social infrastructure and program assistance were complementary to inclusive human development in Africa (Asongu and Odhiambo, 2019). Yet, research has not provided evidence that answers whether any of the abovementioned concepts could be used to evaluate inclusiveness in aid (for instance, can “providing different types of aid” be an indicator of inclusiveness?).

Most existing research on dignity is scattered across different disciplines, including law, political science and nursing. The humanitarian literature concerning the concept is relatively new and limited and has remained at a conceptual level [e.g. Baharmand et al. (2021), Talhouk et al. (2019), and van Wynsberghe and Comes (2020)]. Scholars have examined the use of ICT in humanitarian contexts and highlighted the importance of considering the risks and benefits of technology use in relation to maintaining the dignity of aid recipients (van Wynsberghe and Comes, 2020). Some scholars argue that maintaining dignity should be an inherent aim in the design of humanitarian technologies and call for critical reflection by designers and participants on how technologies can produce tangible outcomes that facilitate dignified interactions (Talhouk et al., 2019).

Moreover, only a few studies (Allahi et al., 2020; Arduin and Saïdi-Kabeche, 2022) have explored the dimensions and components of dignity in the context of humanitarian aid. Examples include focusing on overall health and social security, education and improved coping strategies (Allahi et al., 2020) or concentrating on food assistance as an approach to promote the safety, dignity and integrity of beneficiaries while integrating their protection needs (Arduin and Saïdi-Kabeche, 2022). However, the literature lacks consensus or evidence about how dignity can be evaluated effectively.

Although localized indicators in humanitarian aid are needed to consider the specific needs and resources of the local communities affected by crises, the review shows that universal assessment frameworks in practice do not adequately capture the unique context and challenges faced by different communities [e.g. Matopoulos et al. (2014)]. According to reports from practice, humanitarians try to address the gap by stressing that interventions should comply with the imperative of do-no-harm (Sandvik et al., 2017). The do-no-harm imperative emphasizes the importance of minimizing harm to affected populations and ensuring that interventions do not exacerbate existing vulnerabilities or create new ones (Khaled, 2021). By incorporating the do-no-harm imperative, humanitarians have aimed to ensure that their interventions are effective, ethical and responsive to the needs of the affected populations.

However, the imperative has been criticized for several matters. First, it may not serve the purpose of localized aid as it could imply the preference for indirect harm over direct harm (Sandvik et al., 2017; Khaled, 2021). Direct harm refers to immediate negative consequences that may result from humanitarian interventions, such as physical injuries or violations of human rights (Wessells and Kostelny, 2021). Indirect harm refers to unintended negative consequences from humanitarian interventions, such as negative impacts on local economies or social structures (Sandvik et al., 2017). Moreover, it has received criticism for being normatively insufficient [e.g. Hugman et al. (2011)] – i.e. it does not address inclusiveness. In addition, some suggest that the imperative is subject to irrelevant bias, particularly the action/omission bias (Baron, 1996) – i.e. it has shortcomings in addressing dignity.

Instead, indicators that focus on providing assistance to beneficiaries should be considered (Masud and Yontcheva, 2005). Understanding local perceptions provides concrete insights into the relevance and effectiveness of aid interventions. By understanding the context in which aid is delivered through actively seeking feedback from beneficiaries, humanitarian aid agencies can better tailor their programs to meet the needs of the communities they serve. Hearing the voices of beneficiaries and local people from the beginning of interventions guarantees that their perspectives and experiences are considered, leading to more targeted interventions. Engaging community leadership is necessary for tailoring the aid, specifically to ensure that beneficiaries’ PID are considered in designing and implementing interventions.

The review shows that there are certain challenges with current, widely used evaluation frameworks. The LFA’s rigidity can limit adaptability in dynamic humanitarian contexts (Renzaho, 2007). RBM has been criticized for its potential to oversimplify complex humanitarian situations, leading to a narrow focus on quantifiable outcomes at the expense of qualitative insights (Abidi and Scholten, 2015). The empirical testing of PMS has revealed a low utilization of existing performance indicators, highlighting the need for more robust frameworks (Santarelli et al., 2015).

Moreover, current evaluation frameworks are heavily influenced by universalist assumptions that are deeply rooted in the legacies of colonialism (Tabar, 2016). They assume a universal understanding of what constitutes effectiveness in humanitarian aid and prioritize indicators that can be easily measured and compared across different contexts (e.g. the number of beneficiaries reached) (Easterly and Williamson, 2011; Jia and Williamson, 2019). As such, the frameworks fail to consider the diverse cultural, social and political contexts in which interventions occur (Asgary and Lawrence, 2014) and the voices and perspectives of the affected communities (Roepstorff, 2020). This critical gap calls for a thorough shift from current one-size-fits-all evaluation frameworks toward incorporating historically overshadowed perspectives (e.g. indigenous concepts and localized worldviews).

Furthermore, the review shows the potential of digital tools and data analytics in evaluation frameworks, specifically for enhancing the collection and analysis of indicators. Although the need for digital skills in the humanitarian sector has been noted before [e.g. Birch and Miller (2005)], digitalization of humanitarian evaluation is largely untapped (Jayadi, 2024). Research should explore how technology can improve data management and facilitate real-time monitoring of humanitarian interventions.

Without involving beneficiaries, local ownership and empowerment will not be realized in target communities (Pasha, 2020). Ownership refers to the involvement and participation of the target people in decision-making processes and the overall management of the provision of aid (Pouligny, 2009). It recognizes the importance of local knowledge, expertise and perspectives in shaping effective and sustainable interventions (Pascucci, 2017). Research suggests that ownership is key to promoting empowerment among aid beneficiaries. When beneficiaries have a sense of ownership over the aid programs, they are more likely to feel empowered and take active roles in shaping their own development (Pasha, 2020). This can lead to increased self-confidence, agency, and the ability to make decisions that positively impact their lives (Asgary and Lawrence, 2014). Furthermore, ownership can also contribute to the sustainability of aid interventions, as local communities are more invested in the long-term success of the programs (Pouligny, 2009).

Results from thereview reveal that current frameworks fail to consider a system of involved actors, including beneficiaries, and to see the connection between elements in the aid system. As much as coordination among different international and national actors (e.g. humanitarian agencies, nongovernmental organizations, local authorities, volunteers, beneficiaries, local businesses, etc.) is essential for delivering assistance (Jahre and Jensen, 2021), including the voices of beneficiaries and local communities in the evaluation is crucial. They are the main stakeholders of aid (Kovács and Spens, 2009) and represent a key actor with a deep understanding of the local context.

Beneficiaries’ involvement in the evaluation process empowers them and allows for a more comprehensive understanding of the impact of aid interventions. Therefore, it is crucial to create an enabling environment that promotes meaningful participation and decision-making by the beneficiaries (Kovács et al., 2010). This can be done through inclusive and participatory approaches that involve the beneficiaries in all stages of the aid process, from design to evaluation (Moore et al., 2003).

Despite significance, to the best of author's knowledge, existing humanitarian evaluation frameworks provide no explicit and transparent means to assess whether aid is making (or will make) an impact on PID. This imbalance limits the meaningful participation of local communities in the evaluation process and undermines their agency and self-determination (Banomyong et al., 2019). Moreover, there is a lack of attention to the ethical considerations in humanitarian frameworks (Baharmand et al., 2021). The development and application of new emerging technologies (e.g. ICTs) for delivering assistance raise ethical challenges related to avoiding harm, ensuring privacy and security, addressing inequalities and protecting relationships (Hunt et al., 2016). Current indicators cannot ensure that humanitarian aid is delivered in an ethically responsible manner.

Developing indicators for PID, however, is challenging because these concepts could be interpreted differently based on aid modalities (in-kind vs cash and voucher), delivery mechanisms (in-person vs collectively vs digitally) and disaster types (e.g. refugee crisis vs earthquake) (Young and Maxwell, 2013; Horst and Nur, 2016; Asongu and Odhiambo, 2019; Young, 2023; Allahi et al., 2020; Affolter and Allaf, 2013). For example, observations from cash-based assistance to refugees in settlements using mobile money show that protection might be assessed by looking at the security challenges and risks the targeted refugees face after receiving the money. In a similar context, inclusiveness might evaluate whether all beneficiaries have the digital literacy and capacity to use mobile money daily. Finally, dignity indicators could be whether the states of health and social security, education or coping strategies change before and after the assistance is delivered (Khan et al., 2023).

Furthermore, the definitions and provided sample indicators imply that PID are interconnected and mutually supportive. Protection is essential for upholding the dignity of affected populations, as it ensures their safety and well-being. Inclusiveness is crucial for promoting the participation and agency of affected communities, which in turn contributes to their protection and dignity. As such, it could be possible that the evaluation outcomes of one concept (or two concepts) among PID could provide a good proxy for the other(s). Thus, if the hypothesis holds, there would be no need to evaluate all three concepts of PID in every intervention. This is important because constrained resources (e.g. personnel) and limited access (e.g. conflict and post-conflict settings) are often blamed in humanitarian reports as the main barriers to evaluating PID [see, e.g. Goulart et al. (2021)]. However, such a causality between PID could only be tested with a variety of tested and validated, universally accepted indicators.

Humanitarian aid plays a crucial role in saving lives and alleviating the suffering of disaster-affected people. The effectiveness of such interventions often relies heavily on evaluation frameworks. These frameworks are essential for ensuring accountability and guiding future interventions. This literature review aims to synthesize existing research on evaluation frameworks in humanitarian aid, with a specific interest in the indicators for protection, inclusiveness, and dignity (PID). Protection is essential for ensuring the safety and security of affected populations during humanitarian crises. Inclusiveness in humanitarian aid refers to how aid interventions consider the diverse needs of affected populations, including marginalized and vulnerable groups. Dignity is a fundamental principle of humanitarian action and its measurement is increasingly recognized as vital for effective aid delivery. Assessing PID is necessary for tailoring aid to the context (Guillaumont and Chauvet, 2019), which will, in return, considerably increase its reception as effective among beneficiaries as the main stakeholders.

The review reveals that while incorporating localized, community-driven indicators is essential for tailoring humanitarian assistance to specific contexts, current practices often prioritize universal, easily measurable metrics over locally relevant ones. Existing evaluation frameworks, such as the Balanced Scorecard and Logical Framework Approach, have limitations in fully capturing the complex, context-specific nature of humanitarian interventions. They tend to focus on quantitative, globally consistent metrics without adequately incorporating the perspectives and needs of local communities. The results also show that research on indicators related to PID in humanitarian aid is scarce and fragmented. While these concepts are recognized as critical to ensuring the effectiveness and ethical delivery of aid, there is a lack of consensus on how to evaluate them meaningfully. Furthermore, the meaningful involvement of aid beneficiaries and local stakeholders in the evaluation process is crucial for empowering communities and ensuring the relevance and sustainability of humanitarian interventions. Current frameworks fall short in this regard, too.

The study calls for evidence-based research to address the following knowledge needs in humanitarian evaluation frameworks: the need for the recognition of localized indicators in practice, the need for context-specific evaluation frameworks, the need for the involvement of target people and the need for meaningful and universally accepted indicators for PID. Working on these directions will require close collaboration with local communities to understand their needs and perspectives and the development of widely accepted indicators that can capture the complex, multidimensional impact of humanitarian aid.

For practitioners, this study underscores the importance of incorporating localized indicators into evaluation frameworks to ensure that aid interventions are contextually relevant and effective. Humanitarian organizations should prioritize engaging with local communities to understand their specific needs and perspectives, thereby enhancing the inclusiveness and impact of aid. Training programs for humanitarian workers should include modules on the importance of PID and how to implement these concepts in practice.

Societal implications include fostering greater community resilience and empowerment. Aid interventions that respect and uphold the dignity of beneficiaries are more likely to be sustainable and effective. This approach can also contribute to social cohesion by ensuring that marginalized and vulnerable groups are included and their voices are heard in decision-making (Danquah and Ouattara, 2023).

Moreover, the need for localized indicators and context-specific evaluation frameworks should be reflected in policy guidelines and funding criteria. Policies should also promote the active involvement of local communities in the evaluation and implementation of aid programs, ensuring that interventions are effective and ethically sound. Furthermore, public policy should support the development and dissemination of best practices for incorporating PID into humanitarian aid, thereby contributing to a more accountable and transparent aid system.

This study has limitations regarding collecting documents from academic literature and field reports. A limited number of high-ranked journals were considered in this review. Future studies can expand the list to a broader level. Moreover, other types of relevant documents, such as books, conference papers and PhD theses, could be considered. Furthermore, the considered field reports in this study were collected from only one source, which could have limited the number of accessible documents. Future studies could, therefore, look into additional sources of field reports.

The author thanks the two reviewers for their insightful comments and suggestions. This study was conducted as a part of the research project DigCBA: Responsible Use of Digital Cash-based Assistance in Refugee Crises funded by the Research Council of Norway (grant no. 325437).

The supplementary material for this article can be found online.