From lessons identified to lessons learnt: systemic patterns in Aotearoa New Zealand's search and rescue exercise evaluations

Search and Rescue (SAR) organisations play an important yet historically under-examined role in Aotearoa New Zealand's (NZ) emergency management system (Curd et al., 2025). While New Zealand Police serve as the lead agency for Category 1 (locally-led, land-based) SAR operations (NZSAR, 2021a, b), much of the system's operational capability is delivered by volunteer-based search and rescue non-government organisations (SAR NGOs), including land search and rescue (LandSAR), Coastguard, surf life saving New Zealand (SLSNZ) and amateur radio emergency communications (AREC). These organisations contribute significantly to national readiness through their roles in operational response, planning, training, intelligence support, logistics and community engagement.

Recent research highlights the growing importance of SAR NGOs within multi-agency emergency management environments while also identifying recurring challenges, including variability in training uptake, uneven integration with statutory agencies and inconsistent documentation of lessons management (Curd et al., 2025). These findings highlight the need for further empirical research examining how SAR NGOs operate in practice and how organisational learning is embedded across the sector (Levitt and March, 1988; Neelen and Kirschner, 2020).

New Zealand Search and Rescue (NZSAR) summaries and district search and rescue exercise (SAREx) evaluations repeatedly note recurring issues in doctrine use, planning and information management, suggesting system-level patterns rather than one-off problems (NZSAR, 2020a, 2025a).

This analysis provides a comprehensive qualitative document analysis of 98 Category 1 Police District SAREx evaluations conducted between 2011 and 2025. Analysing these evaluations offers insights into sector-wide strengths, persistent challenges and opportunities for improving SAR readiness and capability, thereby extending the emerging evidence base on the operational functioning of SAR NGOs within NZ's emergency management framework.

Conceptually, this study treats SAREx evaluations as a distributed learning system rather than isolated documents. By synthesising longitudinal exercises across districts, the analysis distinguishes between (1) persistent capability constraints that appear structurally embedded and (2) issues that vary by context, maturity, or local implementation. This provides an evaluation basis for identifying where national-level interventions are likely to produce the greatest marginal gains in readiness and interoperability.

NZ's emergency management system operates as a mixed model in which statutory agencies lead while relying heavily on volunteer-based NGOs to deliver operational capability (New Zealand Government, 2019). In SAR, NZ Police coordinate Category 1 operations, which involve locally managed searches across land, inland waters and near-shore environments. Category 2 incidents, including missing or distressed aircraft and offshore marine emergencies, are managed nationally by the rescue coordination centre New Zealand (NZSAR, 2021a, b). Category 1 operations depend significantly on SAR NGOs, which provide most field personnel, specialist expertise and local knowledge. While this structure provides considerable operational capacity, it also introduces challenges relating to training consistency, resource availability, doctrine integration and alignment with the coordinated incident management system (CIMS) expectations (New Zealand Government, 2019).

Prior research has highlighted the high levels of commitment, local knowledge, and technical proficiency within SAR NGOs, alongside enduring gaps in capability, training standardisation and inter-agency coordination (Curd et al., 2025). Lessons from emergency events also reveal that these organisations frequently shoulder substantial planning, intelligence and logistics responsibilities, yet their contributions are not consistently analysed or documented at a national level, limiting strategic insights into system performance (Bush International Consulting, 2023; Bush International Consulting, 2024; GIRNISWE, 2024).

SAREx evaluations provide structured, practitioner-generated accounts of readiness, coordination and doctrine use. Although produced regularly, they are typically reviewed within districts rather than synthesised nationally, limiting opportunities for sector-wide learning. Research on emergency management exercises shows that after-action reporting alone rarely produces sustained improvement without structured learning loops, clear ownership and resources to translate findings into revised plans, training and doctrine (Donahue and Touhy, 2006; Rostis, 2007). This study addresses that gap by analysing Category 1 SAREx evaluations across Police districts to identify recurring themes and patterns in capability and doctrine adoption. By examining these reports collectively, the study treats SAREx evaluations as longitudinal evidence of system functioning rather than isolated exercise feedback (AIDR, 2019; McCreight, 2023; UK Government, 2025).

A qualitative document analysis was undertaken using the publicly available SAREx evaluation reports (NZSAR, 2025a) to examine patterns across NZ's Category 1 (locally led, land-based) SAREx evaluations. Document analysis is well-suited to structured operational records, enabling researchers to identify recurrent themes, assess evaluative quality and derive meaning from written products generated within practice settings (Bowen, 2009; Salminen et al., 1997). This approach allowed for the systematic synthesis of evaluation findings to understand how SAR NGOs operate within planning and response environments.

The initial dataset comprised 111 SAREx evaluation reports produced between 2011 and 2025 available on the New Zealand Search and Rescue website (NZSAR, 2025a). The chosen date range aligns with the period examined in recent research on SAR NGO roles in emergency events (Curd et al., 2025) and reflects the earliest year from which the NZSAR began making SAREx evaluations publicly accessible.

In applying the inclusion and exclusion criteria, each document was screened for relevance and suitability for qualitative analysis (Bowen, 2009; Meline, 2006).

1. Inclusion criteria: Documents needed to be related to a Category 1 (Police-led) exercise, involved multi-agency participation from SAR NGOs such as LandSAR, Coastguard, SLSNZ and AREC, were a formal evaluation containing identifiable findings or recommendations, and provided sufficient descriptive content for analysis.

2. Exclusion criteria: Documents that were non-evaluative or non-SAREx materials (e.g. planning notes, scenario scripts, attendance lists), related to Category 2 or non-SAR exercises, originated outside NZ, duplicated another report, or were incomplete or unreadable.

Following this screening process, 98 evaluation reports met the criteria and formed the final dataset for analysis.

Analysis followed an iterative, inductive process. Because the evaluation reports span fourteen years (2011–2025), coding and theme refinement explicitly incorporated cross-year comparison to identify persistent patterns rather than exercise-specific findings. All evaluations were read in full and coded for observations, issues, strengths and recommendations. Coding proceeded in four stages: familiarisation, line-by-line coding, code consolidation and theme refinement.

First, all reports were read in full to familiarise with the structure, terminology and recurrent recommendation types. Second, line-by-line coding was applied to reported issues, strengths and recommendations, using short action-orientated labels (e.g. “delayed SITREP”, “demob plan absent”, unclear staging vs ICP”) Third, codes were consolidated into candidate categories through constant comparison across districts and years, with codes merged where they represented the same underlying construct and separated where they reflected distinct operational processes. Fourth, categories were refined into eight themes through iterative review to ensure internal coherence and clear boundaries, with decision rules documented in analytic notes (e.g. “comms infrastructure” coded as Communications; “fragmented COP” coded as Information Management and Situational Awareness) (Bowen, 2009; Scriven, 2007; Yarbrough et al., 2011).

A theme was treated as system-wide when it appeared across multiple police districts and was evidenced in multiple years rather than from a single exercise context. Where themes varied by setting (e.g. marine vs land, alpine vs urban, etc.), this was recorded as contextual variation rather than treated as contradictory evidence.

Scriven's (2007) Key Evaluation Checklist (KEC) and the Program Evaluation Standards (PES) (Yarbrough et al., 2011) were used as guiding frameworks to ensure that the analytical process was systematic, transparent and evidence-grounded. These frameworks did not determine document selection. Instead, they provided information about the accuracy of coding, interpretation and thematic analysis. By emphasising standards such as accuracy, utility, propriety and defensible evaluative judgement, KEC and PES provided a methodological frame that enhanced the credibility and consistency of the document analysis.

KEC and PES were operationalised as quality checks applied during analysis and reporting (Scriven, 2007; Yarbrough et al., 2011). Accuracy and justification were addressed through traceability between paraphrased examples and source documents, and by retaining an audit trail of coding and consolidation decisions. The utility informed organisation of themes around actionable capability domains used in SAR readiness and doctrine. Propriety guided paraphrasing and de-identification to protect operational confidentiality while preserving meaning.

To maintain accuracy, analytical decisions were recorded throughout, and themes were checked for coherence and consistency. Coding and theme development were conducted across the full dataset rather than selected excerpts, allowing patterns to be assessed longitudinally across districts and years. This approach provides a comprehensive, replicable approach for analysing more than a decade of SAREx evaluations, generating new insights into SAR NGO capability, performance and integration within NZ's emergency management system (Bowen, 2009; Salminen et al., 1997).

Coding was undertaken by a single researcher. A second coder was not used because the SAREx evaluations are structured operational documents with clearly stated observations, issues and recommendations, rather than open-ended interview transcripts requiring extensive interpretive analysis. The focus of the study was on identifying recurring patterns across reports, not on reconstructing individual meaning or intent. Reliability was supported through documented coding rules, comparison across districts and years and the maintenance of an audit trail of theme development decisions. Reflexive consideration was applied throughout to ensure that themes were grounded in the written evidence and not shaped by assumptions beyond the documented material (Scriven, 2007). Although the documents were publicly available, ethical principles of respect and responsible representation were observed, and no individuals, teams, or specific exercises are identifiable (Guba and Lincoln, 1981).

Analysis of the 98 SAREx evaluation reports revealed eight recurring themes that consistently shaped exercise performance across the 2011–2025 period. The themes identified were:

1. Training and capability development,

2. Planning and preparedness,

3. Information management and situational awareness,

4. Inter-Agency coordination,

5. Communications,

6. Safety and risk management,

7. Logistics and resource management and

8. Continuous improvement.

Table 1: Prevalence of themes across SAREx evaluation reports summarises the prevalence of each theme across the dataset, demonstrating that all eight themes appeared across a substantial portion of evaluations, confirming their persistence over time.

As shown in Table 1, all eight themes appeared across a substantial proportion of SAREx evaluations, confirming their persistence over the fourteen-year period. Training and capability development (89 of 98 reports; 90%) and planning and preparedness (78 of 98 reports; 79%) were the most frequently identified themes, highlighting consistent challenges in building and sustaining operational competence and readiness. Continuous improvement was also prominent (74 of 98 reports; 75%), reflecting strong reflective practice but, when considered alongside the recurrence of other themes, suggests limitations in embedding lessons over time. Information management and situational awareness (54 of 98 reports; 55%) and logistics and resource management (36 of 98 reports; 37%) appeared in more than one-third of reports, indicating ongoing operational constraints, while inter-agency coordination (45 of 98 reports; 45%), safety and risk management (34 of 98 reports; 34%) and communications (30 of 98 reports; 30%) further indicate that system-level issues extend beyond isolated exercises or districts.

Training and capability development emerges as a predominant theme across the SAREx evaluations, identified in 89 of the 98 reports analysed (90%). Many evaluations reported variable confidence and competency across incident management team (IMT) roles, particularly intelligence, planning and logistics. While experienced personnel were frequently identified as strengths, several districts relied heavily on a small number of skilled individuals. This dependence limits organisational depth, reduces resilience during complex or extended incidents and constrains succession planning (Boin et al., 2013).

Variability in the adoption and application of national doctrine, including CIMS and NZSAR Response Guidelines, also contributed to uneven performance (New Zealand Government, 2019; NZSAR, 2020b). Evaluations showed that personnel were not always confident in understanding doctrine, terminology, or expected intelligence products. In some cases, IMT members defaulted to informal, ad hoc, or locally developed practices rather than national endorsed processes or international best practice and indicating doctrine-practice misalignment and a lack of shared mental models across districts (NZSAR, 2025a).

These findings point to a strong need for targeted development pathways that go beyond general training delivery. Evaluations repeatedly recommended structural monitoring arrangements, scenario-based practice within multi-agency teams and clearer expectations for progression into specialised function-specific roles. Building capability in these functions requires technical training as well as embedded opportunities for deliberate practice, coaching and post-exercise reflection.

Collectively, these capability gaps highlight that current training resources, particularly for land-based SAR, are not consistently enabling personnel to develop and maintain the competencies required for specialised IMT roles. Variability in how doctrine is interpreted and applied suggests that training materials differ in emphasis, accessibility and alignment with national expectations (New Zealand Government, 2019; NZSAR, 2023, 2020b, 2021a, b; NEMA, 2025a, b). As a result, several evaluations called for a systematic review of SAR training resources to ensure that they provide clear, function-specific guidance and support ongoing development beyond introductory courses (NZSAR, 2020a). Strengthening training consistency and implementing advanced, role-specific pathways would help distribute capability more evenly across districts and improve IMT resilience during both exercises and real-world events (Comfort, 2007).

Planning and preparedness quality varied considerably across evaluated SARExs and was identified in 78 of the 98 reports analysed (79%). Many exercises lacked clearly articulated aims, SMART objectives, CIMS alignment, or alignment with the NZSAR guidelines and templates (NZSAR, 2023, 2020b). In several cases, IMTs commenced operations without a shared understanding of operational intent, key decision points, or expected outputs for each planning cycle. Although districts possessed pre-existing plans for high-risk environments (e.g. avalanche, canyoning and coastal response plans), these were often under-utilised or not referred to at all during the exercise, limiting their intended function as accelerators of early operational planning (Kapalo and LaViola, 2019).

Evaluations frequently recommended updating readiness plans, integrating iwi and community partners into preparedness activities and ensuring IMT personnel were familiar with pre-existing plans. This reflects the wider pattern of uneven preparedness at the district level, which directly influenced how efficiently IMTs were able to establish themselves, set initial priorities and initiate structured planning cycles (New Zealand Government, 2019). Where preparedness was weak, IMTs took significantly longer to articulate operational intent, populate critical IMT roles and develop coherent objectives and tasking strategies (Goff, 1987; Gustavsson et al., 2010)

These challenges mirror broader emergency management evidence demonstrating that pre-event preparedness, including clarity of roles, establishing relationships, shared expectations and familiar planning tools, is a key predictor of coordination effectiveness during real-world events (Comfort, 2007). In NZ's SAR context, variability in district-level preparedness represents both an operational risk and a strategic opportunity for system-wide improvements.

Information management and situational awareness was identified in 54 of the 98 reports analysed (55%) and emerged as one of the most consistent systemic vulnerabilities. Many evaluations reported delayed or incomplete logging, inconsistent SITREPs, fragmented information flows and limited analytical use of incident management systems (IMS) and related digital systems. Intelligence functions were often under-resourced or insufficiently defined, limiting the development of a coherent common operating picture (COP) (New Zealand Government, 2019). These challenges mirror evidence that information management failures degrade operational rhythm and decision quality (Boin et al., 2013; Young, 2021). Evaluators repeatedly recommended improved digital literacy, additional multi-screen setups within IMTs and the adoption of visual tools such as A0 planning boards and workflow diagrams to support situational awareness.

Inter-Agency Coordination was identified in 45 of the 98 reports analysed (45%) and was consistently identified as a key relational strength. Across datasets, Police, LandSAR, Coastguard, SLSNZ and AREC generally demonstrated strong relationships and willingness to collaborate. However, despite these positive interpersonal dynamics, operational integration varied. In several exercises, agency roles, responsibilities and decision-rights were not consistently understood, and joint briefings or shared situation assessments were delivered unevenly across agencies. Common terminology within CIMS was also used inconsistently (New Zealand Government, 2019). For example, some IMTs used the term “forward base” to describe an operational location outside the main IMT, even though under CIMS 3rd edition, this function should have been differentiated as either an incident control point, assembly area, safe forward area, or a staging area, depending on purpose. This imprecise use of terminology created ambiguity about where coordination, tasking and safety responsibilities were held, and sometimes obscured the distinction between the location of incident control and forward deployment points.

Evaluations recommended increasing the frequency of multi-agency SARExs and IMTEx events, formally involving iwi representatives and strengthening shared understanding of CIMS, agency mandates and capabilities.

Communication challenges were identified in 30 of the 98 reports analysed (30%). Evaluations documented radio congestion, poor repeater coverage, outdated or incomplete Primary, Alternate, Congingency, Emergency (PACE) plans and inconsistent radio discipline. The interoperability between marine and land agencies was particularly variable, with IMS, logging systems and radio systems not always aligned in practice. Several exercises reported the withdrawal of communications support while field personnel were still deployed, creating periods of reduced situational awareness and increased safety risk.

These issues directly affected tasking efficiency, field coordination and the ability of IMTs to maintain a coherent operational picture. Evaluations recommended district-wide communications planning, geographic mapping of repeater coverage, earlier involvement of comms specialists, such as AREC, in planning, dedicated night-time or low-visibility communication exercises and PACE planning to become mandatory, consistent with international best practice (CISA, 2023).

Safety and Risk Management concerns were identified in 34 of the 98 reports analysed (34%). While safety culture across SAR NGOs was generally strong, evaluations noted increasing attention to formal risk management over the past decade, reflecting legislation changes and organisational learning (HSWA, 2015; Neelen and Kirschner, 2020). However, risk documentation was often incomplete or inconsistent. Dynamic risk assessments were usually undertaken verbally but not recorded, and task-specific hazards were not always documented within IMT processes.

Evaluations noted that specialist environments, such as avalanche, canyoning, caving, alpine cliff rescue and marine night operations, required clearer Subject Matter Expert guidance and stronger adherence to established standards including Safety Management Systems In several exercises, the absence of structured hazard documentation and specialised guidance increased reliance on individual judgement rather than shared operational controls, creating variability in safety practices across teams and context (AREC, 2017; Coastguard New Zealand, 2015; LSAR, 2023; New Zealand Police, 2024; SLSNZ, 2025).

Logistics and Resource Management challenges were identified in 36 of the 98 SAREx evaluations (37%). Common issues included delays in establishing ICPs, gaps in equipment availability, limited numbers of radios and laptops and challenges in staging and tracking resources, including personnel. In several districts, logistics issues constrained the speed and coordination of the IMT, particularly during complex or geographically dispersed exercises.

Evaluations noted limited understanding of key logistics sub-functions as defined in CIMS, which contributed to delays in resource deployment, inconsistent tracking of personnel and assets and inefficiencies during demobilisation. A lack of clarity around responsibilities, such as resourcing, transport, staging, supply and technical support, often created avoidable bottlenecks within the IMT and reduced overall operational rhythm.

Continuous Improvement was identified in 74 of 98 SAREx evaluations (75%), making it one of the most prevalent themes across the dataset. While reflective practice was evident and many reports demonstrated thoughtful debriefing and self-assessment, similar issues recurred across multiple districts and across years. This pattern suggests that lessons identified were not consistently embedded into organisational practice and indicates limited uptake or application of existing lessons management frameworks within the sector (AIDR, 2019; Archer et al., 2021; Donahue and Touhy, 2006; NEMA, 2025; NZSAR, 2021b; Resilience NSW, 2021). Instead of generating continuous, system-level improvements, lessons often appeared to remain within local exercise cycles, with limited mechanisms ensuring their translations into formal doctrine, updating training, or organisational change (Guba and Lincoln, 1981; Rhodes and Dawson, 2013; White and Cohan, 2010; Fullwood and Rowley, 2021).

Multiple evaluations demonstrated that the same capability gaps, such as unclear IMT roles, inconsistent tasking processes, communication breakdowns, or fragmented information management, reappeared across districts and across years. This persistence reflects a well-documented challenge in emergency management, the lessons identified versus lesson learnt gap, where insights are collected but not institutionalised through deliberate organisational processes (AIDR, 2019; Archer et al., 2021; McCreight, 2023; McLennan et al., 2016; NEMA, 2025; NZSAR, 2025b; Resilience NSW, 2021; Sinclair et al., 2012). Recommendations repeatedly called for practices known to support lessons management and adoption, including routine use of IMT exercise follow-ups and structured district learning reviews, systematic updating and exercising of readiness plans and the national standardisation of core IMT forms and templates to reduce procedural variability.

Despite strong local commitment to improvement and the example of high-quality evaluative reflection, the endurance of long-standing issues suggests the absence of a sector-wide learning system capable of ensuring that repeated lessons lead to sustained change. This is consistent with broader finding in the emergency management literature which emphasise that without formal governance, defined accountability and structured feedback loops, lessons remain single loop, correcting immediate issues but failing to alter underlying systems, doctrine or organisational behaviour (AIDR, 2019; Archer et al., 2021; Boin et al., 2013; Fullwood and Rowley, 2021; Guba and Lincoln, 1981; McLennan et al., 2016; Owen et al., 2018; Resilience NSW, 2021; Rhodes and Dawson, 2013; Roast, 2021; Sinclair et al., 2012; White and Cohan, 2010). For NZ's SAR sector, this highlights a need for more coordinated, formalised and nationally supported lessons management processes to ensure that insights from SAREx evaluations are systematically captured, tracked and embedded into training, doctrine and operational practice.

Overall, the patterns observed across 98 SAREx evaluations reveal a SAR system that is mature in its intentions but uneven in its operational execution. The themes show not only isolated shortfalls, but an interconnected set of organisational, doctrinal and structural factors that shape how SAR NGOs function within NZ's broader emergency management system. Interpreting these findings provides insight into three key issues: capability asymmetry, systems fragmentation and the limits of a volunteer-dependent doctrine environment.

What the findings suggest most clearly is that operational capability remains unevenly distributed across districts and agencies, with critical functions frequently concentrated among small numbers of experienced personnel. This dynamic creates a structural vulnerability: capability resides in people rather than systems (Fullwood and Rowley, 2021; McLennan et al., 2016; Pratt, 2002). It means that IMT performance varies significantly depending on who is present, rather than an understanding of doctrine, training progressions, or organisational processes.

For SAR NGOs, whose volunteer ability is inherently variable, this disproportionateness becomes an operational risk (Milbourne, 2013). It reinforces arguments in literature that volunteer SAR organisations require not only technical training but organisational learning systems that can distribute capability more evenly (Catts and Chamings, 2006; Fait and Sakka, 2021; Hammond, 2024; Kapucu, 2006; Waugh and Streib, 2006).

A second interpretive insight emerging from the analysis is fragmentation, where agencies share strong relationships but not always shared ways of working (Espejo, 2002; Kujala et al., 2016; McLennan et al., 2016). Coordination often consistently indicates it relies heavily on interpersonal familiarity and goodwill rather than seamlessly interoperable doctrine, technology and procedures (NZSAR, 2023). This is consistent with literature showing that inter-agency cohesion in emergency management is most effective when relationships are paired with standardised processes, cross-agency exercises and common language (Boin et al., 2013; Catts and Chamings, 2006; Comfort, 2007; Fait and Sakka, 2021; Kapucu, 2006; McLennon et al., 2016).

Although CIMS provides a national coordination framework, its application across logistics, safety, planning and intelligence, remains inconsistent (New Zealand Government, 2019). As a result, strong relationships do not always translate into consistent operational performance. Coordination depends as much on shared methods as on shared intent (Goff, 1987; Gustavsson et al., 2010). This fragmentation is particularly evident in information management. According to a recent report (NZSAR, 2023) there are at least 65 platforms and supporting applications currently used across agencies, with no single system covering all IMT functions in any one organisation. These multiple, non-integrated platforms such as mapping tools, notification systems, communications, document management, tracking apps, etc., introduce an underlying risk of inefficiency, delayed information flows and fractured situational awareness (NZSAR, 2023). This multi-system landscape exacerbates the challenges identified in the evaluations, when IMTs and partner agencies are required to switch between multiple tools, the clarity of the COP is compromised and tasking, tracking and coordination have become less efficient. The resulting doctrine-practice gap is therefore not only procedural but also technological, highlighting the need for sector-wide alignment of systems, training and governance.

The recurring nature of many issues over fourteen years speaks to the limits of lesson management in volunteer-led environments. Although reflective practice is evident in debriefs, the evaluations show that the cycle from “lessons identified” to lessons learned” remains incomplete. This mirrors global emergency management research, where lessons systems often fail due to resource constraints, leadership turnover, unclear responsibilities, or lack of institutional mechanisms to operationalise recommendations (AIDR, 2019; Archer et al., 2021; McLennan et al., 2016; NEMA, 2025; NZSAR, 2025b; Sinclair et al., 2012; Resilience NSW, 2021).

For SAR NGOS operating across diverse districts with varying capacities, time pressures and governance arrangements, embedding lessons requires more than documenting them; it requires national support, stronger readiness planning, formalised feedback loops and strategic investment in shared training and digital systems (AIDR, 2019; NEMA, 2025; NZSAR, 2023; UK Government, 2025). These recurrences of similar issues suggest that learning remains largely document-based rather than operationally embedded (Fullwood and Rowley, 2021; Guba and Lincoln, 1981; Rhodes and Dawson, 2013; White and Cohan, 2010).

These findings reveal a broader issue of doctrine-practice misalignment. While national guidance exists for planning processes, intelligence products, communication, safety management and resource coordination (NZSAR, 2023, 2020b), implementation varies considerably across contexts. This gap between formal doctrine and operational practice is not unique to SAR, it reflects a wider challenge in emergency management where doctrine is necessary but insufficient without practical, routinised training, mentoring and organisational culture that values procedural consistency (AIDR, 2019; NEMA, 2024; NZSAR, 2025b; Owen et al., 2018; Roast, 2021).

Variations on how land-based and marine SAR agencies apply doctrine further suggest inconsistencies in underlying instructional materials and training pathways. While these differences reflect the distinct environments in which each organisation operates, they also indicate inconsistencies in underlying training materials and instructional guidance. To support clearer expectations, strengthen interoperability and reduce avoidable variation across Category 1 and Category 2 support agencies, a coordinated review of national training materials for both land-based and marine SAR is recommended. Such a review would help ensure common terminology, aligned frameworks and shared professional standards (New Zealand Government, 2019), ultimately improving the coherence and effectiveness of multi-agency SAR operations.

Several limitations should be considered when interpreting the findings of this document analysis. The quality, structure and depth of the SAREx evaluation reports varied considerably across agencies, regions and years. While some evaluations provided detailed findings, explicit recommendations and clear rationale, others offered only brief observations or limited contextual information and did not consistently align with established evaluation standards (Scriven, 2007; McCreight, 2023; UNDP, 2021; UNEG, 2008; Yarbrough et al., 2011). This variability constrained the level of analytical consistency possible across the dataset. Because report details and structure varied across districts and years, the findings represent patterns in reported observations rather than direct measures of operational performance.

SAREx evaluations are practitioner-generated documents produced within operational and time-bound contexts (Sinclair et al., 2012). They may prioritise practical observations over theoretical clarity, and their content can be shaped by the experience, writing style, or evaluation confidence of the individual evaluator (AISF, 2013; Gibbs, 2000; Midgley, 2009).

Although the database initially contained 111 evaluation reports, only 98 met the inclusion criteria. While this number ensures analytical accuracy, it also means that findings are based on documents that were sufficiently complete and evaluative to allow analysis. Reports that were incomplete, missing, or non-evaluative may contain insights that were not captured (Meline, 2006).

Document analysis relies on written accounts rather than direct observation (Sinclair et al., 2012). SAREx evaluation reports capture perceived performance, observed behaviours and reported lessons, but they cannot account for unreported issues, implicit knowledge, interpersonal dynamics, or informal learning that occurs during exercises (Bowen, 2009).

Finally, Category 1 SAREx evaluation reports represent only one part of NZ's wider SAR training system. They do not include Category 2 (RCC-led, maritime-based) exercises, incident-level training or internal agency reviews, and therefore provide an important but partial view of SAR NGO functioning within inter-agency planning and response environments.

The findings of this analysis point to several areas where further research is needed to strengthen the consistency and capability of NZ's SAR system. The uneven distribution of IMT competence across districts highlights the need for empirical examination of training materials and pathways, particularly the development and effectiveness of foundational to advanced, function-specific training. Future analysis should assess how national land-based and marine training materials align with doctrine, operational expectations and best practices (Comfort, 2007). Research into the impact of fragmented IMS on situational awareness and decision-making is also warranted, including comparative studies with jurisdictions outside of NZ that have greater system interoperability (NZSAR, 2023). Conceptually, this study reframes SAREx evaluations as a longitudinal organisational learning dataset, demonstrating how recurring operational themes can be used to diagnose systemic capability constraints in volunteer-dependent emergency management systems.

Further, the persistence of recurring issues across fourteen years of evaluations suggests that lessons management processes require closer study, particularly the organisational conditions that support or inhibit the embedding of lessons in practice. Future studies could explore how governance arrangements, resourcing models, leadership continuity and training cycles influence whether lessons are translated into sustained organisational change. Replicating this study in other volunteer-based SAR systems internationally would provide valuable comparative insights into common structural constraints, doctrinal alignment and organisational learning dynamics (Espejo, 2002; Kujala et al., 2016; Levitt and March, 1988).

In parallel, there is a clear opportunity for applied system improvements. Priority areas include the national standardisation of function-specific IMT training pathways, consolidation of core information management platforms to support shared COP, and the establishment of formal governance processes to track SAREx recommendations through implementation (Comfort, 2007; Boin et al., 2013; Owen et al., 2018; AIDR, 2019; Archer et al., 2021; NZSAR, 2023; Resilience NSW, 2021; UK Government, 2025). Embedding these elements would directly address the capability asymmetries, interoperability gaps and stalled lessons management identified in this study.

Importantly, while this study is grounded in the NZ SAR context, its findings have relevance for other countries where SAR capability is delivered through multi-agency, volunteer-dependent systems operating under nationally prescribed doctrine. Many jurisdictions face comparable challenges in embedding lessons and maintaining consistent capability across decentralised, volunteer-dependent SAR systems. As such, the analytical framework and document analysis approach used in this study offer a transferable method for examining SAR performance, training coherence and organisational learning in comparable international contexts, supporting cross-jurisdictional learning and evidence-informed improvements.

The collective insights from fourteen years of SAREx evaluations demonstrate that the issues identified within exercises extend well beyond isolated technical shortfalls. They reveal deeper structural conditions that shape the performance of NZ's SAR system, particularly volunteer SAR NGOs that provide much of its operational capability (Curd et al., 2025; NZSAR, 2025b). While individual skill and inter-agency goodwill remain essential, system effectiveness ultimately depends on consistent doctrine implementation, coherent training pathways and the availability of shared tools and processes across agencies and environments (NZSAR, 2023; Yarbrough et al., 2011).

The findings highlight several interconnected challenges, capability asymmetries between agencies and regions, fragmentation across information and communication systems (NZSAR, 2023) and persistent doctrine–practice misalignment, where national expectations are not always reflected in local training, planning, or evaluative practice (Gibbs, 2000; Sinclair et al., 2012; Bowen, 2009). These issues contribute to variation in IMT performance, uneven quality in exercise reporting and operational inefficiencies during multi-agency responses, patterns commonly observed in practitioner-generated evaluations (Gibb, 2000).

Strengthening national SAR capability is likely to require more than refining exercise design, it would benefit from a deliberate, system-level approach to embedding lessons. This includes formalised feedback mechanisms (Owen et al., 2018), evidence-based doctrine that is consistently applied (AIFS, 2013; Scriven, 2007; UNDP, 2021; Yarbrough et al., 2011) and strategic investment in interoperable systems and leadership development (NZSAR, 2020a, 2023). To support doctrinal consistency and reduce avoidable variation, a coordinated review of training materials across both land-based and marine SAR is also recommended. Ensuring aligned terminology, shared planning frameworks and consistent instructional content will improve interoperability and support clearer expectations across Category 1 and Category 2 support agencies (New Zealand Government, 2019). These improvements will not only raise the quality and reliability of future SAREx performance but also directly influence the effectiveness of real-world SAR operations, where clarity, coordination and timely decision-making are critical in achieving successful outcomes for the lost, missing and injured.