Methodology for examining the challenges in mainstreaming climate change adaptation

Establishing the linkages between climate change adaptation (CCA) and sustainable development has been a growing trend in adaptation research (Huxtable and Yen, 2009). Consequently, CCA and development planning have become interdependent strategies, supporting efforts to incorporate climate-related factors into the development plans, sectoral strategies, decision cycles and policy-making processes (OECD, 2009; Ayers et al., 2014). This adaptation approach is referred to as “mainstreaming”.

Mainstreaming CCA is a long-term adaptation planning measure that addresses concerns about adaptive capacity and the vulnerability of systems to climate change. It is becoming a popular climate change response as it:

• ensures coherence of adaptation and development plans and policies;

• enables efficient use of resources because of the unlikely duplication of efforts; and

• avoids worsening the vulnerability of systems (Agrawala, 2006; OECD, 2009; UNDP-UNEP, 2011).

However, although interest in mainstreaming CCA is growing, there is limited information on how to operationalize the approach effectively, especially at the local scale (Huxtable and Yen, 2009; Olhoff and Schaer, 2010). This paper posits that this slow development is likely caused by the neglect given to the institutional facet of mainstreaming in operationalization. Current techniques and operational procedures for mainstreaming focus on climate-related concerns (i.e. vulnerability assessments, impact analysis, risk screening) (OECD, 2009; USAID, 2009; UNDP-UNEP, 2011) and overlook:

• the existing institutional settings and institutional arrangements into which CCA will be integrated;

• the institutional transformations generated by the integration process; and

• the impacts of these institutional changes on the existing institutions.

Consequently, most studies on the subject deal with the difficulties in applying the approach in practice, rather than illustrate effective mainstreaming actions (Lebel et al., 2012; Ayers et al., 2014; Uittenbroek et al., 2013; Lehmann et al., 2015).

The lack of methodologies to investigate mainstreaming operationalization is another set-back that constrains the capability of researchers, analysts and practitioners to effectively measure and examine the complex processes involved in mainstreaming CCA (Tang et al., 2009; Ayers et al., 2014). This paper argues that mainstreaming operationalization necessitates a methodology that focuses on the challenges in applying the approach and an analytical framework that can examine the mainstreaming process from an institutional perspective. It also advocates for quantitative metrics that can monitor and assess the severity of the challenges in the mainstreaming process. This last point is important especially in planning because CCA is an “abstract concept” (Persson and Klein, 2008, p. 13) that needs to be simplified in a language more familiar to planners. A possible way to address this concern is to develop quantitative indicators that can help planners examine and evaluate the state and progress of adaptation efforts (Engle, 2011; UNDP-UNEP, 2011).

Hence, this paper proposes a mixed methodology for the task. As examining the challenges or barriers to adaptation is context specific (Biesbroek et al., 2011), a case study was undertaken in Albay, Philippines to collect and analyze relevant and robust data (Flyvbjerg, 2011). Accordingly, this paper begins with an overview of the four-stage mixed methodology developed by the research. Afterwards, it introduces the Institutional Analysis and Development (IAD) framework and demonstrates how the IAD was modified to better fit this CCA research. This is followed by the case study area selection process, the data collection design (i.e. survey, interviews) and the development of quantitative mainstreaming indicators. Later, some results are discussed briefly to show the applicability of the four-stage mixed methodology. Then, the issues encountered in applying the mixed methodology and some of the methodology’s limitations are presented. This is followed by the summary and the methodology implications and the paper’s conclusion.

Climate change is recognized as a cross-sectoral issue and an inter-disciplinary concern (Huxtable and Yen, 2009; Nielsen and D’haen, 2014). However, current climate change research has lagged behind the inter-disciplinary nature of the problem in terms of research cooperation, citation and the methods applied (Bjurström and Polk, 2011). Thus, an avenue for collaboration among quantitative and qualitative researchers is needed for effective interdisciplinary communication and cooperation in climate change research (Nielsen and D’haen, 2014). Such synergy can be accomplished through a mixed method approach.

Mixed methods research combines two complementary elements – qualitative and quantitative – to answer a research question (Hesse-Biber and Johnson, 2013). Through a mixed approach, the ability of the quantitative method in establishing or gauging the extent, status or condition of a phenomenon can be enhanced by the capability of the qualitative method to answer queries related to the “whys”, “hows” and “so whats” (Weaver-Hightower, 2014). Thus, a combination of the two generally is viewed as the best approach to accomplish an analytical task and answer complex interdisciplinary research questions (Flyvbjerg, 2011; Hesse-Biber and Johnson, 2013). In particular, the mixed method in this study applied triangulation by data method (i.e. document, interview, survey and key informant consultations) (Yin, 2009).

The research methodology involved four stages, with the use of the quantitative and qualitative methods varying according to the stage of the research. The methodology had two key facets: the modified IAD framework and the case study approach. Stage 1 entailed document reviews and consultations with key informants, and the IAD, as applied to mainstreaming CCA research (IAD-CCA), was developed at this stage [Refer to Section 3]. Stage 2 involved the survey and quantitative analysis, while Stage 3 included the in-depth interviews. Stage 4 involved document reviews and consultations with key informants to validate, verify and further support the results of the survey and interviews. All the information gathered were qualitatively analyzed using the IAD-CCA framework as a guide (i.e. using the modified evaluation criteria to assess the patterns of interactions and outcomes of these interactions) (Figure 1).

The components of the methodology were founded on the notion that mainstreaming CCA (i.e. integrating climate change and sustainable development goals and agendas and creating or amending existing development plans, policies and decision-making structures) is an institutional issue that creates complex institutional changes (Agrawala, 2006; Olhoff and Schaer, 2010). Accordingly, operationalization of the approach needs an institutional perspective.

Four frameworks (i.e. Advocacy Coalition Framework, IAD framework, Adaptive Capacity Wheel and Adaptation, Institutions and Livelihoods) were investigated to determine which can best examine the challenges in mainstreaming CCA. The review showed Ostrom’s (2007) IAD to be the most suitable framework to the task. First, the framework is designed specifically to examine institutional settings, so it is equipped to analyze the setting where CCA is to be mainstreamed. Next, the IAD has a variable (i.e. biophysical conditions) that can represent the impacts of climate change concerns in that setting (Ostrom, 2007; McGinnis, 2011). Furthermore, the IAD is practical with a design that can be adjusted according to the needs of the problems being addressed. It has been applied successfully to a variety of institutional conditions and to an extensive range of problems and concerns, including CCA (Rudd, 2004; Koontz, 2006; Oberlack and Neumärker, 2011).

The IAD focuses on the action arena, which comprises institutional arrangements and the actors who follow these arrangements. The action arena is influenced by a number of exogenous variables, namely, biophysical conditions, community attributes and rules-in-use (Ostrom, 2007). Based on the elements in the action arena, analysts can diagnose, explain and predict the actors’ patterns of interaction (i.e. aggregated individual choices, behaviors and decisions of actors in the action arena) and the outcomes from these interactions (Rudd, 2004). These patterns of interaction and outcomes are, then, assessed through a set of evaluation criteria. The criteria may differ based on the action arena; thus, analysts can determine how current institutional arrangements constrain or facilitate desirable outcomes depending on the specific actor or action situation selected and how the action arena needs to be evaluated (McGinnis, 2011).

When users need to modify the IAD, they generally change only the composition of the framework’s main variables and still maintain the IAD’s general premise in mapping institutional linkages and relationships. Any of the IAD components can be adjusted. A number of scholars – Rudd (2004), Jones et al. (2013) and Ratner et al. (2013) – modified the evaluation criteria, which originally comprised factors used to evaluate policy outcomes (i.e. economic efficiency; adaptability, resilience and robustness; accountability; conformance to general morality) (Ostrom, 2005). Following these examples, this study replaced the evaluation criteria of the IAD with factors that influence the effective operationalization of mainstreaming CCA (i.e. mainstreaming challenges). This paper deemed the evaluation criteria to be key variables, as they guide the users in:

• assessing the patterns of interactions of institutions;

• evaluating which outcomes are acceptable and which need improvement;

• analyzing how the current institutional arrangements constrain or facilitate desired outcomes; and

• formulating ideas on how to attain the preferred outcomes (McGinnis, 2011).

To develop the evaluation criteria of the IAD-CCA, the study analyzed around 80 peer-reviewed papers and over 60 book chapters, conference papers, international agency reports and discussion papers concerning the barriers, obstacles, limitations, constraints or challenges to adaptation (in general) and mainstreaming CCA (in particular). The review revealed that these factors (i.e. barriers) are linked to the adaptive capacities of systems or the ability of a system to adjust or cope with the impacts of climate change. Thus, to further understand these linkages, various literature on the determinants of adaptive capacity and enablers to adaptation were explored (Adger et al., 2005; Brooks et al., 2005; Burch, 2010; Uittenbroek et al., 2013).

The review revealed that the drivers or enablers of adaptation are the opportunities for adaptation, while the barriers are the factors that impede adaptation. Somehow, both factors exist at the extremes of the same scale or spectrum (Amundsen et al., 2010; Oberlack and Eisenack, 2014). For example, the lack of or the growing awareness of climate change are considered, respectively, as barriers to or drivers for adaptation. Similarly, effective leadership can help communities prevail over barriers, while a lack of leadership can be a barrier itself to adaptation (Tang et al., 2009; Moser and Ekstrom, 2010; Jones et al., 2013).

The body of research on the linkages between CCA and institutions is increasing, with several authors emphasizing the significance of developing the institutional capacity of systems to address climate change (Adger et al., 2005; Burch, 2010). Similarly, several studies have identified that the serious barriers to adaptation are institutional in nature (Eisenack et al., 2014; Oberlack and Eisenack, 2014; Waters et al., 2014). Using the definition of Cuevas et al. (2014) of institutions in the CCA context, the following barriers or challenges were identified:

• factors influenced by rule-based institutions such as autonomy of local governments, local government prioritization, commitment to CCA and other institutional issues relating to policies, regulations and the like (Pini et al., 2007; OECD, 2009; Burch, 2010; Ayers et al., 2014; Waters et al., 2014);

• matters linked to social structure-based institutions such as community support, institutional incentives and local leadership (Burch, 2010; Moser and Ekstrom, 2010; Biesbroek et al., 2011; Oberlack and Eisenack, 2014); and

• organizational concerns such as organizational cohesion and organizational cooperation and collaboration arrangements (Pini et al., 2007; Amundsen et al., 2010; Biesbroek et al., 2011; Eisenack et al., 2014).

The challenges related to climate change information also are key factors that affect CCA. These include:

• the extent of knowledge and awareness of climate change issues;

• the availability, accessibility, credibility and reliability of information;

• the manner by which information is communicated and translated by climate change experts; and

• the way the information is received by the users (i.e. planners and decision-makers) (Ekstrom et al., 2011; Ayers et al., 2014; Oberlack and Eisenack, 2014).

Meanwhile, resource constraints have always been a problem for local governments; however, they are highlighted in CCA because resources are crucial factors of adaptive capacity (Pini et al., 2007; Biesbroek et al., 2011). For example, lack of funds is typically among the primary reasons why the implementation of local adaptation is delayed (Moser and Ekstrom, 2010; Lehmann et al., 2015). Local governments have limited capabilities to invest or begin new endeavors, as their budgets are often overextended. With the additional responsibility for CCA, these shortcomings are magnified; local governments become more under-resourced, overcommitted and overtasked (Pini et al., 2007; OECD, 2009). Hence, the availability of funds can be a great barrier to CCA when it is lacking and a significant opportunity when it is sufficient.

This paper summarizes the mainstreaming challenges into three capacity classifications – institutional, information and resource capacities. Institutional capacity pertains to the rules, social structures and organizations involved in mainstreaming CCA. Information capacity deals with the ability of a system to integrate climate change information (i.e. technical and scientific knowledge and data) into the information system of the planning and decision-making processes. Finally, resource capacity focuses on the financial and human resources that ensure the maintenance and continuation of the integration process. The mainstreaming challenges under these capacity groupings are the factors that replaced the evaluation criteria of the IAD, thus transforming the framework into the IAD-CCA (Figure 2).

Converting the mainstreaming challenges into quantitative measures (i.e. mainstreaming indicators) was another significant adjustment in the IAD-CCA framework. In general, quantitative analysis and generating metric tools are relevant research designs to understand and evaluate CCA performances. Accordingly, this need was magnified in this CCA mainstreaming study (Horrocks et al., 2012; Miller et al., 2012). The characteristics for a good adaptation indicator – simplicity, ability to track progress, continuity and timeliness, clarity of purpose, relevance, comparability – were set as the criteria in designing the mainstreaming indicators (Harley and Minnen, 2009; Horrocks et al., 2012; Miller et al., 2012).

The scorecard approach was identified as the most suitable approach for converting the evaluation criteria in the IAD-CCA into quantitative indicators. The approach generates a numerical record of status and condition, which can measure the state-of-play and progress of activities in the setting being analyzed (UNDP, 2005; Bellamy and Hill, 2010). Moreover, (1) it is easy to understand; (2) it can be readily communicated to or interpreted by users with varying backgrounds; (3) it can be updated for timeliness and comparability across time; and (4) it can be applied at the local scale (Frost, 2007). The scorecard approach is also the technique that the United Nations Development Programme (UNDP) uses to generate capacity development indicators. The method has been effective in quantifying the qualitative process of capacity development (UNDP/GEF, 2003; UNDP, 2005).

In designing the scorecard valuation, this research relied heavily on UNDP’s procedures, particularly on the scorecard rating system used in the UNDP/Global Environment Facility (GEF) capacity development indicator methodology. In this method, each capacity result included a number of questions that represented a capacity indicator. The scorecards were in a form of descriptive sentences linked to each capacity development indicator with a numerical rating ranging from 0 to 4. The score for each question under a capacity category was averaged, and the resulting value was considered as the overall rating (UNDP/GEF, 2003; Bellamy and Hill, 2010).

In this study, the scorecard was incorporated in a survey conducted among the significant actors in the action arena of the IAD-CCA, the local land use planning system. First, each question, which represented a challenge in the IAD-CCA evaluation criteria, had three answer choices. The choices were descriptive sentences that exemplified a progressing status of the system’s capacity to prevail over the challenges (i.e. worst condition = 1; moderate condition = 2; and best condition = 3). A “Don’t know” category was also included among the choices to avoid forcing the respondents to make a choice when they had no knowledge of the item (Table I). Accordingly, the possible scores for each indicator were any value 1 ≤ n ≤ 3. The closer the value of an indicator to 3, the more likely the system overcame or turned the challenge into an opportunity that helped mainstream CCA into the local land use plans. Conversely, the closer the value of an indicator to 1, the more likely that the challenge was a barrier to mainstreaming, thus requiring intervention.

Equal weights were applied to each response in the survey to compute the final indicator score. Using various weights usually entails a more complex series of assumptions and validations. However, this procedure is beyond what is required for an exploratory study such as the current investigation (Lebel et al., 2013).

Cronbach’s alpha statistics – the most reported and used method to measure the reliability of estimates for indices – was computed to measure the reliability of indicator estimates. The alpha (α), which can have a value 0 ≤ α ≤ 1, gauges the survey’s reliability by measuring the internal consistency of a test or scale items in the survey (Bravo and Potvin, 1991; Santos, 1999). In general, the accepted value of α is between 0.70 to 0.95, with α > 0.9 as excellent results; α > 0.8, good; and α > 0.7, as acceptable results (Gliem and Gliem, 2003; Tavakol and Dennick, 2011).

The utility of the IAD-CCA framework in examining the challenges in mainstreaming CCA and the effectiveness of the mainstreaming indicators in assessing the state-of-play of the mainstreaming process were tested in a case study.

A three-stage case study selection process was conducted to narrow down this study’s scope and scale. The selection involved choosing a country which is a critical case – one that has strategic importance in relation to the general problem – and the localities in this country to investigate (provincial and city/municipal scales) (Flyvbjerg, 2011).

To be a critical case, the research should be in a country highly affected by climate change, actively applying the mainstreaming approach to adapt to climate change and operationalizing the approach in land use planning at both the national/federal and sub-national/local scales. A web search of the “most vulnerable countries to extreme weather events or climate change” was conducted to narrow the country selection choices. A list of the top ten countries affected by climate change was compiled, based on: the Global Climate Risk Index (for 2012 and 2013) generated by Germanwatch (Kreft and Eckstein, 2013; Kreft et al., 2014); the Climate Change Vulnerability Index (for 2012 and 2014) developed by Verisk Maplecroft (Maplecroft, 2012, 2014); countries at most risk to storm (for 2009) compiled by the World Bank (2008); and the most vulnerable countries to climate change (for 2008 and 2015) based on the methodology developed by Wheeler (2011), from the Center for Global Development. The only country which appeared in all the lists was the Philippines, a developing country in Southeast Asia. Further investigation showed that the Philippines has been institutionalizing a number of adaptation measures, including mainstreaming CCA into land use planning, through legislation such as the Climate Change Act of 2009, the Disaster Risk Reduction and Management Act of 2010 and the People’s Survival Fund of 2011 (CCC, 2011).

Given these findings, the research considered the Philippines to be a critical case, suitable for helping to understand local mainstreaming of CCA (Flyvbjerg, 2011).

In the Philippines, the local land use plan is developed at the municipal and city scales. Meanwhile, the plans are reviewed and approved at the provincial scale (HLURB, 2006). Provinces are the largest unit in the political structure of the Philippines, consisting of either municipalities or component cities or both (NSCB, 2014).

The province of Albay is one of the most vulnerable areas in the country in terms of climate-related disasters, particularly to disasters relating to typhoons (Manila Observatory, 2005). Because of the predicted impacts of climate change, the Provincial Government of Albay has been one of the most active advocates of CCA and disaster risk reduction (DRR) initiatives in the country. It has enacted various local legislation and programs concerning CCA-DRR and is actively promoted mainstreaming of CCA-DRR into the local land use plans (Salceda and Rangasa, 2011; Salceda, 2012). Accordingly, the Albay experience offered a robust set of information concerning the challenges in mainstreaming CCA.

Because of time and financial constraints, only two of the 18 LGUs local government units (LGUs) in Albay province were studied. These areas included one LGU with, and one without, a modified local land use plan (i.e. with CCA and DRR components). The LGU without the modified local land use plan was pre-selected based on the evidence that LGU personnel have undergone training on integrating CCA-DRR into the local land use plan. The accessibility of LGU information (i.e. the LGU was visible on the internet and had a website) was also among the selection criteria. The LGU that best matched the criteria was the rural municipality of Camalig. Meanwhile, among the three LGUs that have already produced local land use plans with CCA-DRR components (i.e. Daraga and Malinao municipalities and Legazpi City) (Personal communication, 2014), only Legazpi City satisfied the following conditions:

• availability and accessibility of LGU information (i.e. existence of an LGU website);

• ease in communication (i.e. responded to the researcher’s attempt to communicate via electronic mail); and

• an urban area (i.e. to pair with the rural Camalig Municipality).

The case study survey applied a purposive sampling technique because of the highly specialized nature of the research. This sampling method ensured that the respondents were knowledgeable about mainstreaming CCA into local land use plans. Specifically, the survey and interview respondents were selected from the actors involved in “Step 7: Preparing the Land Use Plan” of the 12-step process followed by all LGUs in the Philippines to develop comprehensive land use plans (HLURB, 2006).

The respondents at the city/municipal scale included the members of the:

• Municipal/City Planning Development Offices that generate the plans; and

• the Local Disaster Risk Reduction Office (Legazpi City), which is responsible for the CCA-DRR program of the LGU.

At the provincial scale, the respondents included members of the Provincial Land Use Committee (PLUC), who review and approve the plan (i.e. Provincial Planning and Development Office, Albay Public Safety and Emergency, the Provincial Agriculturist and the sub-national offices of the Housing and Land Use Regulatory Board, Department of Interior and Local Government, Department of Environment and Natural Resources) (Table II).

Another set of respondents was selected based on their capability to provide insights with regard to the issues prevailing at the national scale. They were the:

• key project personnel and national institution representatives involved in activities and programs for mainstreaming CCA into the local land use plans (i.e. Housing and Land Use Regulatory Board, Climate Change Commission, National Economic and Development Authority); and

• experts on CCA and/or land use planning from the academe and other institutions with local-level experience (i.e. UN Human Settlement Programme, Philippine Institute of Environmental Planners, University of the Philippines Los Baños).

These respondents were implementers of national climate change policies or CCA projects or advisers of LGUs in the mainstreaming process. Thus, while having a national scale perspective on the mainstreaming concerns, this set of respondents also had knowledge of the issues existing in localities other than Albay. Therefore, they indirectly represented the assessments for other LGUs in the Philippines. With this, the research was able to generate data for two cases – Albay and LGUs other than Albay.

Snowball sampling or chain referral sampling was used to obtain additional survey respondents (i.e. sub-national offices of the Environment and Management Bureau, Mines and Geosciences Bureau, Department of Agrarian Reform and the City Disaster Risk Reduction Management Council [Legazpi] and the Municipal Disaster Risk Reduction and Management Officer [Camalig]). The search for local respondents ceased when respondents started referring people who had already been surveyed/interviewed (Table II).

The survey questionnaires were disseminated using two mediums and in two stages. The first stage involved sending the questionnaires via electronic mail in February 2014, where about 35 percent of the primary respondents replied. The second stage was conducted in the Philippines, in which hard copies of the survey forms were personally delivered to the primary and secondary respondents in April and May 2014.

The survey respondents formed the basis of the interviews. At the national scale, five of the nine survey respondents were interviewed; 11 of the 13 respondents at the provincial scale and 5 of the 7 respondents at the city/municipal scale were interviewed (Table II). The one-hour interviews were semi-structured and involved in-depth discussions of the challenges highlighted in the preliminary results from the online survey, namely, institutional issues, institutional collaboration and cooperation arrangements, organizational cohesion and availability of human resources. The interview schedule was adjusted in the field to focus more on institutional issues and leadership, the indicators deemed most significant based on the partial survey results and initial interviews conducted. Questions regarding the interviewee’s perception on the following concerns also were asked:

• the significance of mainstreaming CCA into the local land use planning process;

• the mindset of the local planners regarding mainstreaming CCA into land use planning; and

• the overall progress of mainstreaming CCA into the local land use plans.

The interviews were recorded and later transcribed.

Key informants from both the local and national governments were further consulted to clarify some concerns raised during the interviews. The informants were representatives of the institutions that played crucial roles in the issues mentioned by the interview respondents (i.e. Department of Budget and Management, the Civil Service Commission and the Climate Change Commission). Documents including government memoranda, national and local laws and regulations, handbooks and guidelines were reviewed for supplemental information.

At this point, the individual stages of the mixed methodology have been discussed, and this section integrates this information into the research’s data collection scheme. Climate change concerns were incorporated in the IAD-CCA framework through the exogenous variables that affect the local mainstreaming setting (action arena). Thus, the data gathered on the biophysical conditions, attributes of the community and the rules-in-use enabled the authors to account for the elements that affected mainstreaming CCA at the local scale, which, by transitivity, also influenced the challenges encountered in operationalizing the mainstreaming approach. The components of the action arena (i.e. actors and institutional arrangements) provided information on the state-of-play of the local mainstreaming activity, whereas the evaluation criteria explained the patterns of interactions and outcomes from operationalizing the adaptation measure.

To gather the data outlined in the IAD-CCA framework, this study applied a strategy for collecting information from multiple data sources and for conducting triangulation by data method (i.e. document, interview, survey and key informant consultations) (Yin, 2009). Through this scheme, the limitations of one data source were supplemented by the strengths of another, thereby enabling the authors to collect robust data and perform extensive analyses on these data (Tables II and III).

The mixed methodology, which was effectively applied in the case study, is briefly discussed in this section. Detailed analyses of the mainstreaming indicators and the underlying issues concerning the mainstreaming process are presented in a separate paper (Cuevas et al., 2015).

The Cronbach’s alpha statistics computed for the entire data set was 0.8595. The subset value for the national scale was 0.8097; provincial, 0.9487; and 0.9001 for the city/municipal scale. These alpha values imply that the survey results presented reliable estimates for the mainstreaming indicators (Gliem and Gliem, 2003; Tavakol and Dennick, 2011).

Based on the respondents’ assessments, the mainstreaming indicator scores were classified into four levels. The first-level mainstreaming indicators (i.e. with scores 1.0 ≤ n < 2.0) represented the primary barriers that constrain the effective integration of CCA into the local land use planning system. The second-level barriers had scores 2.0 ≤ n < 2.25, representing significant barriers to mainstreaming, but second only to the primary barriers. The third-level barriers had scores 2.25 ≤ n < 2.5; because of the actions that addressed these challenges, they were becoming less of barriers and were transcending to becoming opportunities. Finally, the fourth-level mainstreaming indicators (i.e. with scores ≥ 2.5) embodied those that the system had already overcome, had positive effects on the implementation process and thereby had become opportunities for mainstreaming.

The institutional issues indicator was identified as the key obstacle (i.e. primary barrier) to mainstreaming of CCA, with scores equal to 1.00, 1.46 and 1.36 at the national, provincial and city/municipal scales, respectively (Table IV). In contrast, local government prioritization, institutional incentive, credibility and reliability of information and stability of funds indicators were assessed as opportunities at all levels. These results suggest that the challenges related to these indicators had already been overcome. Meanwhile, the availability and access to information indicators were highlighted as second-level barriers at all scales, implying that information-related challenges were still significant impediments to mainstreaming CCA in local land use planning in Albay. This classification system suggests that the mainstreaming challenges exist within a certain spectrum with one end composed of barriers to, and the other, opportunities for CCA. Furthermore, this system illustrates that the challenges have varying degrees of severity depending on the conditions that surround them (Table IV). This set of information can help analysts, planners and decision-makers:

• determine the nature of the challenges in mainstreaming CCA and thereby understand the mainstreaming process;

• prioritize the challenges to address; and

• design strategies that will maximize the use of limited resources (i.e. utilizing the opportunities to overcome the existing barriers), among others (Cuevas et al., 2015).

This section presents the issues encountered in developing the mixed methodology, from modifying the IAD framework, designing the quantitative aspect of the methodology, to the actual application of the plans and methodological designs in the field. For example, the IAD framework views institutions as rules only (Ostrom, 2007). However, the concept of institutions in CCA encompasses rules, social structures and organizations (Cuevas et al., 2014). The IAD-CCA addressed this issue by incorporating the integrated institutional definition into the institutional capacity component of the evaluation criteria.

Because of the methodology’s purposive sampling design – “typically designed to pick a small number of cases that will yield the most information about a particular phenomenon” (Teddlie and Yu, 2007, p. 83) – the survey and interviews involved a small sample size. To gather “greater depth of information from a smaller number of carefully selected” people (Teddlie and Yu, 2007, p. 83), the authors sought all LGU personnel who had intimate knowledge of mainstreaming CCA into the local land use plans. To minimize the non-response in this highly specialized investigation, the authors disseminated questionnaires at two stages (i.e. online survey and field survey). Thus, the target people were successfully included among the survey respondents, allowing for an in-depth investigation of the critical case of the research. According to Maxwell (2009), a systematically selected (for typicality and relative homogeneity) small sample can provide confident conclusion in a qualitative research.

Working with small sample sizes is generally an issue in performing statistical inference and hypothesis testing. Still, the works of Bridge and Sawilowsky (1999), Janusonis (2009) and Fitts (2010) showed that applying standard statistical analyses on small sample sizes can be feasible and valid. Likewise, the scorecard approach also has been applied effectively on a small sample size to create quantitative metrics (Haanpaa and Peltonen, 2007). Still, to ensure reliability of estimates, Cronbach’s alpha statistics was computed on the data generated.

Other issues encountered pertained to the analysis of the mainstreaming indicators. For example, survey results implied that the questions for the availability of funds, experts and human resources indicators should be modified. The computed indicator scores were not able to be taken at face value and the indicators required supplementary information and analysis to determine the “true” conditions surrounding the challenges. In particular, the questions for these indicators were concerned with the availability and regularity of resources. However, the interviews revealed that the issue was not whether the resources were available but whether they were sufficient to operationalize mainstreaming CCA efficiently.

The four-stage mixed methodology was a systematic and practical process. Each stage in the methodology produced its own output, and each individual output significantly contributes to the methods and adaptation literature. Stage 1 developed the IAD-CCA framework, Stage 2 generated the quantitative mainstreaming indicators, Stage 3 devised the varying levels of severity by which the challenges (as represented by the indicators) affected the mainstreaming process and Stage 4 produced the in-depth qualitative analyses of the challenges in mainstreaming CCA. Likewise, each stage was an important part in a chain of actions within the methodology, in which an output of one stage was an input in another (stage).

To illustrate, the IAD-CCA developed in Stage 1 is an output in itself. It has a flexible design; thus, other researchers can adjust (i.e. augment or lessen) its evaluation criteria depending on their respective research needs. Similarly, it is an integral component of the methodology, as the list of mainstreaming challenges guided the design of the survey conducted in Stage 2. The scorecard approach was effective in quantifying the answers of the respondents; the approach was easy to understand, readily communicated to or interpreted by users with varying backgrounds and, most importantly, was applied successfully at the local scale. Accordingly, the resulting indicators are outputs of the methodology that have the potential to be benchmarks for determining the status of the mainstreaming challenges and for tracking the progress of the mainstreaming process. The indicators can be updated for timeliness and comparability across time. Meanwhile, the indicators were the core factors that guided the line of questioning during the interviews in Stage 3.

The assessments during the interviews in Stage 3 explained how the mainstreaming indicator scores should be interpreted, as well as verified and validated, the indicator values. The resulting mainstreaming indicator classification in this stage is an important output that illustrates the severity of the impact of a mainstreaming challenge. This set of information can assist planners and decision-makers to determine which challenges need to be prioritized. This is noteworthy because knowing the primary challenges that need intervention can help local governments use their limited resources efficiently and take advantage of the opportunities they have.

Stage 4 entailed additional document reviews and consultations with key informants to address the additional issues and concerns mentioned during the interviews. Afterwards, all the information gathered from Stages 2 to 4 were organized and qualitatively analyzed using the IAD-CCA framework as guide. Hence, the local land use planning system (i.e. action arena) in Albay was examined, that is:

• the interplays and interactions between and among the rule-based and institutional organizations connected to each mainstreaming indicator were analyzed; and

• the existing and introduced institutional arrangements governing the actions of these institutions were explored.

Overall, the mixed methodology was applied successfully in practice; and the qualitative methods effectively complemented the quantitative approaches. Consequently, the methodology generated results that can assist analysts, planners and decision-makers determine the state-of-play of the challenges in mainstreaming CCA and make informed decisions for overcoming these challenges.

There is a growing interest in mainstreaming as an adaptation approach, and an increasing number of countries are encouraged to implement it (UNDP-UNEP, 2011; Ayers et al., 2014). However, in practice, mainstreaming encounters a number of difficulties; hence, it has been slow to operationalize (Uittenbroek et al., 2013; Lehmann et al., 2015). This paper contends that this setback is likely caused by the lack of attention given to the institutional facet of mainstreaming operationalization. The current techniques and tools for mainstreaming concentrate on climate-related concerns, and they neglect:

• the existing institutional settings and institutional arrangements into which CCA will be integrated;

• the institutional transformations generated by the integration process; and

• the impacts of the institutional changes caused by mainstreaming to certain realities already subjected to the existing institutions.

Thus, this paper argues that mainstreaming operationalization needs a methodology that focuses on the challenges in mainstreaming CCA and an analytical framework that can examine the mainstreaming process from an institutional perspective. To have intensive insights into these challenges, the methodology should be able to monitor and assess the severity of the challenges through metrics; answer causal linkages among challenges; and solve the questions pertaining to the “whys” and “hows” of the subject. Accordingly, this paper devised a four-stage mixed methodology and applied it to examine the challenges in mainstreaming CCA into local land use planning in Albay, Philippines.

The authors effectively applied the mixed methodology in practice and related papers (Cuevas, 2015; Cuevas et al., 2015) identified and examined the primary barriers and other main challenges that hinder the operationalization of the approach at the local scale. Although the methodology is at its initial stages of development, it generated results that have the potential to help analysts, planners and decision-makers determine the state-of-play of the challenges in mainstreaming CCA and make informed decisions for overcoming these challenges. This methodology can be replicated under different conditions and circumstances and can be used to conduct studies in other localities or sectors (i.e. mainstreaming of CCA into infrastructure, agriculture, water or education).

Financial support of the CSIRO-UQ INRM PhD Scholarship is gratefully acknowledged.