The evolution of project portfolio selection methods: from incremental to radical innovation Open Access

Project portfolio selection (PPS) is an iterative process whereby managers select projects from available proposals and current projects, in order to meet organizational objectives (Archer & Ghasemzadeh, 1999). This process is crucial to maintain the competitive advantage, enabling the company to focus on the most relevant and strategic projects. However, firms struggle in managing this process. PPS’ complexity relies on two main facts. First, the information required to determine whether a project could be successful or not is extremely difficult – if not impossible – to know (Baker, 1974). And second, because the project and environment conditions are too volatile, strategic goals might change, promoting high levels of uncertainty (Rice, O’Connor, & Pierantozzi, 2008).

Different methods and approaches have been proposed to address the PPS (Bard, Balachandra, & Kaufmann, 1988; Cooper, 1993; Regan, Ben-Haim, Wilson, Lundberg, Andelman, & Burgman, 2005). Current literature offers a number of increasingly sophisticated project selection approaches, including multi-criteria decision methods, scoring-based and simulation techniques, among others (Zhang, Yang, Dou, & Jiang, 2016). However, the current literature presents an “interesting dichotomy”: although there are an increasing number of frameworks supporting PPS, the adoption by firms and its impact remains limited. In addition, there is a lack of theoretical foundations for these frameworks (Kavadias & Chao, 2008).

Other scholars have also identified similar problems in PPS literature (Beaujon, Marin, & McDonald, 2001; Cooper, Edgett, & Kleinschmidt, 1999; Kavadias & Chao, 2008). Indeed, years of research provided a fragmented and disperse knowledge on PPS, increasing the difficulty of convergence and accumulation of knowledge. To some extent, the existence of this fragmentation might also impact the adoption by firms. The development and adoption of PPS are more difficult for radical innovation, in which managers face high uncertainties in structuring and selecting projects. According to Brasil, Salerno and Gomes (2018), inappropriate selection methods might restrain radical innovation proposals, since, typically, the PPS process has been centered on the assessment of ex ante information, such as estimated costs, profits or market share, that only can be confirmed after launching the project (Zhang et al., 2016). In sum, there are two major concerns in PPS literature, which can render the consolidation of the field more difficult. First, there is a lack of a comprehensive framework regarding how the PPS approaches evolved. And second, there is a lack of understanding of how these different approaches might be associated with different degrees of innovation.

To reduce these two major gaps, the following research question drives our research:

Employing a systematic literature review, we aim to identify which approaches are more suitable for radical innovation projects and what aspects are crucial to manage radical innovation portfolios. Our goal is to go beyond the last review on PPS methods, dated from more than 10 years ago (Cooper et al., 1999), capitalizing on more recent works. Addressing this research question is fundamental to theory and practice. First, there is a lack of understanding regarding when and how to use these approaches to select the specific level (degree) of the innovation project (from incremental to more radical ones). Likewise, without a proper method, firms might not be able to select the opportunities related to radical innovations (Brasil, Salerno, & Gomes, 2018). Using a kind of method that fits all approaches, firms might fail in differentiating the valuation and selection of incremental and radical innovation. Finally, it is also important to determine whether newer PPS methods meet the particular features of radical innovation projects in order to provide a guide for future research.

This paper provides important insights that advance the knowledge on PPS literature. First, we show how PPS evolved. This is especially important to new scholars in the field, who might face difficulties in building a big picture of how this field is structured. Second, we identified the main characteristics of different PPS approaches. This finding provides a rich portrait regarding the approaches, helping scholars and practitioners to identify when and how to use these approaches according to the innovation level. Finally, we provide a theoretical framework in which we associate these approaches according to the innovation level.

This study is structured in five sections as follows. Section 2 presents the methodological procedures including the documents sample selection and the analysis of the methods. Section 3 discusses the PPS approaches, describing each of the 12 taxonomies identified. In Section 4, we discuss the key issues for radical innovation portfolio management and which approaches are more suitable to this end. In addition, we elaborated an integrated portfolio selection method for radical innovation framework, based on the main characteristics and advantages of the approaches. Finally, Section 5 is concluded with the final considerations and future research directions.

To obtain an overview of the evolution of PPS, a systematic literature review was performed. This methodology enforces explicit methods to perform a thorough research and analysis, underlining how a determined field has evolved, and enabling the identification of consensus or divergence among studies and theories (Briner & Denyer, 2010).

The collecting methods’ process started through the search on the Web of Science database, which provides different searching and browsing options to obtain higher accuracy during the research. We applied the keywords: “Project Portfolio Selection,” in order to gather the broadest number of related works. The search returned 175 documents; this sample was refined selecting only the pertinent predefined categories as follows: “management,” “operations research management,” “engineering industrial” and “business.” Additionally, we focused our research only on “article,” because this type of documents is peer-reviewed – an important indicator of quality. We then reached out 75 documents.

Another selecting parameter was the inclusion of articles whose purpose was the proposition or improvement of method or approach for PPS. We read again the titles, abstracts and keywords. Based on this process, 35 articles related to “Project Portfolio Selection” were obtained. Notwithstanding, four documents referenced by Cooper, Edgett, and Kleinschmidt (1999) were added and other works were included too because they are recurrently cited by the studies in our sample. In total, 48 articles were selected.

Once the sample of articles was established, the first stage was to read them identifying the relevant points and the main contributions including proposed frameworks, techniques, tools, examples of application and required data. Based on Cooper et al.’s (1999) portfolio selection methods classification, we defined 12 taxonomy categories to classify the PPS approaches. The categories are: financial, probabilistic, options pricing, strategic, scoring, combinatorial optimization, behavioral, mapping approaches, real options, integrated methods, information gap theory and scenario-based approach. Table I shows the number of documents per year and methods categories. Combinatorial optimization (15) and integrated methods (13) have the greatest number of approaches and also represent the categories more explored in the last 10 years (ten and five documents, respectively).

Afterward, we carefully examined each PPS approach. We then identified four key requirements that PPS methods must meet in order to address radical innovation projects: dynamism, interdependencies, uncertainties and required input data. Based on these analyses, we proposed an integrated portfolio selection method for radical innovation (Section 5).

According to Cooper et al. (1999), portfolio selection methods have progressively changed since the first methods emerged in 1960. At that time, methods were strongly focused on mathematical techniques. Over time, because the business and technology environment became more complex, more sophisticated methods emerged. Scholars identified that some of these approaches consist of prescribed solutions (one fits all), which might not fit with the vast variety of project and firms’ conditions (Chien, 2002; Baker, 1974). Furthermore, these prescribed aggregate methods raise a paradigm wherein a given set of projects (usually competing for limited resources) must be appraised in order to choose the ones which will maximize profits. Although these methods might be useful for more incremental innovation, an increasing research stream suggested that prescribed methods are not sufficient to analyze radical innovation projects due to the large numbers of unpredictable variables and knowledge gaps (e.g. Baker, 1974; Archer & Ghasemzadeh, 1999; Ghapanchi, Tavana, Khakbaz, & Low, 2012). In the following sections, we present 12 methods for PPS.

In these methods, the most important selection criterion is profitability and the objective is to establish a portfolio that maximizes expected returns (Bard et al., 1988). This is one of the most widely adopted methods by managers for screening projects (Chien, 2002; Dutra, Ribeiro, & Carvalho, 2014), and basically, are utilized to assess projects individually. Two documents were identified within this category (Table II).

Among the most popular probabilistic methods are Monte Carlo simulation and Bayesian theory. These methods analyze different scenarios producing distributions of possible outcome values. Some methods are stochastic, using random values. Archer and Ghasemzadeh (1999) indicate that simulation techniques are more effective to analyze risk (Table III).

Option pricing uses mathematical models to calculate how demand varies at different prices’ levels. These models work with three critical pricing elements: pricing strategy, both buyer and seller value of the product, and techniques and elements that affect profitability (Galai & Masulis, 1976). According to Chien (2002), mathematical models are slightly used by managers because these models tend to do not consider the diversity of projects and criteria (Table IV).

Since selecting the right projects is extremely critical, not just to maintain the competitive position, but also to achieve financial goals (Cooper et al., 2002), authors have been exploring selection methods driven basically by strategic objectives. Zeynalzadeh and Ghajari (2011) raise particular awareness about the hierarchy of strategy (i.e. organization, portfolio and project strategy) to be considered during the process of project selection. Strategic Buckets is one of the most widespread methods, in which managers create budget buckets where projects are categorized and prioritized (Cooper et al., 2002). These methods involve both external and internal considerations of the firm (Archer & Ghasemzadeh, 1999), enabling a focused perspective from the entire context (Table V).

Basically, projects are screened based on a set of different requirements and aspects. Projects are eliminated if they do not meet the requirements and selected if they meet most part of these requirements (Archer & Ghasemzadeh, 1999). Scoring methods are a quantified extension of a checklist and rely on subjective ratings of managers. This fact presents a weakness because such ratings and weights are arbitrarily determined (Cooper, 1981), and this is one of the reasons why scoring methods could work better with objective-centered approaches. This method is also popular in practice because of its simplicity of execution (Hall & Nauda, 1990) (Table VI).

This category includes methods such as multi-criteria decision-making methods (MCDM), data envelopment analysis and analytical hierarchy approaches (AHP). According to Razi and Shariat (2017), MCDM are based on the utility theory and assist the human constraints to deal with profits and loss. In AHP models, criteria and alternatives are arranged in hierarchical structures in at least three levels, in order to prioritize the criteria within each level (Amiri, 2010).

According to Archer and Ghasemzadeh (1999), these methods are complex and difficult to apply, and also require a lot of information. Another constraint stems from the use of Technique for Order of Preference by Similarity to Ideal Solution and crisp values which are not recommended for handling uncertainties since these techniques work with single numeric values (Amiri, 2010). However, robust and specialized assessing algorithms may be useful for early screening when there is a large number of potential projects (Beaujon et al., 2001; Abbassi, Ashrafi, & Tashnizi, 2013). The approach proposed by Amiri (2010) incorporates fuzzy numbers, enabling managers to introduce interval judgments improving handling uncertainty issues (Table VII).

These methods aim to create a consensus based on experts and managers’ opinions to decide which projects should be undertaken. For Cooper et al. (1999), this taxonomy is particularly useful during the initial stages of the project selection, in which, virtually, these opinions might be the single input data available (Table VIII).

In this taxonomy, projects are plotted against various parameters in form of matrices or bubble diagram in which it is possible to visualize the strategic position of the firm’s product (Archer & Ghasemzadeh, 1999). Mapping methods have gained wide popularity among companies. For instance, Linton, Walsh and Morabito (2002) mentioned firms, such as 3M, Lucent Technologies or Procter and Gamble, which have implemented such mapping method (Table IX).

These methods incorporate learning models to resolve uncertainties through the experience gained over time. Differently from traditional economic analysis (e.g. discounted cash flow), the real options model is a fluctuating investment decision and assumes that the strategy may evolve depending on the real characteristics of the environment (Mun, 2005). It also helps managers to define a portfolio (which projects and when they should be run) among several seemingly economic outcomes and projects. Brasil et al. (2018) mention some barriers to adopt real options in PPS by firms: mathematical complexity, non-intuitive outcomes, and difficulties to handle unknown uncertainties, among others (Table X).

Integrated methods are those in which two or more different taxonomies are employed in order to guide the portfolio selection process (Table XI).

This method assists the decision-making process in situations under several knowledge gaps. It works with a mathematical process model, performance requirement and uncertainty model (Regan et al., 2005). Through the information gap model, analysts can identify what they believe to be true and important regarding the system, enabling the evaluation of robustness (identifying possible bad outcomes) and opportunity (windfall chances).

Differently from traditional selection methods, which are based on maximum expected benefits, in this method, the focus is on the identification of knowledge gaps, uncertainties and risks, in order to determine the least vague portfolio (Table XII).

One of the main concerns of this approach is to address exogenous uncertainties, which are defined by Liesiö and Salo (2012, p. 162) as missing data or information “which are not influenced by the projects” (e.g. macroeconomic developments or rate of industry growth). This approach proposes to deal with exogenous uncertainties through the scenarios technique. Scenario planning enables a company to imagine possible futures through exploring “the joint impact of various uncertainties” (Shoemaker, 1995, p. 26). For Shoemaker (1995, p. 27), scenarios “attempt to capture the richness and range of possibilities, stimulating decision makers to consider changes they would otherwise ignore.”

The methods rely on the assumption that, through mathematical models and functions, it is possible to complete the information provided by experts, to build a scenario and identify synergies and probabilities. Scenario-based methods require specific and sophisticated knowledge to operate, thus, its applicability and diffusion are restricted (Archer & Ghasemzadeh, 1999) (Table XIII).

In this section, we develop a discussion by bridging incremental and radical innovation levels and PPS approaches. First, integrated methods (i.e. those that enable managers to employ different techniques and different input data) are more convenient to adopt in both cases – incremental and radical innovations (Archer & Ghasemzadeh, 1999; Cooper et al., 1999; Dutra et al., 2014). This is also especially relevant for radical innovation, since no single method has all the features required to respond adequately under uncertain circumstances. Coldrick, Longhurst, Ivey, and Hannis (2005) recognize that optimization methods are powerful, but if combined with scoring, behavioral or mapping methods, they would be more beneficial. Second, qualitative judgment is more appropriate to deal with radical innovation (Archer & Ghasemzadeh, 1999; Cooper, 1981; Fahrni & Spätig, 1990). For instance, mapping or scoring methods are very useful to identify which projects to cut-off regarding the number of uncertainties and risks. Quantitative methods seem to be more suitable for handling risk (Fahrni & Spätig, 1990), rather than uncertainty in which the probabilities are “unknowable” (Brasil et al., 2018). However, some techniques employed by quantitative methods might handle uncertainties, such as is the case of fuzzy theory (combinatorial optimization) or volatility metrics (real options) (Jafarzadeh, Akbari, & Abedin, 2018; Wang & Hwang, 2007).

Another important aspect to consider when handling uncertain projects is the method’s ability to update knowledge, enabling managers to learn through the experience gained. Real options and probabilistic methods that implement Bayesian network enable the aggregation of information at each stage of the process (Chin, Tang, Yang, Wong, & Wang, 2009). Finally, an interesting technique of trade-off between immunity to uncertainty and aspirations is the information gap theory model (Regan et al., 2005). This method focuses on the identification of knowledge gaps, uncertainties and risks in order to determine the tolerance for undesirable outcomes.

Based on the literature review and the considerations presented above, we identified four key requirements that PPS methods must meet in order to address radical innovation projects. First, Dynamism, is defined as the characteristic that enables managers to permanently adjust the portfolio according to the evolution of the firm’s guidelines (Kavadias & Chao, 2008, p. 139). Second, Interdependencies, is regarded as the interrelations among projects and their impacts on portfolio performance (i.e. synergy or incompatibilities); several authors suggest that the identification of interdependencies is crucial for innovation projects (Verma & Sinha, 2002; Chien, 2002; Archer & Ghasemzadeh, 1999). Third, Uncertainties, is perceived as situations where neither the outcomes nor the odds could be known (Sommer, Loch, & Pich, 2008). The concept of uncertainty is central to radical innovation. Leifer, O’Connor, and Rice (2001) explain that radical innovation processes involve unprecedented situations and features plenty of technical and market uncertainties (p. 103). Finally, Required input data that refer to the kind of data required by the PPS method. Some authors (e.g. Baker, 1974; Fahrni & Spätig, 1990) consider that financial input data (e.g. estimated costs, profit margin, investment, sales, etc.) imprecise because information will never be precisely available. For Ghasemzadeh and Archer (2000), qualitative information is more recommended for assessing radical innovation projects.

Regarding these four key issues and based on the information provided in the studies, we built a framework (Figure 1). To do so, the approaches identified were analyzed by considering the adequacy between the approaches and the four requirements – we developed a scoring model in which each approach received a point or more if such approach is suitable to one characteristic or more (see Table AI). According to this assessment, behavioral methods and information gap theory are the two approaches more suited for radical innovation, followed by real options and integrated methods. The methods less appropriate for radical innovation are financial methods, options-pricing theory and scoring methods; these approaches are more suitable for incremental innovation (Fahrni & Spätig, 1990).

After identifying the advantages and disadvantages of each approach, and identifying the methods more recommended to radical innovation projects, we developed a framework for supporting the PPS selection method according to the innovation level. This framework might help managers to choose and to combine different methods among the alternatives according to a specific innovation level (i.e. radical or incremental innovation).

The framework comprises three phases: ideation phase, which includes the strategic goals, resource constraints and project guidelines; the phase ends with the analysis of the characteristics of the project in which the innovation level is identified. The second phase is the PPS process, where the projects are screened using one or more approaches according to the innovation level. In the last phase, decision makers define the project portfolio based on the recommendations of the selection approaches; this phase also comprises the portfolio adjustments stage, in which managers might include, modify or discard projects in accordance with the environment and the strategic goals. Figure 2 presents the framework suggested.

In this paper, we addressed the question of how the selection approaches have evolved to better fit within radical innovation conditions by conducting a systematic literature review. Based on this research approach, we showed how the project selection methods evolved. We present a comprehensive characterization of these methods. We have also identified four key aspects to manage radical innovation portfolios: dynamism, interdependency management, uncertainty treatment and required input data. Based on these findings, we propose a framework for supporting the PPS process according to the innovation level. Moreover, we suggest which PPS approach is more appropriate to incremental and radical innovation projects.

Of course, there is an important opportunity for further studies. We have not discussed in details about how to test our framework. Further studies might test our framework in different industry settings. Also, scholars might test if the methods related to incremental and radical innovation provide superior performance (in relation to the configuration of the portfolio). Scholars might test the best set of methods according to the project’s features (incremental and radical innovation). Finally, scholars might explore empirically how different combinations of methods are more recommended taking into account different industry settings and other features.

Table AI