Firms frequently struggle with measuring the performance of their radical innovation activities. Due to the uncertainty and ambiguity involved, key performance indicators (KPIs) used for incremental innovation projects are often not useful in this context. The purpose of this paper is to explore suitable KPIs particularly useful for radical innovation projects.
This study first reviews commonly used measures for innovation projects, which is then followed by case-study evidence from three industry-leading international firms. This study includes 13 in-depth interviews with innovation managers and directors in these firms, providing insights on how they measure the progress and performance of radical innovation projects.
KPIs used commonly in incremental innovation showed lackluster results in the case firms and were problematic for radical innovation context. A key finding was that radical innovation project performance should be evaluated based on the process rather than on the expected outcome. Concurrently, based on the literature review and the cases, three sets of KPIs with 13 specific KPIs useful for radical innovation projects are proposed.
The paper addresses a core challenge in using established KPIs in a radical innovation context. The paper gathers and synthesizes a range of measurement points suitable for radical innovation projects and provides specific suggestions for appropriate metrics that innovation managers can use.
An important part of large firms’ radical innovation competency is tied to an appropriate managerial mind-set and a system that facilitates experimentation. Accordingly, organizational setup, culture, processes, launch strategies and top management involvement have a major effect on the success of radical innovation. From a management viewpoint, radical innovation projects are characterized by higher uncertainty and absorption of new knowledge for the firm, as well as exploration of new markets, technologies and/or business models. Therefore, suitable managerial practices to support these projects vary substantially from those supporting incremental innovation projects.
Throughout the radical innovation project life cycle of Discovery (exploration), Incubation (experimentation) and Acceleration (development) (O‘Connor et al., 2008), suitable key performance indicators (KPIs) (metrics) must be applied. During the Acceleration phase, radical innovation projects are matured to a point, where they should be measured in line with established metrics for incremental innovation within firms. However, in the innovation front-end (Discovery and Incubation), both the process and expected output have a much lower degree of predictability than in incremental innovation projects. Because radical innovation projects have distinct features differing from incremental innovation projects, commonly used metrics such as time to market and net present value provide little use and may even be harmful for project progress.
Illustrating the above, O‘Connor et al. (2008) discuss IBM’s project for silicon-germanium alloy for integrated circuits (from the 1990s). Here, the radical innovation project led by Bernie Meyerson had to be bootlegged and protected from the rest of the R&D organization to survive. The initial business model was misaligned, and subsequent sales growth and market expectations showed lower than expected fiscal returns. This nearly killed the project. A modification of evaluation metrics had to be made to represent the nature of the project. Silicon-germanium, a highly efficient semiconductor alloy, is still among the best-performing technology platforms for computer chips today. It currently competes with, and possibly outperforms, Intel and their pure silicon-based chips in the computer-chip efficiency race (Armasu, 2015).
Radical innovation projects are uncertain, long-term investments that often target new business areas for the distant future (5+ years). Therefore, metrics should be adjusted to meet demands and criteria of success suitable for this type of project, as commonly used metrics fit poorly if firms are to move beyond an incremental innovation strategy (Christensen et al., 2008).
Previous research has emphasized the importance of using suitable metrics for incremental and radical innovation (Henttonen et al., 2016; Joh and Mayfield, 2009; Paulson et al., 2007; Griffin and Page, 1996). There is, however, a need for a clearer understanding of the damaging effects incurred by using “traditional” product development metrics in the early life cycle of radical innovation projects. Moreover, a further discussion of suitable metrics for the early stages of radical innovation projects’ life cycle is called for. In this paper, frequently used metrics in innovation management are examined and discussed in terms of their usability with respect to the contrast between incremental and radical innovation projects. This is followed by a presentation of challenges with innovation project measurement in three industry-leading global firms. This leads to a discussion of suitable metrics for radical innovation activities.
1. Frequently used metrics for innovation projects
The term “key performance indicator” has been widely used in the management literature and refers to identifying key activities of a value-creation process in the firm and generating a way of measuring those activities (Kaplan and Norton, 1996). For the innovation management literature, key performance indicators have been researched mostly in relation with non-radical innovation projects. The popular measures used to gauge success are either firm- or product-related, which often target markets (e.g. market size and time to market) and finances (net present value or similar) and whether the pre-established plan is followed (Blindenbach-Driessen et al., 2010; Henttonen et al., 2016).
These measures very rarely target specifically radical innovation projects. One implication is that evaluations will be largely focused on fiscal output and expected market performance, rather than the process itself. Frequently used metrics for innovation projects, their applicability and their implications for radical innovation projects are given in Table I.
The metrics mentioned in Table I, covering both pre- and post-launch, either will constitute a poor fit if used or simply will not be applicable because of the different time horizons in project life cycle periods in incremental innovation projects compared to radical innovation projects.
For radical innovation projects, the technological route or market feasibility assessment may deviate from the preexisting assumptions, and new opportunities may occur during the course of five or more years of development. A key challenge in measuring progress in radical innovation projects is, therefore, firms’ inability to follow pre-set goals and measure project performance according to these goals during the project period. Challenges may also arise post-market launch, as new market learning may be needed for successful adoption and impact (Feiereisen et al., 2013).
To further investigate the challenge of finding appropriate metrics for radical innovation, the authors conducted case studies in three large, international firms that are global leaders in their respective fields. All three firms employ more than 5,000 people and have an annual turnover exceeding US$2bn. The firms spend between 6 and 13 per cent annually on R&D, and all of them have a proven track record in incremental innovation (two of the firms spend three times the industry average on R&D, and the third firm is on par with the industry). Pseudonyms of GreenCO, HeavyCO and MasterCO are used. A total of 13 in-depth semi-structured interviews with managers and directors of innovation were conducted at the firms. All interviews were recorded, transcribed and coded. This was further supplemented with documentation from project tools, strategy workshops, seminar work and publicly available information.
2. Understanding the measurement challenge: case evidence from three large firms
As radical innovation projects have different characteristics from incremental innovation projects, and frequently used metrics, such as net present value, introduced challenges once applied in a radical innovation context. According to their innovation director, HeavyCO was experiencing signs of low performance when using their established metrics:
It is flagged as a low net present value project. Senior management can deliberately say, this is a strategic project. But at every point, they will be told, this has no money, this has no money.
Even though firms are following the suggested management practices of radical innovation, metrics stemming from incremental innovation activities provided substantial challenges for HeavyCO. Another example was given from MasterCO:
What really will resonate in 90 per cent of the rest of the organization is: “How much have you sold? What is the bottom line?” “Yes, but we have a strong portfolio…”, or, “Sure, but how much have you sold?” And that is the name of the game.
Applying commonly used metrics for radical innovation projects has created challenges in the case firms especially because the initial focus of radical innovation project management is to conceptualize and experiment and, through this, gradually reduce the uncertainty affiliated with the project. Both the expected financial outcomes and projected time-to-market provide challenges for radical innovation project activities when compared to incremental innovation activities. The time-horizon also created issues in GreenCO, as the cycles of how often projects were measured using the commonly used evaluation criteria were too short:
If I have to choose a radical or an incremental project, it depends on what you gain and what you lose. But the difficulty is, we do these annual evaluations based on results. And if you do something really well, it may take a longer time to see that.
To resolve measurement challenges, the MasterCO respondent argued that they would prefer to use forward-looking rather than backward-looking metrics. MasterCO consolidated the information on what the organization had learned throughout their work with radical innovation. This enabled project managers to improve legitimacy toward sponsors in senior management. A concluding remark from a manager in MasterCO was related to the need for metrics for radical innovation projects:
In the beginning, we said, “Well, we shouldn’t have key performance indicators; we shouldn’t be measured. We cannot create a budget because there is too much uncertainty”. And I think that is nonsense. I think that it is an entirely different perspective than if you look into a factory making 10 million units per year with some well-defined variants. In reality, we need just as much structure to be able to attract resources. We have to find other ways of getting structure out of chaos to be able to communicate with our surroundings.
The illustrative examples above are a part of a larger case study with these three firms. As it is arduous to use commonly used metrics for radical innovation projects, there is a crucial exercise in developing replacement metrics for these projects.
3. Metrics for radical innovation projects
Based on a review of the literature and case study insights, we suggest three applicable sets of KPIs for assessing radical innovation project performance.
3.1 Market orientation
A commonly discussed feature of radical innovation is technological novelty. Certainly, the technological progression radical innovations can introduce may have a breakthrough nature. However, many technologically novel products fail the actual market test, and firms should therefore pay attention to how they are oriented toward the market for their radical innovation activities (Table II).
3.2 Learning and future opportunities
Radical innovation projects are often a part of a larger platform investment in firms. Here, firms have the possibility to cross-fertilize learning and investments across opportunities and business segments. Firms should therefore encourage opportunities that ensure positive conditions for new opportunities to emerge and even to foster growth in existing business segments.
Another aspect will be to go/terminate decisions. In case of project termination, firms should make high-quality termination reports to de-brief key learning from the process.
Finally, firms should also be able to assess how they use and grow core competencies, including the increase in the relevant knowledge base of the firm. Examples of firms that have been actively working with innovation as a holistic innovation system to boost the knowledge base are 3M[3], General Electric and Coca-Cola (Alsever, 2015) (Table III).
3.3 Resource dedication
Radical innovation activities may need to compete for resources from the firm, and it is therefore crucial to have a project overview with a description of resources required to support projects. HeavyCO had been focusing on keeping the proportion of financial resources allocated to their radical innovation projects low. However, these projects involved substantial intangible resources in the form of highly skilled, cross-functional teams. In addition, these employees had contact with the existing resource base of the firm, enabling them to efficiently and quickly gain access to key stakeholders of the firm.
When radical innovation projects mature in the pipeline, they will eventually require an increased allocation of financial resources to get to market. Firms should ex ante decide whether the resources will be available when they are needed. For areas of higher uncertainty, resources can be shared among firms. This includes sharing of the knowledge base but could also represent a pool of test equipment. For example, GreenCO had been sharing resources with external partners on test equipment for a project for a substantial period. Access to these external resources enabled GreenCO to vastly accelerate planned pilot tests, not to mention saving the firm millions of dollars, as they did not have to build a pilot plant (Table IV).
4. Conclusion: measuring value creation rather than value created
Established companies across industries have developed comprehensive toolsets for managing innovation projects and portfolios. Most of these tools and approaches are well suited for incremental innovation, but not for radical innovation. Highly uncertain radical innovation projects demand toolsets that are unlike those that perform well in the realm of incremental innovation.
Building on the radical innovation literature and the case studies of three global firms, three sets of metrics for radical innovation are proposed: market orientation, learning and future opportunities and resource dedication.
KPIs for radical innovation projects deviate from the traditional R&D project measures and should be adapted to fit an uncertain environment. The proposed metrics are targeted for the front-end, i.e. before the commercialization phase. In the commercialization phase, commonly used metrics have better usability. Therefore, innovation managers should pay close attention to the overall portfolio and adopt an appropriate set of measures for different projects depending on their maturity and type.
The findings are based on the literature and a case study on three firms that have been actively working with radical innovation for several years. It is not expected that the metrics presented here are fully exhaustive or provide an immediate “silver bullet” for radical innovation project success. The benefits of adoption of any type of measure depend on who is using the measures. Different kinds of managerial biases (e.g. group thinking, pet projects and confirmation bias) can hamper the potentially useful information available. Nevertheless, the study does discuss immediate and pertaining issues with using established metrics for radical innovation. The study provides useful metrics that can be part of a more holistic and effective assessment of radical innovation projects.
