The hunt for computerized support in information security policy management: A literature review

Today, the not-so-big news is that governments, organised criminals and hacktivists are attacking organisations’ information and information systems. As information has become the lifeblood of modern society it has also become a valuable asset. Therefore, it is not surprising that information security management, where the purpose is to safeguard an organisation’s information assets, has become an important strategic issue (Van Niekerk and Von Solms, 2010). Organisations are in a situation where knowledgeable employees need to be the first line of defence. Unfortunately, breaching organisations still does not typically require advanced technical skills; many victims are tricked into opening attachments and clicking on links (Palmer, 2017; Albors, 2016; Kelion, 2013). In other cases, breaches are caused by employees’ malicious actions, such as in the example of Morgan Stanley (Gara, 2015).

A fundamental method for addressing insider risk is to adopt information security policies. Existing literature uses the term information security policy (ISP) concepts in slightly different ways. Here we associate ISP “with its non-technical, organizational variant” (Cram et al., 2018), focusing on the strategic and operational levels. In other words, policies addressing top management’s strategic direction with regard to information security as well as those including issue-specific guidelines and procedures that employees must comply with on a daily basis (Baskerville and Siponen, 2002; Withman, 2008). However, over the years, we have continued to see that employees’ poor compliance with ISPs is a persistent problem for many organisations (Ernst and Young, 2008; Ernst and Young, 2010; Pwc, 2014); findings that have also been supported by researchers (Herath and Rao, 2009; Nash and Greenwood, 2008; Siponen et al., 2014; Stanton et al., 2005; Johnston et al., 2016).

At the same time, it has been shown that about half of all breaches caused by employees are accidental (Vroom and Von Solms, 2004; ENISA, 2014). Furthermore, research (Adams and Sasse, 1999; Stahl et al., 2012; Karlsson et al., 2017) has shown that the design and implementation of ISPs can sometimes impair employees’ information security behaviour. They have found that policies can be cumbersome, contradictory and incompatible with existing work practices. Hence, managing ISPs to create a solid foundation for employees’ information security behaviour is challenging.

In recognition of this, the number of ISP management studies has increased (Cram et al., 2018) and researchers have contributed towards different kinds of support for the development, implementation and evaluation of ISPs, for example. These contributions address a wide variety of ISP management issues, such as ambiguity (Buthelezi et al., 2016), goal coherency (Karlsson et al., 2017), awareness (Gadzama et al., 2014) and methods to increase compliance (Saran and Zavarsky, 2009). Moreover, these studies target different phases of the ISP management process. Some studies focus on one particular phase of such a process (Doherty and Fulford, 2006; Lindup, 1995). Other studies (Coertze and von Solms, 2013; Knapp et al., 2009) do not treat phases of ISP management as isolated phenomena; instead they recognise that there is interaction between them. Thus, these studies cover more than one phase.

Without question the above-mentioned studies are examples of important contributions. However, they seem to rely on manual support for ISP management. This means that information security managers need to keep track of all the details and coordinate all actors involved, need to be knowledgeable in particular inquiry techniques and have to handle the interaction been different phases of an ISP management process. For example, if they want to carry out a compliance analysis targeting employees they need to be knowledgeable in theories and statistical techniques used in such inquiries. In addition, they need to find a way to reach out to employees and later integrate the results into other phases of the ISP management process.

One way forward is to automate activities associated with ISP management, using computerised support. This kind of support has been around for a long time in information system domains, such as systems development (Jürjens and Shabalin, 2007; Orlikowski, 1993; Pavlidis et al., 2011), method engineering (Efendioglu et al., 2016; Harmsen, 1997; Karlsson and Ågerfalk, 2012; Rossi et al., 2004) and project management (Caniëls and Bakens, 2012; Jaafari and Manivong, 1998; Raymond and Bergeron, 2008; Teixeira et al., 2016). Even though such support is not without challenges, benefits have been reported. For example, when it comes to computerised support for project management, Raymond and Bergeron (2008) concluded that these systems “have direct impacts on project success, as they contribute to improving budget control and meeting project deadlines as well as fulfilling technical specifications”.

Computerised support can also be found in the area of ISP management. For example, already in the start of this century Hoppe et al. (2002) and Vermeulen and Von Solms (2002) presented an example of computerised support for information security management; a software that included components for ISP management. A much later example is Syamsuddin and Hwang (2010). They presented a tool for the evaluation of ISP performance. Even though this kind of support has been reported on, it is not known to what extent researchers have actually suggested computerised support for ISP management as a complement or alternative to manual ways of working.

With this in mind, the aim of this paper is to survey existing ISP management research to scrutinise the extent to which manual and computerised support has been suggested, and the way in which the suggested support has been brought about. To the best of our knowledge, there exist a couple of recent literature reviews that systematise and synthesise existing ISP management research (Cram et al., 2018; Flowerday and Tuyikeze, 2016; Alotaibi et al., 2016). However, the extant reviews have yet to consider to what extent researchers have suggested computerised support for ISP management or to what extent suggested support relies on manual work. Specifically, we pose the following research questions:

Our results are based on a review of ISP management research published between 1990 and 2017. The study is based on a substantial list of papers that initially consisted of 1,880 research papers, including duplicates. Of these, 123 papers were singled out for further analysis (more details are given in Section 3). Our systematic review provides valuable insights into the extent to which manual and computerised support has been suggested by researchers to ease the burden of information security managers. We have also been able to discuss to what extent the interaction between ISP management phases has been addressed and the types of research methods most commonly used in existing studies. This paper thus contributes with a computerised support perspective on current ISP management research and pinpoints areas for future research.

This paper is structured as follows. Following this introduction, Section 2 describes existing literature reviews of ISP management research. Section 3 presents the research method adopted for our literature review. In Section 4, we present the results of our review. In Section 5, we discuss how our findings impact ISP management research. We end the paper with a short conclusion in Section 6.

ISPs are seen as an important managerial tool for regulating employees’ information security behaviours, and this research stream has gained significant attention during the last two decades (Cram et al., 2018). A number of literature reviews (Siponen and Oinas-Kukkonen, 2007; Siponen et al., 2008; Soomro et al., 2016; Zafar and Clark, 2009) have highlighted ISP studies as an important category within a broader information security literature. Several literature reviews have addressed subsets of the ISP management process such as security awareness (Lebek et al., 2013; Lebek et al., 2014), culture (Karlsson et al., 2015) and employees’ behaviours and compliance (Guo, 2013; Sommestad et al., 2014; D'arcy and Herath, 2011; Wall et al., 2015; Siponen and Vance, 2014). Among these subsets of ISP management topics, employees’ behaviours and compliance with information security policies has garnered the most attention. Cram et al. (2018) argue that the previous reviews of the ISP management literature make it difficult for researchers and practitioners to grasp the current state of knowledge on the whole process of the development, implementation, and effectiveness of ISPs in organisations.

In total, we have found only three reviews (Tuyikeze and Flowerday, 2014; Järveläinen, 2016; Cram et al., 2018) of ISP research covering the entire ISP management process. Tuyikeze and Flowerday (2014) review 21 documents with the objective of understanding the ISP development life cycle. Based on that understanding they define a model for the formulation, implementation and enforcement of an ISP in an organisation. They categorise ISP development and implementation methods found in current literature into five phases:

1. risk assessment;

2. policy construction;

3. policy implementation;

4. policy compliance; and

5. policy monitoring, assessment and review.

Thus, the most important contribution of this literature review is to provide a process model for ISP management. The authors do not discuss any possible computerised support for this process or to what extent the ISP development and implementation methods identified in the literature cover the different phases of the ISP management process.

Cram et al. (2018) review 114 ISP-related publications from 34 journals and synthesised the current knowledge in the form of a research framework. The authors categorised the existing ISP literature into five relationships covering the entire ISP development process:

1. influences on the design and implementation of ISPs (e.g., standards and guidelines);

2. the influence of ISPs on the organisation (e.g. security culture) and individual employees (e.g. socioemotional well-being);

3. the influence of the organisation and individual employee factors on ISP compliance (e.g. dispositional traits, sanctions, rewards);

4. the influence of ISP compliance on organisational objectives (e.g. the frequency of security incidents); and

5. adjustments to ISP design (e.g. policy updating and maintenance). Building on the analysis the authors identified research gaps that can be used as a basis for future research.

Cram et al. (2018) conclude that the vast majority of the current ISP literature focuses on understanding the drivers of ISP compliance, while fewer studies consider other aspects of ISP management. Studies considering other parts of ISP management are often based on practical considerations of managers responsible for the design and implementation of policies and not clearly founded in theory. Many of these studies are also conceptual, which according to Cram et al. (2018) may limit the possible impact of these studies for practice. They also argue that almost none of the reviewed studies consider ISP management as an ongoing process where changes occur to policies over time. The authors argue that taking an iterative perspective towards ISP development, implementation, monitoring and adjustment, would uncover new and important insights, but at the same time applying such a perspective adds an additional complexity for information security managers. Cram et al. (2018) show that this type of research focuses mostly on conceptual factors that need to be considered when adjusting information security policies over time. However, almost no empirical research exists on how such adjustment is (or should be) managed in practice, leaving information security managers without support regarding this challenging task.

Although Cram et al. (2018) discuss the need for management support in the process of developing, implementing, monitoring, and adjusting ISPs, they do not assess to what extent existing research has proposed computerised support for this process. Moreover, they do not explicitly discuss to what extent existing studies cover the different phases of the ISP management process. Finally, although Cram et al. (2018) present an appendix containing an account of the research method used in the reviewed studies, they do not discuss these findings.

Järveläinen (2016) conducted a literature review of 46 papers focusing on ISP development. This review acts as a background to putting forth an integrated approach to ISP development and business continuity planning. She concludes that:

• ISPs are supposed to be a comprehensive set of long-lasting, general, technology-independent principles.

• The main objective of an ISP is to ensure the confidentiality, integrity and availability of data for an organisation.

• ISP should be developed based on the assessment of current and previously recognised risks.

• An organisation can have several different kinds of ISPs for different stakeholders and for that reason different groups of stakeholders should be involved in the ISP development.

Järveläinen (2016) does not discuss computerised or non-computerised support for ISP development.

A structured literature review is a “crucial endeavour” (Webster and Watson, 2002) for a research community, as it provides an overview and synthesis of what has been accomplished so far. It is therefore vital that it is carried out in a rigorous and comprehensive manner (Levy and Ellis, 2006). Our research method is quite straightforward on the general level, consisting of four main steps. That said, their implementation has been far from instrumental and below we discuss the details related to selecting and classifying papers. The general outline of the research process is as follows:

1. Elseviers’ database Scopus and Clarivate Analytics’ database Web of Science were used to search for potential papers (see Section 3.1).

2. The abstract of each paper was read and an initial decision was made as to whether the research related to ISP management (see Section 3.1).

3. The introduction, related research and results sections were read for each paper in search of:

   • the kind of research topics that the paper addressed to map them onto different phases of ISP management. We used the process model proposed by Flowerday and Tuyikeze (2016) for classification of the research topics (see Section 3.2).

   • the different types of support – manual and/or computerised – that the paper suggested (see Section 3.3).

   • the number of phases of ISP management that the paper covered, to classify the addressed interaction between different phases (see Section 3.4).

4. The research method section of each paper was read (if such a section was found) and the research methods that the researchers claimed to have used were noted. These were classified using an extended version of Mingers’ (2003a) research method framework (see Section 3.5).

Initially Steps 3 and 4 were carried out in an iterative pattern where the authors individually classified the same set of papers and later compared the classifications made. This was done to arrive at a stable interpretation of our analytical framework, i.e. where the authors classified the papers in the same way. A consistent use of the analytical framework was reached after three iterations. The first author then carried out the analysis of the remaining papers. Ambiguous papers were discussed by the authors and joint decisions were made. The result of the detailed analysis is found in  Appendix 6; a summary is presented in Section 4.

ISP management research appears both in conference proceedings and in international journals. Therefore, we aimed for an inclusive selection of papers. Our search of papers was carried out using Scopus and Web of Science. Scopus is “the largest abstract and citation database of peer-reviewed literature”, indexing 20,000 peer-reviewed journals and 5.5 million conference papers, and Web of Science covers over 12,000 of the most high impact journals and over 150,000 conference proceedings (Franke and Brynielsson, 2014). Scopus provides a good coverage of the journals on the Association of Information Systems’ journal ranking list and specific information security journals and conferences (Karlsson et al., 2015). All in all, these two databases together provide a good coverage with regards to ISP management.

The search included papers published on the databases between 1990 and 2017. The year 1990 was selected because this was when Straub and Nance (1990) published their paper on computer abuse; it is an early example of information security management research that relates to ISPs. Our search included journal papers, conferences papers and workshop papers, regardless of the geographic region to gain an inclusive view of the field.  Appendix 1 shows the search criteria that were used when searching in the databases; search fields included paper title, abstract and keywords. The set of keywords we used to construct the search criteria grew as we learned more about ISP management research based on our searches and reading of papers. The use of multiple search queries resulted in a substantial list of 1,880 research papers, including duplicates. After eliminating duplicates and papers that did not meet our inclusion criteria in Table I we ended up with a net list of 130 papers. In the end we were able to access and analyse 123 papers. The papers we were unable to access are listed in  Appendix 3.

This reduction in relevant papers was due to our inclusive search strategy. It meant that papers were included in the first dataset if they were related to ISPs. For example, an extensive number of studies on employees’ compliance with ISPs were therefore included. However, these papers have different emphases on ISPs. For example, Hedström et al. (2011) used actual ISPs as reference objects in the compliance analysis and are therefore included. In addition, our inclusive search strategy meant that we, in our first dataset, came across papers about computerised tool support for implementing and automating technical information security policies. For example, Subramanian et al. (2011) proposed the PCAL-analyser, a computerised tool to ease the burden of security administrators when analysing technical ISPs; they provided an illustrative example comparing two Unix server security policies. Although such papers are about computerised support, they obviously fall short of the type of ISPs we are interested in. Nonetheless, we chose the abovementioned search strategy to ensure we would not miss any papers by making the search parameters too narrow.

The first component of our classification framework addresses research topics in relation to phases of ISP management. Existing literature provides more than one process model for such an analysis (Howard, 2007; Kadam, 2007; Peltier, 2004; Flowerday and Tuyikeze, 2016), which means that such a classification can be done in slightly different ways. We have chosen to use Flowerday and Tuyikeze (2016) ISP management process model; this is one of the more recent models and is based on a literature review. Hence, it gives us the possibility of comparing our findings with existing wisdom in the field.

As discussed earlier, Flowerday and Tuyikeze (2016) divide ISP management into five phases. We organised each of the 123 papers into one or more of these phases, because a paper can cover more than one part of the process model. The five phases are risk management, policy construction, policy implementation, policy compliance and policy monitoring. The risk management phase focuses on ISP management research that addresses an organisation’s need to “identify the threats, vulnerabilities and risks that need to be mitigated” (Flowerday and Tuyikeze, 2016). Policy construction is about activities and aspects to consider when developing an ISP. This includes challenges related to writing a detailed policy and consultation with stakeholders. Policy implementation represents deployment activities and aspects of ISP management. This phase focuses on policy awareness, education and training. Policy compliance addresses employees’ compliance and non-compliance with ISPs and different ways of assessing compliance/non-compliance. The final phase, policy monitoring focuses on audit and review aspects related to ISPs. Based on the audit and review results changes can be proposed.

Our interest in manual and computerised support for ISP management led us to complement the model put forward by Flowerday and Tuyikeze (2016). We added two categories – manual and computerised support – that cut across the five phases discussed above. This created an analytical matrix that enabled us to identify research that has addressed manual and/or computerised support for the different phases of ISP management. In this study, we define manual support for ISP management as any justified and explicit guidance that assists an information security manager in terms of working with ISP management. For example, in our analysis of Karlsson et al. (2017) we found that they present eight criteria that can guide construction of ISPs. Computerised support for ISP management was defined as a software that assists an information security manager in working with ISP management. Thus, when analysing, for example, Coertze and von Solms (2013), we concluded that the presented software includes computerised support for risk management, policy construction, policy compliance and policy monitoring.

As discussed in the Introduction, there is interaction between different ISP management phases. This is also evident in the framework proposed by Flowerday and Tuyikeze (2016), where the five consecutive phases build on each other. For example, an ISP is developed based on threats and vulnerabilities identified during risk management. Compliance on the other hand needs to be understood and assessed based on the existing ISP. We have therefore analysed to what extent researchers have acknowledged the interaction between different phases in ISP management, i.e. the number of phases that individual studies have covered.

Table II presents this component of our analytical framework, which is a straightforward and intuitive use of the five phases suggested by Flowerday and Tuyikeze (2016). The leftmost column contains the three levels of interaction, and the second column shows the operational definitions. The general idea is that higher interaction includes more ISP management phases, while a lower interaction would focus more on a specific part of ISP management.

The fourth and final component in our classification framework concerns the research method used. Given that several frameworks are available for this purpose (Galliers, 1992; Mingers, 2003a; Palvia et al., 2004; Dwivedi and Kuljis, 2008), it is inevitable that such a classification can be carried out in slightly different ways. We used an extended version of Mingers’ (2003a) framework, even though it adopts a rather inclusive view of research methods. The main reason is that we benefited from the opportunity to make nuanced characterisations of the research methods used. Mingers’ (2003a) original framework includes 13 types of research methods, to which we have added two. The first is design science, which has received increasing attention in recent years, most notably after he developed his framework. Finally, we have added the category “no method” to capture cases where the authors have not explicitly stated or described the research method(s) they have used.

All in all, our extended framework includes 15 types of research methods: action research, case study, consultancy, critical theory, design science, ethnography, experiment, grounded theory, interview, no method, participant observation, passive observation and measurement, qualitative content analysis, simulation and survey/questionnaire/instrument. The operational definitions of these types of research methods are found in  Appendix 4. In our classification, we acknowledged the possibility of studies using a mixed-method (Creswell, 2003) or multi-method (Brewer and Hunter, 1989) approach, i.e. when a study includes the use of more than one research method.

In this section, we present a summary of our literature review, structured into three subsections. The first subsection focuses on what are the most investigated phases of ISP management. We start by assessing research on manual support for ISP management and then continue with research on computerised support for ISP management. In the second subsection, we concentrate on to what extent research has addressed the interaction between ISP management phases. Finally, the last subsection presents the dominant research methods in the assessed studies. In addition, we analyse what kind of research methods have been used to research the interaction between the ISP management phases. The details of our analysis are found in Appendix 5–6.

The overall analysis shows that 117 papers have addressed manual support for ISP management, while only 7 papers have approached computerised support. It is clearly evident that researchers have put more emphasis on manual support than on computerised support for ISP management. Figure 1 characterises existing research further using a bubble chart. The horizontal axis contains the five phases found in Flowerday and Tuyikeze (2016) framework and the vertical axis shows the two types of ISP management support. A bubble chart shows three dimensions of the data. The size of a bubble is proportional to the frequency of papers that are in the pair of categories corresponding to the bubble coordinates. Our analysis suggests that all phases have been addressed for both manual and computerised support for ISP management, although with different degrees of emphasis. The shares have been calculated based on the total number of times the phases in the framework have been addressed.

In the bottom-left of Figure 1, we find risk management. The figure shows that it is the phase that researchers have devoted least attention to when it comes to manual support for ISP management. We found that the majority of papers in this area focused on ways of identifying risks, threats and vulnerabilities, and proposing solutions to mitigate them properly (Palmer et al., 2001; Kadam, 2007; Simms, 2009; Tuyikeze and Pottas, 2011; Ismail and Widyarto, 2016). For example, Kadam (2007) suggests an approach for formulating ISPs: the first stage in such a process is threat identification and vulnerability assessment. Hence, the suggested stage is about risk management and Kadam suggests a tool for risk analysis using confidentiality, integrity and availability. The tool also includes possibilities for adding references to where the vulnerabilities are addressed in the ISP. The goal seems to be increased traceability in the ISP management process.

Construction is the second phase from the left in Figure 1. This phase is the third largest area of research concerning manual support for ISP management. Construction research has addressed what constitutes an ISP and why ISPs should be developed (Al-Hamdani and Dixie, 2009; Hong et al., 2006; Cosic and Boban, 2010; Koziel, 2011). Researchers have also addressed how to formulate ISPs (Lindup, 1995; Tuyikeze and Pottas, 2011; Lopes and Oliveira, 2015c; Niemimaa, 2016), i.e. focusing on construction as an activity. We also identified a number of studies that addressed what factors should be considered during such processes to end up with effective ISPs (Gritzalis, 1997; Siponen and Iivari, 2006; Hong et al., 2006; Renaud and Goucher, 2012). Finally, researchers have emphasised the importance of considering business requirements and goals when formulating ISPs (Doherty and Fulford, 2006).

Implementation is the third phase from the left in Figure 1, and the second largest area of research. We identified implementation research on factors that affect the effective adoption and enforcement of an ISP (Fragos et al., 2007; Hong et al., 2006; Yayla, 2011). Researchers, such as Karyda et al. (2003) and Kadam (2007), have also addressed when it is appropriate to implement ISP and who should be involved in such implementation. Another implementation aspect addressed is dissemination and successful deployments of ISPs (Fulford and Doherty, 2003; HöNe and Eloff, 2002b). Research on implementation has also devoted attention to the importance of employees’ awareness and understanding of ISPs (Gadzama et al., 2014; Ghazvini and Shukur, 2016; Ghazvini and Shukur, 2017). Finally, researchers have acknowledged the importance of paying attention to organisational differences during implementation. For example, Al-Hamdani and Dixie (2009) discuss how ISP implementation in small organisations differs from implementation in larger organisations.

Compliance is the fourth phase in Figure 1, and is by far the phase that researchers have devoted the most attention to over the years. Compared to the other phases, compliance research seems to have a quite unified focus; trying to increase the understanding of what explains employees’ compliance and non-compliance. Such an increased understanding would feed into the other phases of ISP management; for example how to construct an ISP in a way that will improve compliance. Our review reveals that an extensive number of theories have been applied in this type of research. First, we identify a large number of studies that draw on theories from psychology. One such theory that has attracted a lot of attention is theory of planned behaviour (Aurigemma and Mattson, 2017a; Bulgurcu et al., 2009a; Hu et al., 2012; Bulgurcu et al., 2010; Gerber et al., 2016). Protection motivation theory (Pahnila et al., 2013; Herath and Rao, 2009) is another theory from psychology that has been used to explore employees’ compliance and non-compliance with ISPs. Second, researchers have also used theories from criminology to explain employees’ compliance and non-compliance, such as deterrence theory (Chen et al., 2012), neutralisation theory (Bauer and Bernroider, 2017) and social control theory (Hsu et al., 2015). Compliance research has also drawn on theories from sociology. Here we found theories such as social action theory (Hedström et al., 2013). We also identified one theory, value-based compliance theory (Hedström et al., 2011; Kolkowska et al., 2017), which is specifically constructed in the sub-field of ISP management. One thing that is striking with regard to compliance research is that several of the abovementioned theories or parts thereof are frequently combined (Aurigemma and Mattson, 2017b; Ifinedo, 2012; Ifinedo, 2016; Kajtazi and Bulgurcu, 2013; Siponen et al., 2006; Sohrabi Safa et al., 2016; Hou et al., 2011; Humaidi and Balakrishnan, 2015a).

Finally, monitoring is the fifth investigated phase in Figure 1. As is shown in the figure, this phase has attracted relatively few papers, compared to research on compliance, construction and implementation. We have identified monitoring research that assesses and reviews the effectiveness of ISPs (Höne and Eloff, 2002a; Karyda et al., 2003; Vroom and Von Solms, 2003; Corpuz and Barnes, 2010). Researchers have also focused on evaluating to what extent ISP objectives and business objectives are aligned (Mader and Srinivasan, 2005; Knapp et al., 2009; Hong et al., 2006). Finally, researchers have addressed solutions which can improve such an alignment (Talbot and Woodward, 2009; Ismail and Widyarto, 2016; Karlsson et al., 2017).

Figure 1 shows that research on computerised support for ISP management research is far less common than research on manual support. In total we identified seven papers in the former category. Starting with risk management, we identified three papers addressing this phase (Vermeulen and Von Solms, 2002; Coertze et al., 2011; Coertze and von Solms, 2013). Vermeulen and Von Solms (2002) propose a software tool – information security management toolbox – to automate a number of steps of information security management. The software tool they suggest is an implementation of a limited part of a method for information security management. Risk management is among the parts supported by the software tool, providing support in eliciting information security requirements that can be used as input for ISP construction. Later, Coertze et al. (2011) and Coertze and von Solms (2013) seem to have extended this work. However, the software tool has different names and seem to cover our investigate ISP management phases to different extent. In addition, it is difficult to judge whether the toolbox focuses on ISPs in particular or addresses information security management, of which policies are one component.

Construction has been addressed in three papers (Vermeulen and Von Solms, 2002; Coertze et al., 2011; Coertze and von Solms, 2013). These papers all concern the same computerised support – the information security management toolbox. Vermeulen and Von Solms (2002) show in their implementation model of the tool how business-specific requirements help in selecting organisation-specific ISP statements. Although they do not provide any details, Coertze and von Solms (2013) claim that the toolbox supports the “drafting of dynamic policy”.

Implementation has been addressed in one paper by Busch et al. (2016). They designed and implemented a number of persuasive information security features in an interactive Web-based tool. The purpose was to increase employees’ awareness and knowledge of an organisation’s ISP. Hence, this is related to implementation. The goal of the study was to evaluate the effectiveness of these different features.

Figure 1 shows that compliance has been addressed in four papers (Busch et al., 2016; Saran and Zavarsky, 2009; Coertze and von Solms, 2013; Wang and Li, 2015). The above-mentioned paper by Busch et al. (2016), which evaluates the effectiveness of persuasive information security features, also promotes ISP-compliant behaviour. Saran and Zavarsky (2009) studied a non-compliance problem with an Internet policy in an insurance company. Taking an inclusive view on computerised support for ISP management, they used emails to improve the situation. They tested and evaluated different methods for increasing ISP compliance: they investigated whether compliance was increased by sending different types of reminder emails to employees, saying that they needed to re-sign the organisation’s Internet Usage Agreement. Coertze and von Solms (2013) provide the possibility of conducting compliance analyses using their suggested computerised toolbox for information security management. Finally, Wang and Li (2015) used computerised support to visualise employees’ ISP compliance patterns.

Finally, monitoring has been addressed in two papers (Syamsuddin and Hwang, 2010; Coertze and von Solms, 2013). Syamsuddin and Hwang (2010) introduced a framework that guides managers when “evaluating information security policy performance”. The suggested framework, which was demonstrated as an Open Office Calc application, adopts the analytic hierarchy process to structure and understand performance. Their choice of demonstrator was based on their goal to show that this kind of support can be provided without the use of proprietary analytic hierarchy process software. The second paper is the previously discussed paper by Coertze and von Solms (2013). They argue that the toolbox components offer the opportunity to evaluate ISP management efforts carried out. Hence, such functionality is similar to the audit and review aspects put forth in the framework by Flowerday and Tuyikeze (2016).

In this section, we analyse to what extent research has addressed interaction between phases in ISP management. As discussed in Section 3.4, interaction means to what extent researchers have treated the different phases as separate silos (for example focusing only on construction) or have combined them in the same study (for example addressing risk assessment, construction and implementation). The details of the analysis are found in  Appendix 5 and Figure 2 presents an overview using a bubble plot. The figure is divided into an upper and a lower section containing analyses of research on manual and computerised support for ISP management respectively. The vertical axis shows the three types of interaction that we have defined earlier, i.e. how many ISP management phases have been included in a paper. On this axis, we also present the total number of papers that address a certain level of interaction. The horizontal axis shows the actual phases that have been acknowledged in research, using the five phases suggested by Flowerday and Tuyikeze (2016). Thus, in each intersection the bubbles show how much focus a particular phase has received in the context of a specific level of interaction (i.e. how many papers have been written on any particular interaction between phases).

The upper section of Figure 2 presents research on manual support and the acknowledged interaction between ISP management phases. This part of the figure shows that the number of studies decreases when the addressed interaction between phases increases. This might not be a surprising finding in itself, because studies addressing higher levels of interaction are more resource consuming.

We can conclude that most papers, 76 articles out of 117, are found in the low interaction category. This means that these studies have a silos approach to manual support for ISP management (Gritzalis, 1997; Fulford and Doherty, 2003; Fragos et al., 2007; Lopes and Oliveira, 2015a; Niemimaa, 2016), only addressing one phase. For example, Fragos et al. (2007) used the lens of circuit of power to understand the implementation of ISP in a public sector organisation. They showed that power relations are important during ISP implementation and that such relations need to be acknowledged. Although this is an important finding, this study is limited to a specific phase of ISP management, in this case implementation. Moreover, it is evident that research on compliance dominates when the acknowledged interaction is low. This should not be a surprising finding; as discussed above, compliance is the phase that has attracted the most attention in ISP management research overall. At the other end, we found that risk management is the phase that has received the least attention when the acknowledged interaction is low. We only identified one paper (Rees and Allen, 2008) that focused solely on this phase. Furthermore, only a small portion of the research has been devoted to policy monitoring as an isolated phase.

Figure 2 also shows that when the level of interaction is increased to medium – research combining two to four phases – the emphasis on different phases changes. We see that more focus is placed on construction and implementation, and less focus is placed on compliance. The details in  Appendix 5 reveal that studies addressing manual support and two ISP management phases frequently combine construction-implementation (Gaunt, 1998; Abraham and Chengalur-Smith, 2011), construction-compliance (Buthelezi et al., 2016; Choi, 2016) and implementation-compliance (Yang et al., 2011; Mavetera et al., 2015).

However, interest in including risk management and monitoring seems to have been low. Instead,  Appendix 5 shows that more attention has been devoted to monitoring when three or more phases are combined. In such studies researchers seem to have concentrated on combining risk management, construction, implementation and monitoring (Palmer et al., 2001; Simms, 2009; Corpuz and Barnes, 2010; Ismail and Widyarto, 2016). Hence, here compliance research received even less attention.

As Figure 2 shows, we only identified two papers in the high interaction category (Tuyikeze and Pottas, 2011; Knapp et al., 2009), i.e. that addressed a combination of all five phases. For example, Tuyikeze and Pottas (2011) present a detailed roadmap for ISP management, to ensure that ISPs are “comprehensive, effective and sustainable” with regard to organisations’ needs and regulatory requirements. Although the roadmap only includes four steps compared to the five phases found in Flowerday and Tuyikeze (2016) framework, the authors state that policy compliance should be considered part of policy monitoring and maintenance.

The lower section of Figure 2 summarises research on computerised support and to what extent interaction between ISP management phases has been addressed. We can conclude that research on computerised support with low and medium interaction dominates, although the numbers are much lower compared to research on manual support. Actually, we were unable to identify any papers in the high interaction category. Even though papers with such acknowledged interaction are few in the area of manual support, we still found some.

Given the few papers on computerised support for ISP management it is not possible to identify any patterns regarding how phases are combined when the addressed interaction changes. For example, we found that the three papers that use a silos model address three different phases. Saran and Zavarsky (2009) measured whether reminder e-mails of a policy re-release have an impact on compliance, while Syamsuddin and Hwang (2010) address monitoring with their non-proprietary analytic hierarchy process software. Wang and Li (2015) address compliance when visualising compliance patterns. When we examine existing studies with medium interaction, we see that all phases are included at least once. For example, Busch et al. (2016) in their paper on persuasive information security features combine implementation and compliance. The three papers (Vermeulen and Von Solms, 2002; Coertze et al., 2011; Coertze and von Solms, 2013) addressing the information security management toolbox have addressed different combinations of phases. Of these papers, Coertze and von Solms (2013) undertook the study that includes the highest number of ISP management phases, when introducing the ISP management toolbox to small, medium and micro-organisations. In this study, they address the combination of risk assessment, construction, compliance and monitoring.

The bubble charts in Figures 3 to 5 show our analysis of research methods used in existing research on ISP management. Figure 3 contains an analysis structured according to the two types of support – manual and computerised – that we are interested in. Figure 4 shows a more detailed analysis where the use of research method has been structured according to the phases in Flowerday and Tuyikeze (2016) framework. The upper section of this figure shows how research methods have been used in research on manual support; the lower section shows how research methods have been used in research on computerised support. Finally, Figure 5 contains an analysis of how frequently the research methods have appeared in studies with different types of interaction. This figure is also divided into two sections similarly to Figure 4.

In the figures we only present the research methods that have been used in existing research, which means the figures only contain a subset of our modified version of Mingers (2003a) framework. In total, we found that 14 different research methods were used for ISP management investigations. Thus, a wide variety of research methods have been used. That said, there are substantial differences in frequency of use. We can clearly see that four research methods – survey, interview, no method and case study – dominate ISP management research.

We start with an assessment of research methods used in research on manual support for ISP management. As is shown in Figure 3, survey is by far the most-used research method and seems to hold a special position in this kind of research. It is used in 63 out of 117 papers that address manual support. Moreover, as can be seen from the upper section of Figure 4, the majority of these papers focus on policy compliance (D'Arcy et al., 2014; Bulgurcu et al., 2010; Siponen et al., 2006). Indeed, the figure reveals that compliance research seems to have a strong emphasis on surveys. As this phase constitutes a large share of the research on manual support for ISP management it becomes quite natural that the number of survey studies is high. As is shown in Figure 5, the majority of the survey studies do not address any interaction between the phases in Flowerday and Tuyikeze (2016) framework. This is in line with our previous analysis in Figure 2, where we show that researchers have mostly studied compliance as an isolated phase in ISP management.

Continuing in Figure 3, we find three research methods that are roughly equal in size: interviews, no method and case study. They have each been used in about one-fifth of the investigated studies on manual support. Starting with interviews, the upper section of Figure 4 shows that most of this research is concerned with policy construction (Yusufovna, 2008; Karlsson et al., 2017), policy implementation (Karyda et al., 2003; Abraham and Chengalur-Smith, 2011) and policy compliance (Knapp et al., 2009; Hedström et al., 2011). Compared to the use of surveys discussed above, none of these phases have such a clearly dominant position as policy compliance has in relation to survey studies. Furthermore, Figure 5 reveals that interviews have been used to address low and medium interaction between ISP management phases.

Studies where the researchers have not explicitly described the used research method, classified as no-method papers in our framework, also account for about one-fifth of the studies reviewed. According to the analysis in Figure 4, we found that such papers are most frequently concerned with policy construction (Lindup, 1995; Siponen and Iivari, 2006), followed by policy implementation (Palmer et al., 2001; Tsohou et al., 2015). Figure 5 shows that not describing the used research method is equally frequent in studies with low and medium acknowledged interaction.

According to Figure 3, the case study is the fourth most frequently used research method; the fact is that there is not much difference in the frequencies of use of case studies, interviews and no-method papers. Moreover, Figure 4 shows that the use of case studies resembles the use of interviews; it has mostly been used to research policy construction (Yusufovna, 2008; Karlsson et al., 2017), policy implementation (Abraham and Chengalur-Smith, 2011; Lapke and Dhillon, 2015) and policy compliance (Hedström et al., 2011; Buthelezi et al., 2016). The details in  Appendix 6 show that these two research methods are frequently combined. Figure 5 also shows that case study research seems to share the same pattern of addressing low and medium interaction between ISP management phases.

When it comes to the remaining research methods in our framework, they have been used in fairly low numbers. As Figure 3 shows, all of them have appeared in ten or fewer studies on manual support. For example, qualitative content analysis has appeared in ten papers. Figure 4 shows that these studies have included a broad range of the phases in Flowerday and Tuyikeze (2016) framework. The reason is, according to Figure 5, that this research method is often part of studies with medium acknowledged interaction between ISP management phases. Continuing in Figure 3, participant observation and passive observation were only used in four studies each; given this low frequency it is not possible to identify any patterns in terms of how they have been used. Finally, we found a small number of studies that used ethnography (Niemimaa, 2016; Niemimaa and Niemimaa, 2017), action research (Lopes and Oliveira, 2015b; Lopes and Oliveira, 2015c; Lopes and Oliveira, 2016), grounded theory (Knapp et al., 2009; Balozian and Leidner, 2016), experiment (SanNicolas-Rocca et al., 2014), simulation (Doherty and Fulford, 2006), consultancy (Talbot and Woodward, 2009) and design science (Kolkowska et al., 2017). For example, in the last study, design science was used to develop a method for analysing reasons behind employees’ compliance and non-compliance. It should be mentioned that we did not identify any use of critical theory in research on manual support for ISP management.

Turning our attention once more to research on computerised support for ISP management, we can see in Figure 3 that fewer types of research methods have been used compared to those used in research on manual support for ISP management. Given that we have identified few papers on computerised support, it is natural to find a more limited set of research methods both with regard to breadth and frequency. Moreover, the limited number of papers on computerised support makes it difficult to identify any patterns when it comes to how research methods have been combined with different phases (Figure 4) and the addressed interaction between ISP management phases (Figure 5).

As is shown in Figure 3, the most-used research method is survey (Busch et al., 2016; Syamsuddin and Hwang, 2010; Wang and Li, 2015). It is followed by case study (Coertze et al., 2011; Saran and Zavarsky, 2009), and interview (Coertze and von Solms, 2013; Wang and Li, 2015), each of which have been used in two papers. We also identified use of experiment (Busch et al., 2016) and participant observation (Busch et al., 2016); each method has been used in one paper. Finally, we found one paper that did not give any account of the research method used (Vermeulen and Von Solms, 2002). This means we did not identify any papers on computerised support that used the following research methods: action research, critical theory, consultancy, design science, ethnography, grounded theory, passive observation, qualitative content analysis and simulation.

Our literature review considers the entire process of ISP management, similarly to Tuyikeze and Flowerday (2014), Järveläinen (2016) and Cram et al. (2018). It contributes to previous knowledge by explicitly focussing on the extent to which manual and computerised support have been suggested, and the ways in which the suggested support have been brought about. Figures 1 to 5 show the patterns we found in the current literature. Based on our findings, some notable lessons can be learned with regard to:

• the phases investigated in research on manual and computerised support for ISP management;

• to which extent interaction between these phases has been acknowledged; and

• types of research methods used.

In our analysis of existing ISP management research, we identified 117 studies that considered manual support and only seven studies that considered computerised support. This low number of studies considering computerised support was unexpected since the first studies within this area were published already at the turn of the century (Hoppe et al., 2002; Vermeulen and Von Solms, 2002). Taken together, our results show that existing ISP management research has focused mainly on manual support, and research with an explicit focus on computerised support is, to date, very scarce. Indeed, this points towards a considerable knowledge gap in ISP management research and provides an important opportunity for future research: to develop, test and evaluate computerised support. Such research is important to understand what effects computerised support could have on ISP management.

The avenues for future research on support for ISP management were made even more specific when we analysed the existing research using Tuyikeze and Flowerday’s (2014) framework. This analysis showed that all phases have been addressed in papers on both manual and computerised support, albeit not with similar emphases. In papers on manual support, the focus was mostly on policy compliance, followed by policy implementation and policy construction. Risk management and policy monitoring are phases that have received the least attention. This pattern corroborates findings in previous literature reviews (Tuyikeze and Flowerday, 2014; Cram et al., 2018; Wang and Li, 2015). Lack of ISP management research concerning risk management may lead to insufficient understanding of organisations’ needs and/or addressing of organisation-specific threats, vulnerabilities and risks, which in turn makes it difficult to develop effective ISPs. Limited research on policy monitoring, which focuses on audits and reviews of ISPs, may result in difficulties in keeping ISPs up-to date and adjusting them to organisations’ continuously changing requirements and needs. Hence, in general more research concerning risk management and policy monitoring is needed.

In contrast to the previous reviews, our review also analysed the phases investigated in research on computerised support for ISP management. Because of the low number of studies focusing on computerised support, it is difficult to discuss any patterns in distribution between the ISP management phases. We can conclude that computerised support for ISP management is understudied in relation to all five phases of ISP management. Consequently, the research community does not know if and how computerised tools can support information security managers in identifying risks and vulnerabilities, construction, implementation and monitoring of ISP as well as in the analysis of ISP compliance. This lead us to suggest areas for future research. It would be interesting to study how methods and models suggested in research on manual support could be used as starting points for the development of computerised support. It would also be interesting to study which ISP management phase(s) would gain the most benefits from using computerised support and how efficient such support would be for information security managers.

Although a couple of researchers (Tuyikeze and Pottas, 2011; Tuyikeze and Flowerday, 2014; Knapp et al., 2009; Rees et al., 2003) have emphasised the importance of considering the entire process of managing ISP, our analysis shows that the current research focuses mostly on a single ISP management phase. Compliance studies dominate this latter kind of research, which may indicate that existing research treats compliance as a separate area in ISP management. This research does not explore how other phases, such as policy construction or policy implementation may influence compliance and non-compliance. Lack of knowledge on how policy compliance interacts with other phases of ISP management leaves information security managers with limited understanding of how ISPs can be adjusted, modified and updated based on analysis of compliance and non-compliance. Hence, more research within this area is needed.

Furthermore, our analysis shows that the number of studies decreases when the number of addressed phases increases (see Figure 2), indicating that the complexity of the entire ISP management process is not addressed in current research. Taken together, the acknowledged interaction between ISP management phases in research on both manual and computerised support is generally low, which has implications for both research and practice. From a research point of view, more research on the interactions between the ISP management phases is needed to support the iterative nature of the process, i.e. to ensure efficient maintenance and updating of ISP based on input from the other phases. As of now, this limited understanding prevents researchers from uncovering new and valuable insights and providing advice to practitioners. From a practical point of view, the lack of models, methods and tools relating to the entire ISP management process leaves information security managers with fragmented support for this process.

As stated in the Introduction, computerised tools have successfully been used in other information system disciplines to support complex processes. Therefore, we believe that computerised support has the potential to aid in the ISP management process and reduce its complexity by helping information security managers to govern the interaction between the different phases. Yet again, the lack of research on computerised support for ISP management becomes a considerable limitation of current research. Here, several important future research opportunities materialise. It is important to understand how ISP management phases interact with each other; i.e. what should be input and output from each phase, for computerised support to be effective in work involving such interaction. It would also be interesting to investigate how and to what extent computerised support could be provided for the entire ISP management process and the interaction between its various phases. Having computerised support in place would also provide opportunities for studying the effects of such support.

Our review provides details on what research methods have been used in research on manual and computerised support for ISP management. We have shown how the research methods were distributed between the ISP management phases (see Figure 4) and also distributed between studies acknowledging different levels of interaction between these phases (see Figure 5). These findings extend the existing knowledge base, as none of the previous literature reviews explicitly discussed research methods used in research on ISP management (Tuyikeze and Flowerday, 2014; Järveläinen, 2016; Cram et al., 2018). Cram et al. (2018) compiled the research methods used in the reviewed studies; however, it is unclear what analytical framework was used for this categorisation.

In the current review, the research methods were analysed using an extended version of Mingers’ (2003a) framework (see Section 3). Similarly to Cram et al. (2018), we found that ISP management research is to a large extent based on survey methods (66 out of 123 studies). This is mainly because most ISP management research has addressed compliance, where survey seems to be the preferred research method. Our analysis further shows that although 14 types of research methods have been used, only four of them (survey, interview, no method, and case study) were used extensively. The large share of studies (26 out of 123) with no explicitly described research method surprised us, in spite of Cram et al. (2018) showing a similar pattern.

As shown in Figure 4, studies with no method were mainly found in the construction and implementation phases. From a research point of view it is worth noting that not explicitly describing the research method makes it problematic to assess the research results and decreases the studies’ reliability. Thus, we can conclude that researchers in the area of ISP management could be more explicit in their use of research methods. From a practical point of view, studies that do not explicitly describe used research method(s) may be experienced as abstract and consequently provide limited guidance for practitioners; for example, concerning policy construction.

Mingers (2003a) argues that research methods belong to certain paradigms, even if some of the methods can be used in several paradigms. According to Mingers (2003a), a paradigm refers to “the general orientation of a research method and its basic assumptions”. Thus, given the skewed frequency of the research methods used in ISP management research we can conclude that only a limited number of perspectives have been addressed in this area. Mingers (2003a) classifies research methods into positivist, interpretive, and intervention oriented; these methods are related to different research outcomes: prediction, understanding and change. We did not find any studies that used critical theory and only a few studies that used action research, design science or consultancy (see Figure 4). All these methods are categorised by Mingers as methods involving interventions which “inherently involve bringing about change to the research situation” (Mingers, 2003a). The fact that none of the studies on computerised support used design research or action research is surprising since these two methods are considered to be supportive in designing artefacts.

These findings have implications for both research and practice. From a research point of view, a broader use of research methods would help approach ISP management problems from several perspectives and offer different types of research outcomes. From a practical point of view, the Introduction shows a need for improvement (bringing about change) of the complex ISP management process; thus more intervention studies applying research methods such as action research and design science are needed. In particular, these methods become crucial if we are to increase research on computerised support for ISP management.

In this study, we have reported on the extent to which researchers have suggested manual and computerised support of ISP management, and the research methods used in such research. Naturally, the findings rely on our search strategy and on our selection of papers. We have been explicit about our selection of papers, which is based on searches in the Scopus and Web of Science databases. Of course, other search strategies would have been possible, such as the one used by Cram et al. (2018). Thus, we do not claim that we have identified all studies on ISP management; rather, we have used a sample of good size from the relevant outlets.

The use of our analytical framework involved subjective judgment. It was not always an instrumental task to classify papers into different phases or types of research methods. However, our initial triangulation of the classifications, based on the authors’ individual analyses, strengthens our findings. Furthermore, we have tried to make our procedures as explicit and transparent as possible by providing a complete account of our searches and classifications of papers in Appendices 1–6, making it possible to scrutinise the work in detail.

The aim of this paper was to survey existing information security policy (ISP) management research to scrutinise the extent to which manual and computerised support have been suggested, and the ways in which the suggested support have been brought about. To this end we used the ISP management process framework proposed by Flowerday and Tuyikeze (2016) together with an extended version of Mingers’ (2003a) research method framework. We can conclude that the existing research focuses mainly on manual support for ISP management. Computerised support has received very little attention.

The majority of papers on manual support address a single ISP management phase and there are few studies that deal with the entire complexity of the ISP management process. Policy compliance is the phase that has received the most attention; however, it has mostly been studied in isolation. When it comes to research on computerised support, we were unable to identify any patterns in terms of how researchers have addressed the interaction between ISP management phases. This was due to the limited number of papers that have been written on computerised support. With regard to research methods, only a small repertoire of research methods has been used extensively. The majority of the studies used survey, interview, and case study. In a considerable number of papers, no research method was accounted for. Furthermore, it is interesting that we found few studies that employed action research and design science, i.e. targeting intervention and change. This is especially surprising given that supporting ISP management is about providing practical advice and artefacts. Taken together this makes it difficult for practitioners to assess the practical usefulness of many of these recommendations and artefacts.

Our findings suggest that future research on ISP management should:

• to a larger extent address the interaction between ISP management phases, i.e. addressing the complexity of the entire ISP management process; this applies to research on both manual and computerised support for ISP management;

• apply more intervention research to develop computerised support for ISP management, i.e. studies that use design science and action research as research methods. Intervention research would help in assessing the practical usefulness of such support; this recommendation should not be interpreted as meaning that other research methods are not useful; in many cases, they are useful in different steps of design science and action research studies. For example, surveys and interviews can be used during the evaluation of suggested artefacts; and

• investigate to what extent computerised support can enhance the integration of different ISP management phases and reduce the complexity of such a process; as so few studies have been carried out on computerised support it is very much unknown to what extent and in what way this type of support can ease the burden of information security managers when working with ISP management.

Table AI shows the search queries that were used for searching in Web of Science and Scopus, and the results from our searches. For searching in the databases, a combination of search query 1 and search query 2 was used. For example, as the second row in the table shows, the result of searching “Information security policy” and “management” in Web of Science was 74 papers while in Scopus it was 173 papers.

Table AII illustrates the search criteria used in Web of Science and Scopus that did not return any results. The combination of search query 1 and search query 2 was used in searching the databases. For example, as the second row in the table shows, the combination of “Information system security instruction” and “management”, or “Information system security instruction” and “development”, or “Information system security instruction” and “implementation”, or “Information system security instruction” and “design”, or “Information system security instruction” and “requirement”, or “Information system security instruction” and “deployment”, or “Information system security instruction” and “effectiveness”, or “Information system security instruction” and “planning” did not return any results in Web of Science or Scopus.

Table AIII lists the identified papers that we were not able to access.

Table AIV provides operational definitions of the types of research methods that we used in the classification framework.

Table AV shows the coverage of ISP management phases in existing research and how these phases have been combined.

Table AVI presents a detailed analysis of the papers that we could access.