When words lose their edge: a comparative analysis of information security policy keywords in the UK and Sweden

In contemporary organisations, information and information systems are vital information assets that drive efficiency, innovation and competitive advantage. These assets are critical to organisations’ operational and strategic work. Being critical assets, safeguarding these information assets has become essential, as they represent key determinants of organisations’ long-term sustainability and success. In protecting information assets, organisations cannot depend solely on technical measures to guarantee information security. For example, in 2023, an information security breach at the company Okta compromised data across 18,400 organisations after an employee accessed a personal Google account from a corporate computer (Snider, 2024). This information security breach demonstrates how employee actions in managing information assets can lead to widespread implications. Information security breaches, like this one, can cause severe damage to organisations’ reputation, financial stability and even survival (Kör and Metin, 2021).

Both practitioner reports (PwC, 2018; Truesec, 2023) and research (Chatterjee et al., 2019) have stressed the critical role of human behaviour in information security and it has reported as a top-ranked information security threat over the last three decades (Chowdhury et al., 2019; Loch et al., 1992). Organisations develop operational information security policies (ISPs) to provide rules and procedures that employees must follow in their daily work (Siponen and Vance, 2010). However, employees do not always comply with these ISPs. Understanding why employees violate ISPs is complex (Cram et al., 2019), but one aspect seems to be the ISP design itself (Rostami, 2023). Previous research has stressed the importance of designing clear and understandable ISPs (e.g. Höne and Eloff, 2002; Stahl et al., 2012; Lopes and Oliveira, 2015; Karlsson et al., 2017). In addition, the ISO/IEC 27002 standard (ISO, 2022) and other regulations, such as EU directive (Sundt, 2006), provide guidelines for ISP design. Stahl et al. (2012) argued that ISPs should “give specific and actionable advice” and Diver (2021) has in a practical style guide recommended using specific keywords to ensure that pieces of actionable advice are “useable” for employees.

In a recent study, Rostami and Karlsson (2024) examined how congruent keywords were used in Swedish public agencies’ ISPs to pinpoint actionable advice, i.e. “a demarcated part of an ISP, that instructs someone on a task to execute or not to execute regarding information security, and, in case of execution, how to carry out the task” (Rostami et al., 2025). They found that two-thirds of sentences containing the recommended keywords – terms such as “must”, “shall” and “should” that signal obligation levels in policies – were used for other information rather than actionable advice. They suggest this dual use of keywords may reduce the possibility of communicating clear instructions to employees. Their study was limited to Swedish ISPs, and they note that results may vary when applied to different languages and national contexts.

Of course, awareness of such differences is important for international organisations and those that, for other reasons, require ISPs in several languages. Against this background, this study aims to analyse and compare how congruent keywords are used to convey actionable advice in UK and Swedish ISPs. To this end we use the case of higher education institution and examine 30 ISPs, 15 ISPs from each country. In Sweden, universities are predominantly public agencies. In the UK, although they are classified as private entities, higher education institutions often operate as public agencies due to significant public funding and regulation (e.g. by the Office for students and UK Research and Innovation), aligning their objectives with public benefit in education and research. To capture aspects of the national contexts, we have chosen to frame and interpret our results using Hofstede’s national culture model (Hofstede, 2011; Hofstede et al., 2010).

Our study results in several contributions to researchers and practitioners. Firstly, our study reveals differences in the distribution of keywords used in ISPs in the two countries. This suggests different approaches to expressing varying degrees of obligation and prohibition to employees, which is in line with the cultural differences between the two countries. These results contribute to existing research on ISP content, studies that so far is limited to single country studies, by showing that previous results may be contextually bound. This is of importance to international organisations, since the results reveal not only linguistic differences but also underlying divergences in how information security management is framed and operationalised in different countries. Secondly, we found differences between the countries regarding how congruent claimed keywords are used. Our results indicate that Swedish organisations are more congruent than UK organisations when using keywords in actionable advice for employees. Our study thus opens for research into whether institutional factors influence this result. Finally, we developed the ISP Keyword Analyser software to support the analysis of how congruent keywords are use in ISPs to pinpoint actionable advice. This software is a methodological contribution, which can be used by both researchers and practitioners.

The paper is structured as follows. In the Section 2 we presented related research on communicative aspects of ISP design and the national culture model. Section 3 presents our research method. In Section 4, we present the results of our analysis. Finally, the paper ends with a discussion in Section 5 and a short conclusion in Section 6.

Prior literature reviews on ISP research have shown that several studies focus on ISP design (Rostami et al., 2020; Cram et al., 2017). A few of these studies are empirical studies that have analysed the quality of ISPs with focus on clear, consistent and actionable guidance that can help employees manage information assets securely (Rostami and Karlsson, 2024; Rostami and Karlsson, 2023; Stahl et al., 2012; Karlsson et al., 2017). Nevertheless, these studies do not analyse and compare how congruent keywords are used to convey actionable advice in ISPs from different countries. As discussed in the Introduction, Rostami and Karlsson (2024, 2023) examined Swedish ISPs from public agencies using content analysis and found that only one-third of the occurrences of the recommended keywords were used to provide actionable advice. Based on these results, Rostami and Karlsson (2024) offered three practical pieces of advice: (1) reserve specific keywords and use them only for writing actionable advice, (2) document these keywords to keep track of them and (3) do not use synonyms if the level of obligation remains the same. Even if these recommendations are important, they are based on findings from one country.

Stahl et al. (2012) conducted critical discourse analysis on British NHS ISPs and found them often ambiguous, with unclear objectives, vague target audiences and heavy use of jargon. They recommended using clear, accessible language and providing specific, practical guidance for employees. In a similar study, Karlsson et al. (2017) analysed three ISPs as practical tools in employees’ everyday work at a Swedish hospital. They found that the investigated ISPs lacked internal congruence, which made them difficult to use as practical tools. Consequently, they proposed eight quality criteria for ISPs to provide guidance on their development. Two of these quality criteria advise that ISPs should provide “congruent guidelines for actions” and stress the importance of using a “clear and congruent conceptual framework” as the foundation for writing ISPs. These two pieces of advice both emphasise the importance of selecting and applying concepts such as keywords in a consistent way when writing actionable advice. Again, these results are based on studies in individual countries. Furthermore, Asai and Fernando (2011) have demonstrated that translating security guidance into different languages is insufficient because people’s understanding of, and adherence to, such guidance is shaped by cultural context.

Doherty et al. (2009) contributes with an interesting study in that it compares ISPs from universities in eight countries. They have analysed and compared policy structure and coverage of various information security topics. However, they did not perform a detailed analysis of the content at sentence level, nor do they focus on how keywords are used to communicate the content. Thus, the study does not contribute knowledge about differences in how keywords are used in ISPs in different countries.

In addition to existing research, practitioner-oriented literature also offers guidance on ISP design (e.g. Diver, 2021; ISO, 2022; Landoll, 2017; NIST, 2006; Peltier, 2004; Smith, 2010). While much of this literature focuses on ISP structure and content (e.g. ISO, 2022), some sources also address how to write actionable advice. For instance, Diver (2021) provides practical recommendations for making actionable advice clear and easy for employees to understand. These include using concrete language, avoiding abstract terms and steering clear of negative phrases like “never”, which can create ambiguity. Instead, advice should be clear, specific and outline permitted or prohibited actions, including any exceptions. She also suggests using “must” rather than “shall” or “will” to clearly indicate mandatory actions and avoid confusion with future tense. Quite naturally, Diver (2021) does not contain any empirical analyses of how keywords are actually used and whether there are differences between different countries. No such analyses can be found in the other practitioner-oriented literature referred to above, but that has never been their purpose either.

Existing research offers a few metrics to assess the quality of ISPs: policy length (Alshaikh et al., 2015; Höne and Eloff, 2002), breath, brevity, clarity (Goel and Chengalur-Smith, 2010) and keyword loss of specificity (KLS) and total keyword loss of specificity (TKLS) (Rostami and Karlsson, 2024). Of these six metrics, the first four measure the quality of the policy as a document, while the last two metrics focus on how selected words are used in the ISP.

Alshaikh et al. (2015) emphasise that ISPs should be concise, building on Höne and Eloff’s (2002) argument that excessively long policies are likely to be ignored by employees. Measuring the length of an ISP is straightforward and can be applied in various ways, most commonly through word count or page count analyses. However, length is an overarching metric that does not capture the quality of the ISP’s content or how keywords are used. Goel and Chengalur-Smith (2010) have suggested metrics – breath, brevity and clarity – that focus on the content and the words used. Breadth assesses how comprehensive an ISP is, and they use a master glossary as a reference point to evaluate the presence of information security terms within the policy; a greater number of matches indicates a higher level of comprehensiveness in the ISP. Brevity evaluates the degree of word repetition within an ISP, with the argument that lower repetitiveness helps reduce redundancy, excessive wording and unnecessary technical jargon. Finally, clarity focuses on readability of the ISP, where they suggest using established text analysis metrics (Flesch Reading Ease Score, Flesch–Kincaid Grade Level and the Gunning fog index). Although valuable metrics, they do not focus on congruence use of keywords.

The KLS and TKLS metrics suggested by Rostami and Karlsson (2024) have has a slightly different focus compared to the above metrics. KLS is a quantitative measure designed to assess how effectively directive keywords are used to convey clear actionable advice rather than general statements. The metric calculates the percentage of occurrences of the keywords that are not used in the actionable-advice sense. A higher percentage means more “loss of specificity”, i.e. more times the keyword is used for something other than clear instructions. The metric provides a quantitative indicator of how well a keyword is functioning as a directive cue in the ISP. The TKLS is an aggregated metric and is calculated as the proportion of all occurrences of selected directive keywords in a policy that are not used to provide actionable advice.

This study examines ISPs from two different countries. As Asai and Fernando (2011) demonstrated, national culture may influence how ISPs are formulated, communicated and emphasised in different countries. Consequently, it is relevant to explore how differences in cultural dimensions between countries relate to differences in the use of keywords in ISPs. To analyse these potential cultural influences, we select Hofstede’s cultural model (Hofstede, 2011; Hofstede et al., 2010), one of the widely used models for understanding cross-national cultural differences. For instance, this model has been applied in various research disciplines, such as social science (Srite and Karahanna, 2006) and educational science (Cronjé, 2011). The model provides a systematic approach to explaining how cultural values influence behaviours, attitudes and social organisation at the national level.

Hofstede (2015) explains the model and cross-national cultural variation through the following six cultural dimensions:

1. Power distance: describes how strongly people expect and tolerate unequal power structures within organisations and institutions. Nations with a strong power distance tend to accept hierarchical structures, while nations with a low power distance tend to emphasise equality.

2. Individualism: refers to the degree to which people see themselves as independent rather than as part of a larger group, i.e. collectivism. For instance, people with a high level of individualism primarily focus on themselves and their immediate families, while people with a high level of collectivism tend to view themselves as members of a large group and emphasise mutual support.

3. Masculinity: reflects how much a society supports or accepts the use of force. Nations with more masculine cultures tend to value competition, achievement and assertive control, while nations with more feminine cultures tend to emphasise shared responsibility, consensus and work–life balance.

4. Uncertainty avoidance: describes a society’s level of tolerance for the unknown. Therefore, nations with a strong uncertainty avoidance tend to rely on rigid rules and clearly defined norms to regulate belief and behaviour.

5. Long-term orientation: is concerned with how change is managed over time. Therefore, nations with a strong long-term orientation tend to invest in future benefits while compromising short-term gains.

6. Indulgence: focuses on appreciating the good things life has to offer. For instance, nations with a high level of indulgence tend to place greater emphasis on enjoying life, while nations with a high level of restraint are more likely to regulate behaviour through social norms.

Table 1 presents the index scores for each cultural dimension in Hofstede’s model for Sweden and the UK. The index scores show that the two countries have a high degree of similarity across five of the six cultural dimensions. For instance, both Sweden and the UK score relatively low on Power Distance and Uncertainty Avoidance and display high levels of Individualism and Indulgence.

The most noticeable difference between the two countries is observed in the Masculinity dimension. Sweden scores exceptionally low on masculinity, reflecting a strong preference for shared responsibility, consensus and work–life balance, while the UK has a high degree of masculine orientation, emphasising competition, achievement and assertive control. This contrast highlights a fundamental difference in societal values and norms, which may provide insights into the expression of ISPs in these two countries.

This research used a qualitative content analysis (Assarroudi et al., 2018) approach to investigate how congruent keywords are used in ISPs in the UK and Sweden. Qualitative content analysis offers a structured way to study written material using coding and interpretation, which aligns well with our research aim. Moreover, we use a summative approach to qualitative content analysis (Hsieh and Shannon, 2005). A summative approach focuses on finding and studying specific words or pieces of content to understand their meaning based on patterns in how they are used. As Rostami and Karlsson (2024) demonstrate, summative content analysis effectively reveals patterns in how keywords function within ISPs to guide employee behaviour. Firstly, it allowed for systematically identifying predefined keywords across several ISPs (Yanow, 2000). Secondly, it allows us to study the contextual function of these keywords. For example, they may give clear, actionable advice or be used for other purposes (Bowen, 2009), such as providing other information. Thirdly, it allows for quantifying our qualitative analysis through the KLS metrics (see Section 3.2 for details), providing a standard basis for comparing different documents and contexts (Neuendorf, 2017).

As an empirical starting point for our study, we collected operational ISP from UK and Swedish higher education institutions. We decided to use a systematic advanced Google search strategy to find ISPs. We chose this approach because these documents are publicly accessible, providing immediate access without the limitations or obligations that might come with formally requesting such documents. For our Swedish searches, we incorporated relevant keywords such as “policy”, “IT”, “dator” (computer), “handbok” (handbook), “riktlinje” (guideline), “säkerhet” (security) and “regler” (rules). This broader range of terms was used for the Swedish context because Swedish institutions often use the above terms for such documents. This understanding was based on prior experience with the varied naming of Swedish policy documents. For the UK ISP search, the more direct terms “information security” or “ISP” within document titles and institutional domain restrictions were sufficient to identify operational ISPs without needing a similar expansion of supplementary keywords. We adjusted the data parameters systematically, starting from 2024 and working backwards until we collected 15 suitable operational ISPs for each country.

Our interest in how keywords are used in existing ISPs meant that we need a set of keywords as a starting point. As shown in the related work section, not much research exists on the use of keyword in ISPs. We used the keywords in Table 2 as a starting point for evaluating actionable advice in ISPs. These are the same keywords as used by Rostami and Karlsson (2024), which, in turn, draw on the recommendations made by Diver (2021). The leftmost column presents the English keywords, and the rightmost column shows the Swedish translation. Similar to Rostami and Karlsson (2024), we have used synonyms for some of the Swedish keywords. This decision was based on our knowledge of the Swedish language, to be more inclusive. We included the words “ska” and “skall”, which both mean shall, and “ej” and “inte”, which both mean not.

To make our analysis of keywords comparable across ISPs and across countries, we used the two metrics KLS and TKLS (Rostami and Karlsson, 2024). The first metric, KLS for an individual keyword is calculated as follows:

n= total number of occurrences of a keyword in the ISP;

n_not = total number of occurrences where the keyword is not used in line with a defined purpose in the analysed ISP.

The second metric, TKLS is calculated as follows:

kn = total number of keywords;

n= total number of occurrences of keywords in the analysed ISPs.

Before analysing the collected ISPs, data preprocessing was necessary. We learned by reading the ISPs that some organisations had also included strategic parts in their ISP documents. Therefore, these parts had to be removed to not to include data that did not align with the aim of our study. We read the ISPs and removed strategic-level parts (Baskerville and Siponen, 2002) and document housekeeping elements such as tables of contents, forewords and unrelated appendices. We included all parts addressed to employees or end-users – parts targeting anyone in the organisation, from a receptionist to a specialist, to understand and follow.

Rostami and Karlsson (2024) used the text extraction plugin in the data analysis tool Orange Data Mining. However, general-purpose tools like Orange Data Mining are not explicitly tailored to our analytical needs, requiring a time-consuming, monotonous classification process and later manual calculation of the KLS metrics. It is particularly time-consuming to repeatedly open and search individual documents whenever the context provided by the Concordance (a widget displaying keywords within sentences of a maximum of ten words) is insufficient to classify sentences as actionable advice and other information.

We therefore developed the ISP Keyword Analyzer software to support the data analysis. The complete source code and documentation for the ISP Keyword Analyzer software are available on GitHub (Link to the cited article.). When launching the ISP Keyword Analyzer, the user first selects a language and uploads an ISP. Once the first ISP is loaded, the language selection becomes locked to ensure consistency throughout the analysis process. The software automatically highlights keywords and presents sentences containing these keywords for classification. Users classify each highlighted sentence as actionable advice or other information. One limitation of the study by Rostami and Karlsson (2024) was the way in which they operationalised a piece of actionable advice as a single sentence, without taking into account the relationship to other sentences. To address this limitation, the user interface in Figure 1 provides functionality to provide more context to the analysed sentence. The software makes it possible to expand the view to show one to five sentences before and after the current excerpt. The user interface also provides Back and Forward buttons that lets the user step to the previous or next extracted sentence, making it easy to compare and reclassify sentences. After completing an ISP analysis, it is possible to review the classifications in the raw data section before giving a review summary in the form of the keyword loss of specific metrics.

Using the ISP Keyword Analyzer, our data analysis had three clear phases: extracting the relevant sentences, classifying those sentences and calculating metrics. Thus, the analytical process still followed the same approach as described in Rostami and Karlsson (2024). In the first phase, we identified all sentences containing our target keywords in the 30 ISPs. We used the keywords presented in Table 2. For Swedish ISPs, nine keywords were used and for UK ISPs, we used their English equivalents provided in the original study.

The second phase involved classifying each sentence as actionable advice or other information. The criteria for classifying a sentence as actionable advice to be considered clear and directly implementable it had to: declare what to do, how to do it and when to do it (if there was a time aspect attached to the action). Other information encompasses general, ambiguous or non-actionable content. The sentences were classified individually using these criteria. For sentences where it differed in our classifications, we carefully reviewed them against our raw data tables to develop a shared interpretation of the current ISP before making final decisions. Tables 3 and 4 contain illustrative examples of our classifications. Table 3 presents example sentences classified as actionable advice, while Table 4 presents those classified as other information. We have chosen to provide illustrative examples for all the keywords we have used in the analysis. When it comes to the Swedish sentences, we as authors have translated them into English for this paper, the analysis was carried out on the original Swedish sentences.

In the third phase, using the ISP Keyword Analyzer software, we calculated the KLS and the TKLS for each keyword and ISP, respectively. This gave us measurements of how congruent each keyword was used to convey actionable advice in UK and Swedish ISPs. We present these results in Section 4.

This section presents our analysis of the extracted ISP sentences, how the keywords are used and a comparison between UK and Swedish ISPs. In total, we extracted 2,314 sentences from 30 operational ISPs and classified each as actionable advice or other information. Table 5 presents an overview of each ISP’s TKLS. The table is divided into two parts. The leftmost part shows the analysis of UK ISPs, and the rightmost part contains the analysis of Swedish ISPs. By comparing these parts, we see that Swedish ISPs contain a higher proportion of actionable advice (66.3%) compared to UK policies (50.3%). This results in a lower TKLS in Swedish policies (33.7%) compared to UK policies (49.7%). The 16% difference suggests Swedish organisations are more consistent in using keywords to signal clear instructions to employees. A more detailed scrutinisation of Table 4 reveals that the best Swedish ISP (15.6%) outperforms the best UK ISP (29.2%) with some margin. Furthermore, there is a wider spread regarding TKLS in the UK ISPs compared to the Swedish policies.

The TKLS provides an overview but does not reveal any details about the use of individual keywords. To investigate the KLS for each keyword, we analysed the number of times each keyword appeared in actionable advice and other information sentences. Tables 6 and 7 show the distribution of keywords in sentences containing actionable advice and other information, as well as the loss of specificity of keywords in UK and Swedish ISPs. The third column from the left in Table 6 shows that UK ISPs rely mainly on four keywords, with “must” being the most common at 28.6%, followed by “should” (25.3%), “not” (21.9%) and “shall” (17.4%). These four keywords collectively account for 95.1% of all keyword usage, which means the keywords forbidden, “never” and “need” are used very sparingly. The fifth column shows that “must” is the most frequently used keyword in actionable advice at 36.9%. When it comes to other information, in the seventh column, we see that “should” is the most frequently used keyword (29.0%), and that “forbidden” is never used in relation to other information.

When it comes to KLS, the column on the far right of Table 6 shows that it ranges from 0% to 94% for the different keywords. In other words, there is considerable variation in how congruently the keywords are used. The keyword “need” has the greatest loss, at 94%, and “forbidden” has the least loss, at 0%. At the same time, these are two keywords that are not used frequently. If we instead focus on the most frequently used keywords, we see that the most frequently used keyword, “must”, has 35.2% loss of keyword specificity. The other frequently used keywords have significant KLS: “should” (57.0%), “shall” (54.7%) and “not” (51.4%).

In contrast to the analysed UK ISP, Table 7, shows that Swedish ISPs has a more skewed distribution of keywords. As shown in the third column from the left in the table, Swedish policies rely heavily on “shall” (“ska” and “skall”) and “not” (“ej” and “inte”), accounting for 49.1% and 37.2%, respectively. In total, these two keywords account for 86.3% of all occurrences in the Swedish ISPs. The other keywords, “never” (“aldrig”), “should” (“bör”), “need” (“behöver”), “forbidden” (“förbjudet”) and “must” (“måste”), occur very sparingly. By comparison, the keyword “must” accounts for only 3.1% of keywords used, compared to 28.6% in UK ISPs. Based on this, it is natural that the keywords “shall” and “not” also constitute a very significant proportion of the usage in actionable advice. “Shall” (“ska” and “skall”) with its 53.5% constitutes more than half of all identified keywords, while “not” (“ej” and “inte”) accounts for 33.9%. These two keywords also constitute a very significant proportion of the keywords identified in other information. However, the ranking is reversed, with “not” (“ej” and “inte”) accounting for 43.8% and “shall” (“ska” and “skall”) accounting for 40.5%.

The KLS for Swedish ISPs is shown in the rightmost column in Table 6. We see that KLS ranges from 9.5% to 53.8%, which is a smaller range than for UK ISPs. The keyword “should” (“bör”) has the highest loss at 53.8%, while the keyword “never” (“aldrig”) has the lowest loss at 9.5%. At the same time, these are two keywords that are not used to any great extent in the ISPs analysed. When it comes to the two most frequently used keywords, “shall” (“ska” and (“shall”) and “not” (“ej” och “inte”), they have a KLS of 27.8% and 39.6%, respectively.

The final part of our analysis is a comparison between the use of keywords in UK and Swedish ISPs with regard to KLS. Table 8 presents this analysis, where the difference is shown in the column on the far right. The difference is calculated using UK ISP as a reference point. This means that a positive difference value indicates that the Swedish keyword has a higher loss of specificity, while a negative difference value indicates that the UK keyword has a higher loss of specificity. The largest difference is found on the fourth row from the top. Here we find a 53.1 percentage point difference in KLS between the English term “need” and its Swedish equivalent “behöver”. The keyword “need” is also the keyword in English ISPs that has the highest loss. The second highest difference is found between the keywords “forbidden” and “förbjuden”. The English term “forbidden” has a lesser KLS than the Swedish equivalent “förbjuden”. There is a 50% point difference, however, it is also important to note that this keyword is used to a very limited extent by both UK and Swedish ISPs.

After that, there is a significant leap in difference when we look at the next set of keywords. It concerns the English keyword “shall”, which has two Swedish equivalents, both “ska” and “skall”. Here, the Swedish pair of concepts has a lesser loss (26.9 percentage point) than its English counterpart. Furthermore, as shown in Table 6, this is the most common keyword in Swedish ISPs. Moving on, we see that the differences between the keyword combinations “never” and “aldrig” and “not” and “ej/inte” are fairly even. The first combination shows a difference of 13.2 percentage points in favour of Swedish ISPs. The second combination shows a difference of 11.8 percentage points, again in favour of Swedish ISPs. Returning to Tables 6 and 7, we see that “not” and “ej/inte” are used frequently in both UK and Swedish ISPs. Finally, we have the English keyword “must” and its Swedish equivalent “måste”. Here, there is no difference. At the same time, Tables 6 and 7 show a difference in frequency of use, with “must” being the most frequently used keyword in UK ISPs, while “måste” is rarely used in Swedish ISPs.

Our findings reveal differences in how keywords are used in ISPs in the two countries. These findings reveal patterns in keyword usage that have important implications for research. Firstly, there is a notable difference in keyword distributing patterns. Swedish ISPs rely heavily on two keywords – “ska” and “inte” – which together account for most of all sentences identified in Swedish ISPs, while “must” is hardly ever used. This distribution is consistent with Rostami and Karlsson’s (2024, 2023) previous findings regarding Swedish ISPs. In contrast, UK ISPs show a more balanced distribution across four main keywords, where “must”, “should”, “not” and “shall” are found in most of the ISP content. That being said “must” is found in a quarter of all analysed UK ISP statements, playing a prominent role compared to Swedish ISPs. This suggests that UK ISPs differ from Swedish ISPs in their approach to expressing varying degrees of obligation and prohibition to employees.

As shown in Section 3.2, there are many similarities between the national cultures of the UK and Sweden. At the same time, there is a significant difference when it comes to the masculinity dimension. The UK leans heavily towards masculinity, while Sweden leans heavily towards femininity (Hofstede, 2015). This means that the use of force is endorsed differently in the two countries. The UK’s masculine culture favours competition, clarity and assertive control, where clear requirements and direct language, such as “must”, becomes natural. Assertive control also means placing emphasis on accountability, which could lead to a need to express a hierarchy of obligation (must, shall, should). In comparison, Sweden’s feminine culture favours consensus, modesty and shared responsibility. Thus, instead of being driven by assertive control, management is more driven by shared norms. In this case it means not grading the obligations, which seems to result in a direct but understated language.

By indicating a link between the differences in masculinity to differences in the use of keywords, we contribute to previous analysis of ISP content (Rostami and Karlsson 2024; Rostami and Karlsson, 2023; Stahl et al., 2012; Karlsson et al., 2017). These previous studies have addressed ISPs in individual countries and not being able to account for cultural differences. It shows that their results may be contextually bound and that caution is warranted when making generalisations. This finding also echoes the previous finding of (Asai and Fernando, 2011) that translating security guidance into different languages is inadequate due to national cultural differences. That said, cultural differences between countries are a vast and complex area, and researcher need to develop a better understanding how such differences influence ISP content design. Thus, there is a need for more studies comparing ISP content between different countries to identify cultures, countries (or languages) that could potentially be grouped together and potentially link them to patterns in the use of keywords, thereby enabling generalisations to be made. It means researchers should investigate several countries with different cultural profiles to establish this link beyond the two countries we have investigated.

The second identified pattern surrounding keywords has to do with how congruently they are used in ISP statements. Swedish ISPs demonstrated lower TKLS (33.7%) compared to UK policies (49.7%), a pattern that is also repeated when we look at our findings regarding the individual keywords. This indicates that Swedish organisations are more congruent when it comes to using keywords in pieces of actionable advice for employees. It is interesting that there appears to be a noticeable difference even between individual ISPs. This means that researchers need to investigate whether there are institutional differences between countries that have an impact, for example, whether and, if so, how ideas about ISP designs are shared between organisations. Of course, there are structures such as international standardisation work, e.g. the ISO-27000 series, that have an impact. At the same time, there may be additional structures within each country that have an impact. In Sweden, for example, there are networks for knowledge sharing between Chief Information Security Officers (CISOs) in the public sector, which could potentially contribute to creating a shared view on the use of keywords.

Furthermore, when it comes to the results for Swedish ISPs, the TKLS is much lower for the ISPs we analysed (average 33.7%) compared to those analysed by Rostami and Karlsson (2024, 2023) (average 66.9%). One reason for the difference may lie in the sample, where we focused on a specific sector (universities) while Rostami and Karlsson (2024, 2023) had a broad sample of organisations from the public sector. The results indicate that there may be differences between sectors within the same country, not only between countries. Thus, apart from understanding differences between languages and national cultures, there is a need to understand whether there are differences between different industries. Researchers should therefore explore whether ISPs in different industries are differently developed, which could mean that ISPs from certain industries could potentially serve as role models when it comes to the use of keywords to signal actionable advice. However, this requires future studies comparing ISPs from different industries.

Finally, the ISP Keyword Analyzer software we developed offers an important artefactual contribution (Ågerfalk and Karlsson, 2020) in relation to the research method, and responding to the call for computerised tools to support ISP design and management (Rostami et al., 2020). Although there has previously been an analysis process that has been partially implemented in Orange Data Mining (Rostami and Karlsson, 2024), that process still included a number of manual steps. Thus, this type of analysis has been quite resource-intensive and the ISP Keyword Analyzer software simplifies this process through extended support, making the analysis less resource-intensive. Future researchers can use or adapt this approach to analyse ISPs more efficiently. That said, researchers should consider whether this type of analysis can be further automated, for example using artificial intelligence and, more specifically, large language models. This would increase accessibility, for example, for CISOs to use the software as a tool to analyse their organisation’s ISP (see discussion in Section 5.2). However, such a development of the software requires that the large language model used has been trained to identify actionable advice.

While our study focused on higher education institutions, the observed differences in keyword frequency and in use of keywords between UK and Swedish ISPs may have broader implications for international organisations. Firstly, organisations need to pay close attention to variations in keyword distribution patterns between countries. These patterns reveal not only linguistic differences but also underlying divergences in how information security management is framed and operationalised. For example, differences relating to national culture. Such discrepancies highlight the limitations of relying solely on predefined keyword catalogues, such as the one proposed by Diver (2021), for cross-linguistic or cross-contextual translations. Without recognising the socio-linguistic and organisational dimensions that shape ISP language, important nuances in meaning and emphasis may be lost, potentially undermining both policy coherence and compliance across international branches.

Secondly, the observed differences in TKLS and KLS between the ISPs of the two countries suggest that varying degrees of attention are given to the consistent and precise use of keywords when communicating actionable advice. In other words, the extent to which keywords are used congruently to convey for example obligation, recommendation or permission differs across linguistic contexts, which can directly affect how employees interpret and act upon ISP instructions. We therefore recommend that multinational organisations move beyond direct translation practices when localising their ISPs. Instead, they should develop language-specific keyword strategies and guidelines that account for the semantic nuances and pragmatic functions of directive expressions in each language. For example, the English modal verb “must” and its Swedish counterpart “måste” do not always carry the same degree of force or formality, and such subtle differences can lead to misalignment in perceived policy strictness or urgency. Recognising and managing these variations is essential for ensuring that the intended degree of obligation is communicated effectively across all linguistic versions of a policy. This recommendation can be advantageously combined with the recommendations previously given by Rostami and Karlsson (2024). At the same time, it is important to stress that when adapting ISPs to different national contexts, organisations must ensure that they demonstrate coherence in their overall organisational policy, something that is requested in audits.

Finally, to support this process, organisations can use the ISP Keyword Analyzer software to systematically evaluate their ISPs prior to publication. By measuring TKLS and KLS, organisations can identify areas where directive language may have weakened during translation or adaptation and take corrective action to improve clarity and coherence.

As with any study, our study has limitations. Firstly, our focus on document analysis means we examined what ISPs say, but not how employees understand this content in practice. The effectiveness of keywords in real-world settings might differ from our findings based on the document analysis done. Without complementary interviews or surveys with employees, we cannot determine whether the patterns we identified affect employee understanding or actions in reality. While our linguistic analysis highlights the risk of misinterpretation from high KLS it does not capture employees’ subjective perception. Future research should therefore investigate employees’ interpretation, for example using the Repertory Grid (Samonas et al., 2020). Although high KLS may contribute to “mistaken compliance” (Niemimaa, 2024) or “maladaptive responses” (Balozian et al., 2022), future research must empirically test these effects using methods that assess employee interpterion and behaviour to validate these links.

Secondly, this study focused on higher education institutions, which may have unique information security needs and risk profiles, such as balancing academic freedom with data protection. As a result, our findings may not be directly generalisable to other industries. Moreover, our sample was limited to ISPs from 15 institutions from each country. While this allowed for in-depth analysis and comparability with the original study by Rostami and Karlsson (2024), it may not capture the full diversity of approaches to ISPs within Swedish and UK organisations. We therefore call for more comparative studies that include more industries, but also studies that compare ISPs from other countries.

Thirdly, although we have addressed the fact that actionable advice and other information are related to other sentences by introducing an interpretative context in the ISP Keyword Analyzer software, it is important to note that our analytical unit is sentences. Other operationalisations of these two concepts could be possible, which can potentially result in different interpretations of how clearly a keyword is used in a specific text. Furthermore, we used an interpreting context of up to five sentences before and after the excerpt analysed, which may also affect the result. More research is therefore needed regarding other possible operationalisations of actionable advice and other information, and whether an interpretative context should be used and, if so, what its scope should be.

Finally, it is important to acknowledge the set of keywords that we used as a starting point for the analysis. There appears to be no universal agreement on a set of keywords to use when writing actionable advice in ISPs. The set of keywords we have used is an operationalisation of Diver (2021), but we do claim that our operationalisation is the only possible one or that this set of keywords is comprehensive. We therefore encourage similar studies based on other sets of keywords, as well as research that identifies the most common keywords in actionable advice.

This study analysed and compared how congruent keywords are used to convey actionable advice in UK and Swedish ISPs. To this end we used the case of higher education institution and examined 30 ISPs, 15 ISPs from each country. Our results concern two patterns that we found related to the keywords studied. The first pattern concerns the fact that different keywords are more prominent in ISPs from the two countries. In other words, different keywords have different importance in different languages/countries. At the same time, we note a that earlier research has found difference in national culture, when it comes to how these societies support or accept the use of force. The UK favours competition, clarity and assertive control, while Sweden leans more towards consensus, modesty and shared responsibility. This is key knowledge when it comes to developing ISPs for multinational organisations or when multi-language ISPs are developed. The second pattern concerns how congruently the keywords are used to convey actionable advice. Here too, the results show differences between ISPs from the two countries. The results indicate that Swedish organisations are more congruent in using keywords aimed at guiding employees towards actionable advice.