Learning by assessing digital government: a case study of the digital index framework Open Access

Benchmark and maturity models for assessing digital government come in a plethora of shapes and purposes and have a relatively long history of practice and research in the field of digital government dating back to the early 2000s (Iannacci et al., 2025; Okan, 2024; Skargren, 2020; Layne and Lee, 2001; West, 2005). These frameworks for assessing digital government have been criticised and misunderstood according to research (Andersen et al., 2020), yet few studies seem to have looked at their influences when applied in practice. We study the results and influences from a theoretical model, the digital index framework (DIF), designed to assess and learn about digital government in core processes of public administration in Sweden. Further studies are needed on the effects of these types of frameworks in practice, and to view them not as endpoints but as prerequisites for learning and understanding in the context of their application (Okan, 2024; Maheshwari and Janssen, 2013; Kromidha, 2012).

Sweden is a relevant case because its decentralised public governance – with well-resourced agencies relative to the central government – distributes responsibility for digital government across many actors (Digg, 2024). This makes the governance of digital government in Sweden complex. Although such challenges are not unique to Sweden or to assessment frameworks, and have been identified in other areas such as implementation and diffusion (Dwivedi et al., 2012). Governance complexity and bureaucratic structure are likewise identified as key challenges in digital transformation (Pedersen, 2018; Norling, 2025). These conditions motivate studying the application of the DIF in the present context, where development of digital government may depend on agencies’ ability to cooperate and coordinate around shared goals.

We study a project among government agencies in Sweden engaging with an assessment of digital government via the DIF in an area of public administration concerning supervision and control. Our study of the process of using the DIF for assessing digital government is done via a case study of a project evaluating and validating how well the DIF can be used to measure, learn from and develop digital government. The aim is to analyse the application of the DIF in a government project via the results from the assessment and its influence on the agencies participating in the project. We address the research questions:

This research studies a DIF for assessing digital government positioned in the area of research on maturity models (Iannacci et al., 2025; Okan, 2024; Andersen et al., 2020) and benchmarks for digital government (Bannister, 2007; Skargren, 2020). Maturity models and benchmarks normatively assess and compare digital government from a wide array of factors and actors; presenting the outcomes by ranking, index or as developmental stages (Skargren, 2020; Andersen et al., 2020). By positively scoring organisational and/or technological factors, benchmarks prescribe developments for countries and promote what is effective or valuable in digital government (see e.g. OECD, 2024; European Commission, 2024).

Research in recent years has produced a wide array of theoretical frameworks for assessing digital government similar to the DIF studied in this paper. Researchers offer different models and structured taxonomies for developing digital government, such as the maturity of public digital services (Panayiotou and Stavrou, 2019), decision support models based on organisational maturity for co-creation (Jukić et al., 2022) or a e-government acceptance model (Kirat Rai et al., 2020). This category includes research with parameters on core processes in public administration and frameworks regarding “institutional and interorganizational factors” (Chen et al., 2019). The “Enabler-Based Digital Government Maturity Framework” by Renteria et al. (2019) suggests aspects such as data, organisation and the “regulatory regime”. González and Delgado (2021) present a model for evaluating interorganisational collaborative business processes, focusing on evaluating business processes adherence to a compliance requirements model. These frameworks can reflect various demands and opportunities, as in how digital government develops across time and in different stages such as transformation and engagement (Janowski, 2015).

Research also shows quantitative measures such indexes. Including e-readiness assessment for public e-services in Morocco (Benaddi et al., 2023) and a maturity index for local government portals in Mexico (Batista et al., 2022). Other researchers offer methods for assisting decisions on proactive services for “no-stop shops” (Scholta et al., 2022) and a public accounts internet financial disclosure index for digital government and e-participation (Naidu et al., 2022). Regarding theoretical frameworks producing an index concerning core processes, research seems to be scarce. An exception is Hooda and Singla’s (2020) theoretical model addressing “business processes”, and the aspect of competencies in reengineering them.

Past research has called for more case studies of theoretical frameworks for assessment focusing on how to improve public organisations, as well as being open for insights within specific domains (Maheshwari and Janssen, 2013; Andersen et al., 2020). We therefore study a particular framework – the DIF – and its role and value for public sector practitioners as a tool for supporting development and learning. Heeks (2008) suggests using benchmarks in a prospective manner where assessments support decision-making, making priorities and creating a forum for learning. Research has shown the importance of a learning environment and adaptability to local conditions to ascertain the effect of benchmarks and using benchlearning methods to develop digital government (Kromidha, 2012; Batlle-Montserrat et al., 2016). Based on our analysis of learning from engaging with DIF results, the framework can complement past research on digitalisation assessment – especially process-oriented and prospective frameworks.

The DIF is a framework for identifying and assessing the extent of digitalisation in government agencies’ core processes. This means the organised and transparent act of administering and implementing laws and regulations in practice and by extension the exercise of political power (see Wilson, 1887; Peters and Pierre, 2012). A theoretical rationale for the DIF is that it incorporates how public administration carries out core processes mandated by laws and regulations, and core processes are handed out as areas of responsibilities via Government Agency Ordinances. “Core” relates to the tasks of a government agency, in contrast to support processes such as managing the budget and personnel issues, and is carried out to make sure laws and regulations are upheld in society (see Digg, 2021). Core processes primarily highlight the purpose of public administration in terms of what they are to achieve across different policy areas such as education, crime prevention, energy and water management – rather than how it is done (see Hammond, 1990; Gulick, 1937).

The aim with the DIF is to establish a joint understanding of the presence and extent of digitalisation in core processes and to support further learning and development. The DIF was originally designed at the Swedish Transport Agency and is now managed by the Swedish agency for digital government (Digg).

A DIF assessment results in a composite indicator: a digital index representing the presence of digital technology in three analytical dimensions of a process. The theoretical rationale is to provide a comprehensive perspective that includes not only digital services but also back-office processes and data handling, consistent with Heeks (2008p.335) “architecture of e-Government” – channel and recipients, processing and source (government data). Another motivation for the DIF is a process view of the digitalisation, similar to contemporary public administration studies that study public services as “public processes” (see Schedler and Helmuth, 2024). These theoretical rationales are reflected in the dimensions of the DIF, as components for understanding and measuring three levels of digitalisation in public administration, namely, (1) interaction with society: the extent of services offered via digital channels, (2) internal case handling: the extent to which required steps in the administration of a case are carried out automatically and (3) data exchange: the means by which data is collected and shared internally in the organisation, updated and shared with external actors. The DIF assesses both digitalisation in organisations (“technology in government”, stage 1) and transformation (stage 2), as identified by Janowski (2015). Digitalisation is identified by the presence of digital technology in core administrative processes; transformation is reflected in higher scores for technologies that would require reforms and key process changes. Consistently high scores in interaction with society require up-to-date digital payments, field data-collection technologies and responsive digital channels; in internal case handling, administrative activities require advanced data interpretation using for example AI and automatic compilation of suggested supervision objects from high-quality data. Unlike maturity models, the DIF is descriptive rather than prescriptive: it maps technologies in the conduct of governmental core processes and treats development as engaging with results to set priorities that may increase maturity, rather than assigning level-based characteristics. It shares with maturity models a prospective orientation and a holistic view. DIF results can also be combined with maturity models to motivate the need for, for example, transparency and better stakeholder and policy engagement (stages 3 and 4) (Janowski, 2015), or in interaction with society to for example discuss needs and risks of ICT-based co-production and democracy (Muhhina, 2024).

Each dimension in the DIF consists of certain activities representing an ideal type core process (Figure 1). The process representation is built on established definitions in public agencies, as well as Swedish laws and regulations.

The assessment of a core process is done by contacting the responsible civil servant for a core process and its sub-processes. The person responsible answers a questionnaire identifying how information and data are processed during the life cycle of a case in the process under investigation. The questionnaire contains questions relating to the three dimensions regarding the use of technologies in the process. Each item is scored on a predefined scale, and the index is calculated as an arithmetic mean. Note that the values indicate relative levels of digitalisation and should not be interpreted as having equal intervals. The aggregate results from the DIF comprise three ordinal and discrete variables that reflect the presence of digitalisation for the respective activity in each dimension.

The DIF is designed to support learning and development of digital government (see Skargren and Garcia Ambrosiani, 2022) and evaluation theory enables examining how participants use the material and results from an evaluation process. Mark and Henry (2004) – and later Alkin and King (2016, 2017) – describe and offer an analytical framework forunderstanding the mechanisms through which the evaluation process may achieve influence and change. This framework capture the consequences of an evaluation by identifying pathways of socially mediated changes that the evaluation process and findings set in motion (Mark and Henry, 2004). The framework includes three levels of analysis, individual, interpersonal and collective, indicating the locus of the change (Mark and Henry, 2004). The mechanisms are classified into four processes: general influence, cognitive and affective, motivational and finally behavioural. Such processes occur at all three levels and may stimulate changes in beliefs and feelings, motivations and actions. General influence processes are the fundamental architecture of change, likely to set in motion changes in the other processes. Examples of mechanisms on the individual level are elaboration, priming and skill acquisition, which can trigger participants to act as change agents on the interpersonal level and contribute to new standard setting on a collective level (Mark and Henry, 2004). Central cognitive mechanisms on the individual level are that an issue can be judged as more important and become the more salient issue, or that the direction of an attitude is strengthened or changed. Such mechanisms can trigger changes in local descriptive norms at the interpersonal level or agenda setting at the collective level. Motivational processes comprise mechanisms that affect individuals’ personal goals and aspirations. Behavioural processes are for example when individuals act and perform change in practice which on a collective level can take shape in programme cessation or policy change.

Henry and Mark (2003) hold that the mechanisms lead to one another and can trigger a cascade of changes in the organisation and beyond. Consequently, a DIF assessment process can activate different mechanisms, or learnings, that change and develop digital government in practice.

We use mixed methods (Creswell, 2014) in a case study (Benbasat et al., 1987; Walsham, 1995) involving two Swedish government agencies during the period of February 2022 to February 2023. The case study methodology allows to study a phenomenon in its natural setting and to investigate how and why questions (Benbasat et al., 1987). Our case involves the DIF project including two case organisations, the Swedish Estate Agents Inspectorate (EA) and the Swedish Forest Agency (FA). The case is motivated as a critical case (Flyvbjerg, 2006) of understanding the use of the DIF in terms of its results and the influences of the project on the behaviour of the participants. The objective is to understand the DIF in practice, involving two different types of government agencies. We use an explanatory sequential mixed methods design to allow quantitative and qualitative methods to build on one another (Creswell, 2014). Two of the authors in this paper are part of the study as involved researchers and are members of the organisation responsible for coordinating and undertaking the project on behalf of Digg.

The project titled “Digitalisation for cross-government public value” (Digg, 2023) involved five government agencies. Supervised and coordinated by Digg, it had two main purposes: to explore opportunities for cross-government digitalisation and to test the DIF as a framework for assessing digital government.

We selected two government agencies with distinct characteristics. While both perform supervision within their respective policy areas, they differ in size and scope: one operates in real estate, the other in forestry. This contrast allowed us to see how the DIF assesses and evaluates such differences within a learning and cooperation project (see Digg, 2023). This contrast allows us to describe influences from engaging with the DIF in government agencies that vary in size and domain yet share the same core process.

The EA, with 30 employees, oversees registration and supervision of estate agents and agencies across Sweden (FMI, 2025). Its legal advisers and case managers ensure compliance with relevant laws in this area. The EA also informs about rules and regulations for the real estate market. The FA has about 800 employees in 20 districts tailored to local contexts (Skogsstyrelsen, 2025). It supervises forestry practices, ensuring compliance with laws regarding forest management and logging applications (Skogsstyrelsen, 2024).

Data was collected during the project period 2022–2023 and includes a questionnaire, workshops and official reports. The questionnaire reflects DIF results for EA and FA. Workshops were arranged to discuss and engage with the DIF project, and the official reports were produced as outcomes of the project.

Informed consent from both agencies was obtained via e-mail before the study started. The participants were made aware that their contributions would be used for research purposes in anonymised and aggregated form. No personal or sensitive data were collected, and therefore the study was not subject to formal ethical review according to national regulations.

Data from the preparation of the project spans February–August 2022 and includes Digg presentations: one in April to attract agency participation, another in August detailing the DIF and project plan for participating agencies. Observation notes from authors representing Digg, e-mail correspondence regarding sub-processes selection and case volumes, and Digg’s August meeting minutes are also included.

In the first step, the EA and FA were asked via e-mail to propose sub-processes for inclusion in the DIF assessment; the EA selected three and the FA four, all relating to supervision and control activities tailored to specific legal demands. The questionnaire comprised 49 items covering DIF’s three dimensions (interaction with society, internal case handling and data exchange) both in aggregate and per step (Figures 2 and 3 provide examples). It was distributed by e-mail in early September 2022 with instructions to respond by the end of the month. The FA submitted four responses, one for each sub-process, while the EA submitted one response covering their three sub-processes. Respondents were encouraged, both in the e-mail and at the August kick-off meeting, to share the questionnaire internally with colleagues knowledgeable about the relevant administrative processes and digital technologies.

Three workshops were conducted in November and December 2022 and February 2023. Each workshop involved 8–11 participants, including representatives from the two participating agencies, a few invited external contributors (joining online) and two of the authors who participated as representatives from Digg. The workshops were organised around specific themes, with Digg facilitating and presenting objectives, results and purposes of the sessions.

Data were collected through multiple sources:

• official workshop minutes systematically documented by Digg during the sessions;

• workshop presentations provided by both Digg and participants; and

• written summaries of discussions for each agenda item.

Comments and discussions were recorded in the form of detailed minutes rather than audio recordings or verbatim transcripts. These minutes captured participant remarks, questions and agreed conclusions.

Topics discusses in the workshops:

• Workshop 1: Project and DIF introduction (Digg); core process presentations highlighting challenges and successes (participants); DIF questionnaire results and discussion (Digg and Participants).

• Workshop 2: Digitalisation of supervision and international expectations of digital government; concluding discussion on shared challenges.

• Workshop 3: Cross-agency supervision processes; a practical example of supervision; and discussion on which elements could be handled jointly and how to build on these suggestions.

Participation in the workshops was voluntary, and all participants were informed of the purpose of the workshop in advance.

Data also includes three official reports:

1. Digg’s project report detailing purpose, results and suggestions (Digg, 2023), along with e-mail feedback from EA.

2. The Ministry of Finance’s assignment report including DIF project perspective (Ministry of Finance, 2024).

3. Digg’s 2025 final report with future recommendations (Digg, 2025).

While the study involves two participating agencies, we do not treat them as two separate cases. Instead, we conceptualise the DIF project as a single case of government digitalisation. The two agencies serve as contrasting units that allow us to observe how DIF is interpreted across different organisational contexts. Accordingly, quantitative DIF scores are compared across agencies to illustrate variation, while the qualitative workshop data is analysed at the project level to trace shared processes of engagement and influence. The purpose of the workshop analysis is to capture collective learning and influence processes.

The quantitative analysis focused on questionnaire results at two levels. Firstly, we compared EA and FA scores across DIF’s three dimensions using ordinal values (0–6 per item). Secondly, we analysed detailed results by dimension: interaction with society includes 12 interaction points; internal case handling has 22 administrative actions and data exchange includes 9 steps. Outcomes were grouped to assess the presence of technologies enabling each step, aiming to unpack how the DIF reflects digitalisation levels.

The qualitative analysis followed the project’s timeline (February 2022–February 2023), divided into early, mid and late phases. Data from workshops, questionnaires and reports were coded thematically to examine DIF’s influence on participants’ actions and attitudes, applying evaluation use theory (Henry and Mark, 2003; Alkin and King, 2017). The analysis aimed to capture mechanisms of influence on participants that we as researchers identified in different aspects of the assessment process and activities. Firstly, we identified key themes based on the workshop agendas (e.g. digitalisation of supervision, international expectations, cross-agency processes). Secondly, we coded the discussion summaries to capture categories such as perceived challenges, successes and opportunities for collaboration and learning. In this process, we also described our observations of what happened in the workshops, how participants interacted and behaved and what they discussed. These aspects were then related to evaluation influence mechanisms as defined by Mark and Henry (2004) and conceptualised as process use – (how evaluation activities, rather than findings, affect individuals and organisations). Accordingly, we traced key elements of influence, identifying stimuli (oral and written information) that shaped participants’ engagement with the DIF, particularly during discussions of questionnaire results and workshop presentations.

Figure 4 shows that the FA scores are higher than the EA across all dimensions, with the largest gap in interaction with society. The FA’s results are based on four sub-processes (approximately 69,000 cases/year), while the EA’s cover three (approximately 459 cases/year).

In interaction with society, the FA offers more digital options. While FA users access structured, responsive digital services, EA interaction occur more via phone, post, email or static forms.

In internal case handling, the FA scores higher, indicating more automation in back-office tasks. However, both agencies show lower scores here overall, which might suggest digitalisation efforts have mainly targeted external interaction.

In data exchange, differences are smaller but still favour the FA. The EA relies more on manual and analogue data sharing, whereas the FA provides greater digital access to its information.

This dimension includes 12 defined interaction points between agencies and society (Table 1). Empty cells indicate “not relevant” in responses.

The EA has digital services at the start and end of the process – third parties can initiate a supervision digitally. The FA uses digital technologies more broadly: to inform about upcoming supervision, request additional information and communicate results.

Results for the EA is divided into two groups. In six of the twelve steps of interaction the score is zero in one instance and 17 in five instances. The notification of upcoming supervision is done via postal mail, and for the other steps the interaction is offered by means of e-mail, phone and postal mail. Information of results are communicated through a webpage. In the second group, with scores of 50 and 100, serving the results can be done completely digitally, while there is a digital form for third parties to initiate a supervision.

For the FA the results are grouped into three. The first group includes the step of serving a decision, done via postal mail and the initiating of a supervision, done via postal mail and/or telephone. The second group includes the two steps with a score of 58 – for the citizen or company to submit or supplement information and get information about the status of the case. Both are mainly offered via postal mail or telephone, and digital services for one of the sub-processes. The third and largest group includes interactions with a score ranging from 80 to 100. The steps involving personal and technical collection of data during the onsite supervision includes the use of drones and sensors as well as digital photos and films. In notifying about a coming supervision, requesting information about the status and information about the decision includes the use of the national digital infrastructure for digital mail as well as a “my pages” service for the latter two steps.

This dimension includes 22 steps (Table 2); two are marked not relevant by the FA. Of the 21 relevant steps, only four are fully manual. The FA shows variation across sub-processes, though this detail is lost in the combined results. Two steps are fully automated, and at least six involve both manual and automated elements.

The EA’s results are divided into three groups. The largest group includes the steps performed manually (score 0), including all the preparatory stages of a supervision: identifying and categorising risks subjects and planning supervisions; to administering the case in various ways in controlling for relevant and mandatory information and assigning a case officer. In the second group, the logging of the case, as dates, sources and different types of information, is done both manually and automatically, with a score of 50. The third group includes those with a score of 100 and are all done automatically: assigning case-identification, checking ongoing cases and archiving the cases.

Results for the FA are more diverse due to differences in the included sub-processes and can be divided into five groups. Firstly, the almost manual activities (scores 0, 13) include the creation of plans for supervision, assigning a case officer, creating a supervision report and making the decision. For steps where the score is 13, all the sub-processes are done manually, except for one sub-processes which combines manual and automatic processing throughout. The second group includes visualisation of data, given a score of 25 for the static provision of data as graphs and/or tables based on supervision data. The third group of results includes five steps, with a score of 50, revealing the existence of automatic and manual means of performing the administrative activities. The majority of these are all about the stage of analysing and categorising data for planning and identifying patterns and risks before conducing a supervision, except for the step of notifying a case officer of any new developments in a case. The fourth group contains the scores ranging from 63 to 88. This includes the initial activities of using AI to analyse patterns in the “identify risks” step, to fully automated assignment of case identification for some of the sub-processes and instances of combinations of manual and automation for others. All sub-processes except one are fully automated in the process of logging information about a case (score of 88). As for the final group, this contains the steps with a score of 100 and are fully automated and includes making data available for publication and for archiving.

The results are distributed among nine different steps (Table 3). For both agencies scores are low for publication of data and statistics as well as informing external actors on the decision. For the EA the lowest result is for local update of internal information, while it is the highest result for the FA, where also approval of access to information by the information owner is high.

Regarding the EA, local information is updated manually (score 0), while publishing statistics is done on the website in MS Word/PDF format, and informing others about a decision is done via e-mail or publication on a website. Access to external information (a score of 38) concerns giving access upon request and is often done via postal mail, e-mail and digital services. Four of the steps have a score between 50 and 75. Get approval of information access from owner is done via log-in access and local update of external information stored locally is done manually, but also via giving direct access. Creating access to internal information is done via various means – per request via e-mail, postal mail and digital services (a score of 55) – whereas internal sharing of information is done via shared systems, API and digital services.

For the FA the low results include publication of statistics, informing external actors on decisions and requesting information from other agencies, which is done via telephone or e-mail and digital service (score of 18). Four activities have a score from 47 to 58. Access to information from external actors is given both via APIs, digital services, manual file transfers and e-mail (a score of 47). Local update of external information is done both manually as well as via direct access (a score of 50). Making information available internally is done via e-mail, telephone and digital services (a score of 55). As for provision of requested information within the agency, this is done via APIs, digital services and shared information systems. The final group includes the highest results; approval of access to information from owner is done via log-in access and open access, while local update of internal information is provided via digital direct access.

In this section we analyse how the DIF based assessment activities and process influenced participants attitudes and actions applying evaluation influence theory. The interpretative analysis identifies the mechanisms (marked in italics), as described by Mark and Henry (2004), which would indicate that influence had occurred.

This phase began with Digg initiating contact with potential participants. In April, the EA and FA attended separate introduction meetings outlining project goals and the DIF. Each agency identified relevant processes for assessment. The phase concluded with a joint start-up meeting in August.

Both agencies showed strong interest early on, sharing feedback and suggestions via e-mail and meetings in May–June, including case volume data. The start-up meeting and DIF visualisations sparked discussions about digital government and its analysis. Presentations included theoretical models and past results, making the framework concrete and relevant. This reflects what Mark and Henry (2004) term priming meaning that the presented DIF components influenced individuals on how to perceive digital government and what features that are important relating to salience in evaluation influence terminology. The agencies’ active participation and willingness to share materials suggest a positive attitudinal shift. While initial motivation may have been high, the early phase clearly sustained or enhanced engagement, despite the project being voluntary.

The joint start-up meeting in August was the main avenue in this phase for discussions amongst participants and deliberation and questions regarding the challenges of digitalisation. Here they explained, for each other, the work in practice with their core processes, connecting descriptions to the operationalisation in the DIF, making the DIF cognitively approachable in terms of their own core processes. During the start-up meeting participants from the FA described their everyday work in combining digital technologies for risk assessment prior to supervision and control instances, and that they rely in practice on a combination of physical inspections and digital means of collecting data on forests.

A possible stronger finding is that of the influence denoted as elaboration, which concerns how individuals think about the programme outcomes. We found that participants made a collective effort to participate and share their experiences unpretentiously and to be ready to ask challenging questions along with a full focus on how digitalised their core processes are, based on impetus from the DIF. A participant from the EA described their recent work on developing their process for conducting supervision and control in the real estate market and expressed how they “joined the project to get tips and ideas and to learn” (Participant, start-up meeting). The finding of elaboration is perhaps the most consistent throughout all the three phases of the project, although the basis for elaboration changes throughout. This means that in the mid phase elaboration is manifested mainly in how data from the questionnaire is used and how the participants engage with shared challenges of cross-government digitalisation. Elaboration in the early phase is about being able to share and discuss their challenges in structured meetings among peers regarding the potential for core processes coming from different agencies and policy areas.

The mid phase covers the administration of the DIF questionnaire and the three ensuing workshops where results from the questionnaire and concerns and issues raised were analysed and discussed. This period contains the most findings on the influences of the three phases of the project, and during this time the participants moved from individual elaboration and skill acquisition to interpersonal conversations on how to use the acquired knowledge towards making collaborative change in practice.

At the first workshop there was a strong interest in elaborating and discussing the results from the questionnaire. The participants were highly motivated and had acquired skills to understand digitalisation based on the conceptualisation in the DIF to discuss and interpret the role of digital technology in their processes. In this regard there were many comments from the participants based on the results, such as how they note that “the processes with a larger volume of cases are more digitalised” (Participant, workshop 1) or “how come we are not distributing cases automatically to a larger extent” (Participant, workshop 1) or questions concerning access to key data to make better risk assessments and digital tools.

We found that participants started to become more primed towards certain challenges between the first and second workshop, where the idea of cross-government digital integration and cooperation grew stronger, accompanied by discussing what parts of the process might be ripe for such a practice. At the second workshop the participants acknowledged, and reached an agreement, on three key areas of concern:

1. Digital technologies for choosing what object to supervise and associated key questions, and which is based on the idea of risk-based supervision in the DIF. This included shared standards for identifying and categorising risk or choice of different themes for conducting supervision.

2. How to create a common pool and access to key data. This included focus on common needs for basic data to identify objects, more specialised data stored at different agencies and how to better share locally produced data.

3. Which parts of the agencies’ supervision processes are in most need of development, using the results from DIF as orientation.

This third shared concern focused on the participants’ discussion questions, such as how they can help each other in creating common solutions that can be used by a collective of actors. From this discussion we find that participation in the process had triggered a questioning of local descriptive norms and initiated a possible agenda for change where participants attain justification to act as change agents in practice.

In the third workshop we found that the engagement deepened since participants invested time and effort into continuing the discussions, and the basic positive attitude to the purpose and meaning of the project and to continue working with the DIF and raised concerns were upheld. At this workshop one agency gave a presentation showing their work in digital development, using the concepts from the DIF, based on the shared identified challenges identified in previous workshops. This included sharing different types of everyday challenges in the supervision process, as well as sharing ideas and experiences. In terms of influence this shows how collaborative change in practice had started to materialise. In the discussions conceptual questions was raised. As one participant formulated it: “What does the core process of supervision really consist of? How do we categorise it on a more general level?” (Participant, workshop 3) and another participant expressed that “We need to find the shared characteristics of a common pool of data” (Participant, workshop 3), showing that participation in the process had motivated cross-agency exchange and possibly new norms for the execution of supervision. The engagement for new standard setting on the collective level is apparent in that a participant suggested that “we need a test case for more cross-government collaboration” (Participant, workshop 3), showing a motivation for continuing the work and moving forwards towards collaborative change in practice.

Following the third workshop the final phase of the project included work with compiling and presenting the results and lessons learned from the project. This work was led by Digg together with inputs from the participating agencies.

The writing and compilation of the report was done in collaboration with the participants who helped with building business cases and collecting further data on time and cost for selected sub-processes, and further how to identify and understand these processes based on the DIF. In terms of influence, we found continued performance of new skills among participants by how they were providing comments and input to the content of the report, compiling lessons learned, experiences and suggestions for future work developing digital government in Sweden based on the DIF. Their input showed developed structural incentives and skills on a collective level through their ability to elaborate on central questions from the project, now based on a view of themselves on a national level and with cross-government challenges.

The published report delivers answers and suggestions for implementation and funding for how to use the DIF on a national scale, and this was later detailed in another report by Digg, explaining how the work can be continued for three main types of core processes: supervision, granting permits and transferring (Digg, 2025). Relating to influence, the report documenting the project provides a basis for policy-oriented learning and policy change.

The project officially ended with the publication of the report, the influences from the project continued on the national collective action level, adding a policy interest from an actor at the central government level – the Government Offices. The idea and purpose of the DIF is manifest in a government assignment to Digg in June 2024 concerning a request for a proposal and strategy for the future development and financing of a common digital infrastructure for Sweden (Ministry of Finance, 2024). One of the tasked assignments can be traced directly to the DIF project, formulated as: “Which types of horizontal core processes within the public sector can be carried out automatically or simplified with the help of digital technologies?” (Ministry of Finance, 2024. p.3). According to the assignment, this includes core processes with similar functionalities, for example permits and certificates or “where processes for data sharing and development of artificial intelligence are vital for the quality of a core process” (Ministry of Finance, 2024. p. 3).

In terms of influence this show signs of policy-oriented learning, where the assignment applies wording and conceptual understanding of how digitalisation relates directly to core processes. This influence can also be interpreted as a sign of moderate policy change considering that the mentioned assignment is one of many different aspects.

In this research we have used mixed methods in a case study where we have interpreted different kinds of results from using DIF to assess digital government in the core process of supervision and control in two government agencies. Our focus has been to better understand the application and use of the DIF concerning how the quantitative results from the assessment capture digital government in the practice of the involved agencies, and further to understand how participants in the assessment process are influenced and learn from participation to develop digital government.

This discussion outlines how this study contributes to three areas of research in the field. Firstly, we discuss how the study contributes to the general discussion on benchmarks and maturity models, secondly DIF contribute on how to enable a prospective direction to guide development and thirdly the study contributes with an operationalisation of a process perspective on digital government.

Unlike international frameworks and suggestions from research that assess digital government at a macro level (OECD, 2024; European Commission, 2024; UN, 2024; Iannacci et al., 2025), the DIF enables micro-level analysis of core processes. It generates index-based results across three dimensions: interaction with society, internal case handling and data exchange. Macro and micro assessments can complement each other – macro-level indicators such as IT competence or governance conditions may inform DIF expectations or help interpret its outcomes. They also offer hypotheses for what DIF results might reveal.

This study answers Andersen et al.’s (2020) call for case studies on the use of maturity models. It shows how such framework can guide governance and support organisational learning. In addition, the assessment had influence on the national level since the government tasked Digg with exploring how to scale the DIF and identify suitable core processes.

We also contribute to meta-studies on maturity frameworks, highlighting the importance of implementation context, evaluation effectiveness and resource demands (Okan, 2024). Our tracing of participant engagement and influence illustrates howDIF can be applied in practice. Unlike prior studies, we apply evaluation theory to examine how a digital government assessment framework influence practitioners. By using the schematic theory of evaluation influence (Henry and Mark, 2003; King and Alkin, 2019), we show how civil servants engaged with the DIF concepts and results.

Across the three phases, we argue that the framework’s focus on practical administrative activities relevant to daily work enables participants to share experiences and discuss how they operate in conjunction with digital technologies. In the first phase, at one meeting, we note how the DIF made the analysis cognitively approachable in terms of their own core processes. In the second phase, questionnaire results show how they actually work – and how they are not currently working – given the potential of other digital technologies. This is shown in the first workshop of the mid phase, relating to case volume, the extent of digitalisation and the recognition that internal case handling is far less subject to automation.

This study adds to benchmark research calling for better purpose and context (Skargren, 2020; Bannister, 2007). The DIF offers a conceptual model of digitalisation in public administration across three dimensions, as shown in Figure 1 and our results. The application of the DIF validates the frameworks relevance for assessing digitalisation at micro level – down to individual activities. This aligns with other models supporting digital government through taxonomies and conceptual tools (Kirat Rai et al., 2020; Jukić et al., 2022; Panayiotou and Stavrou, 2019). By producing an index of digital progress in core processes, the DIF adds further perspectives on digital government complementing quantitative methods assessing other aspects such as e-services and local government portals (Batista et al., 2022; Benaddi et al., 2023).

Heeks (2008) emphasised that benchmarks can guide future digital government development. Our findings show how structured engagement with a framework like the DIF can influence participants – from fostering shared understanding of technological challenges to enabling reflection on cross-government digitalisation. Rather than using measurement as an endpoint (see Maheshwari and Janssen, 2013), this study shows how assessment results can support learning and development in policy and practice.

We traced positive effects at individual and group levels – participants developed skills for questioning and reasoning about digitalisation in core processes. Participants became attuned to key challenges and asked critical questions, such as “What does the core process of supervision really consist of?”, “How do we categorise it on a more general level?” and “We need to find the shared characteristics of a common pool of data.”

By showing how an engaged application with a framework enhances its purpose, helps address misunderstandings about these types of frameworks as discussed by Andersen et al. (2020). Our results support the importance of localised learning (Kromidha, 2012) and align with benchmarking approaches like benchlearning (Batlle-Montserrat et al., 2016), that help managers and government officers to improve services.

This study contributes to research on frameworks targeting core processes. Chen et al. (2019) advocate cross-boundary digital government systems to streamline administration – an ambition shared in our case. The novel contribution of the DIF is to show how a framework for assessing digitalisation, which operationalises core processes, can enable learning and discussions for such integration. Our analysis shows how an engagement with DIF results across agencies stresses the need for digitalisation progress and areas for horizontal integration.

From another perspective, the structure and engagement with the DIF can complement González and Delgado’s (2021) work in how it frames digital government as the practical execution of core tasks through regulated, technology-enabled processes and includes not only inter-organisational processes but also interactions with society. Our study also relates to Hooda and Singla’s (2020) model on competencies in process reengineering. We believe that the level of detail on processes in DIF, and the effects on learning and preparation for engaging with the results, can contribute to competencies and priorities related to the need to redesign a process. Working with the DIF in practice also raises questions about shared core processes across agencies as shown in the policy effects.

Benchmarks and maturity models are often developed at high abstraction levels for assessing countries (e.g. OECD, 2024; European Commission, 2024). This study demonstrates the practical values of a more detailed, context-sensitive framework like the DIF, which practitioners can apply directly in their work. Our synthesis of how the DIF was used can guide others in conducting similar assessments and offer lessons for international benchmarks aiming to engage practitioners affected by such frameworks. The DIF framework and questionnaire assess digitalisation and transformation in public administration and can be used as a guide in different administrative contexts. The DIF was tested in the Swedish public sectors, where it showed strong practical potential for assessing and advancing digital government. A key question remains on how well it would perform by practitioners in other national contexts. The DIF studied here focuses on supervision and will need adjustments for other core processes (e.g. issuing permits) or administrative contexts. We expect its three dimensions to be relevant across national and local public organisations, though we welcome further enhancements and feedback on how to change or improve identified activities in the DIF as they may differ in different contexts.

A key limitation is the national context: Sweden’s public administration system may limit the transferability. Another limitation is the DIF’s scope which means that some factors – such as leadership and strategic governance – remain outside of the analysis (see e.g. Norling, 2025).

While we identify several influences, there are other perspectives. Working with such a framework for nearly a year – and with a limited set of organisations and sub-processes – can be questioned in terms of investment versus practical outcomes. Moreover, the theoretical framework applied to analyse influences steers the analysis towards positive influences of learning and change, we did not identify possible negative or wasteful effects. Nor did we conduct a controlled experiment to isolate which variables that may have caused the observed influences or assessed whether the identified influences led to practical developments. It may be that, despite a relatively high frequency of influence types, no practical action followed the assessment project. However, our contention is that the results highlight the importance of understanding the assessment process – shifting focus from content and outcomes to how frameworks are used with the purpose of driving change.

The DIF’s theoretical rationale and design might explain the influences we identify. It is built close to public-administration practice and core processes – especially supervision, the primary task of the two organisations studied. Using another digitalisation-assessment framework – with more abstract terms, stronger normative ambitions (e.g. a maturity model) or a greater level of detail as in other process-oriented frameworks – could have produced different frequencies and types of influences, and therefore other possible development trajectories.

This study analysed the application of a DIF in a government project, focusing on assessment results and their influence on participating agencies. We addressed two questions:

1. What do the DIF results reveal about digitalisation in government core processes?

2. How does the DIF assessment influence learning and development in practice?

Our findings show how digitalisation varies across the three dimensions in the DIF, with clear differences between agencies, both at aggregate and activity levels. The engagement with the DIF fostered skill development and deeper reflections on cross-government processes, and we also found indications of policy learning and change.

We call for more studies how to engage with results from frameworks for assessing digital government. This together with critical feedback from participants in these studies can continue to improve digital technologies in public administration. Future research can be for example case studies using evaluation theory to analyse influences from frameworks and on how to apply the DIF in other administrative contexts. We also suggest that the DIF can be developed to for example examine other types of core processes beyond supervision and control, and to study not only the influences from engaging with results from frameworks but also how they may lead to further development in practice.

This work was supported in part by the Swedish Agency for Digital Government (Digg).