Lifelong learner needs for human-centered self-regulated learning analytics Open Access

Self-regulated learning (SRL) refers to the cyclical process in which learners actively and reflectively plan and adapt their learning to achieve their goals (Järvelä et al., 2018; Zimmerman, 2000). Being able to regulate one’s own learning is a critical aspect of successful learning and can enhance academic performance and motivation (Theobald, 2021). Furthermore, SRL is essential for lifelong learning because learners need to be able to monitor and control their learning throughout their lives in today’s rapidly changing world (Taranto and Buchanan, 2020). Despite the importance of SRL, many learners struggle with SRL, particularly in the face of complex and demanding learning tasks (Koivuniemi et al., 2017). This can lead to frustration, disengagement and poor performance (Anthonysamy et al., 2020). Some students may need support to regulate their own learning effectively, and achieving self-regulation can be difficult for them.

However, due to the complexity of SRL processes, its measurement has been a major challenge (Järvelä and Bannert, 2021). To address this challenge, researchers have started to use various data streams and learning analytics (LA) to unobtrusively measure facets of the SRL to provide personalized support for the different phases of SRL, even in real-time (Azevedo and Gašević, 2019). At its best, personalized support promotes learners’ SRL and, as a result, helps them to become more successful learners (Lim et al., 2023). Although the advancement of LA has offered several insights and tools to better understand and promote SRL (Järvelä et al., 2023), a significant amount of effort remains to be put into designing LA systems that support learning with the consideration of individual goals, authentic educational needs and ethical principles (Sarmiento and Wise, 2022).

Human-centered LA has gained prominence as a distinct approach to the design and implementation of LA systems that prioritize the needs and perspectives of learners and teachers. Recent calls have asked for learners’ voices to be considered, along with the involvement of teachers in LA design, to increase LA’s transparency, acceptability and successful implementation across contexts (Buckingham Shum et al., 2019). Building on this foundation, participatory design methodologies have been recognized as an effective means for achieving human-centered LA (Bødker et al., 2022). These methodologies actively involve end-users, namely, learners, in the design process, thereby ensuring that the analytics solutions are tailored to their requirements and contextual realities. Nevertheless, as the majority of current human-centered LA studies focus on higher education, it is essential that human-centered LA studies also address other learning contexts, such as lifelong learning and workplace learning.

In this paper, we respond to these recent calls by examining lifelong learner voice for self-regulated learning analytics (SRLA) in lifelong learning. Moreover, the findings from lifelong learner voice are corroborated with their teacher’s voice for validation. In designing LA, it is important to define the stakeholders who can and should participate (Dollinger et al., 2019). Thus, balancing learner voice and teacher voice can be considered because each perspective offers unique insights. On the one hand, learner voice needs to be considered in the design of LA because students are the ultimate users of the system, and their perspectives on how it can benefit their SRL process are essential. On the other hand, teachers have a wealth of knowledge about teaching and learning and their perspectives can help ensure that the system is designed to meet the needs of both students and teachers. This paper examines learners’ challenges and desirable LA features in each SRL phase from both lifelong learners’ and teachers’ perspectives.

Nevertheless, when it comes to human–computer interaction, it is well-known that users do not always know or are certain about what they want, especially with advanced novel technologies that have the potential to change the way people work (Buckingham Shum et al., 2019; Dollinger et al., 2019; Martinez-Maldonado et al., 2021). This concern also applies to human-centered LA, that learners and teachers may not be fully aware of the effectiveness and undesirable consequences of the LA features. As a result, we argue that triangulating learner and teacher voices with domain expert opinions will ensure that the system meets their needs and provides a positive experience. Furthermore, by considering the pertinent learning theories, human-centered LA can be designed based on a solid understanding of the process of learning and is more likely to be effective in supporting learner success.

In this paper, we seek to address the recent calls by examining learners’ needs and aligning those with SRL theory and LA design principles to formulate design principles for SRLA. To address this aim, first, we propose a methodology that considers SRL theory and integrates both learners, teachers and LA experts’ perspectives for designing SRLA. Second, we apply the proposed methodology in a professional development context and illustrate how the methodology can be used to develop a set of design principles for SRLA. In particular, we seek to answer the following research questions (RQs):

In the following section, we first establish the theoretical foundations for the study by reviewing the literature on SRL in lifelong learning for professional development and SRLA. Then, we introduce our theory-driven participatory design with multistakeholders (TPDM) method for integrating learner needs in establishing design principles for SRLA. We present and evaluate the voice of lifelong learners on their SRL challenges and desirable SRLA features. The design principles for SRLA were established as prescriptive statements that guide the future development of SRLA. After presenting and evaluating the set of design principles for SRLA, we discuss the implications of our study and we conclude by discussing its limitations and future research directions.

SRL constitutes a dynamic framework that describes the processes by which learners personally orchestrate their cognitive, metacognitive and motivational strategies to attain their educational goals (Zimmerman, 2000, 2002; Järvelä et al., 2018). Learners who exhibit proficiency in SRL engage in a systematic sequence of planning, implementation and adaptation, which encompasses the setting of specific learning objectives, the continuous monitoring of their progress and the recalibration of their strategies when circumstances necessitate such changes. The significance of SRL transcends the immediate context of academic settings, holding implications for students’ long-term academic success, motivation and engagement (Theobald, 2021). SRL serves as a critical scaffold that not only augments students’ academic performance but also fuels their intrinsic motivation to learn, thereby offering a synergistic effect on their educational outcomes.

SRL extends its importance to the domain of lifelong learning, especially within the scope of professional development. Lifelong learning is an ongoing process extending beyond formal educational structures, often taking the form of informal or nonformal education embedded in daily activities and occupations (Milligan and Littlejohn, 2014). Within the framework of this study, professional development is defined as a type of lifelong learning aimed at enhancing career-specific skills. Contemporary society, characterized by an accelerated pace of technological advancements, mandates an ongoing learning process for individuals to remain adaptable and competent (Taranto and Buchanan, 2020).

In this regard, SRL emerges as an indispensable skill set for continuous growth and adaptability, empowering individuals to meet the challenges and opportunities presented by a digitized and dynamic global landscape. This can occur in various learning settings such as informal workplaces, nonformal educational platforms, as well as online and offline courses. Previous studies confirm the significance of SRL in empowering learners to actively acquire the requisite skills and competencies in a fast-paced, technology-driven society (Taranto and Buchanan, 2020). These findings align with well-established SRL strategies such as self-assessment, goal setting and self-reflection (Zimmerman, 2000). Accordingly, this study focuses on a nonformal online academy designed to teach digital skills over a period of three to six months to adult learners with diverse educational and professional backgrounds.

Research evidence suggests that the capability for SRL is especially pertinent for lifelong learners in professional development settings (Persico et al., 2015, 2020). Milligan and Littlejohn (2014) identify four key behaviors that are integral to SRL in these contexts as: consume, create, connect and contribute. Furthermore, the educational environment in professional development is often less structured, requiring a higher degree of self-regulation from learners (Milligan and Littlejohn, 2014). In such educational landscapes, students exercise greater agency over their learning paths, choosing subjects and pacing autonomously (Manganello et al., 2021). The design of digital skills boot camps, such as the one examined in this study, often incorporates pedagogical techniques that necessitate active participation from the learner (Moshirpour et al., 2019).

However, it should be acknowledged that not all individuals readily use SRL strategies in their learning processes. Challenges in practicing SRL can manifest in the form of poor task management or emotional regulation, ultimately leading to reduced academic performance or disengagement (Koivuniemi et al., 2017; Anthonysamy et al., 2020). Given these challenges, it is imperative that educational settings provide adequate support to facilitate learners’ SRL strategies.

In this study, Zimmerman and Moylan (2009) SRL model is adopted as the theoretical framework for investigating learners’ challenges in the different phases of SRL and the needs for SRLA. The model depicts the SRL process in three phases of SRL in which the output of the previous phase provides input to the next phase: (i) forethought including task analysis and self-motivation beliefs, (ii) performance including monitoring and controlling one’s learning and (iii) self-reflection including of self-judgment and self-reaction. In the context of professional development, learning is mostly informal and learners are required to take decisions not only about how and when to learn but also about what to learn (Manganello et al., 2021).

The advancement of the LA field has offered unique capabilities for better understanding and supporting SRL. LA refers to “the application of data analytic techniques and tools for the purposes of understanding and enhancing learning and teaching” (Nguyen et al., 2020). The literature has shown that LA delivers remarkable benefits to the learners, such as providing insight in a timely manner to improve outcomes (Knight and Buckingham Shum, 2017), identifying learning needs and providing personalized learning support (Nunn et al., 2016). For instance, Arnold and Pistilli (2012) demonstrated how real-time analytics could be used to identify at-risk students in the early stages of a course by integrating data from various sources, such as grades, course interactions and past academic performance, to generate “traffic light” indicators. Nevertheless, prior studies have called for better alignment between LA design and learning theories to maximize its impact on learning (Gašević et al., 2015). In particular, Heikkinen et al. (2022) noted in their systematic literature review that achieving the full potential of SRLA requires that the different phases of SRL are considered when designing and developing LA solutions.

When learners enact in forethought, performance or self-reflection phases, there are several affordances of LA that can promote SRL (Persico et al., 2015). First, SRLA can help learners to monitor their own learning and to make data-informed decisions on controlling their learning accordingly (Manganello et al., 2021). Second, SRLA can provide personalized scaffolds and guide learners to use specific learning materials or resources that promote SRL (Lim et al., 2023). Third, SRLA could be used to promote self-reflection by engaging learners in self-assessment of their own learning (see Domínguez et al., 2021). While alignment between LA design and learning theories is necessary, it alone is not sufficient to guarantee the design of good LA tools. Therefore, when designing such tools, it is critical to also consider the needs of nontechnical stakeholders, including students and teachers (Giacomin, 2014).

Furthermore, recent research has made significant contributions to the field of LA by offering various strategies that give students and teachers a voice (Alvarez et al., 2020; Buckingham Shum, 2022; Prieto-Alvarez et al., 2018). This has led to the development of human-centered LA in which critical stakeholders (students, teachers), learning and teaching environments and their relationships are identified and considered (Buckingham Shum et al., 2019). Human-centered LA is often achieved through participatory design that offers an innovative approach for ensuring that analytics tools are both effective and closely aligned with the specific needs of learners (Sarmiento and Wise, 2022). First, designing the LA tools with learners and teachers can inform them on the what, how and why the different kinds of data are collected, including how LA tools will acquire and interpret the data to promote SRL (Mangaroska et al., 2021). This increased transparency and acceptability of these tools can also foster their long-term use. Second, by using human-centered LA, both students and teachers may be better equipped to derive meaningful insights and make informed decisions based on SRLA. Namely, learners’ and teachers’ implementations of LA tools may differ from the purposes of designers (Wang and Hannafin, 2005), so designing the interventions with students and teachers may bridge the gap between the desired SRL processes and strategies and those actually realized (Schumacher and Ifenthaler, 2018).

Within the context of SRLA, central to SRL is the notion of learner agency, autonomy and self-reflection, elements that are naturally supported by participatory design methodologies (Zimmerman, 2000). Inviting learners to be active participants in the design of analytics systems ensures that the resultant tools better reflect learner requirements, preferences and contexts (Shum and Ferguson, 2012). Consequently, this participative and multidimensional approach not only yields SRLA tools that are more contextually apt but also reinforces the foundational principles of SRL by encouraging learner input and agency.

Even though the lack of students involvement in the design processes has been acknowledged (Buckingham Shum et al., 2019; Mangaroska et al., 2021), considering only their voice in designing SRLA is questionable because students understanding of effective SRL processes and strategies may be limited (Bjork et al., 2013). Even though the different stakeholders (students and teachers) can have different or even conflicting voices during design processes (Bødker et al., 2022), it is important to investigate both voices to validate design principles for SRLA. Accordingly, this design study seeks to formulate design principles for SRLA by synthesizing the voices of both students and teachers with solid theoretical foundations (Zimmerman and Moylan, 2009) and subsequently subjecting these principles to experts in SRL and LA.

To formulate a set of design principles for human-centered SRLA, this study used the well-known design science research methodology in information sciences research (Peffers et al., 2007) in conjunction with a participatory design approach (Bødker et al., 2022). This approach prioritizes the values and concerns of learners as end-users in the design of LA. However, it is important to note that the needs and preferences of learners may conflict with the instructional values held by teachers; such tensions must be addressed. Additionally, in extreme scenarios, the needs and preferences of learners could potentially have adverse effects on their learning outcomes. As such, integrating the perspectives of teachers is imperative for a balanced approach, and the application of learning theories is crucial for guiding the design of human-centered SRLA. Thus, drawing upon the design science research methodology and the participatory design approach, we present our TDPM approach as a methodological framework for formulating design principles for SRLA.

The design science research methodology by Peffers et al. (2007) serves as a comprehensive set of guidelines encompassing principles, practices and protocols for carrying out studies that focus on the design and development of artifacts for information systems, particularly educational technology (e.g. Nguyen et al., 2021). Its core aim is to develop pragmatic solutions to identified issues by emphasizing a multistage process that entails artifact creation, scientific contribution, design evaluation and result dissemination. The resultant artifacts can manifest in various forms, ranging from constructs and models to methods and instantiations, as well as design principles (Baskerville et al., 2018; De Leoz and Petter, 2018).

In the context of the present study, the research output took the form of design principles aimed at human-centered SRLA. The execution of the study followed the structured steps delineated by the design science research methodology:

• identifying and substantiating the problem;

• formulating solution objectives;

• engaging in design and development;

• demonstrating the artifact;

• evaluating the artifact; and

• disseminating findings (Peffers et al., 2007).

In this study, the initial step involved identifying the research problem, which centered on the necessity of supporting SRL for lifelong learners via human-centered SRLA. The study’s objectives, encapsulated in its aim and research questions, focused on investigating the specific needs of learners and crafting design principles tailored for human-centered SRLA. Following this, the stages of design, development, demonstration and evaluation were carried out using a participatory design method (Bødker et al., 2022), adapted into the proposed TPDM approach to fit the study’s objectives. Finally, the study’s results and findings served as the vehicle for communication in line with the design science research methodology.

In this study, we present an approach for TPDM. Figure 1 illustrates our TPDM approach for integrating learner needs in establishing design principles for SRLA.

Based on SRL theory, learners’ focus group discussions and teacher interviews are triangulated to determine learner needs. Together with design principles for generic learning analytics information systems (LAIS), the findings are incorporated into the conceptualization of SRLA design principles. Both the learner needs and conceptualized design principles are iteratively evaluated by experts based on the following three evaluation criteria (adopted from Venable et al., 2012): 1) evaluate the artefact to establish its utility and efficacy for achieving its stated purpose; 2) evaluate the artefact to identify weaknesses and areas of improvement; and 3) evaluate the artefact to identify side effects or undesirable consequences of its use.

In this section, we focus on the essential design requirements and inputs that will inform the foundational principles of SRLA. These guiding elements are derived from the thematic analysis of Learner Focus Group Discussions and Teacher Interviews. By examining these rich narratives, we identify the specific challenges that lifelong learners face in their learning journeys. We also discern the LA features that both learners and teachers consider most desirable for enhancing lifelong learners’ SRL. This multistakeholder insight serves as the cornerstone for establishing empirically grounded and contextually relevant design principles for SRLA.

The study started with data collection from two groups of participants which are the learners and the teachers. The first group of participants was learners who joined three- to six-months digital skills courses (e.g. Full-Stack Web Development, User Interface and User Experience, Data Science, Digital Marketing) in a nonformal online professional development academy, also known as an online boot camp. Qualitative data collection through focus group discussion was applied to acquire the learners’ voices on the challenges in applying SRL during their learning process and the desirable SRLA features to support them. In total, there were 27 students (n = 27) who formed into ten focus groups consisting of two (three groups) to three learners (seven groups) in each group. All study participants are Indonesian; their demographic characteristics are summarized in Table 1. The topics and guiding questions for the focus group discussion are depicted in  Appendix 1.

The second group of participants was the teachers (n = 5) who facilitated the courses and assisted the learners during their learning process in the mentioned online boot camp. The semistructured interview was conducted to gather the teachers’ voices on the observable learners’ challenges and the desirable SRLA features to support the learners in practicing SRL. Similar questions ( Appendix 1) were asked to obtain teachers’ perspectives on learners’ challenges in SRL and features that may help the learners improve their SRL.

The data collected from the learners’ focus group discussions and teachers’ semistructured interviews were analyzed using a thematic analysis approach. Thematic analysis is a widely used qualitative method for identifying, analyzing and reporting patterns or themes within the data (Braun and Clarke, 2006). The aim of the thematic analysis in this study was to examine the voices of the learners and validate them with a balance with teacher voice. The integrated voices from both learners and teachers were then conceptualized with iterative expert evaluation for establishing design principles SRLA.

The process of data analysis included several stages, and the NVivo-12 software, a commonly used tool for qualitative analysis, was used in the analysis. Initially, audio recordings of focus group discussions and interviews were transcribed into written form and checked for accuracy. The transcripts were then read several times, guided by the research question and the findings from the literature review, to identify initial codes related to the challenges in SRL and associated needs for SRLA. Following the initial code grouping, the themes were refined, combined or divided in the next stage. A series of revisions and modifications were made to the themes as new information emerged during this stage. During data analysis, a conclusion is reached when it appears that no new themes emerged from the data, and the themes seem to have captured the essence of that data. In the final stage, the themes were elaborated on in detail, and evidence of the themes was provided by relevant quotes from the participants.

Understanding learners’ expectations of SRLA is essential for meeting their needs. This study examines the challenges of learners and their expected features and subfeatures of SRLA in the different phases of SRL, namely, forethought, performance and self-reflection phases. The results of the thematic analysis on lifelong learners’ focus group discussions and teachers’ semistructured interviews are presented in Table 2, which summarizes lifelong learners’ voices on SRL challenges and needs, and desirable features and subfeatures for SRLA. The frequency of occurrence and examples of statements for each theme are reported in  Appendix 2.

Each SRL phase is reported with associated challenges and needs as well as desirable features and subfeatures. Forethought is an important part of SRL, and it involves planning and preparation of learning activities (Zimmerman, 2002; Zimmerman and Moylan, 2009). In this phase, learners anticipate features that will assist them in understanding tasks, setting goals and planning their learning journeys. It includes information and visualizations of the learning path, instructions for tasks and end goals, as well as personalized recommendations tailored to the individual’s motivations. As part of this phase, a comprehensive and engaging dashboard will be needed to set effective goals and plans:

During the performance phase, learners need features that will help them manage their time, monitor their progress, seek information and receive help and support. This includes task and time management, real-time monitoring, help-seeking, stimulating interests and excitement and rehearsing and constructing knowledge. The challenges in this phase include managing time, tracking progress, efficiently accessing information and receiving the required support:

The self-reflection phase requires features that will help learners evaluate their performance and make adaptive improvements. This includes self-evaluation analytics based on various standards and recommendations for improvement. The challenges in this phase include objectively evaluating performance and making adaptive improvements:

Finally, ethics and data preferences require features that ensure the information is trustworthy and protect the privacy of the learner’s data. This includes data privacy, transparency and security. The challenges in this phase include concerns about sharing personal data and the need for information on data privacy and security:

In conclusion, learners expect SRLA to provide a comprehensive and engaging dashboard with features that support each phase of the learning process, from understanding the task to evaluating performance and ensuring data privacy. These features should help learners manage their time, monitor progress, seek information, receive help and support, evaluate performance and make adaptive improvements while ensuring that the information they use is trustworthy and protects their privacy.

The conceptualization of design principles for SRLA was informed by the evidence gathered for lifelong-learner and teacher voices (C1–C9 in Table 2) in alignment with SRL theory (Zimmerman, 2000), and guided by the established design principles for generic LAIS (Nguyen et al., 2021) (see Figure 1). The conceptualized design principles were also iteratively evaluated and refined through expert assessment. The conceptualization centers on the activities that the system should afford for addressing learners’ needs as well as promoting SRL to enhance their learning outcomes.

According to Nguyen et al.’s (2021) principle of actionable information, LAIS “should have features that allow for the reporting of actionable information about learners and their learning” (p. 555). Our findings reported that learners need features to keep track of their learning progress toward personal goals and objectives (C1–C4). Importantly, the lifelong learners in this study highlighted the need for personalized recommendations based on their own motivational factors (C2). Accordingly, we argue that SRLA should be able to provide learners with personalized information and visualizations to self-monitor their learning process and prepare for the planned outcomes; thus, the first initial design principle [Init. design principles (DP)] is as follows:

With LA, learners can receive personalized recommendations based on their learning interests, capabilities and outcome expectations (Nguyen et al., 2018; Van Schoors et al., 2021). As a result, learners can set more meaningful and achievable learning goals, and make more informed decisions about their futures. Furthermore, aligning learners’ individual learning progress with their own learning goals could also motivate them to better engage in learning. Motivation plays a significant role in SRL as it drives individuals to take initiative and actively engage in the learning process (Zimmerman and Moylan, 2009).

As a result of our findings, learners need analytics to regularly engage them in learning with features that stimulate their interests and motivate them to maintain their learning progress (C3, C6, C8). Accordingly, our initial DP2 was defined as the principle of continuous engagement that recommends SRLA to provide personalized features to stimulate interests and excitement to boost their motivation and engagement in learning. Nevertheless, Nguyen et al. (2021)’s principle of information timeliness recommends LAIS to deliver analytics at the right time for its maximum impact, and learning sciences and decision sciences can provide insight into the best way to design the time latency between data collection and reporting.

A key component of SRL is reflection, which enables learners to assess their learning experiences and use that information to guide future learning (Zimmerman and Moylan, 2009). It is a process of introspection in which learners examine their own experiences, thoughts and emotions. Nevertheless, learners can also reflect through the peer reflection process in which learners reflect on their own experiences and then share their insights and perspectives with their peers for discussion and further reflection. In addition to self-reflection and peer reflection, reflection on feedback from other sources, including LA tools, can also be a form of reflection. In this study, lifelong learners reported challenges to self-evaluate objectively and comprehensively to remain adaptive (C8). It is suggested for SRLA to involve features for acquiring or generating feedback from several sources, including individual and collective feedback from peers and experts.

Research has shown that social interactions and support could significantly influence learners’ SRL (Kwon et al., 2014). Our findings from learners’ focus group discussions emphasized the need for SRLA to offer access to social and psychological support during the learning process (C5). This resulted in our proposed principle of social support as follows:

The principle of social support posits that SRLA systems should extend beyond mere academic metrics to include recommendations and resources for social and psychological support. This extension appears well-founded, given the gaps in the current literature. Traditional SRL theories (e.g. Zimmerman, 2000) predominantly focus on cognitive elements such as planning, time management and meta-cognition. Likewise, existing SRLA systems, as described by Winne (2017), have largely been confined to these dimensions. However, this narrow focus overlooks the significance of affective and social elements, which have been emphasized in broader educational psychology literature. The inclusion of social support aligns with Vygotsky and Cole’s (1978) constructivist theories, Bandura’s (2001) social cognitive theory and Hadwin et al.’s (2018) SRL model. Empirical evidence from Chen (2022) and Järvelä et al. (2015) supports this, showing improved student engagement and well-being. However, ethical challenges related to data privacy (Nguyen et al., 2023; Slade and Prinsloo, 2013) and measure accuracy (Ifenthaler and Widanapathirana, 2014) should be carefully considered in future implementations. Accordingly, DP5 was conceptualized to address the ethics and privacy aspects of SRLA as follows:

The principle of ethics and privacy calls for SRLA to emphasize anonymity, transparency and robust data protection. This principle complements existing literature that critiques the lack of ethical considerations in traditional educational analytics systems (Slade and Prinsloo, 2013). The call for anonymity aligns with broader trends in information systems emphasizing user control over personal data (Nguyen et al., 2021), while transparency fosters trust, enhancing the effectiveness of SRLA (Buckingham Shum et al., 2019). Additionally, the explicit focus on data protection addresses the real-world risks of data breaches, aligning with regulations like the General Data Protection Regulation (GDPR) in the European Union. In sum, this principle provides a crucial ethical framework for SRLA but demands ongoing commitment to ethical vigilance to adapt to emerging challenges.

Four experts in the domains of LA and SRL were invited for the evaluation of both learner needs and the conceptualized design principles for SRLA. Three of the experts are working in the field of learning sciences (Expert A: Female, 12 years of experience; Expert B: Female, six years of experience; and Expert C: Male, 15 years of experience), while the other one is in the field of information sciences (Expert D: Female, eight years of experience). All the experts have published scientific papers within the domains of LA and SRL. They are presented with the conceptualized design principles for SRLA and the findings from learner focus group discussions and teacher interviews. After their own reading time, the researchers first walked them through the learners’ needs and their desirable features for SRLA. The experts were asked to provide their assessment based on the provided evaluation criteria (adopted from Venable et al., 2012; see above). They were then requested to evaluate whether the set of conceptualized design principles for SRLA has captured all learner needs, and any improvements needed for design principles to serve as useful guidance for SRLA design and development. The expert evaluation sessions lasted between 40 min and 60 min and were videotaped for later examination in evaluating and refining the design principles for SRLA. All the presented learner needs and design principles were sent to the expert after the evaluation sessions for potentially additional examination and feedback via emails.

In the evaluation, all four experts agreed with the importance of this principle of personalized monitoring (DP1 evaluated). Regarding DP2, a learning sciences expert (Expert A) has raised concerns about “continuous engagement” because excessive engagement in learning can be harmful to learners. Studies have pointed to the phenomenon of “academic burnout,” where constant engagement with educational tasks may lead to stress, emotional exhaustion and reduced academic performance (Schaufeli et al., 2002). Furthermore, Kirschner and Karpinski (2010) found that higher engagement in online learning environments could also result in decreased time for other crucial activities, including physical exercise and social interactions, which are essential for holistic well-being. Therefore, while engagement is often considered a positive metric in SRLA, there is a need to balance it with considerations for the learner’s overall health and well-being. Appropriately, after carefully reviewing the issue, we revised the initial DP2 to focus on persuasive motivation for SRLA to stimulate learning interests:

This revised DP2 finds grounding in self-determination theory, which posits that meeting individual needs for autonomy and competence can boost intrinsic motivation (Deci and Ryan, 1985). For instance, the use of “gamification” elements has demonstrated the potential to increase both motivation and engagement in learning contexts (Deterding et al., 2011). Therefore, incorporating persuasive elements into SRLA could serve as a powerful strategy to enhance learner motivation and engagement, albeit with a mindful approach to avoid potential pitfalls such as excessive engagement.

Interestingly, regarding DP3, evaluation with the experts raised a critical discourse about the role of SRLA in improving and/or promoting learners’ SRL. Expert B from learning sciences questioned about the undesirable consequences of SRLA in establishing learners’ reliance on technology for sustaining SRL activities. It is recommended that SRLA should consider a scaffolding approach in developing learners’ SRL skills. Recent research has highlighted the importance of adaptive scaffolding in supporting SRL (Song and Glazewski, 2023). On the other hand, although Expert D from information sciences agreed with these side effects of SRLA, Expert D argued for the nature of living with technology in the age of artificial intelligence (AI). The augmentation was provided for the fact that many learning activities and processes are now cooperating with the use of technology, such as Google or Microsoft 365, which extends beyond the classroom to include daily life tasks that are integral to SRL. For instance, planning and time management – key components of SRL as outlined by Zimmerman (2000) – are often facilitated through digital tools like Google Calendar or Microsoft Outlook. These platforms not only assist in organizing academic deadlines but also help manage personal commitments, thereby enabling a holistic approach to self-regulation (Koehler and Mishra, 2005). Thus, the fusion of SRL activities with ubiquitous technology platforms highlights the increasingly blurred boundaries between educational and everyday digital tools, reinforcing the need for SRLA to be adaptable and integrated across multiple contexts. Future studies on the current topic from different philosophical perspectives are, therefore, recommended.

All the experts reached a consensus on DP4 and also referred to recent literature that highlighted the importance of emotional regulation (e.g. Järvenoja et al., 2019) and its support. They also shared an agreement on the principle of ethics and privacy that advocates SRLA to anonymous and transparent handling of personal data, with protection against unauthorized access, destruction, alteration or disclosure. It is important to consider questions, such as whether the collection and use of learner data are necessary, whether the data is being used in a fair and transparent manner, and whether the benefits of the data use outweigh any potential detriment. The ethical and privacy aspects of LA remain challenging (Ladjal et al., 2022), yet we hope that the findings from this study can contribute to the discussion to push forward the design and development of trustworthy LA systems.

Table 3 summarizes the final five design principles for SRLA as the output from our TPDM. The design principles offer prescriptive knowledge about the design of SRLA systems.

The importance of lifelong learning along with professional development has never been greater than it is today in a world that is fast-changing. SRL has been a key skill for successful lifelong learning (Molenaar, 2022). By providing learners with insights and recommendations based on their learning activities and progress, LA tools and technologies aim to facilitate and enhance SRL (Winne, 2017). The integration of LA into SRL can help lifelong learners make informed decisions about their learning activities, monitor their progress and evaluate their performance. Although research has provided insightful understandings of SRL processes and LA support for SRL, previous studies have been mostly conducted in the context of K-12 and higher education (Heikkinen et al., 2022). Up to now, far too little attention has been paid to SRLA targeting lifelong learners. This study thus attempts to fill a noteworthy gap by examining the challenges and needs of lifelong learners in SRL (RQ1, Table 1) to inform the design and development of SRLA (RQ2, Table 2).

Human-centered LA has grown rapidly in recent years and has the potential to support learners in developing SRL skills and improving their academic performance. In spite of these benefits, it has been questioned about how to consider the voice of learners more into the design and development of human-centered LA (Buckingham Shum et al., 2019). In this study, we present an approach for TPDM (Figure 1) that addresses the well-recognized concerns about the alignment between the needs of the learners and the design of LA tools and systems. Learners’ learning preferences are not always aligned with teachers’ pedagogical intentions or successful learning theories. Our TPDM methodology offers guidance for integrating and evaluating learner needs into the design of theory-grounded human-centered LA.

TPDM approach offers a comprehensive framework that combines viewpoints from various key players in the educational process. approach distinguishes itself from existing methodologies primarily through its comprehensive integration of perspectives, ranging from frontline users like lifelong learners and teachers to theoretical frameworks and expert evaluations. Traditional approaches, such as user-centered design (Norman, 1986) or participatory design (Schuler and Namioka, 1993) often focus more narrowly on user involvement or the inclusion of specific stakeholder groups, respectively, but may lack a structured integration of theoretical constructs. TDPM not only incorporates experiential insights from these stakeholders but also synthesizes foundational concepts from learning theories and design theories to inform its methodology. The inclusion of these nascent theories provides a structured backbone, allowing for data-driven adjustments and refinements in the design process (Baskerville et al., 2018; Heinrich and Schwabe, 2014). Finally, the TDPM approach is subject to evaluation by experts in the field, adding a layer of rigor and validity to the outcomes. This multilayered integration serves to produce more contextually relevant, empirically supported and theoretically robust educational interventions or tools, therefore, enhancing its applicability and impact.

In terms of research impact, TDPM contributes to the empirical rigor by synthesizing insights across a diverse set of stakeholders and theories. It addresses gaps in existing frameworks that may be overly reliant on a single theory or stakeholder perspective. Thus, TDPM fosters a more nuanced and comprehensive understanding of learning contexts, providing fertile ground for interdisciplinary research. For practical applications, TDPM’s multifaceted approach ensures that educational interventions or tools are not only theoretically robust but also contextually relevant and practically effective. By incorporating real-world insights from lifelong learners and teachers and subjecting the design to expert evaluations, TDPM enhances the usability and effectiveness of the final product. This broadens the research artefact’s appeal and adaptability, making it more likely to be adopted in diverse learning environments.

Our findings also suggest some practical implications for promoting SRL in professional development programs. Taking control of an individual’s learning process is an essential element of SRL, which leads to improved outcomes and greater satisfaction (Theobald, 2021). However, it is challenging for lifelong learners in professional development programs to manage their learning time while learning is often not their sole or even main occupation. Accordingly, it is crucial to provide lifelong learners with support for time management as well as to keep up their learning motivation. The results of this study can also shed light on developing SRLA systems to address lifelong learner needs.

The results of this study indicate that it can be helpful to learners to stay motivated by real-time monitoring and tracking of their learning activities and progress toward their personal goals and objectives. The use of motivational notifications and gamification can also aid in stimulating interest and excitement about learning. Previous research suggests that gamified approach embedded with SRL support can leverage students’ intrinsic motivation and lead to better learning (Qiao et al., 2022). In line with prior studies, our findings imply the importance of the self-regulated gamified approach for lifelong learning.

Furthermore, LA has been the subject of controversy for years (Pargman and McGrath, 2021). LA design and use are hampered by a lack of evidence and guidelines for practical ethics from end-users’ perceptions and assessments. Triangulating data collected from both lifelong learners and teachers, this study informs learners’ ethical concerns, relevant data and the conditions in which they are willing to share for SRLA. Learners in this study reported concerns regarding the security, privacy and transparency of their data, which is consistent with the literature. Nevertheless, as long as the LA process is secure and transparent without using sensitive personal information (e.g. income, health, etc.), lifelong learners expressed a high willingness to share their data for SRLA that could support their learning effectively.

It is important to note that this study is limited to analyzing data from a single group or context. This can limit the generalizability of the findings that may not be applicable to other settings or cultural contexts. Further research may conduct multisite studies or comparative studies in different contexts to validate the generalizability of our findings. Furthermore, the limitation of thematic analysis lies in its susceptibility to researcher bias and its challenges in managing large or context-rich data sets, which can compromise the depth and validity of the study (Braun and Clarke, 2006; Guest et al., 2011). Despite these limitations, the method’s adaptability and ease of use make it an effective approach for eliciting and understanding the varied viewpoints of stakeholders. Another limitation of this study is related to the small sample size of teachers and experts involved in this study. Although teacher voices and expert opinions were only applied to evaluate the learner needs, the small sample sizes may not provide a comprehensive validity and generalizability of the findings. Notwithstanding the relatively limited sample, we gained great insights from the triangulation of results from learners, teachers and experts.

In the age of AI, the ability to self-regulate learning for lifelong learning is increasingly becoming important for individuals who want to remain relevant and competitive in the workplace. As an example, the recent release of OpenAI’s Chat-GPT serves as a reminder of the importance of self-regulated lifelong learning as it demonstrates the potential of AI to automate job tasks previously performed by humans. Individuals must constantly learn new skills and adapt to new knowledge due to the rapid pace of technological advancement and the constantly changing job market. Lifelong learners can stay up to date with the latest developments in their field by taking responsibility for their own learning through SRL. Embracing human-centered LA can contribute significantly to supporting lifelong learners’ SRL. The proposed approach for TPDM will allow advanced integration of learning sciences and design science research into the development of effective and trustworthy human-centered SRLA systems for lifelong learners.

This research has been funded by the Finnish Academy grant number 350249.

Statements on open data: The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Ethics: This research complied with the university research integrity and ethics guidelines by the Ethics Committee of Human Sciences of the University of Oulu.

Conflict of interest: The authors have no conflicts of interest regarding this study to declare.