Educational data mining and learning analytics: a review of educational management in e-learning Free

Currently, there is a growing interest in the use of data and information in education e-learning, especially in an attempt to measure the teaching-learning (TL) process and to optimize system performance at different levels, from the TL process itself to the level of management of educational institutions (Rodrigues et al., 2018). According to Gardner and Brooks (2018), the endeavor for the improvement of educational processes has shown that generalizing the results of the analyses proves to be ineffective. This occurs because different educational environments have peculiarities and different characteristics. For example, massive open online courses (MOOCs) platforms are incomparable to other educational modalities because they have different characteristics in terms of structure, model and management. Even in the context of online courses, the objectives, metrics and structuring of MOOCs cannot be compared to the degrees of e-learning modality (Gardner and Brooks, 2018).

The application of data mining in the educational context, known as educational data mining (EDM), is widely applied to identify patterns extracted from collected educational data, with the aim to improve the TL process. When applied, EDM techniques and methods can reveal important information, which are used by administrators who seek to optimize educational processes and increase their assertive decision-making capacity. For example, Kerr (2016) applies EDM to understand how a student learns. Kim et al. (2016) examine the behaviors of educational actors within online learning environments. Burgos et al. (2018) apply EDM techniques, considering activity grades and looking for prognostics on the possibility of student dropout. All of the aforementioned approaches are pertinent at the managerial level of educational institutions.

Similar to the objectives of EDM, for approximately a decade, learning analytics (LA) has drawn the attention of academics, researchers and administrators (Siemens, 2013). LA has an important role in improving educational systems, regardless of its modality (traditional or e-learning) and its different perspectives, both in the actions and perceptions of the actors (educator and/or student) as in the managerial aspect (Bhutto et al., 2020). For example, based on the analysis of students’ behavior and profile, the administrator is able to plan and proceed with appropriate interventions for specific groups or environments, seeking to better understand and, consequently, refine the service to students.

As established by David et al. (2021), there is no significant difference between the concepts of EDM and LA, as both focus on student performance, learning platforms and modeling student behavior. However, there is a subtle difference in the approach in LA, which is focused on the processes that influence learning, whereas EDM is focused on discovery of educational patterns.

According to EDM, the extracted knowledge can contribute to the improvement of the teaching and learning process, the main objective of LA. The areas are congruent and complement each other, and their contributions are of interest to the institutional manager. As the EDM area intrinsically seeks the application of methods/techniques from educational data, in this review, we are interested in works that highlight their contribution to the TL process. In this work, we consider both terms LA and EDM as search keywords of the bibliographic material to be analyzed.

From an administrative point of view, the main functions of managers in any organization involve leading teams and making the strategic decisions necessary to achieve the objectives (preestablished). To perform these functions, it is necessary to: analyze, plan, make decisions, organize, delegate, coordinate, lead and monitor. In the educational field, where the main objective is the effectiveness of teaching and learning, it is important to plan and monitor all the steps of the teaching and learning process to understand possible bottlenecks and failures of the institution.

This work considers as administrator, the person/sector that plays the role of the manager, who analyzes and plans improvements in the institutional scope related to the optimization of the teaching and learning process from planning actions, monitoring and definition of corrective measures resulting in increased performance and student engagement.

In general, educational management involves all processes of an educational institution. From administrative to pedagogical, it is up to managers to optimize routine activities and increase the teaching process efficiency within the institution. Management searches for models and strategies that may help to enhance educational processes, thus reflecting higher-quality teaching. Furthermore, it seeks to improve the quality of services provided to students throughout their journey of consumption of such educational services, which begins before enrollment and ends after graduation, taking into account that the student may become loyal and perhaps buy other services.

It is important to point out that the contributions of EDM and LA should not be limited solely to proposing computational models to meet the different educational demands. These works must show their effectiveness to attract the attention of the manager, who is responsible for implementing the proposal, monitoring and validating the results. This would allow the manager to assess its effectiveness in improving the teaching and learning process, increasing student engagement and reducing possible dropout.

This review seeks to contribute to the management of the top management of educational institutions operating in e-learning environments concerned with improving the TL process, engagement and retention of students. To this purpose, a systematic literature review was conducted in the period of 1994 to 2023, which discusses the research studies in EDM, in the e-learning environment, from the manager’s perspective. This review aims to identify the perspectives, trends and discuss possible key performance indicators’ (KPI) so that the manager can monitor, analyze and infer about the TL process.

From this review, it was possible to observe that managers have been concerned about the effectiveness of the platform used because it contains the entire learning process, and it is through it that all interactions occur that enable the generation of information about students. From the data collected on these platforms, some improvements and inferences can be made regarding the actions of educators and human tutors (or automatic tutoring systems), curricular optimization or changes related to course content, a proposal of evaluation criteria and, also, increased the understanding of different learning styles.

Increasingly, management’s focus is turning to better learning outcomes, as highlighted in Kerr (2016), which proposes a structure that seeks to optimize student’s experience, encourage student’s participation and reduce dropout rates.

Through this review, it was possible to identify a concern on the part of the administrator in regard to the effort for improvement of TL educational models, mainly in the e-learning environment. Thus, the authors of the analyzed works have suggested process optimization models focused on personalized learning, optimization in the recommendation of materials and learning objects (Charitopoulos et al., 2017) and adequacy of educators/tutors’ pedagogy according to the student’s profile (Wongwatkit and Prommool, 2018).

This review is divided into six sections. In Section 2, we summarize the main existing reviews in EDM, analyzing the main topics addressed to show the difference in our work. Section 3 presents the methodology adopted in the review process as the main questions to be answered. The analysis of the managerial view on teaching in the e-learning environment and the perspectives for future research is presented in Sections 4 and 5. Finally, conclusions are presented in Section 6.

In this section, the main and most recent literature reviews in the EDM field are presented to identify their main contributions and differentiate the contribution of the present review. Table 1 portrays simplified the main goals, strategies and/or areas explored by the review studies present in the literature.

Analyzing the review papers, highlighted in Table 1, it can be observed that there is a tendency to produce review papers in more personalized, adaptive and interactive educational environments to improve the results of the learning process. Different perspectives were observed that can be summarized into proposed techniques and methods, visualization of results and/or learning dashboards and MOOC’s.

As noted, the literature reviews specifically address the most commonly used techniques for knowledge extraction in the educational area. However, few works are concerned about the managerial view of educational institutions. The main objective of this review is to highlight and analyze the most effective contributions in the construction of models that optimize student engagement and performance and, consequently, the teaching and learning process, which can be a tool for institutions seeking continuous improvement in their processes. Thus, this paper conducts a systematic review of the literature in the domain of teaching e-learning and focuses specifically on articles to which they propose an improvement or applicability in management.

This review has its domain restricted by the educational concepts of e-learning and management. This review aims to answer the following research question:

To answer this question, we carried out a systematic literature review, following the guidelines presented by Kitchenham et al. (2009), where the selected articles are defined by inclusion, exclusion and filtering criteria.

The search for bibliographic material comprised articles published in journals and papers published in conferences from 1994 to 2023, totaling 30 years of research in EDM. The searches were carried out in English, Portuguese and Spanish using the keywords “Educational Data Mining” and “Learning Analytics”. Queries to the bibliographic repositories returned a total of 1,772 articles. Table 2 shows the repositories in which the searches were performed. The second and third columns of the table indicate the total number of articles per repository and their respective impact.

After the data collection process, inclusion and exclusion criteria were applied to align with the purpose of this review.

In selecting the articles for this review, the abstracts and introduction of each article were read to identify only those articles that suggested improvements to the TL process in the e-learning environment from a management perspective. The case review focus on the teaching process, the learning systems, the content taught and the instructions/interventions in the educational setting.

Accordingly, we emphasize that as an exclusion criterion, those references in which only the application of data mining techniques was performed, without considering the management aspects and improvements in the TL process, were removed from the bibliographic material. As a result of applying the inclusion and exclusion criteria, 78 articles were considered.

In this section, we will answer the research questions by presenting the main trends and perspectives of EDM in the e-learning environment.

With the increase of courses and students in the e-learning educational modality, educational institutions face great management challenges in the search for service quality. From the bibliographic material considered, we can observe that the authors mainly aim to measure and optimize learning to increase service quality, student satisfaction and achieve the objectives sought by the managers of the institutions. This review revealed that most of the works consider students as the main actors in the TL process.

Despite the physical distance between the actors (students, educators and human tutors) in the e-learning environment, the institution managers try to understand the actions of these actors in terms of interaction, thematic comprehension and communication on the platforms to increase the effectiveness of the TL process offered.

With this work, it was also possible to acknowledge the extensive use of mass data generated by online environments such as Moodle. From the actions of the actors in these online environments, through the discovery of patterns, by applying data mining techniques, management strategies for retention, intervention and planning can be purposefully aimed at improving the performance of institutions in the e-learning field.

The articles examined provide results on the type of instruction for educators and tutors, as well as the support materials that should be used based on their respective effectiveness. These results can help managers, for example, in redesigning courses for effective learning.

A literature review process demands great effort in organizing the collected bibliographic material. Typically, this organization is based on the identification of common topics or subjects addressed in the material. In this work, we use OrgBR-M method (Rodrigues and Zárate, 2020) to assist in the organizing of bibliographic material. The method uses the formal concept analysis (FCA) Ganter and Wille (2012) theory to organize this material using a conceptual hierarchy, which reduces the effort required in the organization and facilitates the analysis of the bibliographic material.

In a summarized way, OrgBR-M uses a collection of documents represented by themes or domain concepts related to a study domain. For example, in our study, the domain concepts correspond to the educational concepts involved in the TL process. For this work, we consider the domain concepts proposed by Rodrigues and Zárate (2020), which involve the concepts: teaching, traditional classroom, e-learning, student, educator, tutor, management, coordinator, classroom, virtual classroom, teaching plan, student groups, interdisciplinary, multimedia, monitoring, personalization, evaluation and learning. Those concepts were identified by means of the Sphere-M method (Odon de Alencar et al., 2012) as explained in Rodrigues and Zárate (2020).

From the identified domain concepts and by linking bibliographic material to the concepts, OrgBR-M defines a formal context, which is formed using the bibliographic material as objects and the domain concepts as attributes. Besides, OrgBR-M allows to select domain concepts relevant to a specific subdomain or study theme. For example in this work, we consider the concepts of e-learning and management.

In the next step, OrgBR-M organizes this material by applying the FCA theory to hierarchically organize the bibliographic material in a conceptual lattice. Organizing bibliographic material is obtained by hierarchically organizing concepts, where the higher levels in this hierarchy correspond to sub-domain that contain a recurrent set of domain concepts or themes related to a collection of objects (bibliographic material).

The conceptual organization of this material allows the identification of potential research themes from sub-domains of concepts and their respective lattices. Notice that, OrgBR-M provides researchers with a tool to organize the bibliographic material from bibliographic material previously selected, read and linked to the domain concepts related to a study domain.

After a thorough reading of the bibliographic material and applying the OrgBR-M method, seven subdomains of interest were identified: articles that deal with the manager’s perspective in relation to the educator and the tutor (Sections 4.1.1 and 4.1.2); articles attempting to optimize learning in relation to the study plan and multimedia objects (Sections 4.1.3 and 4.1.4); articles focusing on personalization of teaching (Section 4.1.5); and articles that deal with evaluation processes (Section 4.1.6) and knowledge acquisition through the learning process (Section 4.1.7). These subsections aim to highlight the perspectives and trends of EDM from the manager’s perspective. In Figure 1, we can briefly visualize the main subdomains covered in the articles.

The articles in this subdomain mainly address the actions that can be taken by instructors to improve the TL process and students’ engagement by conducting comprehensive data monitoring that emerges from student-teaching platform interactions.

Based on the application of EDM strategies, it is proposed to develop feedback mechanisms that increase the effectiveness of the TL process for the taught contents (Cerezo et al., 2014; Cao et al., 2009).

Other works aim to enable educators to analyze and regulate their actions (Otsuka et al., 2007), make adjustments to the activities and pedagogical methodology used (Chen et al., 2007; Park et al., 2016; Martinez-Ortiz et al., 2009), identify the points that need improvement (Kim et al., 2016), propose new courses (Marušić et al., 2011) or simply have more control over them (Cao et al., 2009), using adaptive and flexible observation rules defined by the educator body itself.

Based on monitoring data, the educator also suggests interventions for more personalized learning (Paul et al., 2004). Wang and Lin (2012) propose an interactive learning process in which educators can add new topics for discussion when they feel the need for them.

As pointed out by the authors, interactive feedback systems allow educators to better understand students’ difficulties with certain content and monitor their lack of motivation (Rosales et al., 2011). Wongwatkit and Prommool (2018) suggest an interactive instruction recommendation system in which the instructions provided to educators are analyzed. According to Burgos et al. (2018), educators have active roles when the matter is a student. This latter aspect is pivotal to the management of educational institutions and to the proposal of EDM-based strategies.

Kaliisa et al. (2022) prepared a study for higher education about the different aspects used by teachers in the design of practices. This study revealed as main aspects: situational factors, feedback systems and professors’ personal experiences. In addition, professors reported that the design of practices can be influenced by institutional regulations and by biased feedback sources. These findings indicate that support tools for professors, such as LA, can contribute to improving the design of practices, considering the main aspects and challenges that influence the design of practices by professors. Kaliisa and Arild Dolonen (2022) proposes a follow-up panel with important information, such as insights for professors about student participation in online discussion forums.

The articles in this subdomain address to the actions proposed to tutors aiming to improve students’ learning and engagement. It can be observed that the actions presented here are similar to those also proposed to educators.

In this work, the concept of tutor corresponds to the person or computer system that gives support to the educator in the teaching process and helps him/her in decision-making.

According to the reviewed works, tutors should be able to interpret students’ behavior and their learning path. Thus, EDM must provide the mechanisms for such goals. In Cerezo et al. (2020), after proposing learning and intervention models during the tutoring process, it was observed that students who did not follow tutors’ suggestions failed. The review also has indicated that mentoring can be broader, dealing with social, cognitive and affective issues aimed at improving retention in the institutions (Burgos et al., 2018).

Studies suggest that human tutors should receive information and take active actions to provide material recommendations (learning objects) in order for students to master and acquire skills about certain content (Bodily et al., 2018). This recommendation can be made in an individualized manner if the proposal is to provide personalized instruction based on the students’ profile (Charitopoulos et al., 2017; Kausar et al., 2018; Alex and David, 2004), always aiming to optimize the TL process in terms of quality, efficiency and accessibility.

According to the reviewed works, the human tutor should actively participate in the development of methods based on students’ behavior and knowledge to provide more appropriate (Kerr, 2016) and personalized instructions (Ruangvanich and Nilsook, 2018). In this way, EDM could propose clustering-based models to identify groups with similar motivational characteristics in an attempt to optimize the results (Jenila Livingston et al., 2019).

Sun and Zhang (2008) discusses the proposal of automated tutoring systems where instructions are given to assess teaching strategies and learning processes. These systems are characterized in some cases by automated feedback in real-time (Acevedo et al., 2018). Feedback can also be based on rules previously established by the group of educators (semi-automatic systems). Semi-automated systems can also help in the creation of materials that aim to optimize learning based on learning objects (Martinez-Ortiz et al., 2009).

Some works suggest that, to suggest improvements to both the teaching and learning process, automated feedback must be delivered to human tutors so that they can better understand students’ behaviors toward the courses and which are their main difficulties. Based on student interactions with the platforms, Antonio et al. (2010) recommend that human tutors take corrective actions and even reformulate courses when necessary (Pardo et al., 2016).

Students engagement and motivation are a growing concern for the administrator, especially in the e-learning environment. Therefore, it is proposed to create systems that encourage students’ engagement in educational models in which the tutor asks questions and receives answers in real-time (Karachristos et al., 2016). This interaction can also be based on individualized guidance and feedback on task completion. In this case, the intervention mechanism may include instruction, support and motivational components (Muhittin et al., 2019).

Dropout prediction systems can signal which students need special attention from tutors, and corrective action can be taken to prevent these students from dropping out (Acevedo et al., 2018; Nasiri et al., 2012; de la et al., 2017). The systems can provide an alert regarding student performance and behavior so that interventions become more assertive (Shrestha and Pokharel, 2019) or even systematically suggest interventions that can be adopted (Ortigosa et al., 2019).

Many articles address creating or improving feedback for the educator. Cavus Ezin and Yilmaz (2022) examine the effects of feedback in a learning environment based on mobile devices with the intention of verifying the existence of differences between students who received and did not receive feedback. These differences are determined according to acquired skills, motivations and academic achievements. Another perspective to measure the efficiency of feedbacks was proposed by Karaoglan Yilmaz and Yilmaz (2022), where the effect of metacognitive feedbacks was measured (where an attempt is made to increase the individual’s awareness and control over their learning) in relation to student engagement.

With the aim of improving the performance of the TL process, Sahni (2023) proposes a learning management system (LMS) to help tutors and educators to identify students at risk, providing feedback in real-time. The paper also analyzes the relationship between student engagement and academic performance.

The e-learning environment allowed the proportion of students compared to the number of professors to increase. With this come challenges for instructors to provide quality personalized feedback. While significant efforts have been directed toward automating feedback generation, relatively little attention has been paid to the underlying characteristics of feedback. Nicoll et al. (2022) developed a methodology to analyze instructor-provided student feedback and determine how it correlates with changes in student grades.

In this subdomain, the articles propose changes in the study plan aiming to optimize learning and increasing students engagement. To do so, it is important to know the effectiveness of study plans so that if it is not appropriate, possible changes and adaptations can be made (Kerr, 2016; Kazanidis et al., 2020). Often, these changes are made in conjunction with the course’s instructors or faculty so it can be based on systems that may help to create plan of studies based on EDM and adaptive learning processes (Martinez-Ortiz et al., 2009; Lemay and Doleck, 2020).

Based on the review, some strategies for course structuring were identified, listed and explained below:

• The competency-based learning in which courses must be organized into competency topics and leave it up to students to define which topics they want to learn (Umbleja et al., 2017).

• Based on changes in the plan of studies and on the student’s engagement rate (Konstantinidis et al., 2017).

• Based on micro-foundational courses to optimize the learning pathway (Sun et al., 2015).

Recommendation and/or decision support systems can help educators and managers improve their plans of study by making recommendations from evaluating learning patterns (Tang et al., 2012) and creating and monitoring possible modifications (Acevedo et al., 2018).

Due to the distance imposed by the e-learning environment, courses that better involve students in the TL process can be proposed, optimizing participation and learning to consequently reduce (Morales et al., 2019) dropout rates.

Talbi and Ouared (2022) propose to aid educational stakeholders with solutions to understand students’ state of motivation. The authors propose metrics to measure motivation from different paradigms. The work provides, through a mobile application system, automation to stimulate student tasks.

The Nisha and Renumol (2022) article proposes a recommendation system for the learning path (study plan) focused on building knowledge and analyzing learning performance. The model considers static and dynamic learning parameters to generate the study plan. The difficulty level of learning resources is adjusted based on real-time student performance analysis. Learning resources are recommended based on students’ learning preferences and abilities. The model also predicts the learning time and expected score for each student.

Recommending advisors (master’s tutors) is not an easy task in graduate school. For this reason, the work of Chen et al. (2022) proposes, based on modeling tutors of masters and students, a tutor recommendation system based on learning algorithms.

In this subdomain, the reviewed articles propose improvements in the recommendation of materials (learning objects) offered to students to optimize learning. To provide more suitable contents (Cerezo et al., 2014; Brtka et al., 2012), it is important to identify the effectiveness of such learning objects and the teaching methods applied (Kerr, 2016; Park et al., 2016; Kazanidis et al., 2020).

From the accesses via mobile devices, it is possible to determine which type of content is most suitable (Trifonova and Ronchetti, 2004). To do this, it is possible to develop systems to help create materials that can include recommendations for activities, learning content and tools (Martinez-Ortiz et al., 2009). It has also been analyzed how materials have been used (Konstantinidis et al., 2017).

Many studies have proposed content recommendation systems to help educators and tutors optimize course materials in a personalized manner (Charitopoulos et al., 2017; Wongwatkit and Prommool, 2018; Bodily et al., 2018; Alex and David, 2004; Plaban et al., 2007; Niyigena and Jiang, 2020). With regard to this optimization, other types of courses may be proposed, and the materials and methods used in these courses may determine their structure. In competency-based courses, for example, the identification and linking of learning materials for topics by competencies must be. In microfoundation-based courses, the materials provided to students must be divided into small units so that they become short, objective topics (Sun et al., 2015). In combined structure models that integrate traditional teaching methods (face-to-face classes) with in-depth activities, self- assessment and online interaction, integrated systems can be created where supplementary materials are provided on e-learning platforms (Caldirola et al., 2008).

Sun and Zhang (2008) propose a system for modeling instructional materials based on the concept of guided objects to conduct students to learn accordingly to a predetermined strategy, learning skills and assessment results.

Students’ engagement with the materials and activities is fundamental. In this sense, Morales et al. (2019) present a method in which each group of students (active or less active) are assigned to a respective strategy. The purpose is to provide meaningful learning experiences through key contents, varied activities, exercises and interaction in online communities to meet the needs of those specific groups, making courses more appealing.

Design/layout changes to the learning systems tools are also proposed to improve usability and satisfaction when navigating through the learning platforms (Wang and Lin, 2012). Wang and Lin (2012) explores an interactive learning system in which texts and/or images are displayed according to users’ habits. The research also evaluates knowledge exchange between students, educators and tutors and performs real-time updates.

Student interaction, perceived through the data generated by the LMS, is a promising field for blended learning (BL), which combines conventional face-to-face and on-line learning activities. However, applying on-line learning technologies in BL environments is particularly challenging for students with lower self-regulatory skills. The authors Yang and Ogata (Christopher et al., 2022) propose a personalized LA intervention approach that incorporates e-book suggestions and recommendation systems.

This subdomain analyzes the proposals dealing with the personalization of education in online learning platforms from the manager’s perspective. In general, the proposal of a personalized learning system aims to optimize teaching and learning based on the students’ profile and interactions with the content Paul et al. (2004) providing a customizable platform based on students’ preferences, interests and needs.

Plaban et al. (2007) present a system that enables the retrieval of personalized information from knowledge structures. Learner profiles can be captured/identified based on concepts and topics selected from the knowledge structure generated by the system. Based on the user profile and browsing behavior, it is possible to propose a personalized recommendation system.

To match the structure of knowledge to be captured, personalized learning systems fed by learners’ profiles and based on their behavior can provide learning resources and content (Li and Zhang, 2010). By analyzing students’ behavior, it is possible to infer their personalities and outline learning objectives, providing more appropriate and targeted resources and content (Ping et al., 2009).

Adaptive learning, which also works with learning personalization, provides different learning paths and materials defined according to students’ abilities and browsing/accessing data (Cerezo et al., 2014; Sun and Zhang, 2008).

A personalization system, which can group students who have similar habits and study modes, can use navigation/access logs to analyze behaviors and, from these patters, provide individualized services for each student (Sun and Zhao, 2009). In this sense, Yuanyuan and Qian (2009) also proposes a constructive and personalized teaching model.

Rosales et al. (2011) propose a strategy of creating a database from the users’ actions, which contains the resources and activities of the course and tracks the activities in parallel with the studies defined by the knowledge structure, the time spent on each action and other relevant information about the participants’ actions in the LMS. This system identifies problems in the learning process related to the design of the course structure and aims to better understand students’ learning difficulties and their different learning styles.

The concept of personality analysis in an intelligent learning environment can play an essential role in management. Based on this, Ruangvanich and Nilsook (2018) propose a personalized learning system that takes into account students’ personality, which is determined by the interaction between users and the LMS. The authors define five types of interaction between learners and the system (openness, extroversion, agreeableness, conscientiousness and neuroticism) and three types of interaction between instructors and the system (interpersonal content, interpersonal structure, interpersonal complementarity).

This subsection reviews the proposals that address the process of knowledge acquisition and the skills required in e-learning platforms.

Paul et al. (2004) propose cooperative learning in which the components of the learning system are based on the preferences, needs and interests of the learners. Sun and Zhang (2008) proposes different learning materials for students to learn according to their different learning abilities and browsing history.

Considering formative evaluation, which aims to monitor and identify students’ learning development, knowledge acquisition in each part of the learning process should be observed and perceived (Otsuka et al., 2007).

Chen et al. (2007) propose a system that, based on students’ logs, analyzes learning outcomes, based on performance in tasks and questionnaires, with the objective of providing educators a way to infer the outcome of their students’ learning process.

According to Cao et al. (2009), after analyzing the learning process, it is possible to create learning diagnosis systems that would support educators and students with feedback information on the results. Zorrilla et al. (2010) state that based on descriptive mining tasks, such as clustering and association, it is possible to identify students’ profiles and their respective learning patterns to support instructors with information about the teaching and learning process.

In contrast, based on the analysis of students’ learning behavior and trends, Li and Zhang (2010) suggest the use of a personalized recommendation system. Using the same concept of personalized learning, Kausar et al. (2018) propose a Big Data architecture for mining educational data, seeking to optimize the ideal configuration for students to learn more effectively.

The eGraph tool proposed by Cerezo et al. (2014) aims to facilitate the monitoring of student learning through visualizations and interactions with course content (theoretical content, summary, multimedia content, assignments, assignment submission, forum and forum participation). From there, it is possible to carry out an analysis of learning at the student’s level and in relation to the available contents (Cerezo et al., 2014).

Shrestha and Pokharel (2019) propose a system based on student performance that provides alerts to enable self-learning and help tutors take action to improve teaching and retention. In the same vein, Charitopoulos et al. (2017) and Wongwatkit and Prommool (2018) designed the creation of interactive environments using recommendation systems to display optimized materials and instructions to educators.

The work of Perez Sanchez et al. (2022) proposes NeuroK, a learning platform supported by the paradigm of neuro-didactics. This paradigm addresses the optimization of the learning and teaching process from the perspective of brain functioning. The method, after being adapted to capture relevant characteristics corresponding to different contexts, can be implemented in management learning platforms. The method makes it possible to predict student performance, monitor their progress in real-time and predict the risk of dropping out.

With the significant increase in the number of users and the volume of data, current e-learning systems face some technical and pedagogical challenges. Liu and Yu (2022) present an architecture for big data-based e-learning systems to meet the growing demand for e-learning and describe how to use the generated data to support the delivery of more flexible and personalized courses.

The work of Xing et al. (2022) proposes a study of the multifaceted aspects of engagement (i.e. behavioral, social, cognitive and metacognitive group engagement) and their impact on collaborative learning. Results show that behavioral engagement and group cognitive engagement have a significant positive effect on individual and group performance in problem-solving.

Ouyang et al. (2023) propose a model for predicting performance based on several learning analysis approaches. The model aims to improve student learning in a collaborative learning context. Results showed that the integrated approach increases student engagement, improves collaborative learning performance and strengthens student satisfaction with learning.

Varnavsky and Popov (2022) propose the inclusion of chatbots in LMS as an LA tool to interact with students who may have learning problems in the future.

This subdomain deliberates about the evaluation/measurement of the teaching and learning process in the e-learning environment. The outcome of these assessments can reflect the efficiency of online learning, in other words, whether the teaching is efficient or not. With this in mind, strategies and systems for assessing the learning process are proposed:

• Based on test scores and student engagement in proactive learning, an intelligent environment can be developed to assess students’ progress (Paul et al., 2004).

• Through constructive regulation of learning, it is possible to develop a multiagent system to support students’ monitoring (Otsuka et al., 2007).

• The correlation between students’ behavior and their learning progress allows educators to monitor students’ overall and personal learning progress (Chen et al., 2007) and also allows predictions about the student’s performance.

• The development of systems that record all learning activities with their respective status and the student’s behavior in relation to specific material would allow us to measure their assessment (Cao et al., 2009).

• Evaluation of the processes of “Practice Scoring” (from course selection, educators can increase the new exercise or activity and assess students’ tests, students can already do the exercise online and upload the test through the system) and “Interactive Learning”. This assessment is done by formulating a hierarchical evaluation structure that defines the relationship between all assessment elements. Using this structure, a pairwise comparison can be made between the elements. Finally, evaluation rules can be obtained for the analyzed criteria (Wang and Lin, 2012).

• Sales et al. (2012) propose to carry out continuous, formative and mediating assessment practices that are able to help students by contributing to the reduction of educator/tutor work overload. To this purpose, the author designs an evaluation model for the learning and teaching process that takes into account quantitative aspects (performance based on grades) and qualitative aspects (performance based on the positive/negative ratio). The model aims to transform the LMS into a friendlier and more interesting environment, providing an alternative to the limitations and difficulties associated with the distance learning modality.

• Creation of visual tools to facilitate the monitoring of the learning process. Based on the platform’s log files related to the students’ interaction, the activities and the effectiveness of the course content in the learning process could be tracked and evaluated. According to Cerezo et al. (2014), the result could be visualized through graphs, in which the nodes represent the course content and the edges represent the actions and interactions performed by the students.

• Charitopoulos et al. (2017): Run simulated tests to measure which learning materials would be more effective by calculating the duration of actions in each multimedia object available on the platform. Also, evaluate how much specific educator/tutor instructions contribute to the learning process by applying grouping techniques in terms of time spent on the activity, time spent in class and performance of the activity under consideration (Wongwatkit and Prommool, 2018).

• Quantify students’ learning levels based on metrics calculated from data generated via Moodle (user and platform interaction) (Shrestha and Pokharel, 2019; Fábio Tavares Arruda et al., 2019).

The work of Munshi et al. (2022) presents a new hybrid approach based on Elman neural and apriori mining with the purpose of predicting student performance in online education. The incorporation of the Elman neural system aims to eliminate the noisy data present in the databases. Student performance is ranked based on their average score and classified as good, poor or fair.

This literature review allowed us to point out the main approaches and trends from the manager’s perspective to identify the contributions in EDM that can help in the management and improvement of the TL process in the e-learning environment. However, it is observed that the educational process analysis from a managerial perspective is not trivial because the educational process involves many actors, and each one has well-defined characteristics and functions. For example, the coordinators, educators and tutors who work directly with students and have roles of utmost importance for the teaching and learning process.

These activities cannot be confused with the function of the manager because it is the manager who can visualize and analyze the entire teaching and learning process through indicators – KPIs. In order for the manager to be able to analyze and infer about the TL process, the KPIs must be well defined so that they provide management with information that enables strategic planning about the teaching and learning process and allow decision-making and transfer of actions to the actors, to improve and optimize the TL process.

For Gordon et al. (2017), the measure of the success of an organizational strategy is the comparison between the outcome of a specific activity with the goal defined in a strategic plan. When the metric of success is the difference between the expected outcome and the goal, this metric is called a performance indicator. When those metrics are easily observable and adequate to manage are called KPIs. For the authors, a minor quantity of indicators is better to manage the organization. For example, for online courses in higher education, the authors define two main KPIs to reach the following objectives: increase student expectations and increase undergraduate student satisfaction. To define these KPIs, other sub-KPIs are previously determined.

In Badawy et al. (2016), the authors discuss several dimensions to create effective KPIs such as characteristics of KPIs (sparsity: the fewer KPI the better, simplicity: users understand the KPIs, actionability: users know how to affect outcomes, etc.), factors for success and steps for developing them. In Varouchas et al. (2018), the authors present a methodology for the formulation of KPIs based on different quality aspects obtained by qualitative interviews with administrators and professors. The authors establish that the quality and improvement of teaching and learning are sustained by the multi/interdisciplinary aspect of a curse.

E-learning environments have been characterized by generating large volumes of data, allowing the processing and transformation of information into KPIs to measure and evaluate the teaching and learning process in a sustained manner. Caione et al. (2017) establishes that “traditional” way to measure effectiveness in e-learning is not enough because part of the necessary information comes from social networks. For example, the authors propose the following KPIs: ease and quality of service accessibility, use of multimedia services, quality of education, efficiency: of professor and student, technology infrastructure: ease of navigation, availability, etc.

Ashtiani and Valoojerdy (2022) discuss 24 KPIs for an e-learning course. The authors identified several aspects to be considered to create KPIs, such as instructor’s characteristics, learner’s characteristics, learning environment, instructional design, level of collaboration, technology infrastructure, knowledge management, etc.

The main perspectives of EDM and LA in the e-learning environment observed in this review and suggestions of possible KPIs that can be used by management are briefly described as follows:

• Personalization of education, an approach that has received the greatest attention in the bibliographic material considered. It is one of the manager’s main objectives to seek further engagement and a reduction of the dropout rates.

The student’s engagement with the learning platform can be measured using the frequency of access to the learning environments, time spent on activities, number of activities performed, number of clicks and obtained assessment results. Taking the previous measurements into account, it is possible to identify the ideal and minimum engagement levels (KPIs) for good student performance. With this, management can make changes aiming at personalizing learning with more adherent materials (learning objects) and providing more assertive instructions/guidance to educators and tutors to optimize engagement and consequently improve student learning:

• Providing personalized information to the actors (educators and tutors) who are responsible for sharing information with students. This information can be used in the development of intelligent tutoring systems that increase interactivity and recommendation systems for learning objects.

Measurements of the interaction of students with educators and tutors can be used to measure the level of interaction and propose guidelines between these actors. For example, the types of orientations for educators and tutors can be intervention, instruction and suggestion of materials (learning objects) and can be recommended from a KPI indicator. For this indicator, the measures of frequency of access, time on activity, activities performed, number of clicks and assessment results, taking into account when they were performed, can be considered. This way, the administrator can adjust the correct time for each orientation to happen and the ideal material for each type of student, aiming to optimize learning and student engagement on teaching platforms:

• Student performance evaluation used to optimize the personalization of learning with the goal of improving student learning.

Performance (KPI indicator) can be measured from the results of assessments, activities handed in or not and engagement with the platform (linked with the different types of materials) so that personalization can be built in. For this, an indicator can show when a student requires more personalized attention.

Management can use student performance assessment in various stages of the learning process to infer possible problems with educators, students’ adaptation to the platform, courses/subjects and multimedia objects used for learning. The teaching methods that bring the best results for a student profile can be selected, aiming to optimize the quality of the learning:

• Adjustment of the teaching plan, in terms of content to be taken from the students’ previous performance, and the proposal of curriculum structures defined from the course objectives or the type of teaching or interaction expected.

Based on the learning profile in relation to the courses offered, the used type of teaching and materials (learning objects) made available, it is possible to evaluate the teaching plans based on indicators that assess student engagement using KPIs, such as time spent in taking a course or access to materials, clicks quantity, frequency and performance of the students. Management today is concerned not only with graduating its students but also with their placement in the job market. Many professions depend on the expertise acquired by the graduations offered by the institutions. The design of this professional can be defined by analyzing his proficiency (performance) in certain courses/subjects and thus selecting other courses/subjects that should be part of each personalized teaching plan according to the student’s profile and the possible career that the student could follow based on his skills:

• Learning adaptability uses the students’ prior knowledge of the interaction to present topics that are most effective for each student profile.

Based on the learning profiles built from the data generated in relation to involvement, type of materials available/used and subjects studied, the management can propose indicators for adaptable teaching to the student profile that takes into account the optimization of video lessons, materials, platform usability, time of interventions or guidance, so that the student learns in the best possible way and that this learning is pleasant, and that even in teaching in e-learning environment, the student does not feel isolated.

From this review, it was possible to identify several approaches that could bring improvements in Higher Education Institutions. However, the difficulty in implementing these projects remains because it is essential to transform the institutional culture. In this context, the manager must consider that training and alignment with their coordinators, teachers and tutors is extremely important because education is a process of continuous improvement. With regard to the provision of services, changes will always occur in view of the human nature involved. At this point, we find the biggest challenge in the” simple and direct” implementation of the concepts and proposals presented in this review. Each student profile targets different aspects in the educational field, and the manager must be able to identify the strengths and weaknesses of his team and the services offered. Based on this, it will be possible to infer and establish effective processes for continuous learning and student retention, generating revenue and ensuring the sustainability of institutions.

For some institutions, traditional teaching or synchronous e-learning (where the student has a passive role, only listening to the educator and not interacting in class) may be seen as financially expensive, not very engaging and of low performance, unlike teaching modality project-oriented, where the student can perform tasks on digital platforms and hold fertile discussions with educators. By means KPIs, the institutions can optimize their teaching models to understand what is the ideal moment using for each type of material, platform and educational environment and how to optimize face-to-face meetings. This is just one case of how KPIs and engagement actions can be great sources of subsidy for management decision-making regarding the choice of the most effective education model.

Educational management always seeks to optimize its processes, and EDM and LA have provided techniques/tools that may help in this sense. However, there is still a gap between knowledge discovery (related to behavior patterns) and the proposal to improve institutional processes, such as (i) The proposal of methods and procedures for the selection and training of tutors and educators that adhere to the students’ profile. As shown in this review, some works already address more effective forms of content presentation and interventions by educational actors.

The format of offered courses to be made available by the educational institution is another topic much debated by management, of which modalities and courses to be offered are evaluated based on market analysis. (ii) The description of the entire teaching plan must also be adjusted in the portfolio offer, taking into account the student performance and profile and all the logistics of the institutions, which encompasses the infrastructure of the university for the allocation of human (educators and tutors) and physical resources. In this sense, EDM could have significant contributions.

Another process of paramount importance to provide sustainability to educational institutions, especially in private education, is (iii) the retention and recruitment of students, a topic still little debated by EDM. At the managerial level, (iv) there is a concern to know the profile of students by type of course or preferred teaching modality. On the other hand, retention, a topic widely discussed in EDM, (v) there is a concernment about predicting possible evasions, but for institutions, it would be very relevant to have a greater applicability of preventive actions plausible to be measured and controlled to avoid these losses. Note that EDM can contribute to the proposal of intelligent educational management systems.

From the analysis of the bibliographic material, it was possible to identify potential new research topics in EDM from the manager’s perspective, aimed at improving the learning process by monitoring e-learning systems and adapting materials and teaching methods to encourage more engagement from students, and consequently, they may learn with more quality:

• As a result of this literature review, it is possible to conclude that research topics in EDM that have not yet received due attention are within the scope of their applicability in day-to-day management. For example, many models for predicting students’ dropout or engagement rates have been proposed; however, it was not possible to identify suggested clear strategic actions that could be adopted to improve student retention or participation – which could be feasibly achieved by increasing the interpretability of the models. It is also necessary to consider the probability of students dropping out and appraise the possible actions that can be taken to prevent it from happening.

• In terms of student monitoring, most of the work reviewed only uses data derived from interactions with e-learning platforms, internal databases or point-in-time surveys. An unexplored area in this context is cross-referencing external databases, e.g. LinkedIn, Facebook and/or Twitter, to better understand students’ browsing behavior, preferences and profiles. In other words, the students’ engagement on the learning platforms are related to social media.

• Regarding the teaching platforms, it would be interesting to introduce a management point of view, which clearly indicates to each actor (student, educator and tutor) what are the opportunities for improvement in terms of materials, teaching and the best way of learning. In this regard, it is important to identify and perform a simple and summarized visualization of the strengths and weaknesses of the learning process.

• Inclusion of monitoring of hybrid learning environments, where there are classroom and e-learning teaching loads. Carry out cross-reference data and student interactions both in the classroom and on e-learning platforms to create managerial prospects for educators, tutors and managers.

• Faced with so many market changes and the need for multiknowledge professionals, research is needed to optimize the courses study plans, enabling students to act in the market, forming more capable professionals, that is, by using the adaptability of learning models to create courses based on the professional objectives of the market.

• Many articles use EDM to extract the learning profile of each student; however, there are also studies and pedagogical tests that address learning styles. The correlation between these data can provide important information about how to optimize learning and even student engagement on platforms.

Thinking about the implementation of the mentioned strategies, it is essential to consider that the EDM and LA techniques depend on the availability of data. Managers must understand that the transformation begins when there is availability of a comprehensive educational database capable of fully monitoring the student and their evolution within the TL process.

Once the data are available and properly structured, it becomes possible to apply innovative methods and strategies to gain insights into the main problems and bottlenecks within the TL process. The structuring of information would contribute to assertive decision-making by identifying possible problems and necessary changes. By understanding the problems, it is possible to devise strategies to eliminate or reduce them more effectively. A practical example is the dropout approach. Although many institutions carry out studies on the dropout rate and its main reasons, it is based on the available information and with the structuring of an educational big data environment, that it would be possible to monitor the student’s teaching and learning process and understand the actions or the lack thereof, which would contribute to nondropout, involving relevant actors (managers, students, teachers and tutors).

In this work, a systematic literature review was conducted to identify the perspectives, trends and unexplored topics related to EDM from the manager’s perspective.

It was observed that personalization and adaptability of learning are topics that are increasingly being addressed, as they aim to improve student performance and help management identify items that must be modified.

In addition, there is a drive to optimize the content provided so that institutions can better meet the interests of students, with interaction time correlating with their learning. Therefore, it is important to measure and determine how best to optimize learning, whether through materials, multimedia or course restructuring.

Throughout the process, it was possible to observe the extensive use of Big Data in education, generating rich information for institutions to support and be able to make assertive and sustainable decisions, always with the goal of improving educational processes and minimizing operational costs.

Although we are dealing with the manager’s perspective, it was observed that, in general, learning is closely related to students’ retention and institutional health; therefore, the administration must always be mindful about optimizing its teaching. This fact was established through the proposed student retention measures, which are also measures to optimize learning and improve students’ engagement.

Another perception is that on the journey of improving teaching, learning and retention processes, several measures at the level of educators and tutors have been proposed; however, there is no definition and measurement of the actions of all actors in a combined way, aiming at optimization of the process as a whole.

From the review carried out, it was also possible to identify potential new research topics in EDM, which aim to improve the learning process through the monitoring of e-learning systems.

This work contributes to educational institutions to visualize ways to optimize the learning process, engagement and, consequently, retention of their students.

Future research may expand the search keywords to investigate the existence of other research strands and trends of e-learning in the managerial perspective.

Among the limitations of our work, we can highlight the following aspect. After applying the keywords (“Educational Data Mining” and “Learning Analytics”), to search the selected sources, this process returned 1,772 articles. The selection of works had a strong restriction in the selection of contributions that showed, in their results, a clear discussion for the improvement of the teaching learning process in e-learning environments in the managerial perspective, and not only in the application of data mining techniques. In this way, our review had a bias with a more educational vision. If we had applied a more technical view, we could have explored the most used computational techniques to meet the different aspects where EDM and LA are applied. Having this new vision could bring a new contribution to the literature review.