Efficiency and effectiveness of adaptive learning paths: evidence from a large-scale study

Traditional virtual learning environments often fail to adequately address the diverse needs of students due to their narrow focus on specific learning activities (Alshammari, 2020). This limitation highlights the necessity for adaptive learning strategies that can cater to individual needs, considering that learners have varying responses to different learning situations (Rodrigues et al., 2019). However, designing systems that can effectively adapt to students’ unique characteristics presents a significant challenge (Morze et al., 2021, 2023; Truong, 2016). In particular, finding scalable and pedagogically sound approaches for personalising learning experiences – especially in higher and secondary education – remains an ongoing difficulty (Cavanagh et al., 2020; Morze et al., 2023; Truong, 2016). One potential solution to address this challenge is the incorporation of an adaptive learning system (ALS) that considers personal traits. By tailoring tasks and activities based on individual characteristics, learners can experience a more personalised and engaging learning journey (Cavanagh et al., 2020; Morze et al., 2021). Personalised learning paths can not only lead to improved learning outcomes and greater efficiency in acquiring knowledge and skills but also save valuable time. Personalised e-learning requires a deep, data-driven understanding of learners’ prior knowledge and behaviour, which can be achieved through data mining and algorithmic decision-making (Truong, 2016). Despite its promise, existing adaptive learning research has key limitations: many studies use small samples or focus on single courses, rarely examine both efficiency and effectiveness together and provide little evidence on how adaptive and non-adaptive learning paths differ across multiple subjects.

To address these gaps, we designed and evaluated a structurally ALS in Moodle based on pre-test performance and activity locking. In this design, learners who demonstrated prior mastery could skip corresponding content, whereas learners in the regular condition completed the full instructional sequence. Rather than testing a highly sophisticated artificial intelligence (AI)-driven adaptive system, this study focuses on a form of structural adaptivity that is pedagogically transparent, technically feasible in real school settings and scalable across multiple courses.

The study makes three contributions. Firstly, it provides large-scale evidence from an authentic educational context, involving 2,064 upper-secondary students across seven European Computer Driving Licence (ECDL)/International Certification of Digital Literacy (ICDL) modules. Secondly, it examines adaptive learning through a dual lens of efficiency and effectiveness, distinguishing between time-related gains and learning outcomes instead of assuming that one necessarily implies the other. Thirdly, it complements group comparisons with structural and profile-based analyses to explore whether adaptive and regular learning paths are associated with different latent performance patterns and learner profiles.

Guided by these aims, we address the following research questions:

Through these analyses, the study seeks to clarify whether pre-test-based structural adaptivity can reduce time-to-completion at scale while maintaining broadly comparable outcomes, and to identify the conditions under which this approach appears more or less suitable across course types.

ALSs have gained significant attention for their ability to personalise the learning experience for individual students (Morze et al., 2021; Jagadeesan and Subbiah, 2020; Kem, 2022). These systems enable customised sequencing, real-time feedback and content personalisation, allowing learners to follow paths aligned with their preferences, abilities or prior knowledge (Graf and Kinshuk, 2007; Aleven, 2015). Widely adopted in modern e-learning contexts, adaptive learning is closely tied to concepts such as personalised learning, self-paced mastery and student-centred instructional design (Cavanagh et al., 2020; Dziuban et al., 2018; Tariq, 2024; Zlatkovic et al., 2020).

Despite the popularity of adaptive learning, empirical research remains fragmented. Studies often focus on narrow instructional contexts, small sample sizes or single-course pilots, making it difficult to generalise findings to large-scale deployments. This concern is also reflected in recent review literature. For example, du Plooy et al. (2024), in a scoping review of personalised adaptive learning in higher education, identify personalisation, learner support and academic performance as recurrent themes, but also note substantial variation in system design, implementation logic and outcome measures across studies. Their review suggests that, while adaptive learning shows considerable promise, the field still requires clearer evidence on how different forms of adaptivity operate across authentic educational contexts and at larger scales. This reinforces the need for studies that examine not only whether adaptive learning works, but also how relatively simple and scalable forms of structural adaptivity perform across multiple subjects in real instructional settings. Hubalovsky et al. (2019) examined how adaptive elements could be incorporated into e-learning materials in primary education. They compared the benefits of the adaptive e-learning Moodle system with traditional methods. The research findings confirmed the successful implementation of adaptive features in primary education, resulting in more effective achievement of educational goals. The study also revealed that adaptive algorithms enable quicker completion of lower-level exercises in Bloom’s taxonomy, supporting personalised learning approaches in primary school settings. Mirari (2022) conducted a systematic review of 25 studies on personalised and adaptive learning and found strong evidence that ALSs generally enhance students’ academic performance, engagement and motivation. In particular, studies using adaptive technologies – such as adaptive courseware, ALEKS and cognitive tutors – reported significant positive effects on student achievement, especially in STEM-related subjects and higher education settings, where learners demonstrate greater self-regulation and benefit more from individualised pathways. The review also highlighted that adaptive learning technologies increase student engagement and motivation by providing immediate feedback and tailoring learning experiences to individual needs. Recent research further reinforces the value of ALSs in improving learner outcomes. For example, Isaeva et al. (2025) found that adaptive learning platforms – such as Coursera, Khan Academy and edX – significantly enhanced accessibility, flexibility and personalisation in diverse educational contexts. Their mixed-methods analysis showed that the use of adaptive technologies, including AI-driven analytics and real-time feedback mechanisms, increased student engagement and supported more effective learning by tailoring content to learners’ needs. The study also demonstrated that adaptive systems help identify knowledge gaps and provide targeted recommendations, leading to improved learning performance.

While extensive research explores tailoring e-learning to individual student needs, characteristics, experiences and behaviours (Ristić et al., 2023; Arsovic and Stefanovic, 2020; Dziuban et al., 2018; Zlatkovic et al., 2020; Graf and Kinshuk, 2007), some studies have emphasised the need for large-scale validation of ALSs (Hubalovsky et al., 2019; Sutarman et al., 2024; Walkington and Bernacki, 2020; Zliobaite et al., 2012). For example, Walkington and Bernacki (2020) argue that establishing a solid evidence base for personalised and adaptive learning remains challenging due to heterogeneous system designs and limited large-scale evaluations. They highlight that existing evaluations, such as those involving Khan Academy, Cognitive Tutor, ASSISTments, Teach to One and Reasoning Mind, produce mixed findings, underscoring the need for broader, rigorous and longitudinal research to determine under which conditions adaptive systems are most effective. A key question remains: can these adaptive systems function effectively when implemented at scale? The study presented in this paper addresses this gap by investigating the impact of an adaptive e-learning system on a large user base – over 2,000 students across multiple information technology (IT) courses. This allows us to assess the scalability and applicability of these systems in real-world educational settings across multiple IT courses, providing robust validation of their effectiveness.

On the other hand, while adaptive learning holds promise for promoting access and quality in education, its implementation in real-world teaching and learning is still sporadic (Cavanagh et al., 2020), particularly in educational institutions where the best design and teaching practices for adaptive learning courses remain unclear. So, this study examined the effectiveness and efficiency of ALSs in high school students from technical institutes.

Effectiveness serves as a quantitative measure that describes the outcomes and achievements of students when completing tasks (Hubalovsky et al., 2019). ALSs can enhance students’ learning and outcomes by tailoring relevant and meaningful materials and activities to each learner’s unique characteristics (Benkhalfallah et al., 2024; El-Sabagh, 2021). Instead of being a passive listener, ALSs help students to be more active and actively engage in the learning experience (Cavanagh et al., 2020; Kerns, 2019).

In terms of effectiveness, numerous studies have demonstrated that adaptive Moodle learning management systems (LMSs) have a significant impact on students’ learning outcomes, performance and overall success (Chen et al., 2022; Dulkaman and Ali, 2016; Pérez-Pérez et al., 2020; Oguguo et al., 2021). In their study, Oguguo et al. (2021) compared Moodle with the CAI4ME package, a computer-assisted instructional tool developed for teaching Educational Measurement and Evaluation. The authors reported that students who learned through Moodle achieved stronger performance than those who used the CAI4ME package, highlighting the relative advantage of the LMS-based approach. Similarly, Aldila et al. (2024) conducted a mixed-methods study examining the effectiveness of an ALS integrated within an LMS in higher education. Working with undergraduate students across two semesters, the authors compared a traditional LMS with an ALS-enhanced LMS and found that the adaptive system significantly improved student learning outcomes. Students in the adaptive condition achieved higher post-test scores, showed greater engagement (e.g. more frequent logins, longer time spent on learning materials and more interactions), and reported higher satisfaction, particularly valuing the personalised learning paths and immediate feedback. Isaeva et al. (2025) investigated the role of adaptive learning platforms, emerging computer technologies and virtual laboratory systems in enhancing learning effectiveness. Using a mixed-methods analysis of platform features, engagement data and implementation case studies, the authors found that adaptive technologies substantially improved accessibility, personalisation and student engagement. These findings align with broader research highlighting the effectiveness of ALSs in enhancing student outcomes (Almusfar, 2025; Isaeva et al., 2025; Zlatkovic et al., 2020; Morze et al., 2021). However, most studies look at effectiveness on its own, without considering how it relates to the amount of content students study or the time they spend. Because of this, we still do not know whether adaptive systems can offer the same or better results while reducing the learning load. In our study, we measure effectiveness through students’ final ECDL exam outcomes and their scores in seven courses. By comparing adaptive and regular learning paths in a large real school environment, we examine whether adaptive sequencing can preserve broadly comparable learning outcomes while reducing instructional load, and whether this pattern varies across course types.

Evaluating the “efficiency” of the adaptive LMS is another crucial factor to consider. To optimise training in e-learning systems, it is crucial to offer personalised courses and deliver content efficiently, taking into account students’ abilities, knowledge and preferences (Morze et al., 2023; Truong, 2016; Ibrahim and Hamada, 2016).

In the current study, the evaluation of efficiency was carried out through the analysis of several key indicators, including “time saved”, “time attended” and “topics skipped”. By initially assessing students’ prior knowledge and tailoring the learning experience to their individual needs, the study sought to optimise the efficiency and effectiveness of the adaptive LMS. This personalised approach allowed students to progress through the material at their own pace, skipping topics they have already mastered and focusing on areas where they need the most support.

Research across multiple studies consistently shows that ALSs substantially improve learning efficiency. For example, the study by Mettler, Massey, and Kellman (2011) demonstrated that response-time-based adaptive learning significantly improves training efficiency. In a controlled experiment comparing the adaptive response time-based sequencing (ARTS) adaptive system to Atkinson’s classic adaptive scheduling model, ARTS achieved 53.4% higher trial-based efficiency and up to 79% greater time-based efficiency on immediate tests, with similar advantages maintained on delayed tests (Atkinson, 2014). In a real school setting, third-grade students mastered all 12 × 12 multiplication facts in only about 2 h, showing large gains in both accuracy and speed, and outperforming peers receiving standard instruction. These results provide strong evidence that ALSs, especially those incorporating response times, substantially enhance learning efficiency by enabling faster mastery with fewer practice trials and less instructional time. Together, these results highlight the potential of response-time-driven adaptivity to accelerate mastery while reducing practice time. Building on this evidence, Mettler et al. (2020) further confirmed the efficiency benefits of adaptive scheduling through direct comparisons between adaptive and non-adaptive learning schedules. When spacing between learning items was determined adaptively using response times, efficiency more than doubled relative to random spacing in Experiment 1, with adaptive learners showing 102% higher efficiency on the immediate post-test and 50% higher efficiency on the delayed test. Learners in the adaptive condition also required less than half the number of practice trials (197 vs 429) to reach mastery. Even when random schedules included identical mastery and dropout criteria (Experiment 2), adaptive scheduling still produced 31%–33% higher efficiency. These results show that ALSs significantly enhance efficiency by enabling learners to reach mastery faster and with fewer trials than non-adaptive methods. Extending these advantages to corporate and professional training contexts, Pagano and Marengo (2021), in another study, demonstrated that an adaptive learning strategy can dramatically improve training efficiency by personalising learning paths based on prior knowledge. In their simulation of an ECDL module, the adaptive system allowed learners to skip content they had already mastered, resulting in a 54.16% reduction in training time while still preparing learners for the final assessment. They projected that this approach could reduce the total ECDL training time from 28 h to approximately 13 h, without compromising learning effectiveness. Collectively, these studies provide converging evidence that ALSs, especially those leveraging response times and mastery-driven personalisation, dramatically improve efficiency by reducing redundant practice, accelerating mastery and minimising instructional time without sacrificing learning outcomes. Hubalovsky et al. (2019) also demonstrated that a student who takes a longer time to complete an exercise with a 100% score may not use their learning potential as effectively as a student who completes the exercise in a shorter time period. In fact, poor time management can lead to negative effects, including stress, anxiety and ultimately poor performance. Although previous work has identified time-related variables (such as total study time or regularity of learning intervals) as important predictors of achievement, few studies have directly compared the total training time required in adaptive versus non-adaptive courses, especially across multiple subjects in a large population. Moreover, efficiency indicators are often reported separately from effectiveness measures, making it difficult to understand the trade-offs or synergies between them.

Our study addresses this limitation by quantifying efficiency in terms of time saved, time attended and topics skipped for both adaptive and regular conditions, while also examining effectiveness through pass rates and post-test scores. In addition, for structural analyses only, we derive a composite performance index that combines achievement and time efficiency as a synthetic indicator. This index is intended to support latent-variable and profile analyses, not to replace the separate interpretation of effectiveness and efficiency outcomes.

LMSs play a central role in supporting personalised digital learning environments. Among open-source LMSs, Moodle is widely recognised for its pedagogical grounding in social constructivism and its flexible, modular design (Al-Fraihat et al., 2025; Xiao, 2020). Moodle was selected due to its support for adaptive navigation techniques, such as direct guidance and link adaptation, that help learners follow appropriate pathways through course materials. Its comprehensive suite of tools enables educators to build interactive course spaces (Kumar and Dutta, 2011) and has consistently been rated as one of the most robust LMS platforms for enhancing educational quality (Aydin and Tirkes, 2010). Several researchers have demonstrated that Moodle possesses strong capabilities for supporting adaptive learning (Ezzaim et al., 2024; Morze and Terletska, 2025; Nakić et al., 2013; Semerikov et al., 2025). For example, Morze et al. (2021) showed that Moodle offers substantial potential for adaptivity. Through a survey of 119 university students and a detailed analysis of Moodle tools, the authors showed that adaptive content, adaptive sequencing and adaptive assessment can all be implemented using built-in features such as lessons, quizzes, feedback and surveys. Their results indicated that students strongly desire personalised learning pathways, and that Moodle can effectively support such personalisation when instructors design adaptive sequences and assessments. In another study, Kusnendar et al. (2025) found that implementing differentiated learning through a personalised Moodle setup effectively supported diverse student needs. The study showed that students had distinct learning preferences (text, video or audio), and Moodle’s configuration allowed each student to choose materials that fit their needs. Process differentiation, such as repeated formative quizzes with automatic feedback and remedial pathways, helped all students achieve mastery. Students also responded positively, highlighting that Moodle improved their readiness, allowed flexible learning and supported learning at their own pace. Liyanage et al. (2014) proposed an adaptive LMS framework grounded in the Felder–Silverman learning styles model, illustrating how course content can be dynamically tailored to individual learner characteristics.

A key feature of enabling adaptivity in Moodle is activity locking, which allows instructors to conditionally reveal or hide learning objects based on learners’ performance or interaction patterns (Marengo et al., 2012; Quispe-Pari et al., 2023). This feature lets teachers require students to achieve specific criteria, such as completing an activity or obtaining a minimum grade, before accessing the next resource. By guiding learners through a structured sequence and personalising access according to their progress, activity locking supports adaptive pathways that respond to individual needs and learning readiness (Quispe-Pari et al., 2023; Winterhagen et al., 2022). As highlighted by Morze et al. (2021), such functionality is essential for creating personalised learning experiences in higher education. In this study, activity locking was integrated into a custom adaptive plugin that selectively concealed content for which learners had already demonstrated proficiency, thereby streamlining the learning experience and reducing unnecessary instructional load.

Beyond structural adaptivity, the system incorporated learning analytics to further personalise learning pathways. Learning analytics – defined as the collection, analysis and interpretation of learner data to enhance learning processes (Pardo, 2014; Long and Siemens, 2011), enables adaptive systems to respond to learners’ evolving needs. Prior research demonstrates that analytics-driven adaptivity can support appropriate resource allocation, guide learner actions and improve achievement (Kuzilek et al., 2017; Viberg et al., 2018). The integration of a dashboard-based analytics component in this study provided real-time feedback and adaptive support (Chen et al., 2019; Han et al., 2021), allowing the Moodle environment to be dynamically tailored to individual learners’ progress.

Students enter learning situations with diverse backgrounds, including prior knowledge, interests, goals, emotional states and preferred learning strategies, all of which influence how effectively they engage with instructional activities (Aleven et al., 2017). In this sense, adaptive instruction responds to a central problem in educational technology: learners do not benefit equally from the same fixed sequence of materials. A core premise of adaptive systems is therefore that instruction should be aligned, as far as possible, with the learner’s current knowledge state and learning needs. This assumption is also compatible with Vygotsky’s theory of the zone of proximal development, which conceptualises learning as most effective when support is calibrated to what the learner is ready to accomplish with guidance rather than through fully independent performance alone (Vygotsky Lev Semënovič, 1978). In contemporary digital environments, this supportive function can be partially operationalised through adaptive technologies that personalise access to content and progression through learning activities (Gohar and El-Ghool, 2016; Liu et al., 2017).

Within this broad perspective, the present study is grounded in three complementary frameworks: adaptive hypermedia systems (AHS), mastery learning and learning analytics. AHS provide the basic logic for personalised sequencing by combining user modelling, content structuring and rule-based adaptation to regulate navigation through instructional materials (Setiawan et al., 2019; Souhaib et al., 2010). In an LMS environment, this means that learners do not necessarily follow the same linear route because access to specific learning objects can be shaped by prior performance or demonstrated knowledge. In the present study, this logic is implemented through Moodle activity locking, which conditionally reveals or hides instructional content according to pre-test performance. This form of adaptive release is consistent with prior work showing that Moodle can support adaptive pathways through conditional access mechanisms and progress-based sequencing (Morze et al., 2021; Quispe-Pari et al., 2023; Winterhagen et al., 2022).

Mastery learning provides the pedagogical rationale for this sequencing logic. From a mastery perspective, progression should depend on demonstrated competence rather than on mandatory exposure to all instructional materials. Learners who have already mastered prerequisite content should not be required to repeat equivalent instruction, whereas learners with identified gaps should receive targeted opportunities to address those gaps before progressing. This logic underpins the use of conditional release and activity locking in Moodle, where advancement depends on meeting defined performance thresholds (Padayachee et al., 2018; Castelló et al., 2010). In this sense, the adaptive path examined in this study is not intended to represent a highly sophisticated AI-driven adaptive system, but rather a scalable mastery-oriented instructional design that seeks to reduce redundant instruction while preserving access to the content necessary for successful course completion. Learning analytics complements these two perspectives by providing the informational basis for adaptation. In the literature, learning analytics is explicitly described as a conceptual framework for adaptive e-learning systems because it uses behavioural and performance data to monitor learner activity, guide instructional decisions and support timely intervention (Janati et al., 2019; Ngulube and Ncube, 2025). More broadly, learning analytics has been defined as the collection, analysis and interpretation of learner data for the purpose of understanding and improving learning processes (Pardo, 2014; Long and Siemens, 2011). Prior research shows that analytics-driven adaptation can support more appropriate resource allocation, improve guidance and personalise content delivery in LMS environments (Fenu et al., 2017; Kuzilek et al., 2017; Liu et al., 2019; Viberg et al., 2018). In the present study, pre-test results served as the main analytic input used to trigger structural adaptation, while the Moodle environment also provided feedback and monitoring functions consistent with analytics-supported adaptive learning (Chen et al., 2019; Han et al., 2021).

Taken together, these three frameworks explain the rationale of the present study. Adaptive hypermedia explains why differentiated navigation paths are possible, mastery learning explains why such differentiation is pedagogically meaningful, and learning analytics explains how learner data can be used to activate these decisions within a scalable LMS-based environment. On this basis, the independent variable in the study is the instructional condition (adaptive vs regular path), while the main dependent dimensions are learning efficiency and learning effectiveness. Efficiency is reflected in time saved, time attended and topics skipped; effectiveness is reflected in first-attempt pass rates and post-test scores. The composite performance index is used only as a synthetic indicator for structural analyses, not as a replacement for the separate interpretation of efficiency and effectiveness outcomes.

This study used a randomised comparative design to examine the effects of two instructional conditions on large-scale digital skills training for the ECDL. The independent variable was instructional condition, with two levels: an adaptive e-learning path (experimental condition) and a regular full-content e-learning path (control condition). The main dependent dimensions were learning efficiency and learning effectiveness. Learning efficiency was operationalised through time saved, time attended and topics skipped; learning effectiveness was operationalised through first-attempt pass rates and post-test scores. In addition, a composite performance index was used only for structural analyses reported later in the paper.

The study involved 2,064 students enrolled in technical high schools in a medium-sized city in Italy. This number corresponds to the full accessible population of students enrolled in the ECDL school programme during the reference academic year. For this reason, no a priori statistical sample-size estimation was conducted: rather than drawing a subsample, the study included the entire available cohort and then assigned students to conditions. Students were allocated to either the adaptive or the regular condition through controlled randomisation stratified by school year and gender to ensure equivalent group composition on these two variables. Each group therefore included 1,032 students, with identical distributions by school year and gender (Table 1): 608 fourth-year students and 424 fifth-year students in each condition, and 621 males and 411 females in each condition.

Participation was voluntary, and the study was conducted within the ordinary delivery of the ECDL school programme. Ethical procedures included institutional authorisation, protection of participant confidentiality and the use of aggregated data for research purposes

Learning efficiency was assessed through three indicators: time saved, time attended and topics skipped. In the regular condition, learners were required to complete the entire instructional sequence for each course. Across the seven ECDL modules, this full sequence corresponded to 107 learning objectives and a total statutory workload of 1,680 min (28 h), with each module assigned 240 min. In the adaptive condition, learners completed a pre-test at the beginning of each course and only the learning objects associated with unmet competencies remained available. Consequently, the adaptive path could reduce both the number of topics studied and the instructional time required.

Time saved was defined as the difference between the statutory time allocated to a course and the time actually required by the learner’s personalised path. Time attended refers to the amount of instructional time actually spent within the assigned path. Topics skipped refers to the number of course topics that were not assigned to a learner because pre-test evidence indicated prior mastery of the corresponding content.

Learning effectiveness was assessed through first-attempt success rates and post-test scores in each of the seven ECDL modules. Each course concluded with a final test, analytically treated as the post-test, and each module was considered passed when the learner reached the certification threshold of 75% correct answers. To obtain the full ECDL certification, students were required to pass all seven module tests. Because both instructional conditions allowed retakes through different mechanisms, all between-group analyses reported in this study were based exclusively on each student’s first completed post-test attempt, to preserve comparability across conditions and avoid distortions arising from heterogeneous repetition patterns.

For the structural analyses reported later in the paper, we computed a composite performance index intended to summarise, at the course level, both learning effectiveness and learning efficiency. Conceptually, course performance was treated as a joint function of achievement in the final assessment and relative efficiency in the use of instructional time. The composite index was defined as the scalar measure of the two-dimensional vector$[Si,Ri]$⁠, calculated through Euclidean distance:

where $Pi$ is the performance index for learner$i$⁠, $Si$ is the standardised assessment score and $Ri$ is the relative efficiency component.

To place the achievement component on a comparable scale, the raw assessment score was standardised according to its range of variation:

where $Scorei$ is the score obtained by learner$i$ and $Min(Score)$and $Max(Score)$are the minimum and maximum values of the score scale used in the assessment.

Relative efficiency was defined as the proportion of time saved with respect to the statutory duration of the module. In the present application, each ECDL module had a statutory duration of 240 min, and the efficiency component was calculated as:

where $Timeused$ is the time actually required by the learner within the course pathway. The coefficient 0.5 was introduced to reduce the weight of time saving relative to achievement, so that the efficiency component contributed less strongly than the assessment result to the final composite measure.

Under this formulation, the performance index is equal to the standardised score when no time is saved and increases when equivalent achievement is reached with less instructional time. In this sense, the index reflects the study’s interest in time-to-performance rather than in achievement alone.

The composite performance index was used only as a synthetic indicator for confirmatory factor analysis (CFA), multi-group modelling and cluster analysis. It was not intended to replace the separate interpretation of post-test scores and time-related indicators. Because the efficiency component is structurally linked to the adaptive design, the index should be interpreted cautiously and primarily as an exploratory summary variable rather than as a standalone proof of instructional superiority. In addition, consistent with the original analytical procedure, the index was computed only for learners who exceeded the assessment threshold, so as to ensure comparable performance profiles across all seven courses. This choice improves comparability for structural analyses but also limits generalisability to successful completers.

The study used one assessment structure for each of the seven ECDL modules: Computer Essentials (34 items), Online Essentials (34 items), Word Processing (28 items), Presentation (28 items), Spreadsheets (32 items), IT Security (32 items) and Online Collaboration (26 items). All items were multiple-choice questions with five response alternatives. The assessments were aligned with the official ECDL course structure and learning objectives reported in Table 2.

For both the adaptive and regular conditions, pre-tests and post-tests drew on the same item pools, while the presentation order of questions and alternatives was randomised to reduce simple memorisation and answer-position effects. This design choice ensured strong alignment between entry assessment and final assessment across conditions, but it may also have introduced practice effects. We therefore interpret effectiveness findings with appropriate caution.

Because the assessments were embedded in an operational certification-oriented training context and derived from course-aligned item pools, the present study focused primarily on comparative instructional effects rather than on the development of new psychometric instruments. Accordingly, no additional reliability analysis was incorporated into the original experimental design; this should be considered when interpreting the assessment evidence and is acknowledged as a limitation.

To prepare a learning environment suitable for comparison, one LMS administrator and four subject-expert teachers configured parallel adaptive and regular versions of the seven ECDL courses.

In the adaptive condition, each learner began a course with a pre-test designed to assess entry-level competencies. Based on pre-test performance, the Moodle activity-locking system generated a personalised pathway by making available only those learning objects associated with identified knowledge gaps, while concealing content already considered mastered. At the end of the personalised path, learners completed a post-test. If the learner did not reach the passing threshold, the failed post-test functioned operationally as a new diagnostic input, and the system generated a revised personalised path focused on the still-unmastered competencies. This adaptive cycle could be repeated until the learner passes the final assessment (Figure 1).

In the regular condition, learners completed the full non-adaptive instructional sequence for each course before taking the standard post-test. If they did not pass the assessment, they were required to repeat the full instructional sequence before reattempting the post-test.

Although both groups could access retakes, the two instructional conditions differed in the way remediation was organised: the adaptive condition provided targeted repetition of unmastered content, whereas the regular condition required repetition of the entire course sequence. To preserve comparability across groups, the analyses reported in this paper were restricted to the first completed post-test attempt for each learner in each course.

To identify distinct profiles of student performance across the seven ECDL courses, we conducted an exploratory cluster analysis using the subset of learners who successfully passed all seven final assessments. This restriction was adopted because the analysis required a complete and comparable set of course-level performance indicators across all modules for each learner. Accordingly, the clustering results should be interpreted as describing variation among successful completers rather than the full randomised sample.

The seven course-specific performance indices, defined as described above, were first modelled as indicators of a single latent performance factor using CFA. This analysis supported the use of a common latent performance dimension at a general level, while also indicating that the relative contribution of individual courses was not perfectly identical across the adaptive and regular conditions. Building on this latent-variable framework, the same seven performance indices were then used as input variables for the cluster analysis.

We selected the TwoStep procedure because it is suitable for large data sets and supports data-driven identification of cluster solutions. The log-likelihood distance measure and the Bayesian information criterion were used to evaluate candidate solutions. We inspected solutions ranging from two to five clusters and retained the three-cluster solution because it offered the best balance between statistical adequacy and substantive interpretability.

The resulting clusters were interpreted as performance profiles based on the combined distribution of achievement and relative efficiency across the seven ECDL modules. Because the composite performance index incorporates a time-saving component that is structurally associated with the adaptive condition, these profiles should be interpreted cautiously and in conjunction with the separate analyses of learning efficiency and learning effectiveness reported in Section 5.

The ECDL is a certification program specifically designed to assess and validate individuals’ computer skills and knowledge. It consists of several modules or courses that cover various subjects related to computer literacy. The courses included in the ECDL program are Computer Essentials, Online Essentials, Word Processing, Spreadsheets, Presentation, IT Security and Online Collaboration. Each course focuses on specific areas of computer skills and knowledge that are relevant to individuals seeking the ECDL certification. In our study, students completed the official ECDL certification modules, administered in accordance with the standards and procedures of the certification authority. Each module included a formal exam with a duration of 240 min. The total instructional and assessment time across all seven modules was 1680 min (28 h). (Table 2)

By successfully completing these courses and meeting the requirements set by the ECDL program, students can obtain the official ECDL certification. This certification serves as evidence of their competence in the areas of computer literacy covered by the program.

The study used a unique questionnaire for each of the seven ECDL course: Computer Essentials (17 topics, 34 questions), Online Essentials (17 topics, 34 questions), Word Processing (14 topics, 28 questions), Presentation (14 topics, 28 questions), Spreadsheets (16 topics, 32 questions), IT Security (16 topics, 32 questions) and Online Collaboration (13 topics, 26 questions). The pre-test and post-test for both the regular and adaptive groups consisted of multiple-choice questions, with five possible response alternatives. The order of both the questions and the response options was randomised to reduce straightforward recall and answer-position effects. The use of the same item pools across pre- and post-assessment ensured close alignment between diagnostic and summative measurement across conditions, thereby supporting comparability of content coverage. At the same time, this design may have introduced practice effects, and this limitation should be considered when interpreting effectiveness results.

The courses with the highest number of topics skipped were Computer Essentials and IT Security, with learners skipping an average of 9.41 and 8.26 out of 16–17 topics, respectively. This resulted in substantial time savings for all learners (Table 3; Figure 2, Figure 3). However, the Spreadsheet course had the highest number of topics attended (9.38) of all the courses, with an average saving of 1.91 h per learner (Figure 2). As shown in Table 4, the topics skipped and time savings are considerable and similar for each course analysed. Out of the total 28,896 h needed for the learning path, adaptive learning paths successfully optimised the process, resulting in a reduced time of 15,096 h (52.24%); therefore, the total savings were 13,800 (47.76%). Figure 4 shows the time saved and the time spent on each course. Figure 3 and Table 4 show the total time saved compared to the time spent attending to the learning objects.

“Topics skipped” and “Topics attended” represent the average number of topics skipped or completed by learners in each course.

The study’s data revealed that by personalising the learning path through adaptive learning activity locking, the adaptive group achieved a high optimisation of the time needed to complete the learning path. The learners in the adaptive group experienced time savings across all courses, with an average of approximately 2 h saved per learner. The implementation of adaptive learning paths through adaptive learning activity locking resulted in a significant reduction of the total required time.

Figure 5 illustrates the relationship between test scores (0–30) and the percentage of the maximum training time used by learners in the adaptive group. The x-axis shows the assessment test scores (out of 30 points), while the y-axis represents the ratio of time spent to the maximum allotted time. This pattern indicates that the adaptive approach itself is highly efficient, as many learners achieved high scores while using considerably less than the maximum allotted time. The variation in time use among learners with similar scores shows that the adaptive system allows students with stronger prior knowledge or a faster learning pace to progress more quickly. Demonstrating that learners can reach the same level of competency in significantly less time provides strong evidence that the adaptive method supports more efficient learning – an important consideration when evaluating educational systems.

The results of the adaptive post-test show a strong first-attempt pass rate of 72.36%. As shown in Table 5 and Figure 6, this is slightly lower than the 73.88% first-attempt pass rate observed in the regular group. Performance varied across courses: in some areas (e.g. Online Essential and Online Collaboration), the adaptive and regular groups performed similarly, whereas in others (e.g. Word Processing and Spreadsheet), the regular group achieved significantly higher pass rates. These differences are reflected in the p-values reported in Table 5. The overall pattern suggests that while the adaptive approach supports high levels of achievement for many learners, its effectiveness may differ across content areas, warranting further investigation.

To evaluate the learning effectiveness of the adaptive approach, we compared the mean post-test scores of the adaptive and regular groups across seven courses (Table 6).

Overall, the results show a mixed pattern rather than a uniform advantage for either instructional condition. The Z-test allows us to reject the null hypothesis of equality between the two groups for three courses: Computer Essentials, Word Processing and Spreadsheet. In Computer Essentials, the adaptive group achieved significantly higher scores than the regular group. In contrast, in Word Processing and Spreadsheet, the regular group outperformed the adaptive group, with statistically significant differences. For the remaining courses, no statistically significant differences emerged between conditions.

These findings indicate that the adaptive pathway did not produce a consistent effectiveness advantage across all modules. Instead, the two instructional approaches yielded broadly comparable results in several courses, alongside course-specific divergences in which one or the other condition performed better. This pattern is consistent with the first-attempt pass-rate analysis reported above, where overall pass rates were also similar but not uniformly higher in the adaptive condition. Accordingly, the evidence supports a strong efficiency advantage for the adaptive path, but only mixed evidence with regard to learning effectiveness. The same pattern can be further supported by the non-parametric analyses, which corroborate the course-specific differences already identified by the Z-tests.

The TwoStep cluster analysis suggested a three-cluster solution that was both statistically adequate and conceptually interpretable. Two of the clusters were composed predominantly of learners from a single instructional condition. Cluster 1 (n = 819) consisted almost exclusively of learners from the adaptive condition, whereas Cluster 2 (n = 803) was composed predominantly of learners from the regular condition. Both clusters were characterised by generally high composite performance indices across the seven courses, indicating favourable combinations of assessment results and relative time efficiency within the subset of successful completers (Figure 7).

A third cluster (n = 422), included a balanced mix of adaptive and regular learners and showed consistently lower performance indices across the seven modules. Learners in this cluster tended to obtain borderline or barely sufficient results while using a comparatively large proportion of the available instructional time. More than half of the learners in this group obtained insufficient or barely sufficient scores in at least five of the seven ECDL courses, suggesting the presence of a sub-group of systematically struggling learners (Figure 7).

These findings should be interpreted with caution because the clustering analysis was conducted only on successful completers and because the performance index incorporates a time-saving component that is structurally related to the adaptive design. Even so, the cluster solution remains informative in showing that structural adaptivity does not benefit all learners equally. In particular, the mixed low-performance cluster suggests that a sub-group of students in both conditions may require additional pedagogical, motivational or metacognitive support beyond adaptive sequencing alone.

The cluster analysis provided an initial differentiation of performance profiles among learners in the adaptive and regular conditions. To investigate whether these observed profiles reflected deeper structural differences in how students performed across the seven ECDL courses, we conducted a CFA followed by a multi-group structural equation model (SEM). Addressing RQ2 required examining the underlying performance structure, which is inherently multivariate. Preliminary diagnostics indicated that none of the score distributions approached normality. Because traditional multivariate approaches (e.g. analysis of variance, analysis of covariance and Hotelling’s T²) rely on multivariate normality and would not yield reliable results under these conditions, we used SEM using Browne’s (1984). Asymptotically distribution-free estimator, which is appropriate for large samples with non-normal data.

To examine whether the seven course-specific performance indices could be interpreted as indicators of a broader latent construct, we specified a CFA with a single latent factor labelled performance. The model showed good global fit according to commonly used fit criteria (standardized root mean square = 0.016; Tucker-Lewis index = 0.95; root mean square error of approximation = 0.055; adjusted goodness-of-fit index = 0.977; normed fit index = 0.956), indicating that the seven observed indices can reasonably be represented as manifestations of a common underlying performance dimension.

All seven courses loaded significantly on the latent performance factor. The strongest loadings were observed for IT Security, Word Processing, Spreadsheet and Presentation, suggesting that these modules contributed most strongly to the common performance dimension. Computer Essentials and Online Collaboration showed comparatively weaker, though still meaningful, associations with the latent factor. Overall, the CFA supports the use of a common performance construct, while also indicating that the contribution of individual courses to that construct is not uniform (Figure 8).

To examine whether the latent structure of performance differed between instructional conditions, we conducted a multi-group SEM, comparing the adaptive and regular groups. The unconstrained model indicated that a broadly comparable one-factor structure could be specified in both groups, suggesting that the same general latent performance construct was meaningful across conditions.

However, when equality constraints were imposed on the measurement weights and residuals, the chi-square difference tests indicated that these constraints should be rejected (p < 0.001). This means that while both groups can be described in terms of the same general latent performance dimension, the relative contribution of individual courses to that dimension is not fully identical across the adaptive and regular conditions. In other words, the overall structure is comparable at a general level, but not fully invariant in a strict measurement sense.

Inspection of the unconstrained model shows that IT Security displayed very similar factor loadings across groups, whereas courses such as Computer Essentials, Online Essentials and Online Collaboration differed more markedly between the adaptive and regular conditions. Word Processing, Presentation and Spreadsheet occupied an intermediate position, with partially similar but not identical loadings. These differences also affected the associated R² values and the pattern of residual correlations across groups. For example, some residual correlations that were meaningful in the combined model were not equally relevant in both instructional conditions, further indicating that the internal composition of the performance construct varied by group (Tables 7 and 8).

Taken together, these results suggest that adaptive and regular learners share a common broad performance dimension, but that the internal weighting of courses within that dimension differs across conditions. This finding does not imply that one instructional condition is uniformly superior in effectiveness; rather, it indicates that adaptive and regular pathways may organise course-level performance differently, especially in modules with different instructional and competency demands.

The present study examined whether a structurally adaptive learning path implemented through activity locking in Moodle could improve the efficiency of large-scale digital skills training while maintaining learning outcomes broadly comparable to those achieved through a regular full-content pathway. Taken together, the findings show a clear and consistent efficiency advantage for the adaptive condition, but a more differentiated pattern with respect to learning effectiveness. Across the seven ECDL modules, adaptive sequencing substantially reduced instructional time and content exposure, yet post-test performance was not uniformly higher in the adaptive group. Instead, the results suggest that structurally adaptive learning can reduce time-to-completion at scale while preserving broadly comparable outcomes overall, although this pattern varies across course types.

From an efficiency perspective, the adaptive group, by following personalised pathways, was able to eliminate a substantial amount of educational content and reduce the time required to complete the courses by nearly half on average. All courses showed time savings, and each learner in the adaptive condition spent approximately 2 h less per course than learners in the regular condition. This pattern indicates that the activity-locking mechanism was generally effective in identifying topics that students had already mastered and removing them from the learning path, thereby allowing learners to focus on new or difficult material without unnecessary repetition. In this respect, the findings are consistent with previous research showing that adaptive instructional systems can improve training efficiency by reducing redundant practice and accelerating progress towards mastery (Mettler et al., 2011; Sottilare and Goodwin, 2017). The present study extends this evidence by showing that such efficiency gains can also be achieved at a large scale in an authentic school setting. In operational terms, the total time savings observed in the adaptive sample are also noteworthy, suggesting that adaptive sequencing may have practical value not only for learners but also for schools and training providers managing large cohorts.

At the same time, the effectiveness findings require a more cautious interpretation. The adaptive pathway did not produce a uniform achievement advantage across all modules. Rather, the two instructional conditions produced broadly comparable results in several courses, a significantly better result for the adaptive group in Computer Essentials, and significantly better results for the regular group in Word Processing and Spreadsheet. This means that the main contribution of the study lies not in demonstrating a generalised effectiveness advantage of adaptive learning, but in showing that a relatively simple form of structural adaptivity can generate substantial efficiency gains in a large and authentic educational setting without systematically undermining learning outcomes.

The variability across modules is particularly informative. The adaptive condition appears to work more smoothly in courses where prior knowledge can plausibly reduce the need for repeated exposure to content, whereas it may be less advantageous in modules requiring more extended procedural practice or operational reinforcement. This interpretation is consistent with the weaker adaptive outcomes observed in Word Processing and Spreadsheet, which may depend more strongly on practice-oriented engagement than on content recognition alone. In such contexts, reducing instructional exposure on the basis of pre-test performance may save time, but it may also reduce opportunities for consolidation of procedural fluency. By contrast, in modules such as Computer Essentials, where prior knowledge may more directly support successful progression through the course, adaptive sequencing appears more compatible with both efficiency and satisfactory achievement. Overall, these findings suggest that adaptive learning is not a one-size-fits-all solution and that the educational value of structural adaptivity depends partly on course characteristics and on the balance between conceptual knowledge, recognition tasks and practice-intensive competencies.

The findings of this study are broadly consistent with previous research suggesting that adaptive learning can support learner progress and improve the organisation of instruction, while also extending that evidence to a larger and more realistic educational context than is often found in the literature (Hubalovsky et al., 2019; Mirari, 2022). At the same time, the present results also suggest a more nuanced interpretation than is sometimes implied in the broader adaptive learning literature. Rather than showing a uniform improvement in outcomes, the study indicates that the main strength of structural adaptivity in this context lies in efficiency gains combined with broadly comparable, but not uniformly superior, learning outcomes. This distinction is important for interpreting the real educational value of relatively simple adaptive systems.

The structural analyses add a second level of interpretation. The CFA supported the use of a common latent performance factor based on the seven course-specific performance indices, suggesting that achievement and relative efficiency across modules can be meaningfully summarised within a broader performance dimension. At the same time, the multi-group analyses showed that this structure was not fully invariant across the adaptive and regular conditions. In practical terms, this means that the two groups can be described in relation to the same general construct, but that the internal weighting of individual courses within that construct differs across instructional conditions. Thus, the difference between adaptive and regular learning is not simply a matter of more or less performance overall; it also concerns the way course-level outcomes contribute to the broader pattern of learner performance.

The cluster analysis further highlights the heterogeneity of learner responses. Two high-performing clusters were identified, each dominated by one instructional condition, alongside a smaller mixed cluster of systematically low-performing learners. Because the clustering procedure was restricted to successful completers and relied on a composite index that includes time saving, these results should be interpreted cautiously. Even so, the cluster structure remains informative in showing that structural adaptivity does not benefit all learners in the same way. Most notably, the mixed low-performance cluster suggests that some learners struggle across multiple modules regardless of instructional condition. This sub-group may therefore require additional forms of support that are not provided by structural adaptivity alone, such as guided practice, tutoring, feedback regulation or metacognitive scaffolding. This interpretation is consistent with literature suggesting that personalisation alone does not resolve all learning difficulties and that technological or pedagogical barriers may still prevent some learners from fully benefiting from adaptive systems (Walkington and Bernacki, 2020; Barrera Castro et al., 2025).

From a practical perspective, the results suggest that structurally adaptive pathways can be valuable in large-scale certification-oriented training because they substantially reduce instructional time and limit redundant content exposure. This is especially relevant in school and institutional settings where time, staffing and curriculum space are constrained. However, the mixed effectiveness findings also indicate that adaptive sequencing should not be assumed to work equally well across all subject areas. Designers and instructors should calibrate adaptive thresholds to the nature of the target competences and should be cautious in modules where repeated procedural practice is likely to be educationally important. The successful implementation of adaptive learning in practice also requires appropriate technical and educational infrastructure, including modular content design and teacher preparedness for supervising learners moving through different pathways (Cavanagh et al., 2020).

Overall, the present findings support the educational value of structural adaptivity primarily as an efficiency-oriented approach. They show that, in a large scale and realistic school context, adaptive activity locking in Moodle can meaningfully reduce time-to-mastery while preserving broadly comparable outcomes overall. At the same time, the findings also indicate that adaptive learning should not be interpreted as a universally superior solution. Its effects depend on course characteristics, on how efficiency and performance are balanced and on whether vulnerable learners receive support beyond adaptive sequencing alone.

This study provides large-scale evidence that a structurally adaptive e-learning path implemented in Moodle can substantially improve the efficiency of digital skills training by reducing instructional time and redundant content exposure. Across the seven ECDL modules, learners in the adaptive condition were able to complete their pathways with markedly lower time investment, while achieving learning outcomes that were broadly comparable overall to those of learners in the regular condition. At the same time, the results do not support a uniform effectiveness advantage for the adaptive pathway. Instead, they indicate a more differentiated pattern, with similar results across several modules, a significant advantage for the adaptive group in Computer Essentials and significantly better outcomes for the regular group in Word Processing and Spreadsheet.

The main contribution of the study lies not in proposing a highly sophisticated adaptive algorithm, but in showing that a relatively simple and scalable form of structural adaptivity can reduce time-to-performance in an authentic educational setting involving a large number of learners. The study also contributes methodologically by combining separate analyses of efficiency and effectiveness with latent-variable modelling and learner-profile analysis. In this respect, the findings suggest that adaptive and regular pathways do not differ only in the amount of time required to achieve results, but also in the internal configuration of course-level performance across the curriculum.

Several limitations should be considered when interpreting these findings. Firstly, the study was conducted within a single national context, a single LMS environment and a single certification-oriented digital skills curriculum, which limits external generalisability. Secondly, pre-test and post-test assessments were based on the same item pools, although question and answer order were randomised. This design ensured strong alignment between diagnostic and summative assessments, but it may also have introduced practice effects. Thirdly, the composite performance index and the associated CFA, multi-group and clustering analyses were restricted to successful completers. Although this choice improved comparability across the seven modules, it also introduced a selection effect and limited the generalisability of these structural findings to the full randomised sample. Fourthly, the adaptive mechanism examined here is structurally simple and based primarily on pre-test performance; it does not represent a richer AI-driven adaptive system. Finally, the study did not include additional behavioural, motivational or psychometric analyses that might have further clarified why some learners benefitted more than others from adaptive sequencing.

Future research should therefore pursue at least four directions. Firstly, similar large-scale designs should be tested in other subject domains and educational contexts to assess the transferability of the efficiency gains observed here. Secondly, future studies should incorporate behavioural, motivational and self-regulatory measures to better explain learner heterogeneity and the presence of low-performing sub-groups across conditions. Thirdly, more refined adaptive logics should be explored, especially in practice-oriented modules where pre-test-based content skipping may not be sufficient to sustain strong performance. Finally, future work should compare relatively simple structural adaptivity with richer AI-driven or analytics-driven personalisation models to clarify under which conditions more complex adaptive solutions produce added educational value.

The present findings support the value of structural adaptivity primarily as an efficiency-oriented educational strategy. They show that adaptive activity locking in Moodle can meaningfully reduce time-to-mastery at scale while preserving broadly comparable outcomes overall, but they also indicate that adaptive learning should not be interpreted as a universally superior instructional solution. Its educational value depends on course characteristics, on how efficiency and achievement are balanced and on whether vulnerable learners receive support beyond adaptive sequencing alone.

During the preparation of this work, the authors used ChatGPT to provide language editing and proofreading support.