Investigating the Drop-Out Rate from a BIM Course

Building information modelling (BIM), as a process, has been stated as one of the solutions to improve the construction industry’s poor productivity (McKinsey&Company, 2017) and enhance construction professionals’ career paths (Wu and Issa, 2013; Uddin and Khanzode, 2014). BIM implementation itself has been reported as primarily affected by “people issues” (Won et al., 2013) and therefore, one of the key focus areas to look at, is to analyse how BIM is taught (Sacks et al., 2018) to increase the knowledge of BIM.

Numerous research studies have investigated BIM education program development (Sacks and Pikas, 2013; Pikas et al., 2013; Succar and Sher, 2014), which has changed owing to industry needs in terms of BIM related topics, including new job roles (Abdirad and Dossick, 2016; Mayo et al., 2018; Succar et al., 2013; Uhm et al., 2017). BIM-related technology is tightly interconnected with the software and its availability for university course programs (Russell et al., 2014; Tuchkevich et al., 2015), which, in turn, define the topics to be learned. BIM subjects are often defined as courses that do not need any previous knowledge of computer aided design (CAD) (Sacks and Barak, 2010), and therefore, they can be incorporated into first- or second-year study programmes, and this has been tested by various universities (Vinšová et al., 2015). Developing courses that cover industry needs places considerable pressure onto current faculty members (Russell et al., 2014; Peterson et al., 2011; da Silva Vieira and Neto, 2016) and this could be the most significant barrier to integrating BIM into current university degree programs in some countries (Abbas et al., 2016).

To overcome these “people issues”, the learning process should develop various skills and be available for each student not only for those who are more active and respond to teachers’ questions in the classical learning environment. By using classical lecturing methods, it is difficult to simulate the so-called all-involved learning experience and active learning methods have offered a possible solution for that. Active learning embraces various teaching methodologies. For example, peer instruction (PI) as an active teaching method was popularised by Harvard Professor Eric Mazur in the early 1990s (Mazur, 1997). The main idea of peer instruction is to help students learn more from pre-class reading and to increase students’ engagement in discussion sessions (Crouch and Mazur, 2001; Alcalde and Nagel, 2018). Applying peer instruction may dramatically increase the teacher’s interventions (as more questions should be answered than usual) but the teacher availability and readiness to answer questions from the whole group is widely valued by students as it helps them to develop during the course and this reduces the failure rate (Zingaro and Porter, 2014). Peer instruction as a teaching method is very similar to other methods that share the same logic as flipped or inverted classroom (Bates and Galloway, 2012; Hsieh et al., 2015; Wang et al., 2018) or active learning (Lassen et al., 2018). Classical PI assumes that people in a PI session are equally involved (everybody is participating in discussions), but this is not always the case because it is not made compulsory. Therefore, special versions of PI have been developed in which each participant should ask a question before entering the discussion group as in the stepladder technique (Rogelberg et al., 1992; Michinov et al., 2015).

In this research, BIM course development over the period of 2017-2018 is described. The focus is to analyse the effect of course layout, which, from one side, aligns with industry needs but, from the other, may cause higher drop-out rates. The development of various skills usually means a higher rate of assessment types, which, in turn, may influence the motivation to finish the course. Peer instruction as a method has been chosen to give better support throughout the course and as a tool to get initial insights into the reasons for drop-outs.

The typical BIM course, if not based on one software package only, is usually a high educational density course, which means that it involves more than four different tools to be learned (Fonseca et al., 2017). Three separate BIM courses were developed during 2017-2018 in Tallinn University of Technology (TalTech). One with a focus on buildings, one for infrastructure and one for advanced BIM studies that combines both buildings and infrastructure topics. In this research, the analysis is carried out for the first of these, BIM for buildings, which was firstly designed for fourth year civil engineering students but later moved to the second year of studies. Table 1 shows how the “BIM for building” course evolved from an initial, lower density study program to a new, high educational density course.

The course has been designed as an active learning course in a university wide e-learning system called Moodle. It is divided into several modules (Table 2), each one of which has its own learning outcomes and assessments. The core idea of the course is to teach BIM process from the modelling perspective but include collaborative exercises.

The course begins with introductory lectures (usually during first 2 weeks, 3 × 45 min at one time). At the same time, the students start to carry out individual tasks and assessments (Table 2). Peer instruction is simulated in an e-learning environment, where each student must formulate at least one question at the end of the course module and answer or discuss another student’s questions. These questions and answers are assessed based on their constructiveness.

The data used for analysis were obtained from two sources: (a) the university’s study information system (number of declarations and final grades) and (b) student’s individual activities with the completion date from an e-learning environment. In addition, a feedback system at the end of the course was in place and responses were gathered. However, these course feedback responses cannot be fully used in the current research as key information, for example, whether student respondents “passed” or “failed” cannot be distinguished and, with the high drop-out rate, this may introduce bias into the analysis.

The course in both years was taken by different study groups. As such, the main interest was to understand if student performance depends on a study group itself. Division was made into two separate groups: (a) study program students and (b) open university students. Although there could be many other variables affecting the drop-out rate (for example, a previous background or knowledge) these are not taken account in this preliminary analysis. In fact, the course did not have any compulsory prerequisites. The course in both years has been an elective subject for all student groups.

The preliminary analysis was carried out at two different levels of detail. Firstly, the key performance indicators of student accomplishments were compared during the BIM course editions of 2017 and 2018. Secondly, as more data was gathered during 2018, it is analysed against learning outcomes where suggested learning module completion times should be followed but are not made compulsory except for the final deadline at the end of semester. These data give insights into students’ capability to manage their time.

The BIM course layout has evolved in time (Table 1). Table 3 summarises the core statistics of the BIM courses that were delivered during 2017-2018. We can conclude from Table 3 that the number of various assessments can positively drive the student’s performance (final grade) as he/she is more engaged with the study content. The difference is small but visible. On the other hand, we see a moderate drop-out rate in terms of the course completion rate when compared with the previous year (2017). We explain this by the fact that the course needs more engagement and work during the semester and if students fail to do it on their own, their motivation reduces. The increased load of the course (see Table 1) is directly connected with the nature of the course (elective or compulsory course).

We take a closer look, how the motivation differs between those who finish the course and who don’t. Hereby, the motivation is defined through a student’hx2019;s continuous activity. The analyses are divided into two main groups. Starting with those who took the course through the full-time study program. Table 4 presents students' performance in terms of activity completion on-time. Positive numbers indicate the number of days ahead of time that an activity was finished and negative numbers refer to learning activities completed after the suggested completion times (in days). It should be noted that the course has only one deadline, at the end of semester. Therefore, each student was able to apply his own time management to finish the course on-time.

We can clearly see that some drop-outs are already visible before the first assessment is made (Student IDs 17 and 25). We could only explain these as attempts to see what the course is about and what the requirements are and, if those did not align with the students’ own expectations, they discontinued. We can clearly see that being behind the suggested completion date, doesn’t affect the possibility to finish on-time (for example, Student ID 35 began late and lagged the recommended schedule by 70+ days). The second study group, “open university”, is shown in Table 5.

We see a similar pattern here. There is one who does not start the course and those who decide to give up. With “open university” students, we clearly connect this with their daily duties because these students are participating in the course while having a full-time job. When we compare “best-in-class” students in terms of their time management, both study groups are similar: 25 days behind the schedule is quite normal in the middle of the course.

During 2017-2018, new BIM courses were developed as active learning courses. To reflect the industry needs for students to develop various skills, the course was expanded in 2018 (in terms of the number of assessments and assessment types). This research was carried out to analyse the preliminary drawbacks if the increased course learning density in terms of the number and types of assessments affecting student performance and drop-out rates. Overall, we see an improvement in the performance of students who finish the course (higher grade).

We note the high drop-out rate and that we can’t fully explain its increase from 2017 to 2018 (as a percentage of those who started the course). This may be partly explained that in 2017 students were able to present the homework that aligned with the video learning content. In 2018 all videos were meant to be warm-up exercises and each student should present their original work that should not be based on previous works (they declare the chosen project beforehand which, in turn, should be unique).

Further research will investigate student motivation and the influence of active learning techniques on their performance more closely. Owing to the course electivity in the study program to date, the future analysis will also focus on how the subject becoming compulsory affects the drop-out rate.

This research was supported by the “Integrating Education with Consumer Behaviour Relevant to Energy efficiency and Climate Change at the Universities of Russia, Sri Lanka and Bangladesh (BECK) project co-funded by the Erasmus+ Programme of the European Union. The European Commission support for the production of this publication does not constitute an endorsement of the contents which reflects the views only of the authors, and the Commission cannot be held responsible for any use which may be made of the information contained therein.