Chapter 19: Are Executable Exams Executable?: An Analysis From Pedagogical, Technical, and Psychological Perspectives

This chapter describes a process of changing the evaluation format of an Introduction to Computer Science (CS1) course, which is a mandatory course in all academic high-tech oriented engineering programs. Specifically, this chapter presents a two-phase teaching experience in the CS1 course offered to computer science (CS), electrical engineering (EE), and data science (DS) freshmen at the Technion–Israel Institute of Technology.

In this change process, the traditional paper-based exam format was replaced with an executable exam format (Phase I), which eventually created a natural and obvious opportunity to integrate professional skills content into the course (Phase II). During an executable exam, students work on a computer in a designated programming environment that enables them to compile their code and run tests. The professional skills content supports the students in their preparation process for the executable exams and while taking them.

Specifically, Phase I examined attitudes of three groups—students, teaching staff, and industry representatives—towards executable exams. In addition, data was collected and analyzed from CS and EE students from other academic institutions in Israel and from CS lecturers from Israel and around the world. The attitudes are categorized into three aspects: pedagogical, technical, and psychological. The groups’ attitudes are supported by the pedagogical advantages and disadvantages of executable exams that were mentioned.

The examination of Phase I’s results encouraged a reconsideration of the course’s pedagogical approach, objectives, content, and assessment. This examination eventually led to Phase II of this teaching experience, in which the CS1 course content was modified in a way that integrates theoretical content with professional skills from the students’ very first exposure to programming courses. Specifically, the following skills were added to the course content: time management of software development processes, testing, ethics, and integrity. This chapter presents this approach to the CS1 content as well as findings related to the students’ attitudes toward the new skills taught in the course.