Effect of Student Location on Assessment of Instruction and Grade Assignment

Distance/remote education has evolved since 1982 when Penn State offered a distance education program that utilized the U.S. Postal Service to deliver instructional material (Carnevale, 2000; Mirakian & Hale, 2007). Distance education encompasses interactive audio, video, and lecture material between multiple locations. Numerous authors have investigated different distance learning environments (Blackwood 1968). Spooner (1999) define distance education as any form of education that geographically separates students and instructor and requires communication through media. Nowadays, media is often interaction through the Internet via video, audio, Web technologies, and e-mail.

The University of West Florida (UWF) offers Accreditation Board for Engineering and Technology (ABET) accredited undergraduate programs in electrical engineering and computer engineering. Until December 2008, the engineering programs were joint programs with the University of Florida. UWF, with a student body of approximately 10,000, has its main campus in Pensacola but has a strong presence in areas to the east through its Emerald Coast branch campus locations. The largest of these branch campuses is located in Fort Walton Beach (FWB) due in large part to the proximity of Eglin Air Force Base and associated contractor companies that require a highly skilled engineering workforce. Many degree programs offer courses at UWF Emerald Coast locations (e.g., criminal justice, teacher education, hospitality management, computer science, etc.), but, due to a unique delivery method, the engineering department offers a complete degree program that includes all required laboratory courses.

Approximately seven years ago and primarily due to the needs of the Air Force as well as supporting contractor companies, UWF began offering its electrical engineering and computer engineering programs in FWB using a synchronous distance education model. At the outset, no faculty were located in FWB and all classes were delivered from Pensacola. Now, three full-time and one adjunct faculty out of an engineering faculty body of seven full time and two adjuncts are resident in FWB.

Due to the demanding nature of an engineering curriculum, all lecture courses in the UWF engineering program are offered in an interactive distance learning studio (IDLS) and not in the increasingly common asynchronous, self-paced learning environment. The UWF IDLS, which employs commercially available equipment from AMX, CTGaudio, and Polycom, uses a dedicated data connection that reserves 3Mb/s for real-time audio and video and an additional 3Mb/s for data. In this setting (Shaer & Fuchs, 2008), the instructor, with the help of facilitators at both locations, simultaneously delivers a lecture course to students in Pensacola and FWB while being present in Pensacola or FWB. The audio and video connection allows the faculty member to see, hear, and interact with students at both ends with the same capability afforded to the students. Using a Tablet PC, the data connection allows the instructor to present lecture notes via an electronic whiteboard or PowerPoint as well as to utilize engineering software through use of a commercially available projection system in both classrooms. The system also allows lecture audio and data to be recorded so that students can review lectures at a later date.

Some ECE faculty observed that there appeared to be a difference in how students evaluate instructors and how instructors assigned course grades depending on whether the students are at the “near” or “far” location. The “near” location is defined as the location where the instructor is physically present and from which the instruction originates. The “far” section is the remote location connected via the IDLS system. As such, the purpose of this study was to gather empirical evidence on the impact of the students’ location in an IDLS setting on their assessment of instruction and on the course grade they earn. The study investigates the following research questions:

1. Is there an empirical difference in the way students at the near and far locations assess the course instruction?

2. Is there a difference in the way instructor assign course letter grades to students on the near and far sides?

The study was conducted at the ECE department at UWF. It includes a total of 59 engineering courses taught using the IDLS classrooms at Pensacola and FWB during 5 semesters from Spring 2006 to Spring 2008. Each course included in our study had two sections: the near section and the far section. Out of the 59 courses considered in this study, 31 were taught from Pensacola and 28 were taught from FWB. The total enrollment in these classes was 1691 students: 877 on the near side and 814 on the far side. The average enrollment in each course was 29 students: 15 at the near side and 14 at the far side. The classes were all ECE classes and ranged from the introduc-tory-level to the senior-level. The classes were taught by a total of 15 different instructors.

The data related to assessment of instruction were collected from the summary forms of the standard Student Assessment of Instruction (SAI) surveys that students fill in for each class at the end of the semester. A total of 1,077 SAI forms were completed: 583 on the near side and 494 on the far side. This constitutes a response rate of 66.5% on the near side and 60.7% on the far side. At the end of the semester after final grades are assigned, instructors are given a summary of their SAI survey for each class they taught and a copy is made available to the public at UWF library (a sample is shown in Figure 1). As can be seen in Figure 1, the SAI survey contains eight entries pertaining to how students felt about the instruction of the class and their overall assessment of the instructor. These entries are:

1. Item 1: Expression of expectations for performance in this class

2. Item 2: Description of course objectives and assignments

3. Item 3: Communication of ideas and information

4. Item 4: Stimulation of interest in the course

5. Item 5: Facilitation of learning

6. Item 6: Respect and concern for students

7. Item 7: Availability to assist students in and out of class

8. Item 8: Overall assessment of instructor

Students rate each item on a scale of Poor to Excellent. In our study, we assigned the numeric value 4 to the rating of Excellent, 3 to Very Good, 2 to Good, 1 to Fair, and 0 to Poor. The average section response (AR) to any of the eight items in the SAI form was computed as a weighted average of these equivalent numeric values:

where E, VG, G, F, and P represent the number of students who rated that item as Excellent, Very Good, Good, Fair, and Poor, respectively.

Data related to student grades in the classes included in our study were gathered from official student grade summaries that were obtained from UWF registrar office without any student identifiers to protect the privacy of the students. In our study we (and UWF) assigned the numeric value 4 to a grade of A, 3.3 to B+, 3 to B, 2.3 to C+, 2 to C, 1.3 to D+, 1.0 to D, and 0 to F. Because the University of Florida grading system does not accept “minus” grades (e.g., B-), instructors were asked not to assign “minus” grades in engineering classes. If assigned, “minus” grades were automatically rounded up to the next full letter grade. The average section grade (AG) is computed as:

where A, Bp, B, Cp, C, Dp, D, and F represent the number of students in the section who were assigned the course grades of A, B+, B, C+, C, D+, D, and F, respectively.

We analyze and compare the means of the data collected on the near and far sides using the T statistic test (Milton & Arnold, 1990) with n = 59 and α = 0.05.

The means (µ), standard deviations (α) and obtained t statistics for the near and far sides of items 1-8 on the SAI and the course grades are shown in Table 1. As can clearly be seen in Table 1, there is very strong statistical evidence (t values ranging from 3.041 to 4.508) that students at the near site rate instructors/instruction higher than students on the far side of the same class. Consistently, students at the near side rated items 1-8 of the SAI 0.4 to 0.6 points higher than students on the far side. This trend was consistent throughout the 5 semesters covered in this study (see Figures 2-6). If we combine the averages of items 1-8, we can also see the same trend every semester during the period of the study (Figure 7).

As for class grade assignment, there is also statistical evidence, even though not as strong as that for the assessment of instructions (t value = 1.754), that students on the near side were assigned, on average, higher course grades than those on the far side (µ = 2.996, σ = 0.515 vs. µ = 2.808, σ = 0.645). This trend was consistently observed throughout the 5 semesters covered in this study (Figure 8).

Based on our statistical analysis of empirical data gathered from spring 2006 to spring 2008, we can confidently say that there is a difference in the way students on the near and far sides evaluate course instruction. The difference was consistent over the five semesters we conducted this study. What is really interesting is that this trend was observed in courses that did not involve actual instruction. For example, in the senior design classes, there is no instruction per se; the role of the instructor is usually scheduling dates for project demonstrations, oral presentations, gathering student work, mentor evaluations of the projects, and so on. As can be seen in Figure 9, there is similar difference in “instruction” evaluation for this type of class as seen in other “regular” classes. Even though it was not our goal in this study to investigate the reasons behind this trend, one has to wonder why. Is there an inherent bias against instructors on the far side compared to those on the near side? Do students dislike not having direct, face-to-face access to the instructor? Does the IDLS setting feel impersonal?

Even though we also observed a consistent difference in grade assignment to students in the near and far sides, it was not as pronounced or as statistically significant as the difference of student evaluation of instruction. As with the first finding of this study, our goal was not to explain the reason behind this trend. Is there an inherent bias against students on the far side compared to those on the near side? Are students on the far side not learning as much as the students on the near side? Do they feel left out as they are not sitting in the same room as the instructor? Does the lack of direct, face-to-face access to the instructor hamper their understanding of the course material?

Our study showed that there is a difference in how students evaluate instructors and how instructors assign course grades depending on the location of the students. However, our study did not investigate the reasons behind this trend. This study answered two important and crucial questions, but left many questions unanswered. These questions are definitely worthy of future research.

We would like to thank Diana Feltner for her help in gathering some of the data used in this study.