THE EFFECTS OF IMMEDIATE AND DELAYED FEEDBACK ON SECONDARY DISTANCE LEARNERS

Brigham Young University (BYU) is one of the nation’s largest postsecondary providers of independent study middle and high school distance education courses. The Department of Independent Study was established in 1921 but did not begin to offer secondary courses until 1961. At the time of this writing (August 2006), the department offered 225 middle and high school courses, available in Web-based and paper-based formats (except two courses that are in Web format only). For a 12-month period ending on December 31, 2005, the department enrolled 99,280 students, of which 62,490 (62.9%) selected the Web-based format. These students represented addresses of record from all 50 states and 62 foreign countries. The Independent Study program at BYU utilizes an open enrollment system which allows anyone who wishes to enroll in a course to do so. Students are given 12 months from the date of their enrollment to complete a course.

In the mid-1990s, the Independent Study Department at BYU began to implement an immediate grading and feedback program known as Speedback in its independent study courses for many of its graded assignments and exams. The program includes a feedback feature for lessons and assignments that are electronically scored. Elaborative feedback responses are drawn from a data warehouse; these responses mark students’ answers correct or incorrect, and then offer a hint for further instruction and remediation for any incorrect selections. Some responses in this program are even distractor-specific and provide feedback customized to each possible response, as contrasted to item-specific feedback which provides more generic feedback at the question level.

The quick turnaround time of Speedback has not only reduced transmission delays of feedback to students, but has also greatly improved student and faculty service. Use of Speedback has reduced instructor grading time as well as shipping and handling expenses for student and institution alike. At the same time, students’ complaints have decreased because they receive timelier—even immediate— responses to assignments they have submitted. However, the effects of receiving immediate or delayed feedback on high school students in an independent study setting have not been previously researched.

This study explored the impact of immediate versus delayed feedback on secondary students enrolled in independent study high school courses by examining the performance of two groups of students. Students who enrolled in a Web-based version of a particular course received immediately generated electronic feedback to their submitted assignments. Students who enrolled in a paper-based version of the course received delayed feedback, since they did not receive written feedback until they mailed assignments to their instructors and then waited for them to be graded and returned using the postal service. The dependent variables that were analyzed— and by which student performance was measured—included final exam grades and length of time between the first submitted assignment and course completion; all other instructional materials and feedback responses remained the same across groups.

In an instructional setting, the term feedback describes the practice that informs a learner whether a response is right or wrong. It can also be used to provide additional information to the student, explaining why a response is correct or incorrect (Morey, 1992). Feedback is most effective when it corrects student mistakes or misconceptions received during instruction (Bangert-Drowns et al., 1991; Cyboran, 1995; Kulhavy & Anderson, 1972; Kulhavy, Stock, Thornton, Winston, & Behrens, 1990; Kulhavy, White, Topp, Chan, & Adams, 1985; Lhyle & Kulhavy, 1987; Phye & Andre, 1989; Surber & Anderson, 1975); it does not appear to have much impact as a reinforcing tool (Clariana, 1993; V. B. Cohen, 1985).

Instructional feedback can generally be classified as either verification or elaboration feedback. The simplest type of verification feedback is knowledge of results (KR) feedback. With KR feedback, the student is only given an indication of the correctness of a response, such as “yes/no” or “right/wrong.” Knowledge of correct response (KCR) is the next level of verification feedback. Here, a student is given a corrective hint or suggestion in addition to a “right” or “wrong” response, such as, “Incorrect. Remember to solve that part of the equation found in parentheses first.”

Elaboration feedback provides the student with an even greater amount of information. It is possible for the material included in elaboration feedback to become so complex that it begins to take on the role of providing new instruction. This can be especially helpful in a distance learning setting where a teacher is not always present to answer questions or clear up misunderstandings in a timely way. T. Anderson (2002) has propounded a theoretical construct that requires compensating interactions, like elaboration feedback, for distance learners. He wrote, Sufficient levels of deep and meaningful learning can be developed as long as one of the three forms of interaction (student-teacher, student-student, student-content) is at very high levels. The other two may be offered at minimal levels or even eliminated without degrading the educational experience. High levels of more than one of these three modes will likely deliver a more satisfying educational experience, though these experiences may not be as cost or time effective as less interactive learning sequences. (p. 4)

Research findings indicate what many educators have known intuitively—that any amount of feedback is better than none at all and that, in general, the more feedback provided, the greater the impact on the student’s comprehension and resulting performance (Anderson, Kulhavey, & Andre, 1972; Cyboran, 1995; Morrison, Ross, Gopalakrishnan, & Casey, 1995; Olina & Sullivan, 2002; Whyte, Karolick, Nielsen, Elder, & Hawley, 1995). This implies that KR is the least effective type of feedback, followed by KCR, with elaborative feedback being the most effective (Bangert-Drowns et al., 1991; Clariana, Ross, & Morrison, 1991; Kulhavy, 1977; Lee-Sammons & Wollen, 1989; Pridemore & Klein, 1991). Although elaboration feedback requires greater effort to develop than other types of feedback, it can also provide a better learning experience for the distance learner.

Feedback is also classified according to how soon it is provided after a student responds to a question or completes an exercise or exam. Some researchers argue that immediate feedback (i.e., supplied as soon as a student has responded to a question or completed an assessment exercise) promotes retention of learned information. Others believe that delayed feedback (i.e., feedback withheld for a period of time after the student’s completion of an assessment exercise) is more conducive to learning.

Those who argue for immediate feedback assert that a test procedure that does not employ immediate feedback is likely to foster misconception rather than further learning (Dihoff et al., 2003). One of their studies reported that immediate feedback, coupled with an answer-until-correct format in unit tests, promoted the “most learning and retention, facilitated the most involvement in the testing process, and corrected the most inaccurate assumptions” (Dihoff et al., 2003, p. 542).

Advocates of delayed feedback believe that it is more effective than immediate feedback because of the delay retention effect (DRE), a phenomenon in which learners who have feedback delayed for some period of time purportedly recall significantly more than subjects who receive feedback immediately following a learning assessment activity (Kulhavy, 1977; Schroth & Lund, 1993; Swindell & Walls, 1993). However, other research findings have questioned the impact of DRE on student retention and performance. Peeck, van den Bosch, and Kreupeling (1985), for example, demonstrated that final exam scores for students in a typical instructional setting were very similar whether they received immediate or delayed feedback.

How much learners use the feedback they receive may explain some of the discrepancies between the results of the various studies mentioned above. A study by Webb, Stock, and McCarthy (1994) indicated that delayed feedback was superior to immediate feedback in allowing a greater probability for more correct responses on a delayed final exam. However, they also discovered that this may not have been due to the likelihood of students either remembering or forgetting an initial response, but rather because of the tendency on the part of learners in the delayed feedback group to study the feedback for longer periods of time. By taking additional time to study, the students learned the material well enough to identify the correct answer on the final exam regardless of their performance on earlier unit tests.

This study was a retrospective investigation based on archival data maintained by the Independent Study Department. It used a quasi-experimental design to examine the impact of immediate versus delayed feedback on high school independent study course final exam grades, and on the amount of time required for the students to complete the courses.

The sample (n = 352) for this study was taken from the students enrolled in the following high school level courses offered through the BYU Independent Study program: (a) Twelfth-Grade English 1 (ENGL 055), (b) United States History after 1851 (HIST 043), (c) Basic Health: Choose to Be Healthy (HLTH 041), and (d) Character Education: Exploring Values 1 (XPLR 041). These courses were selected because they represented diverse disciplines and enrolled large numbers of students.

The students in each sample completed these courses between January and July of 2005. They were divided into two groups. The first group consisted of the total number of students enrolled in the paper-based versions of the courses, who submitted and received graded assignments by mail. The second group was identified using a computerized random number generator to identify an equal number of students from a numerated list representing the larger pool of students enrolled in the Web-based versions of the courses, who submitted and received graded assignments by computer. Assignments for both groups were identical and were graded using the BYU Speedback program. The same elaborative feedback was generated for both groups, but was delivered to each student using his or her self-selected delivery method (i.e., mail service or Web-based) thereby determining whether the feedback was immediate or delayed.

Required registration information included only that which is necessary to identify a student and provide requested course materials, such as name, address, and birth date. No ethnic or religious preference information was requested. In this study, both groups of students were evenly matched in terms of gender, with 47% of the immediate feedback group and 48% of the delayed-feedback group being male and 53% and 52% of the respective groups female. The majority of the students in both groups were between 17 and 18 years of age, with 71% of the immediate feedback group and 67% of the delayed feedback group falling into this category. Students in the immediate feedback group represented 21 states within the United States, while students in the delayed feedback group represented 22 states.

Final examinations for the four courses included in the study were designed to use either a machine- or electronic-scorable format (i.e., optical character recognition for “bubble sheet” forms and electronic processing for Web-based submissions). The exam questions tested different levels of learning, though most were at the recall, comprehension, or analysis levels (Bloom, 1968). The final exam for the English course consisted of 50 questions (49 multiple-choice questions and 1 true/false question). The history final exam consisted of 100 questions (96 multiple-choice and 4 true/ false questions). The health final exam consisted of 100 multiple-choice questions. The exploring values final exam consisted of 55 questions (54 multiple-choice questions and 1 true/false question). A critical instructional element of two of the courses, submitted just prior to the final examination, was an instructor-graded course portfolio.

Because of the retrospective nature of this study, it was impossible to randomly assign students to experimental groups. The group that received delayed feedback by mail was classified as the comparison group. The group that received immediate feedback by computer was considered to be the treatment group. A pretest and posttest are also commonly used for comparison purposes. This study utilized a posttest-only control (comparison) group design with nonequivalent groups to test the effects of the treatment (type of feedback) on students’ scores and time-to-completion of courses. Research design specialists classify this design as quasi-experimental, albeit a weak one (Shadish, Cook, & Campbell, 2002). The data for this study were gathered from archival records maintained by the department. The data included course final exam grades as well as the end-of-lesson assignment and final exam submission dates used to calculate time to completion.

For data analysis this study employed Statistical Package for the Social Sciences statistical software. The analysis utilized an independent sample t test (α = .05) to compare the differences in average final exam scores and the average number of days used to complete a course between high school independent study students provided with either immediate or delayed feedback. The t tests were used to determine whether the observed differences in group means were statistically significant with the sample sizes of the two groups being equal (Glass & Hopkins, 1996).

Once a student enrolled in either the paperbased or Web-based version of one of the courses identified above, he or she was provided the lesson materials. Both groups received the same instructional materials and content delivered in either paper or electronic format. The paper-based text was printed exclusively in grayscale, while the Web-based text was primarily in grayscale with some color highlights. Any information supplemental to the electronic text, (e.g., audio, video, and hyperlinks) was also made available to the students enrolled in the paper-based version using a CD-ROM that was sent to them with the paper-based course. (An unmoderated Web-based discussion board was also available for students to communicate with other students. This board received minimal use and was available to both groups. However, the students in the paper-based group would have had to go online to access it.)

At the conclusion of each course lesson, the students submitted an end-of-lesson assignment using either the postal service or secure electronic transmission to BYU Independent Study for grading. The assignment was scored by computer, and distractor-specific elaborative feedback was generated for each question on the assignment. The graded assignment and feedback were then returned to the student by the same method the student had selected for submission (i.e., postal service or electronic transmission).

Once the student had completed the lesson material and submitted all end-of-unit assignments for grading, he or she requested a final exam. The request was processed, and exam materials were either held for the student to complete in a proctored environment at the Independent Study Testing Center on the BYU campus, or mailed directly to the certified proctor. The student then met with the proctor and completed the final exam under the proctor’s direct supervision. After the student completed the exam, the proctor collected and returned the exam materials to BYU Independent Study by mail. When received, the exam was graded by computer, and the grade was then posted and made available to the student. BYU Independent Study retained the hard copy of the exam on file.

The mean scores, standard deviations, t test results, and effect sizes are summarized in Table 1 for students who received immediate feedback and those who received delayed feedback.

A visual comparison of the pretreatment and posttreatment means indicates that the difference is positive for each course. The results of the t tests indicate that the mean difference between the immediate and delayed feedback groups was statistically significant (see asterisked values) for the English and exploring values courses, but not for the history and health courses. The entries in the far right column of Table 1 are effect sizes and provide a way to describe the practical significance of the mean difference between the two types of feedback. The effect sizes indicate that students in English who received immediate feedback scored .82 standard deviations higher on the final exam than those students who received delayed feedback. Similarly, students in the exploring values course who received immediate feedback scored .61 standard deviations higher on average than students in the same course who received delayed feedback. Cohen (1988) has recommended the following guidelines for interpreting effect sizes in terms of their relative magnitude: d = .2 is a small effect, d = .5 is a medium effect and d = .8 is a large effect. Accordingly, the size of the effect for the English course should be considered large, while the effect size for the exploring values course is medium. The effect sizes for the History and Health courses are considered small.

The mean number of days required for completion, the standard deviations, t-test results, and effect sizes for the independent study courses included in this study are displayed in Table 2.

Visual comparison of the pretreatment and posttreatment means is negative and shows that students in the delayed feedback group tended to complete their course work in less time than those in the immediate feedback group. The t-test results indicate that the difference in the total means between the immediate feedback and delayed feedback groups was statistically significant (see asterisked values), as was the difference in means between the two feedback groups for the English, history, and health courses. The difference in means between the two feedback groups was not statistically significant for the exploring values course.

The effect of delayed feedback on course completion times for the English and history courses should be considered large effects according to Cohen’s guidelines. The effect size for the health course is considered medium, and the effect size for the exploring values course is considered small. The effect of delayed feedback on the course totals was .532, which is considered a medium effect size.

The final exam grade, commonly accepted as an indicator of student success in a course, was the dependent variable used to determine whether the use of immediate feedback was more successful than delayed feedback in helping a student succeed in an independent study course. Immediate feedback students enrolled in English scored over an entire grade higher than their delayed feedback counterparts, while students enrolled in Exploring Values who received immediate feedback scored over one-half of a grade higher than those who received delayed feedback. Although the differences in mean final exam scores between the immediate and delayed feedback groups enrolled in History and Health were not statistically significant, students who received immediate feedback still scored higher than those who received delayed feedback—an important consideration for most students. Immediate feedback students enrolled in history scored almost one half of a grade higher than students who received delayed feedback, while immediate feedback students enrolled in health scored one fourth of a grade higher than their delayed feedback counterparts. Additionally, a review of the effect sizes of all four courses indicated that the use of immediate feedback had a greater impact on course final exam grades than the use of delayed feedback.

A constant matter of concern for independent study administrators is the need to keep students involved in a course and progressing toward completion. This study hypothesized that immediate feedback would encourage students to complete a course more quickly than would delayed feedback. In fact, results of the study indicate that this is not the case, as students in the immediate feedback group took a significantly greater amount of time to complete their course work than students in the delayed feedback group. The average amount of time to course completion for students receiving immediate feedback was 35 days longer than for those students receiving delayed feedback.

Analyses of the English, history, and health courses showed a statistically significant difference in the average number of days required for completion between the immediate and delayed feedback groups. Although the difference in the completion times between the two groups was not statistically significant for the exploring values course, students receiving immediate feedback still took 13 days longer to complete their courses than those students who received delayed feedback. Again, a review of the effect sizes indicates that the use of delayed feedback had a greater impact on average course completion time than did the use of immediate feedback for all courses.

The students included in the study self-selected whether they received immediate or delayed feedback. Because random assignment was impractical, a posttest-only control (comparison) group quasi-experimental design was used in the study. The obvious flaw in this type of study design is the absence of pretest data (Cook & Campbell, 1979). This presents the possibility that any differences discovered between the study groups could be caused by selection differences rather than the treatment effect being tested. In this case, such a possibility is an area of concern, because no information was available as to why the students selected one course option or the other. A small number of students in the delayed feedback group were contacted by telephone, and all explained that they had chosen to take the courses by mail because they did not have convenient access to a computer or the Internet. Lack of access to a computer may be an indicator of low socioeconomic status, which can in turn be a risk factor for poor academic performance. It may also limit access to the unmoderated Web-based discussion boards, which see minimal use anyway, that are available to students in both groups.

A second limitation was the lack of naturalistic and evaluative data about the student population. This study assumed that students both received and used end-of-unit feedback. However, without direct input from the students, the researchers are uncertain as to whether or how the feedback was used. Feedback may or may not have been used based on the student’s motivation for taking an independent study course. A student who only needs to pass a course may exhibit different study behavior than a student who has a strong desire to learn as much as possible or achieve a higher grade. A third possible limitation was the time constraint which was arbitrarily imposed by the researchers. Independent study students at BYU are given up to 1 year to complete a course. However, because of the researchers’ desire to conclude the project in a timely manner, only those students who completed course work in a 7-month period were included in the study. Again, this was a retrospective study based on archival data, which conferred both advantage and limitation.

This research has raised more questions than it has answered. Additional research is needed to learn how, and how much, students use the feedback and whether or not they use it differently depending on which subject matter is taught. The information could provide insight to instructors and course designers on how they might better customize and present feedback to students.

Another area of interest for administrators of independent study programs is the rate of student completion—not just completion percentages. The unanticipated finding that students who received delayed feedback also completed their courses more quickly—on average 35 days sooner—than those who received immediate feedback merits further research. Why students in the immediate feedback group took significantly greater amounts of time to complete their course work than students in the delayed feedback group defies any rational explanation known to the researchers. They consider this unanticipated result the most important finding of this study; it has also given the research team cause for a follow-up study to investigate this inexplicable occurrence.

A learner in an independent, self-paced instructional environment moves at his or her own pace, unlike those students in cohortbased settings who work together with incremental deadlines. While advantages of immediate feedback are known to the academic community generally, this study examined the effects of immediate and delayed feedback in a specific context of distance learners in a non-cohort, independent study setting. Students who received immediate feedback in this specific instructional setting performed better on their final examination but took longer to complete their courses. While this study and research design had limitations, it has advanced the study of feedback timeliness in a distance learning setting and has also encouraged further research in the field.