If you have ever taken a course in instructional design, you know all about formative evaluation. Instructional designers are trained to ask all manner of questions about a project, and formative evaluation questions are arguably the most important questions that can be asked as a project is developed.
In a course development environment, a formative evaluation scheme might involve asking a representative sample of learners a series of questions about a module in a course. The questions are generally very specific and are directly related to the content of the module being evaluated. Once a course is developed, formative evaluation is often used in an ongoing manner, to diagnose problems and inform curricular redesign.
I have known many instructors who have conducted a mid-course formative evaluation. But let’s consider applying the concept of formative evaluation to an existing online course. Further, let’s apply the concept of formative evaluation to an online course continuously as it is being offered. It does not matter whether the course is totally asynchronous or a hybrid course consisting of both face-to-face and online elements.
I first learned about using formative evaluation in this way from Robert Paugh, whose course “Measurement & Evaluation in Education” was selected for the WebCT Exemplary Course Project in 2001. Paugh, of the University of Central Florida, asks his students to complete a brief, ungraded online survey at the end of each learning module. The questions he asks are simple, but very much formative in nature. Here is a shortened list of the survey questions:
How long did it take to complete this activity from the time you started reading the activity until you completed the activity?
Were the activity directions in the “Procedures and Submissions Instructions” clear? (“Very clear,” “clear,” “somewhat clear,” “not clear”)
How would you improve the directions?
What was the most useful aspect of this activity? ( . . . least useful)
Were the learning objectives stated on the Web page for this activity met through the assignment? (“yes,” “partially met,” “no, not met”)
What academic content problems (if any) did you have completing this activity?
How did you resolve the prob-lem(s) you described?
Paugh has had excellent results with this process. While he does not require that his students complete the anonymous survey at the end of each course activity, he claims that most do. The feedback he receives from students is used to continually improve his course. And who can take issue with that?
If you are interested in engaging your students in an ongoing formative evaluation of your online course, here are a few suggestions:
Implement the evaluative process for the smallest common denominator for your course. If your course content is delivered in week-long units, evaluate at the end of each week. If your course is topic-based, evaluate at the end of each topic.
Use the same questions for each evaluation. Doing otherwise may discourage your students from participating.
Do not require students to participate in the evaluation. Doing so might encourage faux evaluations.
Consider ways to encourage students to participate, and be sure to let them know that you are asking for their assistance in improving the course.
Make the evaluations anonymous and encourage students to contact you via e-mail if they have particular concerns or problems. However, you need to ensure that only students in the course are responding to the evaluation questions. Course management platforms like WebCT and Blackboard have built-in survey features, which are anonymous.
If you receive support from your campus course development team, share and discuss the results with them.
A few online resources related to the concept and process of formative evaluation appear below. As you review these sites—and think about my suggestions—keep in mind a simple question that we should all be asking ourselves about the courses we teach: “How am I doing?”
About the author
