RANDOMIZED STUDY ON THE IMPACT OF COOPERATIVE LEARNING: Distance Education in Public Health

The purpose of this study is to explore the effects of cooperative learning on cognitive outcomes in a public health graduate level Web-based distance education course. Specifically, we use a randomized control trial to determine the impact of two teaching pedagogies on learning effectiveness in three areas of the cognitive domain: declarative knowledge, procedural knowledge, and higher cognition. Examining effective teaching in public health requires asking three questions: (1) how do students learn, (2) how effective is our teaching, and (3) how can we apply the scholarship of teaching and learning to the discipline of public health?

Education researchers have long established the techniques of effective teaching and learning pedagogy in higher education. However, these concepts have been slow to diffuse to graduate-level public health education. A literature review reveals that empirically-based pedagogical approaches are only sporadically applied and curriculum based on modern educational theory is rare in both the traditional and nontraditional public health classroom. This study examines various learner outcomes in distance education as influenced by cooperative learning.

We will test the effectiveness of cooperative learning on knowledge acquisition in a distance education course. Specifically, we will examine whether cooperative learning is associated with improved learner performance in three levels of knowledge: declarative knowledge, procedural knowledge, and higher cognition. The following research questions explore this association:

1. Can declarative knowledge acquisition (remembering) be achieved equally through independent-learning and cooperative learning pedagogical techniques?

2. Does cooperative learning improve procedural knowledge (knowing how to inquire and present knowledge in public health)?

3. Does cooperative learning improve performance on higher cognition (setting problem solving strategies and assessing lines of inquiry)?

The need for understanding teaching pedagogy based on evidence is becoming more apparent in public health education. Graduate-level public health education involves more than acquisition of specialized knowledge. As a practice discipline, public health focuses on skills, critical thinking, problem solving, and inductive reasoning. While it is fairly easy to impart and assess knowledge acquisition through objective tests in a Web-based course, the development of deeper thinking skills is a particular challenge with graduate public health students in distance education because of the time gap between the learning point of the lesson and the learning moment of the student. In other words, the asynchronous Webbased medium delays instructor content amplification and application, in contrast to the more immediate clarification available in a traditional classroom setting.

This research addresses several pedagogical theories, including: cooperative learning, levels of knowledge, the scholarship of teaching and learning, constructivism, and deep learning. Each of these concepts is defined and elaborated in the discussion below.

Cooperative learning is a pedagogical approach which involves learners in their own learning by helping others learn and learning from others. Cooperative learning is a form of active learning designed to enhance individual learning via student group interaction (Johnson, Johnson, & Smith, 1991; Slavin, 1987). The foundation of cooperative learning is a structured, systematic instructional strategy, in which small groups work together to achieve a common goal (Cooper, McKinney, & Robinson, 1991). Students work together to maximize their own and each others’ learning within this small group configuration.

Successful cooperative learning relies on instructors who carefully attend to the structure of learning groups, design of interaction processes, and accountability (Cooper, 1990; Johnson et al., 1991). The collaboration inherent to the structure of cooperative learning provides “scaffolding” for mutual support and enables students to learn from each other (Millis, 2002). Considerable research demonstrates that cooperative learning produces higher achievement, more positive relations among students, and healthier psychological adjustment than do competitive or individualistic learning experiences (Johnson & Johnson, 1979). Also, students in cooperative structures perform better on questions involving higher-level thinking than their peers in traditional classrooms. Although cooperative learning techniques are used by thousands of teachers and have been thoroughly researched as instructional methods (Johnson et al., 1991; Slavin, 1987, 1990), its use is not widespread. Execution remains the primary challenge to successful cooperative learning (Benjamin, 1991; Hibgee, Ginter, & Taylor, 1991). In summary, cooperative learning provides a nonthreatening learning context for interaction among students who exchange alternative perspectives, ideas, and critical feedback. Cooperative learning is amenable to distance education when structured to provide student interactions and critical feedback.

Researchers have developed a number of models to distinguish different levels of knowledge over the last 5 decades. Bloom (1956) provided the first educational taxonomy to identify educational outcomes in the cognitive domain which deals with recognition of knowledge and the development of intellectual abilities and skills. He identified three main cognitive outcomes: remembering, thinking and problem solving. Building on Bloom’s cognitive outcomes, Kurfiss (1988) suggests that three types of knowledge interact in the process of thinking critically and solving problems: (1) declarative knowledge—knowing the facts and concepts in the discipline; (2) procedural knowledge—knowing how to reason, inquire, and present knowledge in the discipline; and (3) higher cognition—setting problem-solving strategies and goals, assessing lines of inquiry, determining when additional information is needed. More recently, Shavelson and Huang (2003) propose four types of domain-specific knowledge:

1. declarative knowledge—knowing that;

2. procedural knowledge—knowing how

3. schematic knowledge—knowing why; and

4. strategic knowledge—knowing when.

The three models by Bloom (1956), Kurfiss (1988), and Shavelson and Huang (2003) over the last 5 decades make the same distinctions on three levels of cognitive knowledge. This study will use the Kurfiss model as its conceptual framework. A self-paced independent study distance education course in public health may be well suited for declarative knowledge, but other pedagogies are necessary to help improve procedural knowledge and higher cognition.

Ernest Boyer (1990) precipitated a critical examination and vigorous discussion of higher education pedagogy by proposing a powerful new paradigm of teaching and research. This groundbreaking work went beyond the common “teaching versus research” argument and highlighted the need for scholarship of teaching. Boyer distinguishes between the research of a discipline (which he calls the scholarship of discovery), and the education in a discipline (which he calls the scholarship of teaching). The former encompasses the conventional focus on research and scholarly investigation within a core discipline, while the latter refers to the transmission of knowledge and examines teaching effectiveness and student learning within a discipline. This focus on teaching, now known as the scholarship of teaching and learning, advocates research of effective teaching and learning on a discipline-specific basis and the development of new teaching processes consistent with current knowledge about how students learn.

After Boyer (1990) started the debate to move discussion beyond teaching versus research, others provided a means to assess scholarship. For example, Glassick (2000) and his colleagues developed six criteria to study scholarship in both research and teaching:

1. Clear goals: To be successful, the educator must show clarity of goals. Did the professor clearly state the objectives of the course? Did the proposed objectives agree with those actually taught?

2. Adequate preparation: Was the instructor well prepared for each class? Were class materials well prepared?

3. Appropriate methods: Methods and procedures make all the difference from the logic of the syllabus to pedagogical procedures to evaluation.

4. Significant results: The scholar should clearly seek to measure the significance of the results of teaching.

5. Effective communication: Ideas and methods must be shared. Did the instructor speak with good expression?

6. Reflective critique: The scholar should reflect, be self-critical, ask for and value the opinions of peers regarding teaching methods. Careful evaluation enriches teaching projects by using lessons learned to inform new projects.

Glassick’s six criteria apply readily to the conduct of general research in public health, but have yet to be applied to research of teaching and learning in public health.

In applying Boyer’s (1990) concepts to public health, Aday and Quill (2000) observe that schools of public health (like most disciplines) have a focus on research, rather than teaching effectiveness. Aday and Quill took a major step toward scrutiny of public health graduate education with the introduction of a framework to assess practice-oriented scholarship in schools of public health. Specifically, they identified criteria and evidence for teaching effectiveness within the three core functions of public health: assessment, policy analysis, and assurance. Consistent with Aday and Quill, the Association of Schools of Public Health (Demonstrating Excellence, 2004) recently released its recommendations for demonstrating excellence in practice-based teaching for public health. Practice-based teaching is intended to make student learning meaningful and relevant, which results in the development of critical thinking and problem solving skills.

To effectively apply scholarship of teaching and learning concepts, public health can begin by recognizing the synergy present between the research of the field and the substance of the scholarship of teaching. The scholarship of teaching and learning movement shows promise to bridge the divide between public health research and the teaching of public health. It does so by treating teaching as a form of inquiry (Huber, 2002). While most faculty are concerned about the quality of student learning, the principles of scholarship of teaching and learning encourages educators to examine teaching and learning in their own classrooms, and to use what they discover to improve their teaching practice (Huber & Morreale, 2002). Scholarship of teaching and learning advocates promote the same research rigor and inquiry in teaching efforts as in research of the discipline. Many educators continue to use traditional ways of teaching and learning and are relatively unaware of education research in the backwaters of their field (Huber, 2002). While public health researchers have developed powerful methods of scientific inquiry in all areas of public health, we have not achieved similar progress in developing inquiry into our effectiveness of teaching.

Two fundamental concepts of scholarship of teaching and learning that deserve closer discussion for this study include constructivism and deep learning.

Constructivism is a learning theory that has profound implications for pedagogy. Constructivism is based on the precept that students learn by constructing knowledge, not by receiving knowledge from others. This is not to say that lectures are ineffective, but that other strategies are more effective in light of the underperformance of traditional pedagogy (Huba & Freed, 2000; Shulman, 2002). The constructivism process of learning begins with student engagement, which leads to knowledge and understanding, which produces a capability of performance or action (Shulman, 2002). Constructivism advocates engaging students to be actively involved in learning, to take ownership, to explore what they already know about a topic, then to explore how new information fits into their existing framework. Under constructivism theory, students talk about new ideas and practice new skills (Huba & Freed, 2000). Learnercentered constructivism uses rubrics, learner assessment, and activities designed to necessitate interaction and cooperation. This critical reflection on practice and understanding leads to higher-order thinking, which ultimately leads to judgment and commitment (Shulman, 2002).

A prominent trend in higher education is to achieve deep learning, as opposed to surface learning (Leamnson, 1999), because deep learning produces more long-term learning results (Rhem, 1995). When students enter the learning process they bring preexisting concepts into the experience. Deep learning engages students in reshaping these concepts, discovering new connections, and integrating new ideas. Students learn better when they discover their own connections rather than having such connections imposed (Bloom, 1956). The best combination for deep learning is: a well thought-out assignment, cooperative activity, and targeted feedback that achieve repetition of various approaches. As such, deep learning is most effectively achieved through cooperative learning.

From the above discussion, it is becoming clear that processes of teacher-centered lectures, traditional course evaluations, and objective tests alone do not form the optimal teaching and learning process. The three concepts—cooperative learning, constructivism, and deep learning—mesh to form a powerful combination for developing pedagogy to improve the effectiveness of learning in public health and have immediate application to public health distance education.

Distance education is becoming a major component in graduate-level and professional education offerings. Distance education is broadly defined as a “course delivered off-campus using live or recorded TV or audio, CD-ROM, or a computer-based system, such as the Internet” (Jensen, 2000, p. 44). A study of 2.7 million graduate students and professional students profiles the extent to which graduate and professional students enrolled in U.S. institutions in the 1999-2000 academic year participated in distance education. The report, released by the U.S. Department of Education National Center for Education Statistics, found that 10% of graduate and professional degree students participated in distance education. Of those, 38% reported that the entire program was taught through distance education (Jensen, 2000). Among graduate and professional students who participated in distance education, the greatest number used the Internet (67%), followed by live TV or audio (43%), and prerecorded TV or audio (28%) (Jensen, 2000). When asked about levels of satisfaction in distance education courses, as compared to other courses, 51% of students were equally satisfied, 22% were more satisfied, and 27% were less satisfied (Jensen, 2000).

However, despite the growing popularity of distance education courses, it appears that a small number of faculty and educational institutions engage in a disproportionate amount of distance education. While educational institutions’ participation in distance education has grown rapidly (Lewis, Snow, Farris, & Levin, 1997-1998), the percentage of instructional faculty and staff at degree-granting institutions who taught distance education classes remains relatively small—5% in public doctoral institutions and 4% in private doctoral institutions (Bradburn, 2002).

Distance education in graduate-level public health programs has also grown rapidly, and in addition to the usual motives for distance education, has the added impetus of addressing workforce development goals. Research concerning distance education in public health has two separate themes: (1) the factors creating demand and why distance education is important, and (2) evaluation and outcome studies of distance education programs. The breakthrough report by the Institute of Medicine, The Future of Public Health (1988), demonstrated a significant need to develop formal management skills among public health professions. In response, public health graduate programs developed comprehensive management curriculum both for the traditional classroom setting (Porter, Johnson, Upshaw, Orton, Deal, & Umble, 2002), and for Internet-based distance education (Umble et al., 2003). Students cite the flexibility distance learning gives to students to manage their time (Blakely & Curran-Smith, 1998; Umble et al., 2003) and easy access to the course Website from home, office, or other remote location (Steckler, Farel, Bontempi, Umble, Polhamus, & Trester, 2001) as the leading motivations for enrolling in public health distance education courses.

Evaluation studies of public health distance education programs are generally quite positive. Studies report consistently favorable results in the efficacy of distance education for meeting the needs of public health students and professionals (Cannon et al., 2001; Farel, Umble, & Polhamus, 2001; Steckler et al., 2001; Umble et al., 2003). However, the evaluation studies focus on student satisfaction rather than the pedagogical effectiveness of learning. For example, a set of articles describing the program development and evaluation of an interdisciplinary masters of public health program contains an excellent description of the program development and a subjective, retroactive assessment by the participants who rate their confidence in performing certain skills before and after the program (Cannon et al., 2001; Umble et al., 2003).

The literature review indicates that current research in public health distance education lacks studies that address the effectiveness of teaching pedagogies on learner outcomes. It is recognized that instructional methods and course designs are important components (Umble et al., 2003) and that some teaching methods do not foster interaction among students and faculty (Cannon et al., 2001); however, studies in public health distance education have not been designed to research these issues. It is also recognized that the efficacy of teaching methods for online distance education must be pursued (Farel, Pfau, Paliulis, & Umble, 2003) and special effort must be made to foster substantive student-student and student-instructor interaction because some teaching methods and designs fail to do so (Cannon et al., 2001).

Distance education is popular, in part, because it promotes self-study and self-pacing. However, distance education may be problematic when considering the three levels of cognitive knowledge. For example, it is fairly easy to assess remembering through objective tests; however, it is difficult to create experiences leading to critical thinking and problem solving. Moreover, it is challenging to systematically assess the effectiveness of this pedagogy. The above discussion suggests that students who participate in cooperative learning groups will achieve deeper learning, higher critical ability, and improved problem solving when compared to a control group engaged in selfpaced independent study with no cooperative learning. However, creating cooperative learning and determining its impact in a controlled study have been lacking.

This study is an experimental design to assess the impact of cooperative learning on different types of knowledge acquisition in a Webbased distance education course. This section deals with the research methodology and consists of four parts, including (a) research hypotheses, (b) course description, (c) study design, and (d) measurements.

The discussion above identified three research questions that can be converted to three testable hypotheses.

Can declarative knowledge acquisition (remembering) be achieved equally through independent-learning and cooperative learning pedagogical techniques?

Declarative knowledge acquisition will be comparable between the two groups. Meeting learning objectives is a lower level type of knowledge and learning skill that does not require critical thinking. We expect to see no difference in performance on traditional evaluation forms of objective testing, such as multiple choice tests.

Does cooperative learning improve procedural knowledge (critical thinking, problem solving)?

Learning in distance education is better accomplished via cooperative learning than a self-paced independent learning environment. Cooperative learning communities result in a more worthwhile educational experience, create conditions for procedural knowledge acquisition, and improve critical thinking skills. We expect to see improved learning in paired and group activities as compared to independent study.

Does cooperative learning improve performance on higher cognition (setting problem solving strategies and assessing lines of inquiry)?

Critical reflection requires learner connection and engagement with the material. We expect to see superior performance on assessments that require analysis, synthesis, critical thinking, problem solving, and professional presentation of ideas from students who participate in cooperative learning.

The study population for this research study is a graduate class of 47 students enrolled in an 8-week course in public health management offered online through the University of Minnesota School of Public Health in the summer of 2004.

The public health management course forms part of the core courses students must complete to receive a masters of public health degree. This course specifically addresses the need for qualified, adequately-trained public health managers and leaders, as described in the Institute of Medicine’s 1988 report, The Future of Public Health. Through seven teaching modules, the course addresses the four basic functions of management: planning, organizing, motivating, and controlling. At the conclusion of the course, students are able to:

1. Develop strategic plans, determine organizational missions, and develop operational plans.

2. Use theories of organizational structure and professional practice to design effective and efficient organizations, and explain how organizational concepts are important to achieve public health goals.

3. Monitor program performance; apply human relations skills to motivating, communicating, coordinating and resolving conflicts; evaluate and manage individual performance; lead organizations and teams.

4. Apply variance analysis, manage programs within budgetary constraints, and prepare proposals for funding from external sources.

The online distance education course is delivered via WebCT, a University of Minnesota-wide Internet interface used to deliver online course information to undergraduate and graduate students. WebCT tools used during this course included: document uploading, discussion boards, internal e-mail, and quiz and survey functions. All students had access to the professor and teaching assistant via WebCT internal e-mail, University of Minnesota e-mail, telephone, or face-to-face contact. For learner comprehension measurement, the course relied on short activities, in-depth case studies, quizzes, and a final project. During the study, all 47 students, regardless of group assignment, received the same course material, assignments, and tests, and were graded on the same scale. The pedagogy differed on the basis of independent learning and cooperative learning through groups.

This study is an experimental model with random assignment into a control group and treatment group. Each student (n = 47) enrolled in the distance education course in public health management during summer session 2004 was included in this study. Each student was notified of the study about pedagogical methods, but was not informed about the specific research objectives or into which group he or she was assigned. All students were offered the option to not participate in the study. No student declined to participate. Through a table of random numbers, subjects were randomly assigned to the control group of independent self-study (n = 25) or the treatment group of cooperative learning (n = 22). Two students discovered the course was conducted differently and inquired. They were reminded of the informed consent and explained that different methods of instruction were used. They were offered the opportunity to switch to the other groups, but both declined.

The 25 students in the control group completed the distance education WebCT course as a self-paced independent study. At the onset of the course, the control group students received instructions on how to navigate the course Web site, how to complete lesson and reading materials, how to take online quizzes, and how to complete and submit assignments. Students worked completely independently. A recommended schedule was provided, but no deadlines imposed. Students were responsible for self-pacing their course activity. Students completed all assignments, case studies, activities, and the course project individually, and submitted everything directly to the instructor on an individual basis. Students did not interact with each other in the course and did not have the ability to contact each other via WebCT to ask questions, group problem-solve, or learn by role modeling. All questions, concerns, and problems were communicated to the instructor or teaching assistant through WebCT, e-mail, phone calls, or face-to-face meetings. To deliver feedback on assignments, the instructor used the comments window included in WebCT’s upload documents function. Only individual students had access to personal feedback. Performance feedback was delivered either via WebCT’s internal e-mail or the University of Minnesota e-mail system.

The 22 students in the cooperative learning group participated in a group learning environment. The pedagogical goal of this group was to create a community of learners who learned from each other, despite asynchronous interactions influenced by distant geographical locations.

Six cooperative learning groups were developed for the study. Each cooperative learning group was created by a two-stage random assignment. First, the 22 students were randomly assigned to the treatment group. Second, the students were assigned to work groups of four students each, with six work groups total in the experimental group. All coursework was completed within work groups, with the exception of the final project assessment. For activities, each student was assigned to a partner from within his work group with whom he discussed and submitted assignments as a partnership. For case studies, the entire team worked together to analyze the case and jointly submit the case study analysis. Students were responsible for discussing and submitting assignments on a public discussion board, which was divided into minidiscussion boards specific to each of the six work groups. Students could access the discussion board at any time to view their group members’ and other classmates’ work. Table 1 shows an analysis of recorded interactions within each cooperative group. The table shows a total of 675 discrete interactions occurred among students during the 8week course; there was an average of 31 interactions per student, ranging from a low of 18 interactions per person in a cooperative group to a high of 48 interactions per person in a cooperative group (see Table 1).

The course instructor actively participated in the online discussions, providing encouragement, suggestions for further inquiry, and feedback. The instructor delivered feedback on assignments by privately replying to group assignment submissions on WebCT’s discussion board. Thus, feedback responses were available to individual groups, but not the class as a whole. The instructor also communicated with students about individual class performance via WebCT’s internal e-mail or the University of Minnesota e-mail system.

Education researchers have done extensive work on models of learning, and all have some type of gradation from lower-level to higher-level learning. This study examined the effect of teaching pedagogy on three different levels of cognitive knowledge: (1) declarative knowledge (remembering), (2) procedural knowledge (knowing how to reason and inquire in public health management), and (3) higher cognition (critical analysis and problem solving). A distinct measurement technique was developed for each knowledge level to determine the impact of learning.

Declarative knowledge was assessed through multiple-choice quizzes designed to test objective knowledge of course content. The declarative knowledge assessment dealt with recall and recognition of numerous public health management concepts. Objective quizzes were administered five times throughout the course and completed individually by all students in both the control and the treatment groups. All quizzes were delivered at the conclusion of lesson modules, administered online, and reviewed by the researchers.

Procedural knowledge was assessed through seven activities designed to elicit skill development in reasoning and inquiry. The activities addressed practical problems faced by public health managers, such as public health ethics, environmental audit, organizational structure, employee motivation, managerial control, planning, and strategy. The activities were rated on a 5-point scale, graduated from low to high. Throughout the course the instructor gave feedback on the activities to all students in both the control and the treatment groups.

The study design lacked blinded assessment for the procedural knowledge measurement (activities). Although the course instructor was not blinded to the students’ identities, no effort was made to separate the students into their respective pedagogical groups during performance assessment. Therefore, in measuring performance, the course instructor was unaware to which pedagogical group the student belonged.

Higher cognition was measured by four case studies designed to elicit skill development in problem solving and assessing lines of inquiry. Similar to the activities, the case studies were rated on a 10-point scale, graduated from low to high. The case studies were extensive descriptions of significant problems confronted in public health management in these areas: (1) strategic planning for a safety net health care system in a large metropolitan area; (2) a problem with organization structure and design and lack of accountability and communication in an environmental water department; (3) a motivation problem with merit pay and lack of team functioning in a city department; and (4) a continuous quality improvement problem in a family planning clinic in international health context. The learners in both pedagogies analyzed the four case studies and developed solutions based on applications of public health concepts in the respective module.

The case studies present real-life public health problems through the presentation of an authentic problem encountered in health care management supplemented with data and relevant facts. Each case study includes specific questions to be addressed by students, whether individually (independent learners) or as a group (cooperative learning). Learners are responsible to analyze the problem and develop a problem statement. Using a problem-solving methodology, learners then develop solutions and present an executive summary. Measurement of performance is based on how well the students assess the situation, address the problems at hand, and develop practical, pragmatic solutions. An objective rubric was applied by the course instructor to assess how adeptly the student(s) present their findings in a cohesive, professional manner.

Similar to the procedural knowledge measurement, the measurement of higher cognition also lacked an element of blinded assessment. However, again reflective of the procedural knowledge assessment, the course instructor made no effort to sort students into their respective pedagogical groups prior to assessing their performance. As such, potential assessment biases were mitigated.

We conducted several analyses for the three research questions to examine the performance results in the three levels of knowledge to determine the effectiveness of the cooperative learning group pedagogy in distance education.

Five of the learning modules had multiplechoice tests to measure declarative knowledge (learning comprehension and remembering). Table 2 shows a comparison of cumulative scores for multiple-choice tests administered following completion of five of the unit modules. Both groups performed very well on the multiple-choice tests and indicate high mastery of the lesson concepts and content for public health management. The five multiple-choice tests included 49 questions total. The control group answered 97.9% of the questions correctly, and the experimental group answered 98.9% of the questions correctly. No significant difference in performance existed between the two groups as students in both the control group and the treatment group performed at high levels. The analysis of multiple-choice tests independently produced results consistent with this finding (see Table 2).

The lack of differentiation could suggest that the pedagogies had a similar effect, or it could reflect the shortcoming of the measurement methodology. There was no time limit on the multiple-choice quizzes and no method to determine the extent of review that could have occurred while students took the quizzes. In any circumstance, the recall learning goals were achieved by both pedagogies, as reflected by the high scores by both groups.

The impact of cooperative learning on procedural knowledge (learning how to reason and inquire within public health management) was examined on the basis of seven activities performed throughout the entire course. These activities were designed to provide the learner with a challenging public health problem requiring reasoning and inquiry using public health methods and concepts. Activities were rated on a 5-point scale, graduated from low to high. The performance comparisons between self-study and cooperative learning pedagogies for activities are shown in Table 3. The self-study group had a composite score across seven activities of 4.7, which differed significantly from the cooperative learning group’s score of 4.9. This is approximately 5% higher performance by the cooperative learning group (see Table 3). As shown in the Table 3, the cooperative learning group performed better than independent study learners in six of the seven activities dealing with practical problems faced by public health managers in the areas of public health ethics, environmental audits, organizational structure, employee motivation, managerial control, planning, and strategy.

It is also noteworthy that the performance on the cooperative learning is less on the first cooperative activity in Lesson 1, (4.7 compared to 4.9) but is greater on all remaining activities. Upon reflection, this finding is not surprising. In cooperative learning, it take stime for group members to assign roles, perform duties, and develop an effective working collaboration with others in the group. In a self-study approach, none of the activity is necessary. Therefore, the lower performance for the first activity could reflect, in part, the start-up challenges working in a cooperative group, as opposed to working independently. However, once the start-up coordination difficulties are addressed, the cooperative learning groups performed better than self-study groups on both activities and case studies.

Although the difference in procedural knowledge acquisition is slight between the cooperative learning group and the self-study group (mean 4.9 compared to 4.7), this difference is in the expected direction and does indicate some influence of cooperative learning. However, both the treatment group and the control group scored high on this assessment. Given that the control group scored 4.7 out of 5 on the task, there were only .3 possible points of improvement for the experimental group to attain. This suggests that the assessment instrument may lack sensitivity and discrimination in measuring procedural knowledge. An important implication for further research is to develop assessments with greater discrimination of procedural knowledge.

The impact of cooperative learning on higher cognition (skill development in problem solving and assessing lines of inquiry) was examined through case studies of significant problems confronted in public health management in four areas: strategic planning, organizational structure, team functioning, and continuous quality improvement. Case studies were rated on a 10-point scale, graduated from low to high. Table 4 shows the performance on higher cognition as measured by the four case studies. The cooperative learning group had a consolidated score of 9.5 while the self-study group had a consolidated score of 9.0. Again, this represents approximately 5% improved performance in the cooperative learning pedagogy for procedural knowledge (see Table 4).

Similar to the finding in procedural knowledge acquisition, the difference in higher cognition is slight between the cooperative learning group and the self study group (mean average 9.5 compared to 9.0). Likewise, this difference is in the expected direction and does indicate some influence of cooperative learning. However, both the treatment group and the control group scored high on this assessment. Given that the control group scored 9.0 out of 10 on the case studies, there was only 1.0 (10%) possible points of improvement for the experimental group to attain. This suggests that the assessment instrument may lack sensitivity and discrimination in measuring procedural knowledge. An important implication for further research is to develop assessments with greater discrimination for higher cognition knowledge.

Multiple choice tests in a distance education class have useful applications for determining the extent of declarative knowledge acquired. The popularity of multiple-choice tests among graduate school faculty in public health stems from a variety of reasons: multiple-choice tests measure mastery of content, check that learner’s review material, and are easily administered and processed. In the absence of face-to-face interaction between instructor and student, the multiple-choice test helps determine if a distance education learner is involved, engaged, and covering the material.

Learners in both learning pedagogies (selfstudy and cooperative learning) showed the same level of performance on declarative knowledge acquisition. We expected declarative knowledge acquisition to be comparable between the two groups. Meeting declarative knowledge learning objectives is a lower-level knowledge and learning skill that does not require critical thinking, so there is no reason to expect differences in performance based on the pedagogical approach.

The high performance on multiple-choice questions throughout the course by both groups suggests a conscientious effort on part of the learners to master the material in the modules and acquire a solid base of declarative knowledge—knowing the facts and concepts of public health management.

It is noteworthy to mention there was no cooperative learning component associated with the multiple-choice quizzes. In both groups, the declarative knowledge measured by the five multiple-choice quizzes reflects productive independent self-study of the module content material.

Procedural knowledge is considered a higher level knowledge than declarative knowledge. It consists of the ability to know how to reason, inquire, and present knowledge in a discipline. Unlike declarative knowledge, we expected to find differences between the cooperative learning and self study pedagogies. After a slow start in the first activity, the cooperative learning group showed consistently improved performance when compared with the independent study group. We hypothesized that learning in distance education is better accomplished via cooperative learning than a self-paced independent learning environment. The literature suggests that cooperative learning communities result in a more worthwhile educational experience, create conditions for procedural knowledge acquisition, and improve critical thinking skills. The research findings in this study support that when learners are paired to perform activities together and share results with their study group, they perform slightly better than learners who in the independent study group.

Higher cognition in Kurfiss’ (1988) scheme is the ability to set problem-solving strategies, assess lines of reasoning, and determine when additional knowledge is needed. Analysis of cases represents the most extensive form of assessment in this research study. We reason that the students in the cooperative learning group acquired higher level critical thinking skills to a greater extent than the self study group by virtue of performing seven activities and four case studies together. In other words, the cooperative learning pedagogy created conditions for better higher cognition capability than the self study group. When the work products are done collaboratively and results made public, higher cognition appears to improve a result. Consistent with Shulman’s criteria, learning begins with student engagements, which in turn leads to knowledge and understanding. Once someone understands, they become capable of performance or action. Critical reflection on practice and understanding leads to higher-order thinking. This leads to judgment and commitment (Shulman, 2002).

We expected the cooperative learning groups to display better performance in distance education because cooperative learning has been shown in the classroom setting to result in improved “scaffolding” of problem solving skills and higher cognition abilities. The study findings show that cooperative learning did produce improved performance on higher cognition when compared to the self-study pedagogy. When learners participate in cooperative learning, there is improved learner connection and engagement with the material, which results in superior performance on problems that require analysis, synthesis, critical thinking, and problem solving. The case studies present real-life public health problems encountered in public health management, and learners in the cooperative study group exhibited better performance on higher cognition activities compared to the self-study group.

Distance education using Web-based technology is an important trend in public health graduate-level education. It is important to examine the effectiveness of teaching pedagogies as this trend grows and becomes more commonplace. This study shows that cognitive learning outcomes are affected by cooperative learning when compared to self-study. While there was no difference in declarative knowledge and procedural knowledge, the cooperative learning pedagogy resulted in a 5% improvement in procedural knowledge and higher cognition capabilities compared to the self-study pedagogy.

Web-based technology has created a tremendous opportunity to make public health education more accessible and responsive to public health professionals. However, there is also a challenge for educators to discover and document the effect of different pedagogies on teaching effectiveness and learner outcomes in distance education. Graduate-level public health education involves the acquisition of specialized knowledge, critical thinking, the ability to assess lines of inquiry, and problem solving strategies. Cooperative learning is a pedagogical approach that gets learners more involved in their own learning to a greater extent than self-study. This research shows that cooperative learning experiences can be incorporated into distance education and results in advanced knowledge acquisition in the areas of public health inquiry, critical thinking, and problem solving when compared to self-study approaches.

Distance learning based on Internet instruction has challenges when compared to traditional classroom instruction, foremost of which is the amount of communication lost by nature of the media. Two points stand out. First, a substantial amount of communication is nonverbal, including voice intonation, facial expression, and body language. Second, learning is both a social and individualistic process. A cooperative learning pedagogy in distance education may help mitigate or compensate for such communication lapses and social loss by increasing interaction among learners.

Graduate-level educators in public health endeavor to do more than simply transmit facts and concepts about the discipline. As a practice discipline, it is also essential to help learners acquire critical thinking and problem solving skills. In essence, we acculturate students to become practicing professionals or in the case of professional education, move professionals to higher learning and skill levels. The scholarship of teaching and learning agenda has the potential to produce improvements in public health teaching in higher education and distant education. This study suggests that it is possible to design, implement, evaluate, and disseminate research results on effective teaching techniques and what helps students in public health learn.

The authors thank staff at the Digital Media Center of the University of Minnesota for their assistance in developing teaching innovation and encouraging the study of teaching pedagogy. We also express our gratitude to the anonymous reviewers who provided very helpful comments and suggestions.