ENHANCING GRADUATE STUDENTS’ PERFORMANCE AS THREADED DISCUSSION LEADERS IN A WEB-BASED PROPOSAL WRITING COURSE Free

Data presented in this study were collected over four academic years in a graduate course titled “Grant Writing in Instructional Technology” at a mid-sized southern university. Participants attended a weekly lecture and were assigned reading from the following texts: Winning Grants Step by Step, by Carlson (2002), the Grantwriter’s Internet Companion, by Peterson (2001), Developing Your Case for Support, by Seiler (2001), and a Course Handout Packet (2001) developed by a professor at the university. Threaded discussions were then used to ask questions about the lectures and readings. Students were given a minimum number of posts that were required each week. Each student received a rubric that would be used to grade his or her posts, based on: (1) showing reflection on the topic of the week, (2) being related to the flow of the discussion, (3) being of sufficient length to support forward an argument and support an idea, and (4) making reference to the reading for the week or other resources that are on topic. Each student served as discussion leader once during the semester. Before serving as discussion leader, students were trained how to use the telementoring taxonomy (see Appendix A).

The course was designed to give students an opportunity to find grant funding sources, write a grant, and learn how to submit a completed funding proposal to a governmental agency, corporation, or foundation for consideration. The class investigated current issues and topics related to research and grant writing in educational technology.

All students in the classes were given an opportunity to participate in the study with the assurance that participation would have no effect on their grade. Participants were mostly graduate students seeking degrees from the College of Education and Health Professions. However, students from business, agriculture, computer sciences, and music also attended. There were also staff and board members from local nonprofit organizations who audited the course. Students who audited the course were not part of the study. Thirty of the 39 students volunteered to participate. However, only 29 were interviewed, and only those results are reported here. Threaded discussions were available on WebCT, which is an online course shell. Of the 30 participants interviewed, all logged onto WebCT at home and 18 also logged on at work. Thirteen of the students who reported logging on at work reported using the computer at lunch or staying late to use the Internet connection at work because it was much faster than their Internet connection at home. Most of the interviewed students (24) took the course because it was required as part of their graduate course of study; the remaining students cited a desire to learn more about grant writing for their current job, plans to go into academia where the skill will be needed, and requirement by supervisors, as reasons for taking the course.

A standardized open-ended interview format was used to collect data. The interview consisted of three demographic and 19 open-ended questions. Two of Patton’s reasons for using the standardized open-ended interview were important to this study: first, the exact instrument will be accessible for scrutiny and, second, the interview should be highly focused so that interviewee time is carefully used (Pat ton, 2002). Participant responses to the interview questions were analyzed to ascertain perspectives on the discussion groups. Each participant interview was assigned a number to protect the confidentiality of the participants and to make the documents easily identifiable.

In addition to the interviews, data were gathered from “threaded discussion groups.” Threaded discussion took place during 14 weeks of the semester. Students serving as online discussion leaders used the taxonomy to guide them in making posts. Students were assigned to serve as “student of the week” at least once during the semester. Due to the size of the class, occasionally two students would share the discussion leader responsibilities. Students of the week were selected from the class roster by a purposeful self-selection. Data included the text of the threaded discussions and transcripts of the interviews.

Discussions were started with a case study provided by the professor that was based on the reading for that week. The case studies put the students in a real-life situation as a grant writer in which they would need to apply information from the class readings and class sessions up to that date in the course. A task was assigned requiring a product of some type to be produced from the discussion, such as a memo to the executive director on possible funding sources, or a project budget. The student serving as discussion leader led the discussion of the case study and tried to reach a consensus of how to address the key issues and what the final product should look like. Discussion topics ranged from editing an existing letter of intent to role-playing assigned foundation board member characters when making decisions about which proposals to fund.

All participants were interviewed during the 3 weeks after they served as discussion leader. The interviews focused on: (1) their experience as discussion facilitator, (2) the usefulness of the mentoring taxonomy, and (3) their experience as discussion participant. Students were given an opportunity to read transcripts of the interviews and make changes. Online discussions were captured and analyzed for effectiveness and usefulness in facilitating discussion and learning.

The constant comparative method was used to analyze these data, and open codes led to axial codes and, ultimately, to higher-level axial codes. This method required constant comparison of data to and against itself to ascertain its meaning and to generate theories and genres. As each element of meaning was chosen for analysis, it was compared to all other elements of meaning and then categorized and coded with similar elements (Glaser & Strauss, 1967).

Qualitative software was used in the analysis of these data. Discussion group postings and transcripts of interviews were searched for common terms and patterns. The results of these searches led to recognition of emergent themes existing in these data.

A taxonomy (see Appendix A) was developed as a scaffolding system for threaded discussion facilitators to support student learning (Brescia, 2003). This taxonomy was developed from an analysis of mentoring characteristics that are commonly used in traditional face-to-face mentoring. The first version of this research was developed for use by university faculty (Brescia, 2003).

The telementoring taxonomy was presented to the students during the first class. The purpose of the strategies was explained, and they were told when and how to use the taxonomy to help them meet their responsibilities as discussion leaders. Students were given an opportunity to practice the strategies and ask questions during this training session. Also, students learned how to use the WebCT threaded discussion technology. During the first week’s discussion, the instructor took the role of discussion leader and modeled effective use of the taxonomy. Before each discussion started, the student of the week for the next discussion was reminded, via e-mail, that a new case study was posted and he or she should begin the duties of discussion leader.

During the 42 discussions covered in the study, discussion leaders made an average of 7.2 telementoring posts. Those posts stimulated an average of 32.1 posts from other students, with an average length of 53.0 words. The ratio of discussion leader posts to student posts was 1 to 4.4. Every discussion leader made a post that was less than a dozen words, these being “feedback” posts, and six posts were made that contained over 300 words. The average number of words per discussion leader post was 55.3. Word counts did not include any words quoted from previous posts. In all instances, the discussions were divided into three or four major questions, developed by the instructor to foster examination of the main issues presented in the readings and the case study. There were a few occasions in which students started threads outside these major questions, but those threads were created to address a particular issue that related to the issues presented in the case study.

Table 1 illustrates the wide variation in number of posts made during discussions by the discussion leader. If one discussion leader made an usually low or high number of posts, it affected overall totals. One discussion leader posted only three times, and another posted a high of 16 times. Also, the discussion leaders who posted the fewest times were also the ones who made the shortest posts.

Table 2 shows the students’ responses to the various discussion leader posts. There was no association between the number of posts made by the discussion leader and the number of student responses, as the discussions with the most and least posts had the lowest number of student posts, 22 and 24 respectively. Also, the discussion that had the most telementoring posts had a low average number of words per student response.

Table 3 shows how many of each telementoring strategies were used by the discussion leaders over the course of the discussions. The total number of posts was recorded at more than 100% of the total number of posts due to discussion leaders reporting that they used more than one strategy per post. The discussion leaders used Nourish Good Ideas (NGI) more than any other strategy. In some cases, it was used almost to the exclusion of other posts. Almost one quarter of the posts were NGIs. The second most frequent strategy used was Summarize (SUM), used at least twice in each discussion. Discussion leaders saw summarizing as an important way to show how much ground had been covered in the discussion to that moment and explaining what still needed to be done to complete the task. The third most-used strategy was Feedback (FEED), a strategy similar to NGI, except it contains only a simple positive feedback message (e.g., “good post”). On the other end of the spectrum, the Frame Task (FT) strategy was used sparingly. Modeling Good Analysis (MGA) was used 20 times, and Recommending Resources (RR) was used 57 times. In those discussions, the MGA and RR strategies were closely linked because discussion leaders often used an MGA posts to RR, particularly resources that could be found on the Web by clicking on an active link created in the post.

Finally, two strategies were not used at all; Champion Lost Ideas and Stating Opinion. The category of Stating Opinion was added to the taxonomy because, in previous research, students often pleaded with the professor to obtain a personal opinion, and it may be that when students know the discussion leader is another student, they stop seeking a “right” answer.

The percent of usefulness of posts indicated that discussion leaders were thoughtful about the effectiveness of their posts. Several said “perhaps I could have done better here.” They were ready to accept the blame that the discussion they led was not as successful as it could have been due to their ineffectiveness as a discussion leader. Ninety-two percent of the discussion leaders said that they thought the taxonomy made it possible for them to be more effective as discussion leaders and believed that using the taxonomy would make them more effective as member of future online discussions. As one commented, “This was great. If I ever do a discussion again I’ll dig this up” (pointing to the taxonomy handout).

Analysis of the interview data indicated that the more closely students followed the taxonomy, the more successful they believed their performance was and the more likely they were to make posts. Students were of the opinion that, by using the taxonomy, they received guidance in “making posts that went beyond just being another student. It (the taxonomy) made me feel like I could take charge of the discussion.”

Students agreed that NGI was an important strategy. They used it often because they thought they would be more productive if a discussion leader tried to nourish their ideas. One student discussion leader said:

Another student summed up her use of the taxonomy as “helping me see the difference between participating and leading online discussions. These computer discussions are so new I’m not sure what to do” Several students who were novices to threaded discussions expressed similar thoughts, as not every student used the taxonomy for every post. “I forgot I was the discussion leader ... and when I remembered I just got on line and posted a few times before I remembered I was supposed to use the taxonomy. So, these first posts might not be part of the taxonomy.” Another student said he “just thought a post would be good here and I didn’t think about which strategy it was.”

While many discussion leaders used FB, most did not have much to say about those posts: “Oh, I was just telling them that they did a good job” and “that was a good post so I told her. I like just saying good job.” “Just trying to be a pal, ya know.” However, when they were the recipients of FB, they were more positive. “Yeah, that really let me know I was on the right track. I thought (student’s name) really gave us some great Web sites that week and I was happy the he (discussion leader) thought I did good.” Another student said “he knows this stuff better than I do and it made me feel good for him to say that.” Only one student expressed a negative feeling about FB posts, “I don’t know what that means. What did I do right? That was a long post (she had made) and all I get is, ‘good work’?”

When using SUM, the discussion leaders commented on how effective they thought it was in helping students come to some sort of consensus.

Another student said SUM “helps you know where you have been and what you can expect from the rest of the discussions.” As discussion leaders, a few students asked other students to summarize the discussion to that point so that they could get another student’s interpretation of what was happening. In those cases, the students were high performers and recognized expertise in another student that would help them as discussion leaders. “[Student’s name] had made some really intriguing posts and I thought she might like to summarize the discussion so far. She took the opportunity to send us off in a direction she was interested in but that was fun.” In every case in which SUM was used, the discussion leader thought it was helpful in continuing and improving discussion.

While Challenge Hypothesis (CHY) was used fewer times (n = 6), the students thought it worked well. “That just wasn’t right. I thought (student’s name) just missed the point there and I called her on it. The discussion really took off after that. I thought we did a good job of working on that one.” One student said “This thing (pointing at the taxonomy CHY strategy on his handout) really worked here because I wasn’t sure what to do and this really worked.” Another time it was used was by a discussion leader who only made three posts. He said “I didn’t like being discussion leader but I guess that was a good one (post). After I did that I didn’t have to post again.”

Some students were good at using RR and, as one student put it,

Another student said, “I just thought that (RR) was part of what I wanted to do as leader. It made me feel like I was able to show the other students that just because I’m not in education I know some stuff.” However, some students didn’t like when other students used RR. “What is the big deal with all these Web sites? I thought it was supposed to be a discussion.” Another said “I didn’t have time to look up additional resources. I just stuck with keeping the discussion going.”

The use of Modeling Good Analysis (MGA) was problematic for some discussion leaders. “My discussion was number twelve. I just felt that by that time in the semester if they didn’t know how to analyze the material it was too late. So I didn’t use it.” But other students appreciated the opportunity it gave them to take the role as discussion leaders. “I used that (MGA) a couple times as I remember. I thought we needed to think more systematically about what we were talking about.” One student said “This really put the pressure on me. As student of the week I didn’t want to mess up. I read a lot more about the topic and tried to help get things going.”

Some students had difficulty distinguishing between Question (QUS) and Asking for Clarification (AFC). In the end, it came down to if they intended to ask specifically for a clarification of a previous post. “I’m not sure. What do you think? (Interviewer indicates that it is not his place to tell them what their post is). Well, it must be questioning because, like, in the other one I asked about that other post and here I’m just asking a question.” Then some students used it several times. “You know me from class. I like to ask questions. That’s what I did.” Another said she was not comfortable with the idea of discussing something online and “asked for a lot of clarification” because she could not see body language or hear inflection.

In directions given to discussion leaders during the first class of each of the semesters, Encourage Reflection (ENR) was covered in great depth and for longer than any other strategy because students asked the most questions about it. However, it was not used often. “I just wanted her to reflect on what she had said and to back it up with a quote or back it up with something, maybe a link to something on the Web or something. I just didn’t want it to hang there.” Others felt the other students quickly learned that starting a post with “I believe,” and not providing support and showing reflection would get a quick response from the discussion leader.

As participants in discussions, graduate students let the discussion leader show the way. If a discussion leader was active, they posted more and better posts. If a discussion leader was mainly invisible to them, they responded with fewer and shorter posts. The only discussion in which this was not the case was during the first semester, when there were two discussion leaders and, due to the low number of student posts early in the discussion, they began to talk to each other and praise virtually any post made by students. This was also the only discussion in which the discussion leaders made almost as many posts as the rest of the class, 19 to 22. This discussion had another anomaly, in that one discussion leader post contained more than 300 words. That single post pulled up the overall average for number of words per post. Many of the discussion leader posts were short and only included positive feedback. The low number of responses and low number of words per post, in this particular discussion, may be attributed to this being at a point in the semester when students were focusing on their major projects and concerns in other classes. It may be that any threaded discussion will suffer due to outside concerns of students related to other courses and outside issues.

Students were asked during the interviews if success of their posts would be judged on how many replies followed that post and, if the posts included reflection, continued the discussion, and provided support for the arguments presented, how would they judge each post? As their response to the second research question, students indicated that they were most successful and felt comfortable using the three strategies Nourish Good Ideas, Feedback, and Summarize. Other strategies that were reported to be most successful were Question and Recommending Resources. In the case of Questioning, this strategy was used to help other students who were having trouble with a particular concept, and explanation often took several exchanges of posts before the issue was cleared up to the students’ satisfaction. Students also took pride in providing resources to the other students. Most of these resources were Web sites that had information about the topic being discussed, and that information often made its way into the discussions.

The data relating to the third question, “How do graduate students use the taxonomy?” are unambiguous. Students consistently do the following: first, they use it to provide praise and positive feedback to their classmates; second, they use it to summarize the discussion. Students believe that their classmates and they themselves need to know that they are making good posts and are moving toward addressing the important issues in the case. None of the feedback used was of a negative variety. When they saw that the group had reached consensus or were at an impasse on an issue, they summarized and encouraged the group to move on to other issues that needed to be addressed.

Overall, the number of posts made by discussion leaders was low. This is partly because no minimum number of posts was assigned, and some students only posted the minimum; two did not even make the minimum requirement. Monitoring and facilitating a threaded discussion takes time, and these discussions indicate that, without more directions, some students will take the opportunity to post only a few times. Seventy-six percent of all posts were made in the categories Nourish Good Ideas, Feedback, Recommending Resources, and Summarize. No other category was used more than five percent of the time. While some students used specific strategies more than did other students, this pattern was so prevalent that no other trends could be discerned.

Finally, results indicate that the more closely the students followed the taxonomy, the more successful those discussions were in reaching consensus, producing a product, and leading to students’ satisfaction with their experience as discussion leaders. During the interview process, those students who used the taxonomy sparingly were less happy with their performance as discussion leaders and believed that they would use the taxonomy if they had it to do over. Those students who used the taxonomy believed that it was useful for them and made it possible for them to lead the discussion to the desired conclusion.

These data reflect each student serving as discussion leader once. Perhaps, with practice, graduate student discussion leaders might become more proficient at implementing the telementoring taxonomy and improve both their number of posts and the effectiveness of those posts.

• Model good analysis: Show other students the kinds of behaviors you are looking for. Provide examples of good posts. For example, one of the attributes of asynchronous conferencing is the ability for conference participants to find references and quote from them online. By linking student ideas to the readings it is possible to model what is expected of the students.

• Clarify: Provide keys, links, framing statements, focusing statements, and examples from the readings and other resources that help the discussion avoid vagueness. The discussion should include references to the readings and other research, rather than discussing what students did on the job.

• Challenge hypotheses: Students will be developing hypotheses relating to actions to be taken relating to the Web projects they are working through. You may introduce some conflict into the proceedings by challenging those hypotheses. For example, bring up an instructional theory that is counter to the one the students are proposing. Another possibility is to challenge the context the students have constructed.

• Question: Ask students to think some more about what they are saying or conclusions they have reached. You can ask any number of questions to help the students explore an idea, clarify their thinking, reach a conclusion, or work on an assignment. Ex: Can you give some examples of this? How might you go about doing that?

• Asking for clarification: Similar to questioning but usually more specific. A student may have made a specific assertion that the class or the instructor is unclear about the meaning or theme. This is a simple request to make the post more clear, hopefully by providing more supporting references.

• Frame tasks: I will devise tasks and allow appropriate time for discussion and completion of those tasks. You must provide structure for the discussion to proceed toward a reasonable end. Once the task is clear, the Student of the Week should help the students to stay on task.

• Summarize: The Student of the Week guides the discussion along by summarizing key points, topics covered and those still needing to be covered, bringing the group to a point of convergence which they can move forward form. The Student of the Week should not always be the summarizer. The responsibility for this might pass to another student who see an appropriate place to summarize.

• Encourage reflection: Student should think about what should be done as much as how to do it. Help the students to reflect on what they have said in the discussion and how it may contribute to an outcome. By stating your opinion students can compare their own thinking to other students. Ex: What were you thinking about when you suggested doing that? Did your actions change after trying that design strategy? Did you ever stop to question your assumptions?

• Nourish good ideas: The Student of the Week markets the good points a student has made to the group or may intervene in a conflict to defend points a student has presented. The Student of the Week is trying to convince the discussion participants of the significance of particular ideas and is attempting to gain admission of those ideas into the conversation. By showing appreciation of student ideas, the Student of the Week gives support to those thoughts.

• Champion lost ideas: You may provide support for a student’s ideas as expressed in the conference. You can help a student’s ideas have visibility in the discussion by returning the discussion to ideas a student has presented but were not followed up on by the conference.

• Feedback: The Student of the Week provides simple positive or negative feedback. The Student of the Week is attempting to support or correct specific student behaviors. These posts provide encouragement and supportive feedback to the student. Ex: “Good post. I particularly like how you brought in the reading for the week”.

• Recommending resources: In face-to-face classes, students often provide other student with specific references to additional resources that are recommended for further, but not required, reading in the area being discussed. As Student of the Week point students to Web sites or readings that apply to the discussion.