DETERMINING E-LEARNING COMPETENCIES: Using Centra^™ to Collect Focus Group Data

E-learning provides educators the opportunity to deliver instruction to large numbers of learners or professionals who are unable to access residential programs due to time or place restraints. The increased use of computers and the Internet for information delivery has made online instruction possible (Savenye, Olina, & Niemczyk, 2001). E-learning, also known as online learning, is the means of delivering instruction via the Internet. E-learning has been defined as “the appropriate application of the Internet to support the delivery of learning, skills and knowledge in a holistic approach not limited to any particular course, technologies, or infrastructures” (Henry, 2001, p. 249). Text, graphics, audio, video, animation, and interaction are brought together in such a way that instruction can be provided via the Internet right to the desktop of the learner. Moore and Kearsley (1996) noted that a course is good or poor depending on how well it is designed, delivered, and conducted. This is especially true in regard to the delivery of courses online.

Instructional designers and course developers involved in the delivery of these courses require unique skill-sets that have often been associated with distance education but may be different given the delivery mechanism. The need for documentation of these skill-sets has been articulated in previous studies (Dooley & Lindner, 2002; Irani & Telg, 2001; Telg, 1995). In order to better prepare educators to develop and deliver instruction for delivery via e-learning, clear identification of competencies is critical.

E-learning offers many opportunities and challenges. Companies and agencies are turning to e-learning as a means to promote their products (Barron, 2000); educational institutions are turning to e-learning to deliver courses; and not-for-profits are turning to e-learning to reach clientele. The potential to reach across geographic distances irregardless of time is paramount. However, as noted by Oliver and Herrington (2003), too often the advantages of using the technology are not achieved due to poor design. As increasing numbers of graduates seek employment as e-learning instructional designers and developers, the identification of competencies required by elearning specialists is essential.

Competency identification and measurement are useful for educational practitioners, clients, and potential employers. Determining competencies helps to “improve human performance and unify individual capabilities with organizational core competencies” (Rothwell & Lindholm, 1999, p. 104). Formal degree programs and certificates that provide an organized means of obtaining identified competencies help individuals become more competent and more effective in their chosen career.

As we look at identifying competencies for e-learning, it is important to clarify terminology as we consider the theoretical framework for this study. Knowledge is the fundamental basis for each competency and consists of the body of information that is applied directly to an activity, while skill relates to an observable competence to perform a learned psychomotor act. Ability is a present competence to perform an observable behavior or a behavior that results in an observable product. Competencies, therefore, establish the behavior requirements needed to be successful in a given profession or task. Buford and Lindner (2002) define competencies as a group of related knowledge, skills, and abilities that affect a major part of an activity.

Based on a competency model developed by ASTD (formerly known as the American Society for Training and Development), Thach and Murphy (1995) identified roles, outputs, and competencies of distance learning professionals within the United States and Canada. Their top 10 competencies portray the dual importance of both communication and technical skills in distance learning. These competencies, in rank order, were: (1) interpersonal communication, (2) planning, (3) collaboration/teamwork, (4) English proficiency, (5) writing, (6) organization, (7) feedback, (8) knowledge of the distance learning field, (9) basic technology knowledge, and (10) technology access knowledge.

Williams (2003) replicated the Thach and Murphy study in higher education institutions and found these to be the top 10 competencies: (1) Collaboration/teamwork skills, (2) basic technology knowledge, (3) interpersonal communication skills, (4) English proficiency, (5) knowledge of distance learning field, (6) writing skills, (7) questioning skills, (8) skills in development of collaborative, student-focused learning environment, (9) adult learning theory, and (10) knowledge of support services.

Egan and Akdere (2004) explored roles and competencies from the perspective of graduate students specializing in distance education and compared them to previous works (Thach & Murphy, 1995; Williams, 2003). They determined that agreement exists among and between experts but differ in prioritization of the competencies. Graduate students felt that technology competencies were most important. Their top 10 were: (1) basic technology, (2) technology access knowledge, (3) computer networking, (4) knowledge of distance learning field, (5) multimedia knowledge, (6) software skills, (7) adult learning theory, (8) organizational skills, (9) collaborative/ teamwork skills, and (10) data analysis skills.

Given that the clients for professional certificates are often the graduate students served in higher education, the researchers for this study sought to expand on the competencies explored in distance education studies by looking at competencies unique to e-learning based upon the perceptions of graduates and current students focused on the area of e-learning. Are the competencies for distance education and elearning the same, or are there different knowledge, skills, and abilities required for e-learning?

A Web-based e-learning development certificate program is currently being developed to serve graduate students at Texas A&M University. The certificate will provide students with the unique knowledge and skills required to develop sophisticated e-learning courses and training programs for their disciplines. Students will gain a substantive foundation in learning theory, adult education, instructional design, and computer programming from an elearning perspective.

The purpose of this study was to conduct a needs assessment to reveal those competences perceived by graduates and current students focused on the area of e-learning to be of most value to an e-learning specialist. A secondary objective was to compare these findings to other studies conducted in regard to distance education competencies.

A focus group was conducted with past and current graduate students who were either currently working in the area of e-learning or had plans to work in this area. Purposeful sampling included students who had completed the graduate course, “Advanced Methods in Distance Education.” Using purposive sampling, eight respondents participated in the focus group (two males and six females).

The focus group session was conducted using Centra™, a live online classroom, to capture data both on-campus and at remote locations. Four on-campus respondents participated in person by coming to the designated room and four respondents participated via Centra™. The session lasted approximately 2hours and consisted of an informal discussion regarding the knowledge and skills needed for individuals working in the area of e-learning.

The researchers followed a protocol that included open-ended questions to enable individuals to be as informative as possible in their responses. The session consisted of two parts. The first part consisted of open-ended questions that were projected on the screen and to remote locations to allow all participants to view the questions being considered. Distant participants utilized conferencing tools within Centra™ to raise hands, indicate agreement, and provide verbal feedback. Questions were presented in a neutral format to encourage additional information and not suggest specific answers. Encouraging questions such as “How is that?” or “In what ways?” were used to support conversation. Follow-up questions were employed to obtain further information. After the researchers obtained complete input from the participants in regard to their thoughts and reflections, participants were provided a complete listing of certificate competencies that had been created by the guiding body for the development of the e-learning certificate program. Distant participants were e-mailed the document and on-campus participants were provided a handout. Appropriate time was allowed for participants to read and review the document. Once all participants indicated that they had completed their review of the document, the researchers guided the participants through a series of questions in which participants were asked if any items should be added or deleted from specific sections of the competency domains. Once again, distant participants utilized tools within the Centra™ program to communicate their agreement/disagreement and thoughts/reflections.

The meeting was recorded to allow accurate transcription of all important points shared. Field notes collected during the live session were coded by letters in the order participants spoke so that names would not be associated with items shared in the report or in future publications involving the research. Member checking was used to confirm data as needed. Data collected from the focus group was analyzed by the research team.

The constant comparative method was used for data analysis (Lincoln & Guba, 1985). Researchers analyzed notes taken during the focus group to determine trends in the data. Each unit (idea) was initially listed, without placement into categories. Tacit knowledge was employed in making initial judgments for categorization. Colored markers were used to identify themes so that the data could remain in context and provide a visual indication of emerging categories.

The researchers summarized the findings into categories and broad themes. A peer debriefing was held in September 2004 with a Distance Education Work Group to review and provide feedback on findings. This group is familiar with online learning and serves as the guiding body for the development of the elearning certificate. This debriefing and analysis allowed the researchers to further identify themes and constructs. It also allowed appropriate revisions to the e-learning competency list.

This research study was reviewed and approved by the Institutional Review Board— Human Subjects in Research. Analysis and coding records are available for an audit trail to ensure trustworthiness of the data.

The results of this focus group will be discussed in three areas based on the major themes and constructs identified in the data analysis: Description of E-learning Activities, Critical Competencies Necessary for Working in E-learning, and a Comparison of E-learning Competencies to Distance Education Competencies.

Of the eight respondents who participated in this study, five were currently working in elearning and three had completed coursework in this area with future e-learning career plans. In qualitative research, it is important to capture the most divergent views, due to small sample size. The respondents ranged from those who had continued directly from a bachelor degree into a master’s degree program with only part-time work experience, to those who had been working over 30 years.

The e-learning activities described by respondents were clustered into three categories: instructional design, technology/media, and administration. For instructional design, respondents referred specifically to their role in content development and delivery strategies (E, B). An additional respondent was creating online labs to assist learners with practical skills using asynchronous technologies. One participant, a public school teacher, was integrating e-learning into areas in the core curriculum and creating interactive activities to assist learners outside of the traditional classroom (B). Creating assessment tools, such as online testing features, was another instructional design activity mentioned (D).

In the areas of technology skills and media development, respondents mentioned activities such as creating Websites (B), learning to write software programs (B), and developing CD-ROMs (C, E). Video editing was also mentioned (E), although there was a belief among participants that e-learning specialists cannot be proficient in all media applications and need to capitalize on other media professionals for specialized skills.

Due to their expertise in e-learning, respondents were also involved in administrative components at their institutions and agencies. One respondent indicated involvement in creating quality standards for his or her campus, promoting e-learning to help with expansion of offerings, and creating incentives for faculty participation (A). Faculty apprehension and inhibition was expressed as preventing e-learning from expanding. A need to develop strategies to help faculty become confident and competent using e-learning tools for instruction was articulated. Promoting courses and evaluating outreach potential was mentioned by two respondents (D, E). Selecting the management and learning system to deliver elearning was another administrative activity shared (D). Reduction in travel time and costs was mentioned as a benefit that would encourage the adoption of e-learning (C).

In the area of critical knowledge, skills, and abilities needed to work in the e-learning arena, respondents identified seven core areas: (1) proficiency with computers and programs, including interface design, (2) organizational skills, (3) instructional design, (4) evaluation and assessment strategies, (5) adult learning theory, (6) written communication skills, and (7) student/teacher relationships to build a sense of community.

Competence using a variety of computer applications was expressed as a necessary skill when working as an e-learning specialist (F, A, H). Some level of technical skill was emphasized as a requirement of all e-learning specialists, but there was recognition that technology is always changing. Respondents expressed that a desire for continuous learning is imperative to work in this field. It was noted that specialists working in this area should be aware of supplementary media services available, and access specialized technical skills as needed. Activities such as developing timelines, schedules, communication channels, and project management for a team were mentioned as being important. This skill set was labeled “organizational skills” (E, G). Organizational skills in combination with instructional design skills (C) were noted as allowing the e-learning specialist to develop appropriate instructional materials geared to the target audience and delivered through appropriate media channels. Developing goals, instructional objectives, and content were expressed as primary functions of an e-learning specialist. The design of evaluation instruments and assessment strategies to measure learning was mentioned as an additional competency required (D).

A specific target audience that respondents felt was the primary focus of e-learning materials was adult learners (D, C). It was believed that e-learning specialists should incorporate adult learning theory into their design and delivery of materials. Some research knowledge on distance education effectiveness was also noted as being paramount to promote elearning to decision makers and adverse faculty (A, D).

Since e-learning is an asynchronous delivery strategy, it was believed that specialists should have excellent written communication skills (E). Respondents expressed that the primary message learners receive in e-learning is through e-mail and written documentation; thus, clarity, precision, and correctness are required. It was expressed that these writing skills translate into a virtual environment that should develop a sense of community (C). Therefore, respondents felt that the instructor cannot simply place content online and expect a “class” to feel engaged. It was noted that the instructor should develop meaningful relationships with the learners and provide communication channels and feedback on course content and assignments (G, H, A). Participants expressed that delivering instruction for e-learning is time consuming for instructors and learners; requiring a strong commitment by both (A, C).

Although the intent of this focus group research was not a replication of previous distance education competency studies, it is interesting to note the similarities and differences (see Table 1). The e-learning focus group was not given the distance education competencies to rank, and thus used different descriptions. The e-learning descriptions were matched with the distance education competencies by the researchers during the peer debriefing. In the Thach and Murphy (1995) study, all of the top ten competencies were perceived as necessary competencies for e-learning specialists. The elearning respondents mentioned adult learning theory (which is also indicated in the Thach & Murphy study, but ranked #22). The Thach and Murphy study makes no specific mention of instructional design skills.

From the Williams (2003) study, 8 of the 10 competencies could be matched with the elearning focus group. It was not clear if questioning skills could be combined in the evaluation and assessment strategies or student/ teacher relationship. Respondent quotes did not specifically indicate questioning skills as important, but rather the need for specialists to create environments where students felt engaged. Organizational skills were not in the top ten in the Williams study, but were ranked #12. Williams’ top 10 did include adult learning theory, but instructional design was also absent. One could argue that skills in development of collaborative, student-focused learning environments includes components of instructional design and student/teacher relationships.

The Egan and Akdere (2004) study emphasized technology skills in their top ten competencies. The e-learning focus group also recognized the importance of computers and software, but in addition mentioned interface design as critical. No specific mention of computer networking or data analysis skills was included in the e-learning discussion. Adult learning theory, organizational skills, and collaborative/teamwork skills were in agreement with e-learning competencies.

The findings that resulted from the focus group re-emphasize the importance of key competencies required by an e-learning specialist: (1) proficiency with computers and programs, including interface design, (2) organizational skills, (3) instructional design, (4) evaluation and assessment strategies, (5) adult learning theory, (6) written communication skills, and (7) student/teacher relationships to build a sense of community.

Based on these findings, it can be concluded that graduates and current students working in the area of e-learning do believe there are skills and competencies unique to the field of e-learning. As shown in the comparison with previous research in the area of distance education, which is similar to e-learning, it can be concluded that while some similarities exist, there are differences. As one looks at the comparison of competencies across studies (see Table 1), it is interesting to note that the current study lends further support to the importance of competencies related to computer skills, organizational skills, and writing skills. However, it also reveals the importance of understanding evaluation and assessment strategies which could relate to the need to justify and document the value of e-learning from both a learning perspective and a financial perspective. This study also reveals the importance of instructional design. While previous studies have included aspects that could be considered a part of instructional design, this study very pointedly emphasizes the need for e-learning specialists to have competence in instructional design. Kidney and Puckett (2003) state that the “skills needed to develop a Web course, including instructional design and Web and multimedia development, do not exist in isolation” (p. 210). Quality implementation of e-learning requires individuals who possess a broad range of competencies as revealed in this study.

Direct implications for these findings exist. In order to better prepare educators to develop and deliver instruction for delivery via e-learning, identification of e-learning competencies is critical. Competencies that have been identified and confirmed in this study will guide the development of a newly designed e-learning certificate program at Texas A&M University. Further, the findings are beneficial to others who seek to educate individuals to work in the area of e-learning or improve their own skillset by providing a robust picture of the skill set needed. Gaining an understanding of student/ employee perspectives provides tangible insight that enables course developers and instructors to provide courses that meet the needs of the students and the needs of future employers.

The methodology employed in the study also has implications. The significance of using software programs such as Centra™ are far reaching. Focus group research is typically limited to participants in close proximity to the institution and researchers. The ability to include respondents virtually worldwide removes geographic sampling limitations and allows broader perspectives. The researchers found Centra™ to be an effective researchgathering tool. Participant input was similar regardless of location/technology. Use of this technology provided an effective and equivalent mechanism for gathering needs assessment data for the development of the elearning certificate program. This research effectively utilized technology to gain participation and serves as an example to others seeking to collect data using focus group techniques.

As shared by Waight, Willging, and Wentling (2004), the significance of e-learning cannot be overlooked, as emphasized by the multitude of e-learning reports that have been conducted in recent years. Documenting the competencies needed by e-learning specialists provides critical input that will shape the future of education through the preparation of individuals who will create the instruction to be delivered through e-learning.