Student Feedback on Tutor and Academic Support of Wawasan Open University: A Four-Semester Study Free

Wawasan Open University (WOU), a distance education provider in Malaysia, completed five semesters of operations with nearly 3,000 active students by June 2009. Wawasan means “Vision” in the Malay language and it aims to provide a “second chance” for working adults who for a variety of reasons did not enroll into the traditional tertiary institutes when they were younger. It also practices an “open entry” system of student intake for students who do not have the requisite academic qualifications but are willing to go through an “open entry test” to ascertain that they have had the potential to follow an academic programme leading to the conferment of a first degree. Those who still cannot be accommodated through this “open entry test” mechanism are encouraged to do a six-moth preparation programme called Head Start, and upon successful completion they will be channeled into WOU's first degree programmes. In addition students with an A-level qualification or equivalent can be accepted into the first degree programme direct. It thus has a rich mix of students with vastly different academic preparations in the programmes. Starting in the third semester on January 2008, WOU welcomes the first batch of students into the Commonwealth Executive Master's of Business Management (CeMBA) and the Commonwealth Executive Master's of Public Administration (CeMPA) programs. Apart from the normal requisite first degree qualification with 2 years of working experience, again there are “open entry” mechanisms to this postgraduate course through successful pre-entry tests.

Like most distance education providers, WOU has adopted a flexible approach to ensuring students receive quality education through various support structures, commonly known as student support services (SSS). SSS, however, has been described in many different formats and inclusions. For example, Wallace (1996), in analyzing demographics and motivation of distance learners, contended that reducing barriers to success in distance education hinged also on SSS and policies on teaching resources and fees. SSS, she noted, includes services like library search, reprints at no cost, free resources, and toll-free telephone contacts with the academic staff. This is thus quite a narrow interpretation of SSS.

While SSS is appreciated as an “assurance” as well as a determinant for students to stay on with the distance education programme, it must be pointed out that even the traditional universities are also elbowing their way into distance education. This is made more possible with the integration of technology into teaching and learning, chief among which are the Internet technologies, iPods and MP3s, and social networking systems and communities that have grown very fast. Fearn (2009) reported the technology to support instruction in institutions had blossomed to created different facets of instruction and learning, attesting to the rapid changes. Students thus have a wider net to cast to choose their institutions for learning.

The recent recession has also encouraged institutions to focus on new income sources and, using new technology, has reached out to more learners. Blended or hybrid instructional systems will become more dominant, and purely ODL institutions have to do more to retain their students. For example, The Open University (UK), a specialist ODL institution, had managed to secure a twofold increase in student enrolments from 34,420 in 1998-99 to 64,759 in 2007-08 (Fearn, 2009). Comparatively, Allen and J. Seaman (2007) predicted hybrid environments to be increased to over 3.18 million based on a research survey of over 1000 colleges and universities. In fact, in fall 2007, more than 3.9 million students were enrolled in an online course (Allen & Seaman, 2008). With so many players in distance learning either as a hybrid entity or a pure ODL player, it is thus imperative that institutions have proper SSS care to ensure that their students stay on and attract more students to the program. So what constitutes a fair measure of a SSS?

The University of Wisconsin System Administration (UWSA) educates more than 170,000 students on 26 campuses, and also conducts distance education programs. In September 2000 the UWSA Board of Regents issued “Standards for Academic and Student Support Services in Distance Education Credit Course, Degree and Certificate Programs.” Among the board's recommended standards on academic and SSS, (condensed here) are:

• Curriculum and Instruction: Academic rigour and quality instruction, courses are current, use of appropriate technology, training of faculty, faculty support services, interactions between faculty and students.

• Evaluation and Assessment: Orientation and meeting expectations of students, evaluation of effectiveness of distance education, course evaluation including student satisfaction.

• Library and Learning Resources: Student access and effective use of resources, institution monitors usages.

• Student Services: Adequate access to range of student services like admissions, financial aid, course materials, and placement and counseling; means to resolve student complaints; timely and complete information to students on equipment and knowledge; help desks, and so on.

• Facilities and Finances: Appropriate equipment, facilities, technical expertise etc. available and sustainable over a period of time; faculty has access to instructional design support to ensure quality of student learning experiences; institution has longrange planning, budgeting, and so on.

Based on these standards for academic and student support services, does WOU meet these criteria? A quick look will tell us that all these standards have been met, and are here annotated:

1. Curriculum and Instruction: The program and course curricula have been subjected to academic accreditation mechanisms such as the academic peer group after the Outline program proposal is approved by the school and senate. Also the program will have to be approved by the Malaysian Ministry of Higher Education. Preparation of instructional/course materials proceed upon final approval of these accrediting bodies. The program then is launched with course coordinators, tutors, and online learning management system called WawasanLearn, to support the students (Ng & Kong, 2008). Tutor and faculty support is just a phone call away or by way of discussion forums in the WawasanLearn.

2. Evaluation and Assessment: The course evaluation is carried out through continuous assessments in the form of tutor-marked assignments (TMAs) and a final examination. The TMAs are in various modes, such as, essay and multiple choice questions, short duration projects, computer code writing, laboratory reports, and so on, depending on the objectives and the nature of the courses. Then there is the “Student Feedback on Tutor and Course” (SFTC) by way of a questionnaire, and the parallel face-to-face student dialogue with the university officials (course coordinators, deans, and heads of academic units) to add onto the collected data from the SFTC as well as to confirm trends in tutor and course conduct. This is the main mechanism to determine effectiveness of courses and the delivery systems used for WOU.

3. Library and Learning Resources. The library and its regional libraries located at the regional offices (RO) provide basic support for all courses offered by WOU in the form of copies of textbooks, past years' examination questions, samples of past TMAs, as well as providing a 24hour online learning resources. The WOU library subscribes to online databases such as Book24x7, ebrary, NetLibrary, Emerald, EBSCO Host, and leading journals and reviews. Also, registered students can search through these databases from the comfort of their home as their student identity and password are confirmed for access to these databases when they log in. The library staff also provides search assistance, guidance on academic research, and writing as well as document delivery. In addition, for many courses WOU even provides the textbook to the registered student! Further, WOU courses are complemented by an active Learning Management System, called WawasanLearn, that provides a structured digital resource-based support to students registered for their specific courses. This operates on a 24/7 basis, with academic contents like tutorial information, additional files in Adobe, PowerPoint, Excel, and MS Word, links to current resources on the web, as well as active forum sessions for students to collaborate and interact with each other, as well as tutorsupervised forums for tutorial groups.

4. Student Services. The most obvious student service is seen at the regional office (RO) level, where students can seek help in submitting their TMAs physically, use the computers there to submit their TMAs electronically through WOU's “Online Assignment Submission System” (OASS) if they do not have computers at home or place of work, and receive administrative and counseling help when required.

5. Facilities and Finances. The RO and the nearby learning centers (LC) are the main physical support for the students, as they do not need to be present at WOU's headquarters in Penang. Instructional design supports on developing quality learning materials are available at the headquarters and through the intranet system to academics and course writers located at the RO. As the RO grows in size and operations, additional computer laboratories will be provided. Also, WOU provides scholarships to deserving students on a means as well as merit basis. In fact, over 60% of the current registered students do get access to this mode of financial support!

Thus, it can be confirmed that WOU provides sufficient academic and SSS to the student, based on the criteria set up by UWSA Board of Regents. An academic audit will thus confirm these services are available. However having academic and SSS in place does not necessarily mean that the ultimate users of the services are satisfied with them. Thus, the educational technology unit, of which the first author of this paper is the director, is assigned the task to evaluate and determine if the students are satisfied with the academic and SSS available?

With decidedly nontraditional working student demography for open and distance learning (ODL) institutions like WOU, it becomes quite unwieldy to gather information on students' satisfaction with the programs, especially when even the allocated five tutorials per semesters are not fully utilized by the students, as tutorial attendance is not made compulsory. This will thus affect the sourcing of data on student satisfaction with the tutor and academic support. The declining tutorial attendances for the five regional learning centers for the third semester are illustrated in Figure 1. The regional learning centers are located in the main Malaysian towns/cities of Penang (PG), Ipoh (IP), Kuala Lumpur (KL), Johor Baru (JB), and Kota Bharu (KB). Thus the data collected will only come from the remaining 50 to 60% of the students and, in fact, the actual return rates are consistently around 30% for all four semesters.

The “Student Feedback on Tutor and Course” (SFTC) questionnaire was developed and used for all four semesters to gather selfreporting of satisfaction with the tutor and academic and student support of WOU. The questionnaire was locally developed and tested on samples of students groups for validity and reliability. All items are statements about specific areas of concern in ODL environments, and the students responding indicate their perception about the implications/impacts of the statements using a 5-point ordinal/interval scale together with an alternate choice of NA (Not applicable). In addition, in the last row (“other comments”), students can offer qualitative or written impressions about the area of concern. The guide to the rating used in the questionnaire sample statements appear in Figure 2.

A 5-point Likert scale instrument was seen to be the best compromise between selecting from a yes/no selection and from having to make different choices based on different response protocols (Page-Bucci, 2003). Also, Dumas (1998) wrote that a Likert scale “is the most commonly used question format for assessing participants' opinions of usability” (p. 5). Here the student chooses based on the agreeableness or otherwise of the statement towards a specific area of tutor or academic support to the student with the rating that reads “5 = strongly agree,” “4 = agree,” “3 = neither agree nor disagree,” “2 = disagree,” and “1 = strongly disagree.” In this study, there is a known mid-point of 3, referring to “neither agree nor disagree.” Hence, students can opt to agree or disagree to the statements in the questionnaire. In addition, the column for “not applicable” provides an avenue for the student to express that the statement does not matter at all or it is not suitable to comment on it. Immediately after the end of the statements for the specific area of tutor or academic support, such as, face-to-face tutoring, there is an “other comments” box for the students to express in their words their perception or feelings, in this instance, about face-to-face tutoring.

However, there were issues in relation with the use of this type of Likert scale. Is it ordinal or interval? We consider this type of Likert scale to be ordinal in nature (Allen & C. A. Seaman, 2007), and that treating it as interval will be doubtful (Hodge & Gillespie, 2003). However, rather than looking at the standard deviation of the summated scores for that statement, we suggest it would make better sense to equate the mean as being the median score for that statement. Thus, Allen and Seaman stated that some considerations must be observed when treating a Likert scale as interval. They imply that that the data must have continuous sense of measurement and the assigned numbers are not merely labels as used in nominal scale data. To use a Likert scale in measuring latent variables, Cronbach's alpha test of reliability must be conducted prior to other procedures (Allen & C. A. Seaman, 2007). Cronbach's alpha coefficient is a measure of the internal consistency reliability of items in a test, and when items (here the questionnaire statements) in a scale are calculated for internal consistency, and if the alpha derived is high, it can be assumed that the test is consistently measuring the intended object of interest (Streiner & Norman, 2004). As a rough guide, an alpha of 0.6 to 0.7 indicates acceptable reliability and an alpha of > 0.8 indicates greater reliability. In this study over four semesters (coinciding with the first to fourth semester) we also sum these ordinal data for a number of statements that represent, say, face-to-face tutoring, and the summed mean will only represent the summated ordinal data.

As WOU is a new ODL institution, it will have to constantly improve on its academic and student support services. Whenever clear-cut defects are detected, immediate actions have to be taken so that the problems do not linger too long and affect the reputation of the institution. We surmise that given the increasing number of students, we need to be very alert to any changes in satisfaction with the academic and student support services. Thus, for the first four semesters, we suggest that the level of satisfaction of the students with the academic and student support services are relatively strong and positive and that only “fine tuning” may be necessary to manage the services at the present state of operation.

Our hypotheses for this study are that:

There are a number of assumptions about the student satisfaction survey. First, the respondents volunteer their satisfaction about the WOU's presentation consistently for the four semesters. With new students being added to existing students as the semesters progress, it is assumed that the mode of response and perceptions thereof remain uniform. This, however, cannot be proven, but if the means of the statements are relatively stable or even trending slightly, the assumption about respondents can be upheld compared to when the means are changing rapidly and “bouncing” unevenly through the four semesters' students.

To accommodate the areas of concerns about tutor and academic support for students, the questionnaire was designed to include the areas 1 to 6 below represented by the statements and Cronbach's alpha of these statements are displayed in the tables for each of the area of discussion. In describing the statements in the questionnaire, the context of these statement in relation to the actual operations and/or offerings to the students are discussed to provide the appropriate context. Also, qualitative comments volunteered by the students are randomly selected to clarify the strengths of the statements' Cronbach's alpha.

The second assumption is that as the semesters progress, the number of students would therefore increase and so does the course offerings. Thus, will “combining” the courses to be computed as an “overall” score for the responses to individual statements of the questionnaire be viable? The authors are of the opinion that it would make sense that an overall mean for all courses in each semester may be more inclusive of all responses than using and comparing each individual named course. This become more critical when the numbers of respondents to each course fluctuate wildly or when the course student numbers become too small to be representative. Basic information about number of courses and the number of student involved in responding to the “Student Feedback on Tutor and Course” (SFTC) is provided in Table 1.

It is noted that not all courses or all students are administered the SFTC. Also, as the number of students attending tutorials waned by the fourth tutorial, so does the number of students as well as the number of tutorial classes in operation.

In order to ascertain if the survey instrument using statements on “agreeability” to, an internal consistency and reliability of the statements represented in each of the six areas are analyzed using Cronbach's alpha test. Only when the statement items are deemed consistent and reliable can we test if the hypotheses promoted here are supportable. The following tables report on the Cronbach's tests.

There are 10 statements regarding the behavior of the tutor, the pedagogy used by the tutor, fairness in evaluation by the tutor, as well as the quality of feedback on student efforts. The 10 statements are presented in Table 2. The Cronbach's alphas of these 10 statements are strong at > 0.9 and are similar for all four semesters, indicating that they measure reliably the face-to-face aspects of the academic support.

There are three statements used for this area of tutor support and the statements are displayed in Table 3. Only individual statement's means are reported and there were no attempts to combine the statements into a composite variable. Thus, there will be no Cronbach's alpha reported for this aspect of telephone tutoring and e-mail, as the items are measuring different aspects of the two different supports.

There are three statements used for this area of tutor support and the statements are displayed in Table 4. The Cronbach's Alpha for all statements in each semester is computed. The Cronbach's alpha of these three statements are in the 0.7 to 0.8 ranges for all four semesters, indicating that they measure reliably the aspects of the student support provided by the regional learning centers.

Course materials become the main focus of support to student learning in an ODL environment. This is especially true when compared to regular face-to-face opportunities in traditional institutions with lectures and library facilities as well as more frequent tutorials planned. In an ODL environment, students are more dependent on the course materials, and here the materials are self-instructional modules (SIM) with activities and self-tests as well as web links and supplementary course materials through WawasanLearn. Also, in some courses free textbooks accompany the course materials. Interactions with course materials through reflective thinking (Wikieducator, 2008), regular practice with the activities and redirected access to other learning resources are important for student success in the course (Field, 2008). The SIM of WOU is described in greater detail in Ng (2007), and WawasanLearn is discussed in Ng and Kong (2008). The Cronbach's alphas of these eight statements are strong and almost similar for all four semesters, indicating that they measure reliably the course materials aspects of the academic support (see Table 5).

TMAs are an important component of assessment and evaluation in ODL environments. The tutorials are arranged in a manner to pace the students' progress, and TMAs are scheduled to be collected and marked based on the schedules of the tutorial session. For WOU, in an ODL operation, even when there are planned tutorials there is no compulsion to attend the tutorials, and what is needed of the students is to submit their Tutor Marked Assignments (TMAs) on time, and normally during scheduled tutorials. However, referring to Figure 1, it is evident that even with tutorials, many students choose not to attend and still manage to submit their assignments to the regional learning centers, or more often now by means of WOU's Online Assignment Submission System (OASS). Here the students access their student accounts with the OASS, follow the instructions on that website, and submit their assignments electronically.

The Cronbach's alphas of these five statements are between 0.6 and 0.7 for all four semesters, indicating that they measure reliably the TMAs aspects of the academic support, although at an alpha value lower than the other areas described. The lowering of the alphas could be ascribed to the different emphasis of the statement intents, and that some statements do not run “parallel” with each other—for example, how do we reconcile a statement that reads “The self-tests and TMAs are relevant to the objectives of the course” with another statement that reads “I prefer to submit my assignment online on or by the due date”? (see Table 6).

The learning management system, WawasanLearn, has become an important platform for students to access additional resources such as postings of forum chats, to interact with their tutorial mates through forums, and to submit queries to the tutors. To ascertain the usefulness of its academic support, five statements relevant to WawasanLearn support were used to determine if it has accomplished its roles in academic support. The Cronbach's alphas of the five statements are almost similar at > 0.8 for all four semesters, indicating that they measure reliably the academic support provided by WawasanLearn (see Table 7).

The six areas studied using the questionnaire thus show that most of the UWSA Board of Regents' suggested academic and student support services are catered for. The component on curriculum is not assessed in the student feedback, as the courses are accredited by the Ministry of Higher Education Malaysia. However, implementing the curriculum in terms of instruction, course materials, tutor and learning centre support, TMAs and WawasanLearn are all considered to be important building blocks to provide academic and student support, and the statements used to “measure” responses are all considered reliable or highly reliable.

Thus, the questionnaire, after being subjected to two trials with small groups of students at midsemester of the first semester, was distributed to students on the fourth tutorial and to be collected back on the fifth semester. As reported the Cronbach's alpha of the statements in the six areas representing the standards of a good academic and student support service are all relatively reliable and thus the reported mean scores of the statements as well as the combined statements were determined over four semesters.

The means of all statements in the questionnaire in the six areas of concern in SSS were reported in the next six tables. Also for some areas of concern, consolidated means were used. For example, in order to ascertain the face-to-face experience of the students, all the scores of the ten statements were combined to reflect the “face-to-face experience.” The combined score is divided by the number of statements, here ten. It must be noted that combining of the statements and subsequently dividing them by the number of statements give rise to the assumption that all statements are equal in weightings when that might not be true.

The means of all the ten statements are clustered around 3.9 to 4.3, indicating that most students “agree” with the face-to-face tutorials provided by WOU. A sample of the comments accompanying the questionnaire illustrates positive vibes about the face-to-face tutorials. Here are a few of the comments: “I am very privileged to have a tutor who is knowledgeable, innovative and resourceful.” “Tutor should able to add more examples and question to us, more understanding and for discussion, point out the key words to assist us to remember.” “Two hours of tutorial × 5 is not enough” However, the value of the combined means appear to trend down from 4.27 in the first semester to 4.06 in the third semester but rebounded to 4.23 by the fourth semester (see Table 8).

The means of the statements on telephone tutoring/e-mail are all less than 4 and thus appear to indicate that the respondents neither agree nor disagree with the statements. Compared to the other areas of concern, this would imply that the use of telephone (fixed-line or mobile), with an average mean of about 3.6, had not been as useful or helpful, or even frequent (see Table 9).

The means of all the four statements are clustered around 3.9 to 4.1, indicating that most students “agree” with the regional learning center support provided by WOU (see Table 10). A sample of the comments accompanying the questionnaire illustrates positive views about the support provided: “The staff find their job fun I guess and are really helpful.” “Provide more information/services for IT support centre.” “Sometimes when we ask some queries, RO staff unable to answer us, e.g., the time table for next semester.” “I really love the service level. Very efficient! Keep up the good work.”

The means of all the eight statements are clustered around 3.9 to 4.0, indicating that most students “agree” with the course materials provided by WOU (see Table 11). As course materials are the primary source of student learning, WOU needs to be very watchful of the course materials' impact and acceptance by students. A sample of the comments accompanying the questionnaire illustrates positive views about the support provided: “Overall, the course materials is well prepared.” “The course materials & the textbook was a best match, after i read thru the course material, I get to understand around 40% of the contents & I got the rest thru text book.” “Need more examples and case references.” “Some of the topics are too confusing.” “Some paragraphs are too long.” “The topic in course material some doesn't match the text book used.” “The course materials are in very heavy files. They could make parts 1 to 5 into separate booklets so we can carry them easily.” “Should have more exercise to evaluate.”

As the TMAs are considered critical in contributing to student achievement, it can be viewed that, as a whole, most students are relatively supportive of how they perceive TMAs to be usefully abetting their study's progress. Most statements average about 3.6 to 4.1 (see Table 12). The statement that reads “I have difficulties comprehending the TMA questions” has a lower mean, as it is negatively phrased, and was later reversed for computation purposes. A sample of the comments accompanying the questionnaire illustrates positive views about the support provided, and also some reservations: “The assignments are good for our practice and assessment. Only through these exercise we can learn of our true capabilities.” “It is good job to post the best answer in the LMS that others can refer to it and how marks are allocated.” “TMA1 cover Unit 2, TMA2 cover Unit 3 and 4, TMA3 cover all Units. The marking for the TMAs is different from tutor to tutor; some tutors are too kind to give good mark & some too strict.”

The learning management system, WawasanLearn, was launched in conjunction with the first semester offerings, and it has demonstrated to be quite well attended to and rated consistently at above 3.8 for all statements (see Table 13). This shows that the university was proactive in ensuring better interaction with the course materials and with courses mates digitally. A sample of the comments accompanying the questionnaire illustrates positive views about the support provided, and also some reservations: “Many of the questions posted by the students are very low level. But the tutor and Course Coordinator answered well. Very good support is given to weak students and also to encourage advanced students.” “Too many ‘thank you’ and complimenting messages in the forum. I think it's useless and flooding the forum. Please get the administrator to clean it up.” “The participation of fellow course mates in Forum is very low and I lament at the standard of the English language nowadays!” “The Course Coordinator had done a good job in contributing much in the LMS as do the many tutors also from other regional office.”

The study examined student support services (SSS) as per the criteria set up by UWSA regents. The focus on the SSS is on six areas identified. Based on the analyses of the responses to the six areas (viz., face-to-face tutorials, telephone tutoring/e-mail, regional learning centers, course materials, tutor-marked assignments, and learning management system), it can be seen that although all the means of statements across the four semesters are about the same, there is a decided “down-trending” as the semesters progress. This implies that students have rated the statements a bit lower over the semesters. This should alert the service providers on what had caused the slight declines. Thus, we can say that the first hypothesis that student satisfaction of the academic and support services are positive and remain positive can be supported. As long as the means of statements are > 3.5, they imply the services are agreeably well received. However, the slight decline is a trend to be studied closely, and whilst the second hypothesis of student trends on the statements may be valid, we need to further clarify why the “decline” happened. A closer look at the qualitative responses in the six areas indicated that there are some views volunteered that need studying. Samples of these statements have been included in the elaborations after Tables 8 to 13.

The University of Wisconsin System Administration Board of Regents Standards for academic and student support services (SSS) in distance education were applied to the SSS provided by the Wawasan Open University (WOU) on an area-for-area basis and a survey instrument developed for the purpose. A student feedback survey conducted over four semesters confirmed that WOU's SSS had been positively received, with means of statements as well as combined statements being > 3.5 generally. The statements used and the responses to the statements were analyzed for internal consistency reliability of items in the questionnaire. All the statements in each area of SSS have a Cronbach's alpha in excess of 0.6, meaning that the items (here the questionnaire statements) in a scale have high internal consistency. However, there was a slight down trending of means of statements representing the SSS by the third semester, and the trend had reversed to a slightly higher mean by the fourth semester. It is concluded that the SSS represented by the six core areas of SSS for open and distant learning institutions (i.e., face-to-face tutorials, telephone tutoring/e-mail, regional learning centers, course materials, tutor-marked assignments, and learning management system) are providing adequate support to the students. Samples of the qualitative feedback also attest to the support provided. It is suggested that the university look into the causes of the slight down trending of the means of the statements and also to evaluate the qualitative response provided by the responding students in the four semesters.