This article describes the effect of teacher comments, students' demographic information, and utilization of learning management system (LMS) on student academic performance in a K-12 virtual learning environment. Students who completed biology courses in a Midwestern state virtual school during 2007-08 and took the end of course test participated in this study. The hierarchical linear modeling (HLM) technique was employed for data analysis. The results show these factors can influence student academic performance in biology courses in different ways. The implications for teaching were addressed. Further research is proposed based on the results and limitations.
Introduction
Studies have shown there is a gap in student science achievement between the United States and other developed countries and even some developing countries (Morrone, 2001). In 2007, American students in 12th grade scored very low in international science tests as compared to their counterparts in other countries (Brainard, 2007). Even on the domestic test of science knowledge, the National Assessment of Educational Progress, about half of U.S. 12th graders did not reach the basic level of proficiency in 2005 (Brainard, 2007). Some international studies, such as Program for International Student Assessment (PISA), Trends in International Mathematics and Science Study (TIMSS), and national assessments such as the National Assessment of Educational Progress (NAEP) showed that U.S. students did not meet the academic standards in subjects such as mathematics, science, and literacy (Gonzales et al., 2004; Hampden-Thompson, Johnston, & American Institutes for Research, 2006). In a study on the integration of environmental health science into K-12 curriculum, Morrone (2001) warned there was a science education crisis in the United States. Science has been considered a very important force to push a society forward. Many countries emphasize the improvement of science knowledge and develop policies to attract more people into this field. However, a series of recent reports warned that the number of students in science fields and students' test scores in some countries are surpassing the United States (Brainard, 2007). The underachievement of U.S students in science subjects at K-12 level could lead to the lack of preparation for students to pursue advanced degrees in these fields. One National Academy of Sciences report showed only 15% of U.S undergraduate students graduate with degree in natural sciences or engineering, and many other countries have surpassed this number (D'Amico, 2008). This situation causes a shortage in the workforce in science fields, which, in turn, could weaken the momentum for the country to move forward in many aspects. The National Association of Manufacturers suggested that the shortage of workforce in the science, technology, engineering, and mathematics fields can weaken manufacturers' abilities to ensure quality, productivity efficiency, and customers' satisfaction (D'Amico, 2008). Today, the United States faces a challenge: other countries might surpass it economically through mass production of technology-based merchandise (Brainard, 2007). To meet this challenge, actions should be taken to improve student science achievement and increase the science workforce in this country.
Online Education in K-12 in the U.S.
The United States has experienced an extraordinary growth in online education at the K-12 level since its emergence in the late 1990s: from single online course offerings to large virtual schools today. Thousands of students were attracted to online education because of the advantages it brings such as flexible and longer school time, more educational opportunities, and increased access to resources (Cavanaugh et al., 2004). Several surveys have showed that at least one third of high school students had online learning experience (Allen & Seaman, 2006; Setzer & Lewis, 2005). Figure 1shows the dramatic increase of K-12 online enrollment between 2001 and 2008 (Clark 2001; Glass, 2009; Newman, Stein, & Trask, 2003; Peak Group 2002; Picciano & Seaman, 2009; Picciano & Seaman, 2007; Setzer & Lewis, 2005; Tucker 2007; Zandberg, Lewis, & Greene, 2008). By 2016, this number is anticipated to reach 5 to 6 million and will keep growing (Picciano & Seaman, 2009). Only public school students were included in these figures; the number will be higher if all other students are included, such as those in private schools and home-schools (Picciano & Seaman, 2009).
Online courses have been offered at the high school level in the United States for over a decade. Since their emergence, a variety of effective practices has evolved. It is possible to evaluate the effectiveness of online courses because of the current large population of online learners. The investigation of the factors that influence students' success in online learning at K-12 level will be beneficial to educators, researchers, virtual program leaders and designers, policymakers, and society at large.
Success Factors in K-12 Virtual Learning Environments
Many factors could potentially influence student academic achievement in the K-12 online learning environment. Several of these online success predictors were investigated in the present study in light of the relationship established with student academic achievement in other studies. The factors of interest include time students spent in the LMS, number of times students logged into the LMS, participation in free or reduced lunch programs, teacher comments, student learning ability (whether they have an individual educational plan), grade level in the physical school the student attends, race/ethnicity, and student status in the virtual school (full time/part time). Besides these student level factors, school characteristics could influence student academic achievement via school culture, resources available for students, and other factors. Any evaluation of the effect of student level factors must account for the influence of individual differences among different schools on student academic achievement, because the majority of virtual school students take online courses as a supplement to their traditional school programs (Watson, Gemin, & Ryan, 2008).
Learning Management System Factors
The time spent in academic activities has been identified as a very important factor that has strong effect on success in online education (Cavanaugh, 2007), face-to-face instruction (Rocha, 2007), and blended programs (Cavanaugh, 2009). The activities students engaged in during online learning can affect their final scores, with students who participate in academic activities at high level performing better than those who do not (Wang & Newlin, 2000). The number of times students logged into the LMS is believed to be a strong predictor of student academic performance in online learning (Diez, 2002; Dickson, 2005). In the present study, the numbers of times students logged into the LMS and how long they stayed in the LMS were considered the indication of time students spent in academic activities.
Participation in Free/Reduced Lunch Programs
Participation in school free or reduced lunch programs has a correlation with student academic achievement (McLoyd, 1998). Klein, Hamilton, McCaffrey, and Stecher (2000) conducted a research study on the relationship between students' participation in free or reduced lunch programs and school test score and found the percentage of students participating in these programs in a school is a predictor of the school's mean score. Participation in a school lunch program was also frequently used as the measurement for students' family socioeconomic status (SES) in the literature on students' academic achievement (Sirin, 2005). Family SES presents the background for student academic performance. Higher SES families are able to provide students more resources at home and social capital, both of which can improve chances for student academic success (Coleman, 1988). A considerable body of research has been done on the relationship between SES and student academic performance. White (1982) and Sirin (2005) each conducted a meta-analysis in this field and found the correlation coefficients between family SES and student academic achievement were 0.343 and 0.299 respectively.
Teachers' Comment/Teacher-Student Interaction
Teacher comments and feedback on students' assignments and teacher-student interaction are very important factors that influence student academic achievement in online learning (Cavanaugh et al., 2005; Dickson, 2005; Ferdig, Papanastasiou & DiPietro; 2005; Hughes et al., 2005; Peters, 1999; Smouse, 2005; Zucker, 2005). Teacher-student interaction can bring many pedagogical benefits for students during online learning such as higher learning and constructive feedback helpful for deeper understanding (Anderson & Kuskis, 2007). Interaction is “at the heart of online learning” (Cavanaugh, 2007, p.6). Teacher comment and feedback has been identified as a constructive feature within the online learning process (Cavanaugh et al., 2005) and one of the benchmarks for quality online learning (Phipps & Merisotis, 2000). In the present study, the number of teacher comments and feedback could be considered as the indicator of teacher-student interaction.
Student Learning Ability/Individual Educational Plan
The virtual school student body is a diverse population including students with different learning disabilities (Dickson, 2005; Ferdig, Papanastasiou, & DiPietro, 2005). Virtual schools offer or support individual educational plans for these students during the learning process. Therefore, whether a student has an individual educational plan could be a sign of the level of learning abilities. The virtual school learning environment has the potential to bridge gaps between students with disabilities and students without these disabilities with respect to the success opportunities in online learning. For example, for students with different levels of learning disabilities, the technologies such as computer, Internet, etc. could help reduce their disadvantages as compared to students without disabilities to access the course materials during online learning (Coombs & Banks, 2000). The proposition of early adoption of technology-infused education for disabled online students (O'Connor, 2000) also can benefit their online learning. However, even with bridging gaps with regard to online success opportunities for disabled online students, they are still underrepresented in online education (Kinash & Crichton, 2007). For example, some learning management systems (LMS) such as WebCT and Blackboard still have some inherent problems limiting disabled students from fully utilizing some functions therefore limiting their chances of success in online learning (Asuncion et al., 2006).
Race/Ethnicity
Numerous studies (e.g., Bali & Alvarez, 2004; Barth, 2001; Hall et al., 2000; Lockhead et al., 1985) have shown racial gaps in student academic performances. When comparing student academic achievement among different racial groups in the United States, Asian American and Caucasian American students outperformed Hispanic and African American students (Barth, 2001; Lockhead et al., 1985). The student body in a virtual high school is similar to its counterpart in the traditional learning environment (Ronsisvalle & Watkins, 2005). The findings of the studies on the relationship between racial/ethnicity and student academic achievement in a traditional classroom is hypothesized to apply to the K-12 online learning environment in the present study.
School Type: Private/Public
In 2006, the U.S. Department of Education's National Center for Education Statistics (NCES) released a report with regard to a study conducted on the differences in student academic achievement in reading and math at grade level four and eight between private and public school (Braun, Jenkins & Grigg, 2006). This study took into account some selected students' characteristics such as gender, race/ethnicity, disability status. The results show the average private school mean score was significantly higher than its counterpart in public school. Students in private schools achieved at higher levels academically than those in public schools.
Science Achievement in Online Learning Environments
During the past 10 years, few studies have examined student science achievement in online learning environments. Barman and Stockton (2002) conducted a study to evaluate a web-based science course, SOAR-High (science, observing, and reporting-high school) with respect to helping improve students' science process skills and promoting their motivation to learn science. They found positive results in both of these two aspects. Zion, Michalsky, and Mevarech (2005) conducted a research study on the effects of the online learning environment on students' scientific inquiry skills. The researchers found a strong positive relationship between the online learning environment embedded with metacognitive guidance and students' general scientific ability and domain-specific inquiry skills and they concluded the online learning environment can be utilized to enhance students' science literacy.
Leu, Castek, Hartman, Coiro, and Henry (2005) conducted a study on the improvement of scientific knowledge and reading comprehension in online secondary school learning environment. They found student content knowledge gains in the online learning environment were equivalent to those achieved in traditional learning environments and the online learning environment can help increase students' online reading comprehension which is an important element for success in online learning. Wang and Reeves (2006) investigated the effects of web-based learning experience on motivation for students to learn science. They found a well-designed web-based learning environment could motivate students to better learn science content knowledge and help engage them in the learning process. The researchers concluded that the web-based learning environment can be utilized as a tool to engage students in the science learning process. Jayaraman (2002) studied teachers' perceptions of difficulties when teaching science online and the pedagogical skills and strategies they need to overcome these difficulties. Interaction between students and teachers and prompt feedback from teachers were found as very important instructional strategies to effectively deliver science instruction online.
Some research also has compared online simulated science labs and traditional hands-on science labs. The results show no significant difference with respect to learning outcome between students doing hands-on versus online simulated labs (Ma & Nickerson, 2006; Trion & Klahr, 2003). Studies show online simulated labs can provide students real experimental experience and data without many restrictions caused by the limitations of resources and lab space (Nedic, Machotka & Nafalski, 2003; Proske & Trodhandl, 2006). Other advantages of online labs include giving teachers and students more time concentrating on the important scientific concepts learning, providing them more opportunities to collaborate and cooperate with one another during scientific investigation (Jona & Adsit, 2008). These findings form a foundation for the implementation of science courses online.
Significance of the Study
New quantitative online learning research has been called upon to promote effective online courses/programs (Smith, Clark, & Blomeyer, 2005). Rigorous research methods with high priority including correlation research, mathematical modeling such as hierarchical linear modeling used in this study, exploratory research strategies, were highly recommended to help develop research on the efficacy of online learning environments at the K-12 level (Smith, Clark, & Blomeyer, 2005). Smith et al.(2005) emphasized the importance of the empirical studies and the implications of the findings to broad areas in K-12 virtual environments such as the qualities of effective online courses/programs, professional development for online teaching and learning, and student academic achievement. The findings of the present empirical study can add to the knowledge of the exploration to improve the quality of K-12 online education and the adoption of advanced quantitative methodology in online education research design.
At present, no clear set of characteristics have been identified to predict success in virtual learning environments, and no conclusive model has been created to apply in online learning practice (Roblyer & Davis, 2008; Tallent-Runnels et al., 2006). However, two lines of research have emerged to address success factors in online learning: studies focusing on learner characteristics and studies focusing on learning environment characteristics (Roblyer et al., 2008). Learner characteristics include learner's cognitive ability such as locus of control, prior technology skills, learning styles, and self-responsibility. Learning environment characteristics include technology support, course content area, and accessibility to Internet. The success factors investigated in this study such as the learner characteristics including personal effort (participation in academic activities), student learning ability (whether has individual educational plan), race/ethnicity, and family background (participation in free or reduced lunch programs), and learning environment characteristics such as instructor-student interaction (teacher comments have been proved in some studies to correlate to student academic achievement). However, these factors' influences have not been investigated systematically in one single model. The present study will be beneficial to the establishment of one online success model to help improve student academic achievement in online/distance learning environments.
Purpose and Research Questions
The purpose of this study is to investigate the influence of various factors on student science achievement in a K-12 online learning environment. The variables of interest/independent variables in this study include learner characteristics variables including race/ethnicity (RACE), grade level in physical school (GRADE), participation in free or reduced lunch (FRL) programs, whether students have individual educational plans (IEP), status in the virtual school (full-time or part-time, PT/FT), the number of times students logged into the LMS and the time students stay in the LMS, and learning environment characteristic (teacher comments). Students' final score is the dependent variable.
The research questions in this study are also grouped based on the two lines of variables:
Learner characteristics:
Does the level of utilization of the LMS predict online science achievement?
Does student demographic information such as race/ethnicity and grade level predict online science achievement?
Learning environment characteristic:
Does teacher comments/feedback predict online science achievement?
Methodology
Participants, Data Collection, and Data Analysis
The data were collected during the 2007-2008 academic year from one Midwestern state virtual school. This virtual school offers courses at secondary level including math, science, social studies, and communication arts. The student body in this virtual school is composed of students statewide from traditional public schools, private schools, and home school. A single learning management system is utilized to manage all course materials and deliver the instruction.
In 2007, this virtual school collected data about students' academic achievement, LMS utilization, and demographic information within 2007-08 academic year. The data were organized by course and department. Students across the state who completed Biology (first half) and Biology (second half) in this virtual school during 2007-08 academic year participated in this study. There were 211 and 94 students in these two groups respectively.
In the present study, the majority of students in these two groups are White. All the minority groups, including Asian, Black, Hispanic, and Native American had very small sample size. Therefore, all the minority groups were combined as one category of the variable: race/ethnicity. All other categorical variables in this study were also coded accordingly during the data analysis (see Table1).
Coding of the Categorical Variables
| RACE | 0: White student |
| 1: minority student | |
| IEP (individual educational plan) | 0: without individual educational plan |
| 1: with individual educational plan | |
| Student status (part-time, full-time) | 0: part-time |
| 1: full-time | |
| FRL (free or reduced lunch) | 0: not in free or reduced lunch programs |
| 1: in free or reduced lunch programs |
Table 2 shows student demographic information related to ethnicity, whether enrolled in school free or reduced lunch programs, online learning status, and so on. The sample can be described as primarily white, without individual educational plans, part time online students who were not enrolled in school free or reduced lunch programs. For the Biology (first half) group, the majority of students were recruited from Grades 9 and 10 in their physical schools, while for the Biology (second half) group, the majority of students were recruited from Grade 10 and 11 in their physical schools.
Student Demographic Information
| Course | Variable | Category | Number | Percentage (%) |
|---|---|---|---|---|
| Biology (first half) | RACE | 0 | 181 | 85.8 |
| 1 | 30 | 14.2 | ||
| IEP | 0 | 194 | 91.9 | |
| 1 | 17 | 8.1 | ||
| Student Status | 0 | 175 | 82.9 | |
| 1 | 36 | 17.1 | ||
| FRL | 0 | 141 | 66.8 | |
| 1 | 70 | 33.2 | ||
| Grade | 7 | 1 | 0.5 | |
| 8 | 4 | 1.9 | ||
| 9 | 69 | 32.7 | ||
| 10 | 86 | 40.8 | ||
| 11 | 36 | 17.1 | ||
| 12 | 15 | 7.1 | ||
| Total | 211 | |||
| Biology (second half) | RACE | 0 | 78 | 83.0 |
| 1 | 16 | 17.0 | ||
| IEP | 0 | 87 | 92.6 | |
| 1 | 7 | 7.4 | ||
| Student Status | 0 | 75 | 79.8 | |
| 1 | 19 | 20.2 | ||
| FRL | 0 | 68 | 72.3 | |
| 1 | 26 | 27.7 | ||
| Grade | 7 | 1 | 1.1 | |
| 8 | 3 | 3.2 | ||
| 9 | 11 | 11.7 | ||
| 10 | 46 | 48.9 | ||
| 11 | 29 | 30.9 | ||
| 12 | 4 | 4.3 | ||
| Total | 94 |
The physical schools students attend could affect student academic performance via resources school provided for students, technical support, and school culture. Students' final scores within one school are not independent of one another. Any evaluation of the variables at student level such as teacher comments, grade level, and race on student final score must account for the influence of school characteristics on this dependent variable. The HLM technique was carried out by the software program HLM 6.06 for data analysis to account for the clustering of students' final scores within one school caused by the school characteristics. The fully unconditional or Random ANOVA (RA) model was utilized and analyzed to partition the total variance into within-school (Sigma Square) and between-school (Tau) components at the beginning during the analysis for each course. After that, all the independent variables were added into the model. Generalized estimating equation was then applied for the estimation of coefficients of the variables for the two datasets.
Instrument
The measure of success in online learning can be the grades students earn in the courses they are taking and their academic performance on advanced placement exams (Ronsisvalle & Watkins, 2005). Student final score was used by Dickson (2005) as the measure of academic performance in online courses in a study on the investigation of the variability of student performance in online courses. In the present study, the final score students earned in the courses they completed during 2007-08 academic year was used as the indicator of their success in the online courses. The courses were taught by state-certified teachers.
Student final scores on the two biology courses: Biology (first half) and Biology (second half) were collected. The instruments used to collect these data were the tests administered to the students at the end of semesters in
Methodology
Participants, Data Collection, and Data Analysis
The data were collected during the 2007-2008 academic year from one Midwestern state virtual school. This virtual school offers courses at secondary level including math, science, social studies, and communication arts. The student body in this virtual school is composed of students statewide from traditional public schools, private schools, and home school. A single learning management system is utilized to manage all course materials and deliver the instruction.
In 2007, this virtual school collected data about students' academic achievement, LMS utilization, and demographic information within 2007-08 academic year. The data were organized by course and department. Students across the state who completed Biology (first half) and Biology (second half) in this virtual school during 2007-08 academic year participated in this study. There were 211 and 94 students in these two groups respectively.
In the present study, the majority of students in these two groups are White. All the minority groups, including Asian, Black, Hispanic, and Native American had very small sample size. Therefore, all the minority groups were combined as one category of the variable: race/ethnicity. All other categorical variables in this study were also coded accordingly during the data analysis (see Table1).
Table 2 shows student demographic information related to ethnicity, whether enrolled in school free or reduced lunch programs, online learning status, and so on. The sample can be described as primarily white, without individual educational plans, part time online students who were not enrolled in school free or reduced lunch programs. For the Biology (first half) group, the majority of students were recruited from Grades 9 and 10 in their physical schools, while for the Biology (second half) group, the majority of students were recruited from Grade 10 and 11 in their physical schools.
The physical schools students attend could affect student academic performance via resources school provided for students, technical support, and school culture. Students' final scores within one school are not independent of one another. Any evaluation of the variables at student level such as teacher comments, grade level, and race on student final score must account for the influence of school characteristics on this dependent variable. The HLM technique was carried out by the software program HLM 6.06 for data analysis to account for the clustering of students' final scores within one school caused by the school characteristics. The fully unconditional or Random ANOVA (RA) model was utilized and analyzed to partition the total variance into within-school (Sigma Square) and between-school (Tau) components at the beginning during the analysis for each course. After that, all the independent variables were added into the model. Generalized estimating equation was then applied for the estimation of coefficients of the variables for the two datasets.
Instrument
The measure of success in online learning can be the grades students earn in the courses they are taking and their academic performance on advanced placement exams (Ronsisvalle & Watkins, 2005). Student final score was used by Dickson (2005) as the measure of academic performance in online courses in a study on the investigation of the variability of student performance in online courses. In the present study, the final score students earned in the courses they completed during 2007-08 academic year was used as the indicator of their success in the online courses. The courses were taught by state-certified teachers.
Student final scores on the two biology courses: Biology (first half) and Biology (second half) were collected. The instruments used to collect these data were the tests administered to the students at the end of semesters in this virtual school during 2007-08 academic year. According to the virtual school administration, these end-of-course (EOC) tests were designed by the virtual school's course provider and have high correspondence to the state's science content standards. The purpose of the EOC test, as described by the state's department of education (2009), is to:
Measuring student achievement and progress toward postsecondary readiness;
Identifying students' strengths and weaknesses;
Communicating expectations for all students;
Meeting state and national accountability requirements; and
Evaluating programs.
The Biology EOC test includes one session of multiple choice items and one session of performance events (Missouri Department of Elementary and Secondary Education, 2009). The items in the multiple choice session are developed specifically for students in this state. The items in the performance events session are longer, and focusing on more challenging tasks that require students to work through different problems, arguments, or require extended writing.
Results
Descriptive Statistics
Table 3 shows descriptive statistics of the three continuous variables: teacher comments, total logins and total minutes. They all have quite a large range for the two groups, especially for Biology (first half) group. The large standard deviations also demonstrate the big variability of the three variables.
Descriptive Statistics
| Course | Variable | Range | Minimum | Maximum | Mean | SD |
|---|---|---|---|---|---|---|
| Biology (first half) | Teacher comments | 126 | 2 | 128 | 30.28 | 15.175 |
| Total logins | 827 | 0 | 827 | 173.25 | 122.828 | |
| Total minutes | 46,664 | 0 | 46,664 | 9,753.55 | 8,534.783 | |
| Biology (second half) | Teacher comments | 136 | 13 | 149 | 39.85 | 24.397 |
| Total logins | 761 | 2 | 763 | 236.31 | 174.460 | |
| Total minutes | 46,546 | 15 | 46,561 | 14,677.88 | 12,527.161 |
Random ANOVA (RA) Model
RA model was analyzed at the beginning to partition the total variance into within-school (Sigma Square) and between-school components (Tau). See Table 4 for the information.
Overview of RA Model for the Two Groups
| Course | Variables | df | Sigma Square | Tau |
|---|---|---|---|---|
| Biology (first half) | None | 174 | 1,857.87 | 77.92 |
| Biology (second half) | None | 82 | 1,038.67 | 15.78 |
Level-1 Model
Y = B0 + R
Level-2 Model
B0 = G00 + U0
There are no predictors in the RA model. B0 in the RA model is the mean of student final score for one specific school, while G00 is the mean of final score for all students in all physical schools from which the students were recruited into the virtual school. The variances at level-1 (student level) and level-2 (school level) are all due to error. The intra-class correlation (ICC, Tau/(Sigma Square + Tau)) for Biology (first half) and Biology (second half) was 0.08 and 0.01 respectively.
Generalized Estimating Equation
After the analysis of RA model, all predictors were added into the model at level 1. Least-squares estimate with robust standard errors was applied to estimate the effect coefficients for this generalized estimating equation.
Level-1 Model
Y = B0 + B1*(GRADE) + B2*(RACE) + B3*(FRL) + B4*(IEP) + B5*(PT/FT) + B6*(TEACHERCOM) + B7*(TOTALLOG) + B8*(TOTALMIN) + R
Level-2 Model
B0 = G00 + U0 B1 = G10 B2 = G20 B3 = G30 B4 = G40 B5 = G50 B6 = G60 B7 = G70 B8 = G80
B0 in this model is the mean of student final scores in one specific school when all the eight variables are held constant at zero while G00 is the mean of final score for all students when all the eight variables are held constant at zero. G10, G20, and so on, in this model are the mean differences in final score for students who are one point apart on the corresponding variables such as G10 is the mean difference in final score for students who are one point apart on grade level (1 grade level difference, 7 and 8, 8 and 9, etc.). The estimates of the coefficients for these two groups are shown in Table 5. The P values of hypothesis testing for factors which have significant effect on student final score were heighted in grey and explained in more detail. To compare among different factors with respect to the importance in determining student final score, standardized coefficient (β) was calculated according to the formula: βk = bk * Sxk * Sy (bk is the unstandardized coefficient, Sxk is the standard deviation of the corresponding independent variable, and Sy is the standard deviation of the dependent variable). Standardized coefficient demonstrates that how increases in the independent variables affect relative position within the group. For example, the standardized coefficient of GRADE was −17.12 for Biology (first half). It means with 1 standard deviation increase in GRADE, student final score decreased 17.12 standard deviations.
Least-Squares Estimates of Fixed Effects (With Robust Standard Errors)
| Fixed Effect | Coefficient | Standardized Coefficien | d.f. | P value | |
|---|---|---|---|---|---|
| Biology (first half) | GRADE | −0.59 | −17.12 | 202 | 0.734 |
| RACE | −0.64 | −6.85 | 202 | 0.867 | |
| FRL | −11.92 | −172.00 | 202 | 0.002 | |
| IEP | 2.54 | 21.20 | 202 | 0.658 | |
| PT/FT | 4.74 | 54.63 | 202 | 0.413 | |
| Teachercom | 0.23 | 106.71 | 202 | 0.132 | |
| TOTALLOG | 0.01 | 37.55 | 202 | 0.666 | |
| TOTALMIN | 0.001 | 260.93 | 202 | 0.001 | |
| Biology (second half) | GRADE | 4.27 | 123.64 | 85 | 0.115 |
| RACE | −9.95 | −122.02 | 85 | 0.052 | |
| FRL | −4.10 | −59.89 | 85 | 0.369 | |
| IEP | 2.41 | 20.65 | 85 | 0.721 | |
| PT/FT | 3.93 | 51.54 | 85 | 0.497 | |
| Teachercom | 0.13 | 102.96 | 85 | 0.384 | |
| TOTALLOG | 0.06 | 339.80 | 85 | 0.009 | |
| TOTALMIN | 0.001 | 406.66 | 85 | 0.003 |
A significant and strong effect of FRL was observed for Biology (first half) (−11.92, p = 0.002), with students who were not in free or reduced lunch programs achieved higher scores than students in these programs. TOTALLOG has a significant effect for Biology (second half) (0.06, p = 0.009), with students who logged into the LMS more performed better than those who logged into the LMS less. A significant effect of the time student spent in the LMS (TOTALMIN) was observed for Biology (first half) (0.001, p = 0.001) and Biology (second half) (0.001, p = 0.003), both of which show students who spent more time in the LMS achieved higher scores. The standardized coefficient shows the TOTALMIN was the most important factor among the 8 variables for both of the two groups.
Discussion and Implications
In the present study, Random ANOVA (RA) model was analyzed at the beginning to partition the total variance of student final score into within-school and between-school components. The intraclass correlation coefficient was calculated for these two groups and it was 0.08 and 0.01 respectively. These show the between-school variance was small in comparison with the within-school variance for both of these two groups especially for biology (second half), which also tells us the students from different schools are not much different from each other with respect to their academic achievement. After the analysis of RA model, all the predictors were added into the final model. This generalized estimating equation was analyzed for the estimation of the effect coefficients. The variables are of interest to the researchers and have been examined in light of the relationship with student academic performance in other studies.
Learner Characteristics
Number of Times Students Logged Into the LMS, and the Time Students Stay in the LMS
The number of times students logged into the LMS and the time they spent in the LMS could indicate the level of student participation in online learning. The influence of the time students spent in the LMS was found to be positive and significant for these two groups. The influence of the number of times students logged into the LMS was positive and significant for Biology (second half). These findings confirm the belief that students participating in online activities at a higher level tend to perform better in online learning (Wang & Newlin, 2000). It echoes the call for sustained time on task for cognitive learning (Gallagher, 2009). The number of times students logged into the LMS had nonsignificant influence for Biology (first half) (0.01, p = 0.666). It is possible that students in Biology (first half) just started their online experience and were still in the process of adapting to the new learning environment, while the students in Biology (second half) already succeeded in first half of the Biology class and they were more independent learners. Nonetheless, this finding, to some degree, contradicts the belief that the number of times students logged into the LMS has a strong and positive impact on the success of online learning (Dickson, 2005; Dietz, 2002). More study is needed on the activities students are engaged in when they logged into the LMS for a deeper understanding of the findings.
Influence of Participation in Free or Reduced Lunch Programs and Status in the Virtual School
The participation in free or reduced lunch programs had a negative and significant impact on student final score for Biology (first half) group (−11.92, p = 0.002), and a nonsignificant impact for Biology (second half) (−4.10, p = 0.369). It could be due to the growing maturity of the students in Biology (second half) group as compared to those in Biology (first half) group. The influence of this variable on student final score is getting weaker. This outcome aligns with the finding in McLoyd's (1998) study: the magnitude of the relationship between eligibility for school free or reduced lunch program and academic achievement is weaker as grade level rises. The strong and significant influence for Biology (first half) group also confirmed the correlation between the percentage of students in free or reduced lunch program within a school and the school's mean test score (Klein et al., 2000). In the literature on student academic achievement, eligibility for school lunch programs is used frequently as a measurement of students' family SES (Sirin, 2005). The negative influence of participation in free or reduced lunch programs on student academic performance found in this study can add to the body of knowledge about the correlation between SES and academic achievement. Virtual schools should be sensitive to the needs of low-SES students and take measures to assist them with gaps in resources that might influence their achievement.
The effect of student status in virtual school was not significant for the two courses: Biology (first half) (4.74, p = 0.413) and Biology (second half) (3.93, p = 0.497). The direction of the effect shows full-time online students tended to achieve higher score than part-time online students. This finding could lend the relevance to online course design by supporting the integration of components such as teacher-student communication targeting part-time students to help improve their academic achievement or providing assistance in study strategies and time management to maximize the efforts of students who balance courses in online and traditional environments.
Influence of Individual Education Plan (IEP), Grade Level, and Race
In this virtual school, some students were taking online courses for enhancement of their education while others were taking the online course as remediation to earn credit lost as result of failing courses in the physical schools. This virtual school offered individual education plans for these students to enhance their learning. Most of the students who had individual educational plans had some learning disabilities. The effect of IEP was found to be positive for the two Biology courses, though not significant: Biology (first half) (2.54, p = 0.658) and Biology (second half) (2.41, p = 0.721). The effect of student grade level in physical school was not found to be significant for the two groups: Biology (first half) (−0.59, p = 0.734) and Biology (second half) (4.27, p = 0.115). Interestingly, the directions of the effects for the two groups were different from each other. The direction for Biology (first half) shows the students in lower grades tended to perform better. The direction for Biology (second half) shows students in higher grades tended to perform better. Many students in higher grades took Biology (first half) in this virtual school as the remediation to make up credit not earned in their physical schools. The different signs of the effect of grade level for the two groups could indicate that the virtual school is helping the higher grade level students to achieve their academic goal of completing high school. The effect of race/ethnicity was not significant for Biology (first half) (−0.64, p = 0.867), nearly significant for Biology (second half) (−9.95, p = 0.052), with White students tending to perform better than minority groups as a whole. The combination of different minority groups into one category could mask important information regarding the differences in student academic achievement among different racial groups. More study is needed to increase the sample size for the investigation of these differences.
Learning Environment Characteristic
Influence of teacher comments
Teacher comments and feedback have been identified as important factors for online success in other studies. Surprisingly, its effect was not found to be significant for the two Biology courses in the present study: Biology (first half) (0.23, p = 0.132), Biology (second half) (0.13, p = 0.384). It could be due to the small sample size. More study is needed with a larger sample size and on the form and content of teacher feedback for insightful explanation. The positive direction of the effect shows students with more teacher comments tended to perform better than those receiving fewer comments. This finding could inform the instructional design process to integrate more teacher-student communication during the development of online courses.
Conclusion
The purpose of this study was to investigate the effect of various variables including student's demographic information and the utilization of the LMS on student science achievement in a K-12 online learning environment. The results of the study show different variables could affect student academic achievement in different ways. For example, the time students spent in the LMS affected student final score in Biology courses positively and significantly. This finding can help improve the development of user-friendly LMS with interfaces that motivate students to spend more time in the system engaging in academic activities delivered via the LMS, as well as teaching practices that foster connectedness among teachers and students. The participation of reduced or free lunch programs affected student final score in Biology (first half) negatively and significantly. This finding could lend relevance to the development of online courses that integrate components targeting improved academic performance for students with lower family SES.
Given the dearth of research on success in online learning at the K-12 level, this study could benefit the stakeholders of online education including educators, researchers, administrators, policy makers, courses designers, online program leaders, and society at large. The investigation provides a richer understanding of student success in online leaning in general and in the science field in particular. It can shed light on the implementation of virtual learning environments to help bridge the gap in student science achievement between the U.S and other developed countries. It is relevant for the policy making process regarding K-12 online education to better prepare the science workforce for this country in the long run.
Limitations And Suggestions For Further Research
The limitations in this study include the nonrandomized selection of the student samples from the physical schools into this virtual school. Additionally, the sample size was small, which could negatively affect the statistical power of analyses given the significance level was set at .05 in this study. The small sample size also caused many physical schools had a very small number of students participating in this study (some only had one participant). Readers should be cautious about the generalization of the findings in this study. Further study can be conducted with randomized sampling techniques for higher generalizability and with a bigger sample size for more powerful hypothesis testing. Factors influencing student science achievement were only investigated for Biology courses in this study; future study can be conducted for other science courses such as physics and chemistry for a richer understanding of the success of online learning in the science field as a whole. Furthermore, qualitative data could be collected and analyzed to seek deeper explanation and interpretation of the phenomena in virtual learning environments.
