An Exploratory Study of Self-Efficacy for K–12 Online Teachers Free

Self-efficacy is a belief in one’s ability to organize and execute a plan of action with intended results (Bandura, 1997). The concept of selfefficacy is a key element in Bandura’s social cognitive theory, which is the belief in human agency or that people can exercise some level of control over their own lives (Goddard et al., 2000). When applying social cognitive theory to a learning environment, one of the exciting areas of research involves teacher self-efficacy. Broadly speaking, teacher self-efficacy is a teacher’s belief that they can positively affect student outcomes and learning.

Research has found that in traditional face- to-face classrooms, teacher self-efficacy is positively associated with student learning (Goddard et al., 2000; Martin & Marsh, 2006; McCoach & Siegle, 2007; Tschannen-Moran et al., 1998). An understanding of this relationship begins with an examination of student success in K-12 classrooms. To better understand the factors that influence student success, Pianta et al. (2008) put forth the Classroom Assessment Scoring System, which assesses overall classroom quality across three dimensions-classroom organization, instructional support, and emotional support. These dimensions are related to improved student outcomes (e.g., Bransford et al., 2000; Hamre & Pianta, 2001; Raver, 2004) and are positively impacted by an increase in a teacher’s self-efficacy.

While we have existing knowledge about the association of teacher self-efficacy in K-12 face-to-face learning environments, we do not have the same level of knowledge about teacher self-efficacy in K-12 online learning environments (Gosselin, 2009). This relationship is essential given the growth in the number of students participating in these environments. There appears to be no comprehensive source of enrollment information that covers both full-time and part-time K-12 distance learning; however, it is estimated that hundreds of thousands of students now receive their K-12 education exclusively online while potentially millions more participate in parttime or supplemental online programs (Digital Learning Collaborative, 2019; Gulosino & Miron, 2017; Miron et al., 2018). With increased attention and enrollments in online learning environments, it has become increasingly clear that research in teacher self-efficacy in these new, virtual classrooms is not only timely but essential to understand better and support student success of our K-12 online learners. One barrier to this understanding is the lack of a tool for measuring self-efficacy that has been validated for use with K-12 teachers that operate in online-only environments (Corry & Stella, 2018). This barrier becomes even more critical when considering that components of instructional design or delivery that may be important for online teachers (e.g., asynchronous vs. synchronous delivery, facilitating safe and productive online interactions) may be underrepresented in a tool designed for use with face-to-face teachers for whom these components may be less important.

With this as a background, this paper reports on an exploratory research study that investigated online K-12 teacher’s selfefficacy over time and explored the measurement properties of a modified version of the Teacher’s Sense of Self-Efficacy Scale (TSES) (Tschannen-Moran & Woolfolk Hoy, 2001). TSES has been extensively used with face-to- face K-12 teachers and is widely accepted as a measure of self-efficacy in teachers. In this study, the TSES instrument was modified to align it more closely with the structure of online learning. This modified instrument was used to measure levels of teacher self-efficacy on four occasions over 5 months. In addition, those teachers who participated in the study were provided in-service training involving how to improve student interaction in their online classrooms. This professional development training was given to stimulate teacher self-efficacy and examine the length of its effects. As this was primarily an exploratory study, few a priori hypotheses were generated. It was hypothesized that there would be an initial increase in teacher self-efficacy (preintervention to immediately postintervention). Additionally, it was expected that the extant TSES items (i.e., those from the published short form of the instrument) would exhibit the factor structure suggested in the literature (i.e., Tschannen-Moran & Woolfolk Hoy, 2001).

One of the more widely tested instruments used to measure teacher self-efficacy in traditional face-to-face classrooms is TSES (Tschannen-Moran & Woolfolk Hoy, 2001). This instrument includes both a long (24 items) and short (12 items) form. It provides an ability to calculate subscale scores in efficacy in student engagement (ENG), efficacy in instructional strategies (IS), and efficacy in classroom management (CM). The instrument is supported by validity and reliability evidence (Klassen et al., 2009; Tschannen-Moran & Hoy, 2001; Wolters & Daugherty, 2007) and has been widely applied in practice (e.g.,Caprara et al., 2006; Klassen & Chiu, 2010; Ross & Bruce, 2007; Sang et al., 2010).

In prior research, the TSES was modified for use in online higher education settings but not for online K-12 settings. Specifically, in 2010, the TSES was modified for use with online faculty in a higher education setting in nursing education. This modification resulted in the Michigan Nurse Educators Sense of Efficacy for Online Teaching (Robinia & Anderson, 2010), which maps onto the long form of the TSES. This scale has not, however, been validated for use with online K-12 instructors. The current study sought to explore the psychometric properties of an adapted version of the TSES short form in a K-12 online setting. Researchers began by comparing and mapping the short version of the TSES to similar items from the Michigan Nurse Educators Sense of Efficacy for Online Teaching. Once mapping was completed, the items were modified, with the permission of the authors of both measures, to match an online K-12 environment better. This modification resulted in 12 core questions mapped to 12 questions used in the TSES short form. Researchers then added five questions that focused on efficacy in student interaction (INT) in an online K-12 environment and were scored on the same Likert scale as the existing items. These items were added because student interaction has been an essential component of student satisfaction in online learning. Lastly, the researchers added 10 demographic questions to the survey. These demographic questions measured age, race, gender, grade level, subjects taught, teaching experience (both overall and online), highest level of education, full or part-time teaching, and a self-reported level of technology expertise. The resulting instrument is called the Online Teacher Self-Efficacy and Classroom Interaction Survey (OTSECIS). Table 1 presents the text of each of the 17 content items and its intended subscale for each. The analyses described in a later section investigated the psychometric properties of this new instrument.

The OTSECIS survey was administered four times. The first administration was immediately before a professional development session to instruct teachers on how to improve student interaction in their classrooms. It was attended by K-12 teachers who work entirely in an online classroom. The second administration of the survey was given to the same group of teachers immediately after and on the same day as the professional development session. The third administration was given to the same group of teachers approximately 1 month after the professional development, while the fourth administration was approximately 5 months after the professional development. The timing of the administrations of the instrument was used to measure a baseline level of online teacher self-efficacy (before the professional development), the immediate impact of the professional development on online teacher self-efficacy, and the lasting effects of the professional development on online teacher self-efficacy.

In total, N = 79 online-only K-12 educators from across the United States responded to a minimum of one time point. The sample sizes at each time point were N = 71 (Time 1; some participants did not provide data at Time 1), N = 57 (Time 2), N = 32 (Time 3), and N = 37 (Time 4). Table 2 presents a summary of the sample characteristics in frequencies as a function of the demographic data collected. A total of N = 72 participants provided demographic data, with one participant not responding to the age or gender variables.

Initial analyses focused on missing data screening to determine the underlying missing data mechanism. Following that initial step, data analysis proceeded along two lines- (a) an exploration of the psychometric properties of the OTSECIS and (b) analyses focused on patterns of change across time and comparisons by demographic categories for each of the four subscale composite scores.

Missing Data Screening. A central problem is the presence of missing data in determining the mechanism that underlies that missingness (Enders, 2010). In the present study, our primary concern was that, with components of self-efficacy being the outcome of interest, teachers with lower self-efficacy might have been more/less likely to drop out of the study (i.e., the missing data mechanism is missing not at random). In order to evaluate the validity of this concern, a count variable was created corresponding to the total number of time points at which each participant provided data(Freq in Table 2; Range: 1-4). The Spearman correlation coefficients between this variable and each of the four outcomes (ENG, IS, CM, and INT) at Time 1 were estimated. Estimates small in magnitude were considered favorable. They suggest no, or at least a limited relationship between the extent of the respondent’s participation in the study and their self-efficacy. Correlation coefficients between Freq and each of the demographic indicators (collected at Time 1) were also estimated to determine whether any of said variables could be of potential use as auxiliary variables.

Psychometric investigation. Exploratory factor analysis (Fabrigar & Wegener, 2011) was used to estimate the patterns of common variance among the 17 items included in the scale using the Time 1 item responses. Analyses were conducted in Mplus (v7; Muth�n & Muth�n, 2002) using default methods of weighted least squares estimation (Asparouhov & Muth�n, 2010) with oblique (GeoMin; Browne, 2001) rotation. While weighted least squares estimation is not as well-equipped at handling missing data as other estimation routines (full-information maximum likelihood, for example), it is computationally efficient when using categorical indicators. Given the small amount of missingness at Time 1 (all items had a 100% response rate save for Q3 and Q13, which had a 98.6% response rate), the use of weighted least squares was considered appropriate. Follow-up analyses were conducted based on the results from these initial models and are detailed below. Given the relatively small sample size, invariance testing by subgroup was not performed.

Patterns of change and group comparisons. Patterns of change were evaluated graphically (via line plots over time) and via dependentsamples t tests (comparing pre/postinterven- tion scores). Some effort was made to estimate latent growth curve models but, given the relatively small sample size and attrition rate, the result of those efforts should be regarded with some skepticism. Finally, a multivariate regression model was fit using maximum likelihood estimation (Myung, 2003) to evaluate the impact of the various demographic variables (age, gender, online/overall teaching experience, level of education, part-time/full- time employment status, and self-reported level of comfort in using technology) on the self-efficacy component scores at Time 1.

Table 3 presents the Spearman correlation coefficients for each pairwise combination among the four outcomes (ENG, IS, CM, and INT), the demographic indicators, and the number of time points for which the respondent provided data (Freq). Of note is that the coefficients pertaining to the relationship between attrition and self-efficacy outcomes were all nonsignificant and relatively small in magnitude. No auxiliary variables were considered for inclusion in any of the models that followed the screening.

Table 4 presents the factor loadings for an exploratory factor analysis consisting of four extracted factors. Applying two rough criteria (Cattell, 1966; see the scree plot in Figure 1 and Kaiser, 1960) suggested that between three and six factors be retained. The four- factor solution is presented as it best aligns with the anticipated structure of four selfefficacy components.

Given the lack of coherence exhibited by the new items intended to measure interaction (INT), the decision was made not to combine scores from these items into a single composite score for the remaining analyses. Instead, fol-low-up item-level analyses were conducted. These analyses looked at change across time and differences as a function of demographics. Results of these analyses are reported in the subsequent sections. Additionally, the decision was made to drop Q11 from the subsequent analyses as it did not load with the other items from the TSES short form with which it would have been expected (i.e., the ENG subdomain). A literature search provided some precedent for removing this item due to potential validity issues in some samples (Daniels et al., 2017; Klassen et al., 2009). The ENG composite scores were recalculated without Q11 for all remaining analyses.

The line plots in Figure 2 show the change in each of the three outcomes (ENG, IS, and CM), while Figure 3 presents the average response to the five new items (Q13-Q17) over the four time points. Table 5 presents the results from a dependent samples t test evaluating the significance of the change in each outcome/item from Time 1 to Time 2 (pre- and posttraining) and Time 1 to Time 4 (pretraining to +5 months).

Multiple regression was used to evaluate the utility of various demographic indicators as predictors of the three outcomes of interest (ENG, IS, and CM) at Time 1. Several of the demographic variables were recoded to be dichotomous in order to aid with sparseness and interpretability. The resulting variables were:

• Age (0 = 20-29 / 1 = 30-39 / 2 = 40-49 / 3 = 50-59 / 4 = 60-69 / 5 = 70+)

• Gender (0 = male / 1 = female)� ExpOn (0 = 0-5 years / 1 = 6-10+ years)

• ExpOvr (0 = 0-10 years / 1 = >10 years)

• EdLevel (0 = bachelor’s / 1 = master’s or higher)

• PT (0) / FT (1)� Tech (0 = novice or intermediate / 1 = advanced or expert)

Table 6 presents information pertinent to the results of those analyses.

The current study aimed to explore using a modified instrument used to measure self-efficacy as it related to K-12 teachers that teach in online settings. Additionally, the study was used to explore the impact of professional development training on teacher self-efficacy over an extended period. This exploration included evaluating a modified instrument for measuring self-efficacy based on an existing measure that is widely applied in research involving face-to-face teachers. This modified instrument offered a unique contribution to the measurement of online teacher self-efficacy in that it (a) was designed to be used with online K-12 teachers, (b) focused heavily on selfefficacy, and (c) consisted of a relatively small number of items thus making it more efficient to administer. The modified instrument was used to collect data from a sample of online K- 12 educators immediately preceding and following an online professional development that focused on improving online student interaction. The instrument was administered at two later time points. These data were analyzed for patterns of change over time and differences by demographic subgroup concerning the three domains of self-efficacy (ENG, IS, and CM) and an additional domain added to the modified instrument (INT). Given the relatively small sample size, it was not expected that benchmarks such as statistical significance would be reached. Instead, the hope was that the results of this exploration would be compelling enough to provide an impetus for further, more targeted research using the new measure. With these goals in mind, the remainder of this section is dedicated to a discussion of the results, including implications and opportunities for further research.

This study provided quite a bit of interesting and helpful information regarding the validity of the new items about the existing carry-over items in the OTSECIS. For example, psychometric analyses provided little evidence for the validity of the new items intended to capture self-efficacy related to student interaction (the INT subscale score). It may have been the case that some items were not specific enough to form a cohesive factor, which is evidenced by the fact that three of these five new items tended to group with items related to instructional strategies in the ENG subscale score for efficacy in student engagement. This information is beneficial in continuing to fine-tune this instrument. An obvious direction for further refinement of the five new items in the INT subscale score would be developing and piloting new or modified items. Another option would be to drop those five items entirely since there is evidence that they are highly related to the existing items found in the ENG subscore. Given the overlap between student engagement and student interaction, eliminating the five new items should be strongly considered. This finding provides the researchers with guidance on how to improve the instrument and better measure teacher self-efficacy in student engagement and interaction.

Change over time was another interesting finding of the study. Participants experienced an initial “spike” in domain scores immediately following the workshop. This spike indicates that participation in the professional development workshop resulted in immediate higher teacher self-efficacy scores across the board. In another worthwhile finding, it appears that the impacts of professional development on teacher self-efficacy declined with time. By the time that last collection point had been reached (5 months after the professional development workshop), the effects of this initial increase had all but dissipated, with Time 4 scores often being lower than their preworkshop (Time 1) levels. The professional development workshop may be the reason for the slow decline in teacher self-efficacy after an initial spike. It is entirely possible that the professional development workshop materials and training were geared toward short-term gains in self-efficacy. An examination and possible redesign of the training with another round of surveys would be an interesting future research project. It would provide greater insight into the role of the professional training materials on the length and magnitude of self-efficacy gains.

One additional explanation for this lack of longer term effect may be that teacher selfefficacy, for online-only educators, is a relatively nonmalleable construct and/or that growth in self-efficacy occurs on a longer timeline (with additional, frequent professional development training) than what the current study was able to capture. This finding would mean that for professional development to have a long-term effect on online teacher self-efficacy, it should be repeated at intervals less than every 5 months. An informative future study could examine the optimal timing for professional development activities related to online teacher self-efficacy. This future study could examine the total length of time that professional education is delivered and the intervals between training sessions, and the length of each session. Having a proven instrument to measure online K-12 teacher self-efficacy is key to moving this research forward.

Another important finding of the study is found in the analyses by demographic subgroups. Only two unstandardized regression coefficients had a magnitude of greater than a quarter of a scale point-online and overall teaching experience predicting self-efficacy in fostering student engagement. Perhaps counterintuitively, though these coefficients were opposite in sign with increasing self-efficacy being indicated by more overall teaching experience but less online teaching experience. Given the study’s limitations (small sample size, potential cross-validations issues with the instrument), it may be that this is an entirely spurious effect. One could hypothesize that there may be something about teaching in an online setting (e.g., the physical separation from students) that, over time, lessens a teacher’s confidence in his or her ability to keep students engaged. It could also be that teachers with more online teaching experience may have been negatively impacted by the less interactive formats that were common when online courses were in their infancy. An interesting follow-up question would be whether engagement self-efficacy is impacted by the balance of time spent teaching in online versus face-to-face formats. This balance could be represented by the difference in online and overall teaching experience-a value not easily derived from the Likert items used in the current study.

Examinations of the demographic subgroup show that overall, teachers who participated in the study were very experienced in their profession. Of the entire group, 21 participants had between 5-10 years of teaching experience, and 51 participants had 10 or more years of teaching experience. None of the participants had less than 5 years of teaching experience. This relatively senior group of teachers, with a wealth of teaching experience, provides us with an opportunity to consider this impact on overall teacher self-efficacy, gains, and long-term retention of increased self-efficacy. The length of experience teaching may have an impact on teacher self-efficacy. Without a subgroup of less experienced teachers in the current study, it is hard to examine this possibility. It would be very interesting to conduct a similar study with less experienced K-12 online teachers as a subgroup. This future study could examine their self-efficacy gains and retention and be used to compare to more experienced teachers. Further study would provide some fascinating and significant research.

Several possible future research studies have already been mentioned in the previous discussion of the findings of this study. Overall, the psychometric work detailed here suggests ample opportunity for further research in refining the modified elements of the current OTSECIS. As noted, having a valid and reliable instrument to measure K-12 online teacher self-efficacy is key to future research.

In addition to the possible future research studies already discussed, another area to be considered for future research could involve using the OTSECIS instrument with other populations. The current OTSECIS contains items that are specific to K-12 online teachers. New items could be written to use across populations (e.g., higher education instructors, corporate trainers). An item pool consisting of items tailored to each target population could be created for researchers to draw from in constructing an instrument for their particular research needs. Constructing a large pool of novel items would, of course, require a large pilot sample in order to calibrate the items as well as investigate issues such as differential item functioning. These new items could allow for more informative items, including response modalities not currently represented in the OTSECIS (open-ended items, for example).

Overall, the exploratory study presented here provides a good starting point for further research in important areas involving self-efficacy. Additional research focusing on the intervention, survey instrument, and student outcomes are critical areas in this ongoing study of online teacher self-efficacy and how it impacts K-12 online student success.