The development of instructor social connectedness instrument to predict student satisfaction, motivation and perceived learning in online courses Free

The growth of online education in higher education has highlighted the importance of instructors connecting with their students to foster student engagement, satisfaction and learning outcomes. Instructor social presence and connectedness have been shown to significantly impact student engagement, satisfaction and learning outcomes in online and traditional educational settings. Studies have found that instructor social presence can be correlated with increased learning satisfaction, engagement, achievement and learners’ perceptions of the instructor (Oyarzun, Barreto, & Conklin, 2018). Establishing instructor presence through frequent communication and interaction with students and providing support throughout the learning process have been identified as a crucial factor in fostering student engagement and improving learning outcomes (Collıns et al., 2019; Oyarzun et al., 2018).

A well-known framework for understanding and facilitating effective online learning experiences is the Community of Inquiry (CoI). It views learning in virtual environments as supported by three interacting presences: cognitive, social and teaching (Al-Saggaf & Rosli, 2021; Swan, 2019). While the CoI model has been widely used to understand online learning environments, recent research suggests that instructor presence may be distinct from teaching presence. This study focuses on the concept of instructor social connectedness, defined as the ability to establish an emotional connection with students through various communication methods (Conklin & Garrett Dikkers, 2022).

Building on previous research, a framework designed to understand instructor social connectedness in online courses has been introduced. This framework is FORCES – Feedback, Organization, Response time, Communication, Empathy and Sociability, and it emerged from studies conducted with undergraduate and graduate students at a four-year public university between May 2020 and March 2021, exploring the facilitation strategies that resonated with students and aided in establishing connections with instructors (Conklin & Garrett Dikkers, 2021, 2022).

This present study aims to validate a survey instrument designed to measure instructor social connectedness in distance higher education, specifically examining how the FORCES framework can assess student satisfaction, motivation, and perceived learning. By investigating these relationships, we seek to provide insights into effective online teaching strategies and contribute to the ongoing discourse on enhancing the quality of distance education.

This research is particularly timely, given the increased reliance on online learning platforms and the need for evidence-based approaches to improve student outcomes in virtual environments. Through a rigorous process of survey development, expert review, and think-aloud protocols, an instrument has been created to assess the multifaceted nature of instructor social connectedness and its impact on key aspects of the student experience.

The most prominent theoretical framework cited in effective online learning studies is the Community of Inquiry (CoI) model (Arbaugh et al., 2008; Garrison, Anderson, & Archer, 2003). The CoI model defines effective online instruction as the overlap between teaching, cognitive and social presence. Cognitive presence refers to the extent to which learners can construct knowledge through discourse and reflection (Swan & Ice, 2010). Social presence encourages a collaborative online learning environment. It is defined as the ability of learners to feel affectively connected with their peers and perceive their full personality through computer-mediated communication (Garrison, Anderson, & Archer, 2000; Swan & Ice, 2010). Teaching presence has been defined as the crucial construct that facilitates cognitive and social presence as “the design, facilitation, and direction of cognitive and social processes” (Anderson, Rourke, Garrison, & Archer, 2001, p. 5). Research has found that teaching presence significantly predicts cognitive and social presence (Garrison, Anderson, & Archer, 2010; Gutiérrez-Santiuste, Rodríguez-Sabiote, & Gallego-Arrufat, 2015; Lin, Hung, & Lee, 2015).

In the CoI model, teaching presence includes everything the instructor does, from structuring the course, providing lectures, explaining assignments, creating assessments, providing feedback, answering questions, etc. Other researchers have argued that instructor presence and teaching presence are distinct (Richardson et al., 2015). Richardson et al. (2015) concluded that teaching presence is focused on the pedagogical design and development of the course, and instructor presence focuses on the implementation of instruction in an online setting and reflects the online personality or interactions an instructor has with a student (Watson, Sullivan, & Watson, 2023). There are two distinct skills in teaching presence: course design/development and facilitation. This distinction has led to the development of new frameworks for understanding instructor–student interactions in online environments.

Many terms, such as instructor presence, instructor connectedness and instructor social connectedness, have been associated with the facilitation or interconnectedness between instructors and their students. Instructor social connectedness is defined as “the ability to establish an emotional connection with students through multiple means of communication.” (Conklin & Garrett Dikkers, 2022, p. 68). This emotional presence can be established through: (1) teaching presence or course design and (2) social presence, the instructor being perceived as real, instructor connectedness or communication.

In conjunction with these theories, instruments have been created and validated to measure the various concepts surrounding instructor social connectedness, such as the instructor social presence scale (Pollard, Minor, & Swanson, 2014), Community of Inquiry scale (Arbaugh et al., 2008), instructor–student relationship connectedness scale (Creasy et al., 2009), online student connectedness survey (Bolliger & Inan, 2012) and classroom community scale (Ravai, 2002). Table 1 summarizes the details of these scales.

Pollard et al. (2014) posited that the social role of the instructor in online courses warranted further examination and suggested that the CoI add a fourth construct to measure student perceptions of instructor social presence (ISP). The ISP instrument was developed based on a review of the literature and validated through two pilot studies.

The SIRS instrument (Creasy et al., 2009) primarily assesses instructor connectedness regarding students’ perceived connection to the instructor rather than evaluating the specific strategies employed to foster such connectedness. The SIRS was designed to capture the student perceptions of the central relationship dimension of connectedness and anxiety and found that these were tied to positive achievement and student confidence.

The OSCS, introduced in 2012, is specifically designed to measure feelings of connectedness among students in online degree and certification programs (Zimmerman & Nimon, 2017). It provides data on students’ perceptions of connections in online courses and has demonstrated evidence of factor validity and reliability. The OSCS has been utilized to assess student satisfaction with a sense of community in online programs and has revealed disparities based on gender and credit hours (Jamison & Bolliger, 2019).

In contrast, the CoI framework developed by Garrison et al. (2000) is a more comprehensive model that examines the online learning experience through three interrelated elements: social presence, teaching presence and cognitive presence (Arbaugh, 2008; Richardson et al., 2012). The CoI Survey has been extensively validated and employed to predict perceived learning and delivery medium satisfaction in online education across various disciplines (Arbaugh, 2008; Bangert, 2009; Stewart, 2019).

While both the OSCS and COI instruments aim to assess aspects of community building in online learning, they differ in scope and application. The OSCS focuses specifically on student connectedness, whereas the CoI framework provides a broader perspective on the entire online learning experience. The CoI model has been applied to various educational contexts, including blended learning environments and different disciplines such as management education and engineering (Chudaeva, Blodgett, Loth, & Somaskantha, 2023; Shea & Bidjerano, 2013).

While the Classroom Community Scale (CCS) (Rovai, 2002) measures connectedness, its primary focus is on the sense of connectedness, cohesion, spirit, trust and interdependence among course participants. CCS provides researchers and educators with a validated instrument to assess the community in online learning environments. This assessment can inform efforts to enhance course design and instructional approaches to foster a stronger sense of community and improve student retention in distance education programs. However, the focus is predominantly on student interactions rather than strategies to cultivate instructor connectedness. Thus, there is a need for a more comprehensive instrument that directly measures instructor connectedness in both the course design and facilitation of online courses.

The FORCES framework provides a comprehensive structure and broader constructs to represent instructor social connectedness accurately. The FORCES framework, which stands for Feedback, Organization, Response time, Communication, Empathy and Sociability (Conklin & Garrett Dikkers, 2021, 2022; Oyarzun et al., 2018), was developed based on qualitative research conducted between May 2020 and March 2021. This study explored online facilitation strategies that resonated with students and facilitated connections with their instructors. The research yielded seven themes that formed the foundation of the FORCES framework. These strategies closely align with the facilitation construct of the teaching presence scale from the Community of Inquiry (CoI) or the facilitation construct from the Students Connectedness Survey; however, FORCES further operationalizes the facilitation construct for a more nuanced understanding of effective strategies to build instructor social connectedness. For instance, the CoI Teaching Presence scale and the Online Student Connectedness Survey (OSCS) contain six facilitation items that are not operationalized into further sub-categories. In contrast, the FORCES instrument includes seven constructs and thirty items.

Feedback has been identified as a crucial component of instructor social connectedness. For instance, students highly value individualized feedback designed to develop mastery of knowledge or skills (Watson et al., 2023). Other studies support this finding (cf., Martin, 2019).

Although primarily associated with design and development, course organization has contributed to students’ perceptions of instructor presence. A well-structured course can serve as the first impression of the instructor and facilitate interpersonal relationships while providing subject matter expertise (Watson et al., 2023). Heilporn and Desrochers (2020) suggest that effective course organization includes dividing content into weekly modules with introductory guidelines. Students also want other organizational strategies, such as weekly checklists (Kelly, Colella, & Sottosanti-Kusnir, 2024; Minichiello, Lawanto, Goodridge, Iqbal, & Asghar, 2022).

Response time is another critical factor for establishing an instructor's social presence. Students enrolled in high-quality online classes expect highly responsive instructors (Van Wart et al., 2020) and value timely instructor feedback (Watson et al., 2023).

Communication plays a vital role in fostering instructor-student connections. Learners value multilevel online interactions, particularly with instructors (Adanir et al., 2020). These interactions can be facilitated through various means, such as comments, forum discussions, presentations and announcements (Heilporn & Desrochers, 2020; Minichello et al., 2022).

Empathy and sociability are also important aspects of instructor social connectedness. Students prefer communication strategies that demonstrate social connectedness, such as the instructor reaching out via email to check their well-being (Kelly et al., 2024).

These themes collectively contribute to the concept of instructor sociability, which extends beyond merely social presence to projecting oneself as a genuine individual in an online environment. The authors posit that sociability encompasses the awareness of students’ specific circumstances and demonstrates concern for their well-being. For example, Kelly et al. (2024) found that students find instructor presence to be the key to their success.

(See Figure 1) Research suggests that instructor social connectedness significantly influences student satisfaction, motivation, and perceived learning in online courses. Lee, Song, and Hong (2019) found that learners’ satisfaction depends on their motivation to participate, perceptions of the learning process and engagement in applying acquired knowledge.

This study aimed to validate a survey instrument designed to measure instructor social connectedness in online higher education. Specifically, this study aimed to assess the extent to which this instrument could accurately predict students' satisfaction, motivation and perceived learning. By examining the constructs within the FORCES framework – Feedback, Organization, Response time, Communication, Empathy and Sociability – this study seeks to determine the effectiveness of these facilitation strategies in fostering meaningful instructor–student connections, thereby enhancing the overall online learning experience.

1. Does the FORCES instrument reliably measure instructor social connectedness in distance higher education?

2. To what extent does the FORCES instrument demonstrate content-related validity?

3. Does FORCES produce reasonable face validity?

4. To what extent does the FORCES instrument prove construct-related validity, as evidenced by the confirmatory factor analysis (CFA)?

The survey instrument was developed based on the findings of two foundational studies (cf., Conklin & Garrett Dikkers, 2021, 2022). These qualitative studies investigated instructor social connectedness and strategies facilitating connectedness between instructors and students in asynchronous online learning environments. Based on a review of the two foundational studies, the research team crafted survey items to assess the impact of the FORCES framework on student satisfaction, motivation and perceived learning. Content experts reviewed the initial survey items to ensure the instrument's content validity. The survey items were further refined based on qualitative feedback from students.

After completing the initial survey draft, subject matter experts were invited to review the survey items thoroughly. The team comprised two experts in instructional design and online education. Both experts have been in the field for over 10 years, and their research has focused on online learning. The team used a content validity index (CVI) tool to assess the content validity of each item. The survey items were evaluated for clarity and relevance using a 4-point scale, where 1 = Not relevant, 2 = Somewhat relevant, 3 = Quite relevant and 4 = Highly relevant. The Item-Level Content Validity Index (I-CVI) was calculated by dividing the number of experts who rated the item as either 3 or 4 by the total number of experts. Given the limited number of reviewers, we used this index to identify items that were potentially ill-written or did not represent the construct of interest. Thus, for any item with reviewer disagreement (I-CVI <1), we revisited those items for clarity and relevance and revised the items to improve quality.

The survey underwent a think-aloud protocol following the expert review to further refine the questions. This qualitative method allowed participants to verbalize their thoughts as they read each survey item, providing insight into their comprehension, interpretation and reaction to the survey content. The goal was to identify ambiguities or misinterpretations that could compromise the clarity and effectiveness of the items.

Participants. Five participants were recruited for the think-aloud sessions, including two graduate and three undergraduate students from a southeast university. The participants were selected from various backgrounds and disciplines, including a Master of Instructional Technology, Doctorate of Coastal Engineering, environmental science/biology and two exercise science students. To ensure that the survey was clear, it was important to have students who were unfamiliar with educational jargon.

Procedure. Each participant completed a think-aloud session conducted virtually via Zoom. Participants were instructed to read each survey item aloud and share their thoughts or reactions that came to mind, including whether the item was clear and how they interpreted the wording. At least two researchers attended each session and took comprehensive notes.

Once the instrument was finalized, it was distributed to online students at two different United States Southeast region institutions. The survey was administered electronically, and participation was voluntary. The target population included students enrolled in various online programs, ensuring a diverse sample in academic discipline, level of study (i.e. undergraduate or graduate), and demographic characteristics. The data collected from this survey were used to validate the instrument.

To evaluate the internal consistency reliability of FORCES, the Cronbach's alpha coefficient was computed for each subscale as well as the entire set of FORCES. A high-reliability consistency ensures that items hypothesized to evaluate the same construct (or subconstruct) consistently measure it.

Construct-related validity was evaluated using confirmatory factor analysis (CFA). CFA tests the plausibility of the factor structure identified by the theoretical framework used to develop an instrument. FORCES was designed to encompass six distinct sub-constructs, so a six-factor solution was assessed.

To draw valid results, CFA requires a set of assumptions that need to be checked prior to the analysis. We examined the data for extreme outliers and did not identify any. To evaluate factorability, the Kaiser–Meyer–Olkin test was conducted, producing a KMO value of 0.95. KMO values greater than 0.60 are considered adequate for proceeding with factor analysis (Hutcheson & Sofroniou, 1999). However, the Shapiro-Wilk multivariate normality test was rejected, w = 0.630, p < 0.01, suggesting that a typical maximum likelihood estimator may not be adequate. Thus, the robust maximum likelihood estimator with Huber–White standard errors, which does not require multivariate normality, was used. CFA analyses were conducted using the R package, lavaan (Rosseel et al., 2017).

Previous studies have established thresholds to determine the appropriate level of model fit for various fit statistics. This study used the chi-square goodness of fit test, comparative fit index (CFI), root mean square error of approximation (RMSEA) and standard root mean squared residual (SRMR) statistics to evaluate model fit. Specifically, RMSEA and SRMR values less than 0.06 were regarded as satisfactory fit (Browne & Cudeck, 1992; Kline, 2015). CFI values greater than 0.90 were considered adequate (Kline, 2015).

High internal consistency values were found for all subscales of FORCES despite the small number of items measuring each sub-construct: Feedback (⁠$α$ = 0.96), Organization (⁠$α$ = 0.92), Response time (⁠$α$ = 0.96), Communication (⁠$α$ = 0.92), Empathy (⁠$α$ = 0.95) and Sociability (⁠$α$ = 0.96). The alpha coefficient was even higher for the entire set of FORCES, $α$ = 0.98. Kline (2015) and Nunnally (1994) considered alpha values greater than 0.70 as acceptable, greater than 0.80 as adequate, and greater than 0.90 as good. This demonstrates the reliability of FORCES in measuring the same construct across items in the instrument.

Items five through eight focused on the organization of the course. One expert felt that course organization was not related to facilitation. Although the organization of a course is typically considered course design rather than facilitation, previous research has found that students connect with instructors who have well-designed classes since they perceive the instructor cares about their teaching (Conklin & Garrett Dikkers, 2021). Based on the foundational studies, organization was a clear theme for creating instructor social connectedness; thus, those questions remained in the survey.

Seven items received an I-CVI rating of less than 1 for their relevance. Additionally, nine items were marked by the content reviewers for clarity regarding wording or presentation. The experts also provided qualitative feedback for each question. Based on the feedback from the expert reviewers, many of the unclear items were changed based on their suggestions. For instance, Item 1 was changed from Provided feedback using a rubric to help me understand how to improve in the future to Provided feedback using a detailed rubric. Item 17 was double-barreled, valued me as a person and appreciated my perspective. This was changed to Valued my perspective in the class. Item 21 verbiage was changed from using a reassuring tone to used supportive language. Item 25 used the word personal, which was changed to sincere (See Table 2).

Students who engaged in the think-aloud review process found many questions straightforward. A few students suggested adding some examples for additional clarity. Based on their feedback, four questions were edited to include examples. For example, Question 12 - Responded to me according to the posted response times in the online environment was changed to Responded to me according to the posted response times in the online environment (e.g. syllabus, orientation modules or listed in the course). Question 23 underwent the most change as students felt that humor alone may not be appropriate for some instructors. The original question, Incorporated humor into the learning environment, was changed to Incorporated positive tone (e.g. humor, visuals and icons to express emotions, wittiness, etc.) into the learning environment. It was noted that although humor can be powerful, some instructors are not humorous in and throughout their courses and, therefore, not a must to connect with the instructor.

The six-factor solution generally produced an excellent model fit, as summarized in Table 3. While the Chi-square test was rejected at the alpha level of 0.05, it is known to be overly sensitive to large sample sizes. Thus, other fit statistics often aid in making a reasonable evaluation of the model. The CFI value was found to be very high, with a value of 0.965. Additionally, the SRMR value was smaller than 0.06. In contrast, the RMSE value slightly exceeded the threshold value of 0.06. RMSEA is known to be sensitive to model complexity (Hong & Asgari, 2023), which is the case in our study, given the large number of subscales (six subscales). Some implications of this finding will be discussed later in the subsequent section.

Given the overall satisfactory model fit, we examined the local model fit for each parameter. Figure 2 depicts the hypothesized six-factor structure with standardized factor loading coefficients. All factor loadings and parameter estimates were significant at the alpha level of 0.05, confirming the appropriateness of the hypothesized model. As shown in Figure 2, all standardized factor loadings were 0.80 or higher.

In terms of factor correlations, the six factors showed moderate to strong correlations, ranging from 0.763 to 0.971. Notably, the correlation between Empathy and Sociability was quite high at 0.971, suggesting that these two sub-constructs essentially measure the same underlying construct. Coupled with the high RMSEA value for the six-factor model, this finding may indicate a potential merger of the two subscales.

The findings of this study underscore the importance of instructor social connectedness in online education and provide evidence supporting the validity and reliability of the FORCES framework. Each subscale (Feedback, Organization, Response time, Communication, Empathy and Sociability) demonstrated high internal consistency. The high correlation between Empathy and Sociability warrants further investigation as it may indicate an overlap between these constructs. This finding could reflect students’ perceptions of empathy as a core component of sociability in online education, which is in alignment with the definition of online sociability presented by Osler (2021).

Confirmatory factor analysis (CFA) results confirmed the hypothesized six-factor structure with satisfactory fit indices. The RMSEA value slightly exceeded the threshold for an acceptable fit, which may be attributable to the complexity of the model. Future studies should explore the potential simplification of the framework by merging highly correlated constructs, such as Empathy and Sociability to enhance parsimony without compromising its explanatory power. At the same time, the authors of this article viewed them as a distinct, unique factor given the relevant literature and theories.

Additionally, this study highlighted the importance of course organization as an aspect of instructor social connectedness. The CoI places course organization as a design strategy under the teaching presence construct, which has been identified as one of the most important components of teaching presence (Kappel, 2022). Although traditionally associated with course design rather than facilitation, students perceived well-structured courses as reflective of instructor care and commitment, contributing to their sense of connectedness. This finding reinforces the need for instructors to consider organizational elements as part of their efforts to build rapport with their students. Quality course design can increase student motivation, usage and acceptance of online learning systems (Almaiah & Alyoussef, 2019). Indeed, when instructors organize and tailor the course design to their students’ needs, they are “humanizing” the online instruction, which is essential to support the individualized needs of all students (Pacansky-Brock, Smedshammer, & Vincent-Layton, 2020).

The development and validation of sound measurement instruments are essential for advancing research in online education, particularly in the realm of instructor social connectedness. At the time of this study, no comprehensive, validated instrument specifically addressed the multidimensional nature of instructor–student interactions in distance learning environments. Existing measures, such as the Instructor Social Presence (ISP) scale and the Online Student Connectedness Survey (OSCS), capture aspects of peer social presence, but lack the granularity needed to provide actionable insights for educators. Kehrwald (2008) acknowledged the need for a robust theory of social presence due to the benefits of online teaching and learning. The FORCES framework fills this gap by offering a theoretically grounded and empirically tested instrument that operationalizes the key elements of instructor connectedness. By establishing the reliability and validity of this framework, this study provides a crucial tool for researchers and practitioners seeking to enhance instructor presence and improve student outcomes in online learning.

Although this study provides valuable insights, several limitations must be acknowledged. First, the sample was drawn from students at two universities in the southeastern United States, which may limit the generalizability of our findings. Future research should replicate this study across diverse institutional contexts and student populations to validate the broader applicability of the FORCES framework.

Second, reliance on self-reported data introduces potential response bias. Although using multiple validity checks, such as expert reviews and think-aloud protocols, enhanced the instrument's rigor, future studies could complement self-reported measures with observational or behavioral data to triangulate the findings.

Third, while CFA supported the six-factor structure of the FORCES framework, the high correlation between Empathy and Sociability suggests a potential overlap. Future research should explore the conceptual distinctiveness of these constructs and assess whether combining them can simplify the framework without diminishing its explanatory power.

Finally, this study focused primarily on student perceptions of instructor social connectedness. Future research could examine how instructors perceive and implement the FORCES framework in their teaching practice. Additionally, longitudinal studies could explore the long-term impact of instructor social connectedness on student outcomes such as retention and academic performance.

This study contributes to the growing body of research on online learning by introducing and validating the FORCES framework as a tool for measuring instructor social connectedness. The results demonstrate that the FORCES instrument is a reliable and valid measure that can predict key student outcomes, including satisfaction, motivation and perceived learning (see  Appendix for full survey). By operationalizing the dimensions of social connectedness, this framework provides actionable insights for instructors seeking to enhance their presence in online courses.

The findings underscore the importance of fostering instructor social connectedness to improve the quality of online education. The FORCES framework offers a structured approach for instructors to build meaningful connections with their students, promoting engagement and positive learning outcomes. As the demand for online education continues to grow, this framework can serve as a valuable resource for educators and institutions that aim to create supportive and effective virtual learning environments.

This study investigates student perceptions of instructor online facilitation strategies related to student motivation, satisfaction, and perceived learning.

Think about one online asynchronous course you have recently taken. Please write the name of the course (e.g. UNI-101) and think about it while you are taking the survey.

To what extent did the instructor implement the following in the online course you took? (Strong Disagree – Strongly Agree)

1. Provided feedback using a detailed rubric.

2. Provided feedback with a grade to help me to improve my work (e.g. substantial, meaningful, etc.).

3. Provided personalized/individualized feedback (i.e. feedback specific to my submission, addressed me by name).

4. Provided constructive feedback with a balance of positive and areas for improvement comments (e.g. verbal, written, video).

5. Created an online environment where materials were easy to find.

6. The online environment had a consistent and organized navigation.

7. Divided materials into manageable modules or units.

8. The modules clearly presented the learning objectives, content (e.g. readings and videos) assignments and due dates.

9. Responded to me in a timely manner.

10. Provided feedback on assignments in a timely manner (e.g. before the next assignment is due, within a week).

11. Responded to me according to the posted response times in the online environment (e.g. syllabus or listed in the course).

12. Adhere to the response time for student communication channels and feedback in the course (e.g. email, announcement, discussion, etc.).

13. Used various communication channels (e.g. announcements, emails, video-conferencing, phone, instant messenger) to interact with me and other students.

14. Communicated with me individually or in small groups.

15. Kept me informed about my progress (e.g. online gradebook, email, etc.).

16. Sent out regular communication (e.g. weekly).

17. Valued my perspective in the class.

18. Fostered positive relationships with me and/or other students in the class.

19. Demonstrated empathy with myself and others.

20. Demonstrated flexibility when an unforeseen circumstance occurred.

21. Used encouraging language when communicating.

22. Approachable and used conversational language when communicating with me or the class.

23. Incorporated a positive tone in the learning environment (e.g. humor, icons used to express emotions, wittiness, etc.).

24. Encouraged social interaction with the instructor.

25. Made my online interactions with the instructor feel sincere.

To what extent do you agree with the following statements? (Strongly disagree–Strongly agree)

1. The instructor motivated me to learn the content.

2. Overall, I am satisfied with the instructor

3. Overall, I am satisfied with the design of the learning environment.

4. Overall, I feel I have achieved the learning objectives.

5. Overall, I feel connected to the instructor.