Student Perceptions of Factors Influencing Success in Hybrid and Traditional DPT Programs: A Q-Sort Analysis

Hybrid learning, the blend of online and face-to-face learning experiences, is a novel but rapidly expanding instructional approach in physical therapist education. Our Hybrid Doctor of Physical Therapy (H-DPT) program was designed to provide flexibility for working adults and those with geographic or financial constraints. In the H-DPT Program, students learn via online instruction for 3 weeks followed by 4 days (32-36 hours) of intensive face-to-face learning on-campus each month. During the 3 weeks of online instruction, students watch screen-captured lectures and videos, practice skills, upload videos of themselves performing skills to web-based platforms such as blogs, receive feedback from peers and faculty, take quizzes, read texts, and submit assignments related to course content. Most learning activities occur at self-selected times asynchronously, although there are occasional synchronous activities such as webbased meetings. All courses have weekly deadlines for assignments and quizzes. The 4 days of face-to-face instruction consists of psychomotor skill practice accompanied by immediate feedback, honing clinical reasoning and problem-solving skills, applying material learned in lecture-based courses (debates, journal clubs), and taking high-stakes practical and written exams.

Garrison and Vaughan (2008) refer to blended learning as the “thoughtful fusion of online and face-to-face experiences” (p. 5). Hybrid instruction is a distinct instructional modality. It is distinct from traditional classrooms, where all instruction is done face to face; it is different than online courses, where all instruction is done online: and it is dissimilar to “flipped” classrooms, where there is a shift in passive learning activities (lectures) to online while bringing application (case studies) into the classroom (Berrett, 2012; Boucher, Robertson, Wainner, & Sanders, 2013; Murray, McCallum, & Petrosino, 2014; Wong & Chu, 2014). While some researchers evaluated factors influencing student success or retention in online and blended environments (Smith, 2005; Smith, Murphy, & Mahoney, 2003), none to date have investigated factors impacting success in hybrid DPT programs. It is also unclear if there are differences in student perceptions regarding what factors impact student success in traditional versus hybrid DPT programs.

Academic success can be viewed through a variety of lenses. Alexander Astin (2012) suggests that student outcomes (O) are a result of “inputs” (I) or personal qualities that students bring to the educational experience (academic achievement, study behaviors, aspiration, financial status, life goals), as well as “environment” (E) factors that the students experience during the academic experience (program policies, curriculum, facilities, instructors, friends, family support, teaching styles). This “IEO” model reflects the milieu of factors impacting student outcomes.

Similarly, Rovai’s persistence model in distance education distinguishes pre-admission variables such as student characteristics (academic preparation, age, ethnicity) and student skills (computer literacy, time management) from post-admission variables such as learning community, interpersonal relationships, study habits, advising, teaching/learning styles, finances, hours of employment, and family responsibilities (Rovai, 2003). In physical therapy education, predictors of academic success have been analyzed using student inputs prior to and during the professional curriculum such as grade point average and standardized exam scores (Kosmahl, 2005), as well as program variables such as accreditation status, number of faculty with PhD or EdD degrees, and total years of preprofessional and professional coursework (Mohr, Ingram, Hayes, & Du, 2005).

In the context of blended learning environments, Garrison and Vaughan (2008) describe a community of inquiry framework depicting factors that influence learning in blended classrooms. This framework describes three realms influencing student learning in the hybrid classroom: social presence, cognitive presence, and teaching presence. Social presence refers to the personal communication and camaraderie between students and between students and faculty; cognitive presence is the exploration and exchange of information and new ideas; and teaching presence refers to the educational design, direction, and focus created by the instructor (Garrison & Vaughan, 2008). These elements, in addition to those variables put forth by Astin and Rovai, suggest there are many variables that could influence student success in a hybrid DPT program.

Student perceptions of variables impacting success have been investigated in health professions education, although not related to online or hybrid programs specifically. A qualitative study investigated factors influencing academic achievement in high achieving medical students (Abdulghani et al., 2014). Using focus groups and grounded theory analysis, the researchers identified four primary themes related to academic success: learning strategies, resource management, motivation, and dealing with nonacademic problems (Abdulghani et al., 2014). These four themes were further broken down into 17 subthemes, such as lecture attendance, prioritization of learning needs, mind mapping, and learning from mistakes (learning strategy theme), time management and family support (resource management theme), internal motivation and exam results (motivation), and language barriers, homesickness, and stress (dealing with non-academic problem theme).

Similarly, student perceptions of dental school including morale, strengths and challenges, and ranking of content area importance were researched using a survey of students from five western dental schools (Cardall, 2008). The findings from over 740 student participants revealed the top five positive influences on their school experience included faculty, clinical experiences, classmates, curriculum, and facilities; whereas the most frequent negative experiences related to curriculum, clinical experience, organization, student/faculty ratio, and patient pool (Cardall, 2008). Interestingly, clinical experience and the curriculum were viewed as both positive and negative influences.

In summary, there is a dearth of literature in physical therapy education regarding factors students believe to influence success, and little to no literature describing such beliefs related to hybrid education in the health professions or specific to physical therapy.

The overarching purpose of this investigation was to determine students’ perceptions of factors they believe influence their success in a hybrid DPT program and to compare their perceptions with students in a traditional DPT program. Understanding students’ beliefs about success is analogous to understanding patients’ health beliefs. By identifying student perceptions about success, programs can potentially identify potential barriers and facilitators in designing hybrid classrooms and curricula, improve the admissions selection process to better match applicants with the hybrid program, share findings with prospective applicants to improve their understanding of the program, and assist faculty who advise students.

Following institutional review board approval, subjects were recruited from two programs within one university—a hybrid DPT program and a traditional DPT program. Students were recruited verbally and via email. No incentives or rewards were provided to participants.

Based on a review of the literature regarding students’ perception of success in online/hybrid education and in health professions education, we identified variables potentially impacting student success (Rovai, 2003; Park & Choi, 2009; Smith, 2005; Smith et al., 2005). Our goal was to include variables related to both the individual student (I) and educational environment (E) as described in Astin’s work. We also wanted to ensure that E variables reflected all areas the community of inquiry model (Garrison & Vaughan, 2008) for hybrid learning: social, instructor, and cognitive presence. Three faculty reviewed the variables for face validity, and as a result 36 variables were identified (Table 1).

Since this study was conducted to investigate student perceptions of success, we used a descriptive methodology called Q-Sort because it characterizes opinions through comparative rank ordering (Portney & Watkins, 2015). According to Dennis (1986), Q-Sort is particularly valuable in research that explores human perceptions and interpersonal relationships. The Q-Sort procedure requires participants to sort a set of items (in this case, 36 factors influencing their success) into five ordinal categories, ranging from most influential to least influential, regarding student success. The number of items permitted in each category is fixed in advance so the shape of the distribution of item scores is constant for all students and reflects a bell-shaped curve.

To begin the Q-Sort, each participant received a set of 36 index cards, each index card describing one variable that potentially influenced student success. Participants were not given an operational definition of success, and if they asked, were told, “whatever you feel is success as a student. Participants were provided a symmetrical bell-shaped grid on poster board consisting of columns numbered one through five; each column representing a different degree of influencing student success (Figure 1). Column 1 was labeled “least influential,” column 2 “not very influential,” column 3 “somewhat influential,” column 4 “very influential,” and column 5 “most influential.” To maintain a bell-shaped curve suggested for Q-Sort methodology, participants were limited in the number of cards they could place under each column: 4 cards for “most” and “least” influential, 8 cards in “not very” and “very” influential, and 12 cards in the “somewhat influential” column (Figure 1). Participants were instructed to take as much time as they needed to complete the Q-Sort, placing all 36 cards (variables) in each column as they saw fit. They were also told that the order of the cards (variables) within each column did not matter, just the selection of which column to place each variable. As students finished, they notified the PI or coinvestigator, who then double-checked that all the spaces were filled and no cards were left over.

In order to maintain the appropriate groups for proper data input, the cards from each column were placed in a corresponding envelope marked 1–5. When all cards were placed in envelopes from a particular students’ board they were wrapped together with a rubber band and placed in a larger envelope until they were input for data analysis.

The Q-Sort analysis looks for correlations between subjects across a sample of variables (Portney & Watkins, 2009), reducing many individual viewpoints of the subjects down to a few factors. The term factor in the Q-Sort is defined as representing individuals with similar views, feelings, or experiences (Akhtar-Danesh et al., 2013). In our Q-sort, factors represent groups of individuals with similar views about which variables influenced student success.

For this study, analysis of the Q-Sorts was conducted using the PQ Method 2.33, a free downloadable software program (Schmolck, 2014). This is the program most recommended by the International Society for the Scientific Study of Subjectivity. The PQ Method is a statistical program tailored to the requirements of Q-Sort studies. Specifically, it allows the user to enter each Q-Sort individually, identifying which variables were selected in each of the 5 columns. The PQ Method program computes intercorrelations among Q-Sort, which are then factor analyzed and rotated for simplification (Schmolck, 2012). The analysis step produces an extensive report with a variety of tables on factor loadings, statement factor scores, and discriminating statements for each of the factors (Schmolck, 2012).

The number of factors created by the Q-Sort program is based on a correlation matrix between all Q-Sorts followed by factor matrix analysis. Each factor thus had its own set of scores, expressed as z scores for each variable/statement. Since there so many variables represented in each factor, we reported each factor using only those variables with z scores greater than or equal to 1.5 (representing most influential) and less than or equal to -1.5 (representing least influential). The individual statement factor z scores were used to develop an understanding of the profile for each factor and to make comparisons among the factors.

This Q-Sort methodology cannot compare two different data sets (hybrid vs. traditional) directly as one might do in an independent / test. Instead, we entered the data for the H-DPT program and DPT program separately and compared the factors identified by each group.

Two cohorts completed the Q-Sort: Hybrid DPT Students (H-DPT) and Traditional DPT students (DPT). Results for each Q-Sort are reported separately based on the way the Q-sorts were analyzed.

Participants. A total of 54 H-DPT students participated, including 20 second-year students and 34 first-year students. Although demographic data were not collected, the average age of the two cohorts combined (based on admission statistics) was 26.2 years, 58% female and 42% male.

Factors. The Q-Sort data from the Hybrid sample created four factors. Again, a factor is a subgroup of responses representing groups of individuals with similar views about which variables influenced student success ranked in a similar manner using correlations. For each of the four factors, the statements with z scores of greater than +1.5 (“strongly impacting success”) or less than -1.5 (“least impacting success”) are identified (Table 2). Among the statements with z scores of greater than +1.5 (most influencing success), “self-initiation in learning” was identified in all four factors, and “time management” noted in three of the four factors, whereas “academic work ethic” and “student ability to identify study focus” were identified in two of the four factors (Table 2). Among the statements with z scores of less than -1.5 (least influencing student success), “access to student services” was identified in three of the four factors, and “beliefs about teaching and learning” was identified in two of the four factors (Table 2).

Consensus Statements. The Q-Sort program identified consensus statements, meaning those statements that did not significantly distinguish one group (factor) from any other group (factor). For the Hybrid program, there were six consensus statements (Table 3). There were three positive consensus statements (students perceived it strongly impacted success) including: “self-initiative in learning,” “problem-solving skills, and “organization of courses in the curriculum.” There were also three negative consensus statements (students perceived it did not impact success) including: “prior academic performance,” “writing skills” and “access to resources outside the classroom.”

Participants. Seventy-one traditional DPT students participated in the Q-Sort. Although demographic data were not collected, the average age of the two cohorts combined (based on admission statistics) was 24.3 years. Gender was not identified in the traditional cohort.

Factors. The Q-Sort data from the Traditional sample created four factors. For traditional students, “time management” was identified as strongly influencing success in three of the four factors, while “academic work ethic” and “ability to prioritize study focus” were identified in two of the four factors. The statement with the highest positive z score for the traditional group was “instructor’s ability to organize course material.”

In the Traditional sample, there was little similarity between the four factors in the statements ranked as “least influential” for impacting student success. “Access to student services” was identified by two of the four factors as being among the least influential. No other statements were similar across factors; however, statements with the highest negative z scores were: “hours worked per week,” “finances,” and “preadmission major.”

Consensus Statements. Again, consensus statements are those statements that did not significantly distinguish one group (factor) from any other group (factor). For the traditional group, there were no consensus statements within the four factors, either positive or negative.

The purpose of this study was to determine students’ perceptions of factors they believe influence their success in a hybrid DPT program and to compare their perceptions with students in a traditional DPT program. The Q-Sort methodology required participants to rank order the most and least important attributes related to their success, forcing them to consider all attributes potentially related to success rather than selecting a single attribute as one might do in survey methodology. In this discussion, factors represent groups of individuals with similar views about which variables influenced student success. The H-DPT and DPT student Q-Sorts were analyzed separately based on the capability of the PQ Method 2.33.

There were several differences between the hybrid and traditional DPT student perceptions. Self-initiation in learning was identified in all four factor groups in the Hybrid program, with z scores ranging from 1.57–1.98; whereas it was not included in any of the four factor groups from the Traditional program. This finding was not surprising, given the nature of hybrid instruction. Although self-directed learning is a characteristic of professionalism in physical therapy (May, Morgan, Lemke, Karst, & Stone, 1995) students in a hybrid program must be self-directed to independently navigate the learning experiences during the online portion of each month. For example, in most traditional programs, students attend classes at set times each week, following the pace of learning set by the instructor. In contrast, students in a hybrid program access online lectures and videos at self-selected times and a self-selected pace. Further, in our hybrid program, students have weekly assignments to keep them engaged and must then be self-directed to complete the assignments on time.

Another difference between the hybrid and traditional DPT students’ perceptions of success was the perceived role of the instructor. In the traditional DPT cohort, one of the four factor groups included “instructor’s ability to organize course material,” whereas none of the four factor groups in the H-DPT program included this variable. In fact, the highest z score for the traditional DPT students across all factor groups was this variable. Although it was only a factor in one traditional DPT factor group, it may indicate that some traditional DPT students perceive the instructor’s organizational skills in organizing class material as being more important than their own attributes.

In contrast, three of the four factor groups in both the Hybrid and Traditional students identified time management as a variable strongly impacting success. Given the intensive nature of physical therapist education, this was not surprising. These findings are like those reported by Abdulghani et al. (2014) in medical students. It is possible that the reasons why time management was rated as strongly influencing success may be different between students in each program. For example, most hybrid DPT students juggle work/family along with school demands, while the traditional students manage a more-intensive course load and must be on-campus every day. Time management is an attribute however that all DPT students emphasized, and this should be shared with prospective students in all programs.

Two variables, “academic work ethic” and “ability to prioritize study focus” were identified as strongly impacting success by two of the four factor groups in both cohorts. These findings are similar to prior research for online learning identified study habits as an important post-admission variable impacting student retention (Rovai, 2003; Smith, 2005).

Consensus statements are those statements that did not significantly distinguish one group (factor) from any other group (factor). Interestingly, the Hybrid cohort had six consensus statements while the Traditional cohort had none. One reason for this difference might be the shared experience of being in a hybrid DPT program. While all students have had similar experiences in traditional classrooms, few students in the Hybrid program had previous exposure to hybrid learning. The uniqueness of being in a hybrid program may have more strongly shaped their perceptions related to what it takes to be successful. For example, DPT students in traditional programs use the same learning strategies they’ve employed prior to PT school, whereas hybrid DPT students were potentially forced to develop new or different strategies. Further, their experience was unusual or different than most other professional programs, and so they may be more aware or reflective of factors impacting success.

The three consensus statements rated as strongly related to success in all hybrid factor groups included: self-initiative in learning, problem-solving skills, and organization of courses in the curriculum. As stated previously, the importance of self-initiation as an attribute makes sense in a hybrid curriculum because students must organize, plan, and engage in learning on their own time frame. Problem-solving may have been rated consistently high across all factor groups for several reasons. First, the technology itself is an area in which students must problem-solve in online and hybrid environments (Kowalczyk, 2014; Stott & Moser, 2016; Talcott, O’Donnell, & Burns, 2013). For example, in the hybrid program, students must create and upload videos demonstrating a psychomotor skill, and can encounter problems uploading the video in a correct file format to the course management system. It is also possible that time management issues, clearly identified in this study, require problem-solving to decide how to prioritize study and work time. Also, in the H-DPT program, students have more graded assignments than in the Traditional program. When students encounter questions about the assignment or pertaining to the lecture, they are free to contact faculty; however, many first try to problem-solve on their own before emailing or calling faculty. This is different than being in a classroom where students can raise their hand and easily get a question answered.

There are several limitations of this study. First, the population from which the sample was drawn represents only those students in one university, albeit two distinct programs, so findings cannot be generalized to other programs. Second, students volunteered to participate in the study, meaning they may be those individuals with stronger feelings/perceptions about factors that influence success. Third, the H-DPT program was only in its second year, and it is possible that the perceptions of students would be different in a more mature, stable program. As the program matured, instructional practices became more sophisticated and consistent than they were in the first 2 years of the program. Last, the definition of “student success” was not operationally defined for the participants, because it was the authors’ intentions to allow students to selfselect their perception of success. This may have led to a lack of uniformity in analyzing variables impacting success.

Suggestions for future research include the following: (1) replicate the Q-Sort in the H-DPT program to see if student perceptions have changed now that the program is more mature; (2) conduct a qualitative study to probe students lived experiences in the H-DPT program regarding the variables identified in this study; (3) compare student perceptions in the first versus final year of the program; (4) compare student perceptions between those students in the top versus bottom quartile measured by final grade point average.

While both hybrid and traditional students perceive time management, academic work ethic, and ability to prioritize study focus as strongly influencing student success, self-initiation in learning, problem-solving, and organization of courses were perceived by only H-DPT students as factors influencing success whereas the instructor’s ability to organize course material was only identified by the traditional DPT students. Further, there was consensus in the H-DPT cohort regarding factors influencing success compared to traditional DPT students.

This article was also published in Volume 14, Issue 4 of Distance Learning.