The effectiveness of applied learning: an empirical evaluation using role playing in the classroom Open Access

In the era of globalization, the functions of business are dynamic, and their success depends on how efficiently companies manage their inter-firm relationships with different channel members within the supply chain. Be it a manufacturing or a service company, supply chain management (SCM) plays a vital role in determining the performance of the business. The current research is focused on preparing supply chain leaders to best establish and maintain proper business relationships through the use of a selected applied learning technique – role playing. This work will compare evolving thought on applied learning vs traditional lectures and recommend an alternative way to prepare future managers to serve their supply chain.

With the rise in competition in the global market, SCM activities are becoming more dynamic, complex and diversified. Therefore, it has become imperative to improve and enhance the skills of managers to deal with the rise in challenges in the supply chain (Kovacs and Pato, 2014; Kotzab et al., 2018). Further, the pace of change, explosion of knowledge and technological revolution have had a great influence on the field of education, raising awareness of the need for new pedagogical and learning approaches (Bates, 2005; Zhang and Nunamaker, 2003). In order to prepare students as ready professionals, it is vital that instructors of business courses like SCM know how to best prepare their customers – the students.

Motivating students so that they can understand the importance of integrated-topic scenarios is another issue of concern (Chang et al., 2009). Therefore, the courses taught at educational institutions are changing their teaching patterns and utilize comprehensive learning solutions instead of focusing on subject knowledge only (Ten Eyck, 2011; Preziosi and Alexakis, 2011; Tran and Lewis, 2012; Westrup and Planander, 2013; Tabak and Lebron, 2017; Paul and Ponnam, 2018). For instance, in order to determine the quantities to produce, managers need to consider their decisions with regard to demand forecasting, allocation of resources and inventory policy as well as scheduling of manpower and machinery. Hence, it is important that future managers first study different aspects of a decision, then consider them in totality and not in isolation. Several opportunities do exist for advanced learning in applied settings, such as internships, residencies and practical learning courses such as capstone projects with industry clients (Brown et al., 2018; Silva et al., 2018; Bhattacharya and Neelam, 2018). However, this option of applied learning is also available for basic and intermediate courses to meet the changing demands of the student.

A vast body of literature evaluates the effectiveness of different practical and applied approaches to learning that can be embraced by teaching institutes so that they can contribute significantly in producing ready managers. The research by Alessi and Trollip (1985), Martocchio and Webster (1992), Quinn (1996) and Li et al. (2017) shows the use of applied gaming tools in teaching different subjects resulted in better knowledge retention and have acted as useful pedagogical tools supplementing the traditional classroom course material.

In light of the changing business environment and the complex dynamics of SCM, there is a need to alter the current traditional pedagogy and equip educators with modern, comprehensive, applied, scenario-based teaching techniques. Therefore, this project will test the effectiveness and benefits of the technique of role playing as just one technique that can be used to implement applied learning in the field of SCM.

The development of hands-on skills is challenging for many workers due to lack of coaching, mentoring and applied learning opportunities. In order to improve productivity and implement best practices, businesses require better results than standard classroom lectures. This study consolidates a thorough review of the benefits of applied learning and recommends ways in which traditional classroom sessions can be enhanced by using applied learning solutions:

In the following sections, this paper will present a comprehensive review of literature regarding applied learning, a research methodology, the analysis of the data and final conclusions.

The rise in globalization and need for innovation have marked significant changes in supply chains. The increasing role of SCM is associated with the rise in integration among different members in the supply chain (Christopher et al., 2011). In the case of a manufacturing chain, the manufacturer does not reach directly to the consumers in order to sell its products, and therefore needs channel partners such as wholesalers, distributors and retailers to reach consumers. Being an intermediary, the retailer transfers information about customer tastes and preferences and feedback about the products to the manufacturer. Hence, resellers become a vital part of a SCM. Therefore, the success of the manufacturer is highly dependent upon the relationship between the manufacturer their channel partners in SCM. This project will explore best practices in educating future supply chain managers on the specific process of establishing and maintaining successful business relationships.

Effective learning and education is characterized by rigorous coursework and effective pedagogical approaches that aim at enhancing the success level for all students (Rhodes, 2007). Hmelo-Silver and Barrows (2006) assume that teaching is a complex task and the instructor should make use of a repertoire of strategies in order to make the learning effective. Effective education is not just about teaching but learning. To enhance learning, students should be equipped with the optimal level of information and assignments (Preziosi and Alexakis, 2011). This means that the information imparted should be justified and not overly demanding. In other words, if it is too demanding, the students will be discouraged and if it is non-challenging, the interest level of the learners will tend to be shattered.

Instructors should focus on enhanced learning that is possible by being flexible and assimilating new subject content while adopting a variety of pedagogical methodologies (Douglas et al., 2008). It is also vital that the teaching style of the instructor should be effective enough that they can be well understood, and their lessons can be applied by the learners (Gooden et al., 2009). As provided by Niemi et al. (2010), effective education can be developed only when the students are involved in the decision-making process of learning and their experiences and stories as learners are heard. Ferguson et al. (2011) also supported this argument and focused on involving the students’ opinions in generating meaningful dialogue and informed decision making. Instructors should be flexible to adopt changes in their teaching practices after understanding the experiences of students in order to develop an effective learning forum (Robinson and Taylor, 2007).

Another tool toward developing effective learning sessions and pedagogy is the ability to scaffold the learning of the students (Husbands and Pearce, 2012). The scaffolding for learning can be provided by offering intellectual, social and emotional support (James and Pollard, 2011). This move will motivate the students to develop interest in their studies. Kim and Hannafin (2011) have posited that the students who do not develop an interest in the studies and learning process are not able to progress. Real-world scenarios are forums for creating interest, motivation and scaffolding. These techniques culminate into a new teaching method called applied learning.

Learning is known to be effective when the learner is not a passive recipient of knowledge but is a proactive personality in the entire learning process. Active learning is possible when the learner perceives the existence of a relationship between prior knowledge and new knowledge (Christmas, 2014). Bringing the real world to the classroom is viewed as one of the crucial ways to promote effective learning. Therefore, there is an absolute need to transcend from the traditional approaches and pedagogy to new techniques such that the learners can use their learning to solve existential problems (Hui and Koplin, 2011).

Learning-by-doing is considered an apt approach in today’s technology-driven world, wherein continuous learning should be the premise for overall development of the students (Serbessa, 2006). The pedagogical activity in the case of applied learning should be active and involve concrete processes by focusing on the motto of learning by doing (Schwartzman and Henry, 2009). The focus of effective education should be centered toward the learners wherein the role of the instructor is to guide and facilitate the learner and the learning process and not assert control or discharge lecturers toward the targeted learning goals (Silcock and Brundrett, 2001). With the rise in globalization and changing demands of the business world, the learning approaches of the students may need to be altered to provide the best possible fit. In the light of this dynamism, the student-centered approach has become a necessity while deciding upon different learning styles to apply in the classroom (Hudson, 2009). Moreover, the minds of the learners are complex and have a heterogeneous nature and therefore, an effective classroom should consider the constructivist perspective of learning. In order to develop a constructivist perspective of learning, it is vital to implement a student-centered model of instruction which should blend differentiated curriculum and assessment paradigms (Klein, 2003). The learning theory of constructivism aims at making learning meaningful. As per this perspective, learning is a self-directed sociocultural process in which the instructor assumes the role of a facilitator (Tobin and Tippins, 1993).

SCM education techniques are similar to those of all other fields. As suggested by Phillips (2005), applied learning can be achieved through chats, role plays and case studies for application-based education (Miller et al., 2016). Further, additional tools of applied learning adopted by Ruey (2010) include submitted artifacts, interviews, surveys and observations. Simulation is another technique used for teaching useful learning concepts using applied learning (Ten Eyck, 2011). Instructors can adopt different gaming tools to facilitate real-time applied learning in teaching SCM, logistics management and business relationship management (Lewis and Maylor, 2007; Quinn, 1996). These gaming tools are effective in motivating active participation in the course as they foster competition and add excitement in the learning process (Elgood, 1997). Further, Dempsey et al. (1996) established that games lead to a playful learning environment that is goal-oriented, challenging and promoting competitiveness.

Using simulation tools and models in teaching SCM is more insightful and helps to evaluate alternatives and outcomes of supply chains which further enhances performance (Persson and Araldi, 2009). Moreover, it becomes possible to use quantifying measures for problem solving, which is a more reliable and objective approach in teaching (Katsaliakia et al., 2014). Simulation-based models should be used with caution, however, as the learners may find it difficult to understand the use and importance of these pedagogies (Chwif et al., 2000). Therefore, it is imperative that the learners become familiar with the use of these applied learning tools and have the competence for successfully implementing these models (Katsaliakia et al., 2014). Hence, the role of the instructor becomes important in explaining the need and the correct way to implement these applied learning tools so that the learning benefits can be maximized. The most popular simulation game used while teaching SCM is the MIT Beer Game, in which production and distribution in a multi-stage distribution channel is facilitated (Chang et al., 2009).

Role playing as an active learning technique involves a high level of participation from students (Howell, 1991; Westrup and Planander, 2013; Barnabe, 2016; Tabak and Lebron, 2017). Role playing is a group activity involving more than one person who assume different roles in a given situation (Rao and Stupans, 2012) with the aim to acquire learning experiences (Sogunro, 2004). As per Yardley-Matwiejczuk (1997), role playing puts the people in “as if” situations through simulations and actions depending on specific events and circumstances such that different behaviors, roles and arguments directly influence and deepen the learning experience. The preparatory part of role playing is crucial as it involves establishing the descriptions of roles along with the pre-requisites of the involved participants (Westrup and Planander, 2013).

Learning in role playing is facilitated by observing as well as acting out the series of events happening in the respective situation. Role playing helps students to understand the dilemmas in situations and highlight the values of interpretation, which are not possible to study in the traditional lecture mode of teaching (Bryant and Darwin, 2004). As per Rao and Stupans (2012), role playing helps to provide a better understanding of situations and augments that are critical to the subject matter. Yang and Quadir (2018) found significant evidence to support language learning during the role playing game. Further, McCarthy and Anderson (2000) undertook a study to measure the effectiveness of role playing as compared with the traditional methods of teaching and concluded that the active learning tool of role playing leads to better absorption and retention of information. Not only is it more effective, role play provides a variation from the usual lecture-based pedagogy and thus provides an applied and practical aspect to the theory taught in the classroom (Sogunro, 2004). Role playing is also recognized as a pedagogical tool that helps to depict the manner in which one acts, reacts and communicates in a social interaction (Daly et al., 2009). Thereby along with critical thinking, it facilitates enhancement of communication and inter-personal skills. Further, as per Rao and Stupans (2012), role playing provides deeper insight about the business world and the students who take part in it can learn about the situations, issues, alternatives and methods to tackle later in their working life. This means of pedagogy hones the skill sets of students that are required in future employment (Ruhanen, 2005). Sogunro (2004) posited that role playing helps students view a specific situation from different perspectives and not just the single perspective of the instructor or any individual. Therefore, it can be said that role playing acts as an important active learning tool. However, there are drawbacks in regard to student learning as related to the different knowledge background of each student (Qiu et al., 2018). Another study explored role playing for learning in the marketing field to help students create favorable customer experience without direct supervision from the instructor (Paul and Ponnam, 2018). Business ethics is another well-suited educational realm for role playing learning (Sauser and Sims, 2018). Role playing also helps students to understand the feelings, values, attitudes and body language that provide the context of the business situation at hand. This is very valuable in business education as role playing supports cross-sectoral integration and help students experience the complexity in decision-making (Ferrero et al., 2018). Moreover, as per Alkin and Christie (2002), role playing is a useful tool in teaching conflict resolution and preparing students with the necessary skills required for effective conflict management.

According to the Accreditation Council for Pharmacy Education (nd), schools and colleges should make use of active learning methods so that critical thinking, problem-solving skills and better retention of knowledge can take place. There have been many researchers who concluded that there is a positive relationship between the collaborative and applied learning approaches and higher cognitive and affective outcomes (Tran and Lewis, 2012; Chang et al., 2009; Li et al., 2017; Ferrero et al., 2018). The research has been supported by Johnson and Johnson (2009) who posit that applied learning tools augment the achievement rates of the students and increase their knowledge retention. It has also been shown that cooperative simulations like role playing lead to better academic, social and psychological advantages (Johnson and Johnson, 2005). As per Beck and Chizhik (2008) and Zain et al. (2009), academic performance improves drastically when the pedagogy contains applied learning tools.

The pre-test and post-test experimental method is the best method to evaluate the performance of students to adjust for previous knowledge of the learners while directly measuring the gain of knowledge for each student (Kilic, 2008; Doymus et al., 2010). The growing literature continues to support applied learning techniques, and this study explores one such option in the realm of SCM education.

The experiment for this project was conducted with 91 graduate students at a private, non-profit university over the period of 1.5 years with seven different sections of the same course, each course lasting four weeks. These seven classes were pre-selected to approximately divide the sample into control and experimental groups. The control group was comprised of 59 students who were first given a pre-test on the subject matter, followed by the traditional mode of face-to-face lecture by the instructor using a PowerPoint presentation, and finally each student was given a post-test to capture their knowledge gained. The experimental group had 32 students, who were also given a pre-test and then the learning was facilitated using the applied learning technique of role playing, and again a post–test was conducted to assess the knowledge gain. The  Appendix shows the questions used for the evaluations, with post-test questions re-sorted.

Traditional teaching for the Control Group used a 30-min lecture on the subject of business relationships in the context of SCM, focusing on the partnership model (see Lambert et al., 2004, 2010). In contrast, the applied learning was facilitated using a role playing exercise for the experimental group. The exercise required a real-world business relationship consisting of a seller and a buyer, as selected by the students. For example, one student group self-selected the Coca-Cola Company selling soft drinks to the McDonald’s Corporation – guidelines were that both companies be familiar to all students in the class. The role playing exercise took 45 min, in which the first 10 min explained the research study and the partnership model. Thereafter, the experimental classes were randomly divided into two groups, whereby the first group played various roles in the Coca-Cola Company (e.g. the supplier), such as the President/Chief Executive Officer, Vice President for Supply Management, Vice President for Operations and Vice President for Distribution, while the other group performed similar roles for the McDonald’s Corporation (e.g. the buyer). Both the groups were then given time to find background data on the company, such as identifying the actual president or CEO, location of the headquarters, number of employees, number of stores, revenues and the mission statement of the company. After this, based on the partnership model, the major drivers – “compelling reasons for partnering with the other firm” (Lambert et al., 2004, p. 23) – were identified and scored by each team. Once the drivers were identified by the teams separately, a facilitator session was conducted, whereby both the teams were asked to rate the facilitators jointly. They then selected the proper level of business relationship based on the partnership model and discussed the appropriate actions that these companies should undertake toward increasing performance otherwise not obtainable without a partnership. With such a process, students played an active role in gathering the data and reaching their own conclusions based on their active participation.

In order to meet the research objectives, the research hypotheses were framed as:

Figure 1 shows the knowledge connections before and after the learning experience for both groups. It should be noted that scoring was evaluated on a scale of 0 to 6 (see  Appendix for sample instrument) with partial credit given as appropriate in increments of 1/2, 1/3 and 1/6 points based on the question, so continuous variables were assumed. However, results were also confirmed with the non-parametric Mann-Whitney U-tests used when the data are ordinal or when the assumptions of the t-test are not met (Boslaugh and Watters, 2008).

This section analyzes the knowledge acquired during the learning sessions, followed by a comparison of knowledge gain between the control and experimental groups. After all experiments were conducted, data were reviewed to test assumptions of normalcy and continuality. Figures 2 and 3 show slight skewness on the lower end for the control group and slight skewness on the upper end for the experimental group, with no outliers except for a single perfect score in the experimental group (role playing).

For a statistical test of the normality assumption, the Shapiro-Wilk Normality Test (Shapiro and Wilk, 1965) was performed, with all four data sets as acceptable at the p<0.05 level (see Table I), which along with group sample size over 30, the normality assumption holds.

It was next appropriate to verify the equivalency of the groups before making any comparisons. This evaluation of the experimental group as compared to the control group must be done prior to the implementation of the treatment. Therefore, the pre-test scores were used as a direct measure of the level of knowledge of subjects prior to the experimental treatment (e.g. students may have had previous experience with the material or may have reviewed course material prior to the class). The below student’s t-test for two-samples assuming unequal variances verifies this assumption in that the there is no statistically significant difference in means between the two group’s pre-test average scores (see Table II).

The next step was to compute the gain in knowledge from both the traditional and applied learning techniques, using each students’ pre- and post-test scores. To verify that learning occurred in each instructional format, scores were tested again using a student’s t-test but this time for paired samples as each subject’s pre- and post-test scores are not independent. The results obtained for the lecture and workshop are given in Tables III and IV; using each group’s average pre-test score as baseline, the control group scored 53 percent higher after the traditional lecture while the experimental group scored 76 percent higher after their role playing activity. A graphical interpretation can be seen in Figures 4 and 5. With these results (Tables II–IV), the experiment is considered valid and H2 can be validated for both groups so further analysis can proceed to evaluate H3, which is to compare between the groups.

To directly evaluate H3 as to the comparison between learning methods, a two-sample comparison assuming unequal variances was utilized. Since the previous analyses confirmed that the control group and experimental group were equivalent prior to their, respectively, learning activities and that learning did occur in both groups, Table V details the analysis showing a statistically significant higher level of learning by the experimental group (p<0.10).

Using this role playing applied learning activity, students in the experimental group reported a 45 percent gain in learning in comparison to those attending traditional lectures. To further evaluate this result based on the relatively small sample and moderate variation as expected in any human-subjects experiment, the effect size should also be considered. A standard Cohen’s d can be computed if the standard deviations of the two samples are roughly the same and therefore assumed they are estimating a common population standard deviation. The computed effect size for this experiment using Cohen’s d was 0.360, yielding a relatively small effect size as d<0.50 (Cohen, 1988):

where $σpooled=(σ12+σ22)/2$⁠.

However, the standard deviations of the two groups did differ slightly (1.094 and 1.398, respectively) so the homogeneity of variance assumption is questionable and pooling the standard deviations may not be appropriate. One solution is to insert the standard deviation of the control group into the equation and calculate the Glass’ Δ (Glass et al., 1981), assuming that the standard deviation of the control group is untainted by the effects of the experiment and will therefore reflect more closely the true population variance. “The strength of this assumption is directly proportional to the size of the control group. The larger the control group, the more it is likely to resemble the population from which it was drawn” (Ellis, 2009). With the Control Group containing n=51, the Glass Δ was 0.413, yielding a similar small effect size as the Cohen’s d for this relatively small experiment:

And finally, it should also be considered that the experimental group (n=30) was smaller than the control group (n=51). Taking this into account, the Hedges’ g will account for sample size differences by weighting the variances by their respective sample size (Hedges, 1981). Hedges’ g was 0.372, re-confirming again the small effect size:

where $σpooled*=((n1−1)σ12+(n2−1)σ22)/(n1+n2−2)$⁠.

From the analysis of this experiment comparing the learning benefits of traditional lectures vs applied learning using role playing, it was first verified that there was a gain in immediate knowledge in both the modes of learning, confirming H2(a/b). The goals of this study were met as the analysis showed significantly greater gain in knowledge (post-test verses pre-test for each student) for the role playing groups (1.463 average gain in score) as compared to those exposed only to the traditional lecture (1.012 average gain in score). This represents a valuable 45 percent improvement in learning scores. Therefore, it is determined that the time investment in preparing and executing the role playing activity was of significant value to the customer – the students.

This research was conducted using a single type of Applied Learning – role playing – and using a single discipline within a business school. In addition, the application was tested in the graduate-level educational environment, so findings are not generalizable to other student levels or fields of study. However, findings are in-line with various other studies that reported similar benefits of applied learning techniques (Brown et al., 2018; Silva et al., 2018; Bhattacharya and Neelam, 2018). Larger samples using a variety of applications should continue to be explored.

Applied learning has a vast potential in the fields of training and education. The scope for this research has been limited to one topic area (SCM) and one tool of applied learning (role playing). The learning assessment measures in this study (i.e. pre- and post-test) were comprised of only a few multiple-choice questions, but real learning may be better captured using a more extensive test including essay questions that are application-based. It is also possible to conduct a qualitative assessment on the participants to study their subjective experience of applied learning in comparison with traditional learning. Further, in the current research, the independent variable of gain in knowledge was considered, but other variables like the learning experience, quality of learning, time efficiency of learning and the capability to implement the concepts in business and managerial situations should also be considered. And finally, long-term retention of knowledge gained in the classroom may be of even greater importance to meeting the goals of education; longitudinal studies could be conducted at the one-month, six-month and one-year milestones as done with many alumni surveys that attempt to assess the value of the education as assessed after graduation. Because the scope of research in the field of applied learning is vast, continued study and implementation of applied learning techniques is highly recommended.