Creating Equivalent Learning Outcomes in a Distance Education Leadership Course: An Action Research Partnership

Leadership education is a vital element in diverse fields, such as business, education, and national defense. While the professional contexts in which leadership is applied vary among professions, the use of foundational theory and its practical application are shared elements of most leadership development programs (Murphy & Riggio, 2003). Taking advantage of the commonality between educational leadership and military leadership education, this action research was conceived as a partnership between a university educational leadership department and a graduate-level school of professional military education. While both partners routinely engage in the study of leadership theory and application, the educational leadership department provided guidance in educational research methodology, while the faculty of the military institution provided diverse subject matter expertise in leadership application.

This research seeks to achieve greater parity between the in-residence and distance education offerings of a professional military education course. The resident course is an intense 5-week program with continuous small group interaction and feedback. The distance learning version is self-paced and minimally interactive. Many of the distance learners serve in austere and remote locations halfway around the world from where the resident course is offered. This fact greatly restricts the degree to which synchronous interaction can be incorporated into the distance learning program. Recognizing this limitation, this research pursues a means of achieving equivalent learning outcomes between the two course formats. Course enrollment consists of a blend of junior officers and equivalent-grade Department of Defense civilians. All enrolled students have at least a bachelor's degree, though a significant number hold graduate and professional degrees in medicine, dentistry, nursing, divinity, and law. The diversity of the student body contributes to a variety of leadership perspectives. This research uses qualitative methods to identify these perspectives and proposes novel ways to feed them back into distance learning content to simulate the effects of exposure to the varied perspectives that occur naturally in the resident course.

In its report on quality in distance education programs, the U.S. Department of Education cited a set of “good practices and red flags” representing best practices and troubling indicators of postsecondary distance education programs. Among the positive indicators was evidence that faculty, assessment specialists, and academic leadership considered course evaluations and student portfolios as inputs to their continuous improvement efforts. Among the red flags were instances where faculty directly converted traditional course content into an online format without consideration of distance education pedagogy or where needs of distance learners were overlooked in the course design (U.S. Department of Education, 2006). These indicators were used as guideposts for this action research, which draws from the principles of Simonson's equivalency theory to create more equivalent learning outcomes for our distance learning leadership course.

The premise of equivalency theory is “that the more equivalent the learning experiences of distant students are to that of local students, the more equivalent will be the outcomes of the learning experience” (Simonson, 2009). As intuitive as that sounds, the definition of “learning experience” requires a more careful treatment. Schlosser and Simonson define a learning experience as “anything that promotes learning including what is observed, felt, heard, or done” (2010). Assuming the institution values its traditional and distance learning students equally, achieving parity in learning outcomes should be the goal of distance education program design. Simonson distinguishes “equal” from “equivalent” by noting that a triangle and a square may have equivalent areas, but they are fundamentally different shapes and therefore not equal (Simonson, Schlosser, & Hanson, 1999). He places responsibility for creating an equivalent set of experiences entirely with the instructional designer:

Before equivalent experiences can be conceived, designers must first recognize the differences in learning environment and situation that exist between resident and distance learners. Once designers understand these differences, they should overdesign distance education learning activities to maximize the potential to achieve equivalency for all learners (Simonson, Smaldino, & Zvacek, 2015). To determine the degree of equivalence between resident and distance learning courses, Karatas and Simsek (2009) tested their students' performance and satisfaction with formal assessments given once at course completion and again 4 weeks later. They found that resident students demonstrated greater mastery of the course's objectives and retained this knowledge longer than their distance learning counterparts. Surprisingly, student satisfaction was nearly equivalent between the two groups. Unlike the Karatas study, where the two courses were similarly structured, the leadership courses in this study are structured quite differently from one another. Although the course objectives are equivalent, the content, activities, and course schedule are vastly dissimilar. A structural approach was used to examine equivalency between the two courses. The set of activities and conditions found in our resident course that are unavailable to our distance learners was considered. Analysis revealed that the distance learners' self-study environment did not provide adequate exposure to diverse opinion sets. Evidence is found in the literature in both leadership theory and learning theory to suggest that such exposure is vital to individual development.

Leadership theory suggests that exposure to diverse opinions and leadership styles enables leadership growth. In their approach for developing transformational leadership teams, Bass and Riggio (2006) describe high-performance teams as those whose members challenge one another's assumptions, and listen to each other's ideas. For leadership education, Yukl's (2009) case study method recommends that programs should allow students to see how their peers may approach problems with different assumptions, biases, and priorities. They should also relate theoretical problems to their actual work experience. Finally, groups should be varied in composition to expose students to different points of view.

Avolio and Luthans argue that leadership development experiences can create “disequilibrium and heightened self-awareness that can lead the individual to challenge his or her basic beliefs and assumptions” (2006, p. 47). These take place in the form of “trigger events,” which can be seen as positive or negative. Learning theorists Young and Wasserman (2005) explain that “one's internal cognitive linkages once formed are often very resistant to change” (p. 162). Significant triggers can modify these linkages leading to further self-development.

In the resident leadership course, students are confronted with numerous challenges in which individual and team leadership tasks must be performed under resource and time constraints. These events are designed in part to create disequilibrium, requiring students to look beyond their individual abilities and experiences and seek solutions from among the collective set of their classmates' skills and abilities. Simply put, these events are designed to develop leaders by challenging their assumptions and demonstrating the value of diverse skills and experiences in addressing complex problems. To create an equivalent learning experience for distance learning students, curriculum designers need to account for the absence of trigger events and consider possible alternatives for achieving the same goal of challenging the students' assumptions and broadening their perspectives.

Action research was chosen as a means to address the disparity between our two course formats for a variety of reasons. In action research or practitioner inquiry, teachers actively participate in defining and solving academic problems. It is a process that simultaneously addresses research questions, encourages faculty development, and empowers teachers to reform their practices (Dana, 2009). In this study, faculty, administrators, instructional designers, and assessment specialists each had a voice in shaping the research and each had a stake in the outcome. The involvement of these stakeholders is aligned with the U.S. Department of Education's positive indicators for distance education programs as previously discussed.

Carver and Klein (2013) used action research to improve a leadership preparation program. In their research, qualitative analysis of student writing was used to identify “turning points” in leader development. The knowledge gained was used to drive curriculum improvement and strengthen faculty understanding of their student population. This research uses qualitative methods to understand how leadership students apply foundational leadership concepts in their personal and professional contexts. Qualitative action research has also been applied to improving distance education. Lamaster and Knopp (2004) used focused interviews and surveys to understand their distance learning students' needs and preferences and then applied this knowledge toward improving course delivery. This research uses student leadership essays as one-sided interviews where the students express personal perspectives on a variety of leadership topics and the faculty analyzes these perspectives in aggregate to better understand the student population.

To promote a more equivalent learning experience for our distance learning leadership course students, we designed an action research to consider which elements of the learning experience are found in our in-residence experience but missing from our distance learning course. We considered how these elements might affect our students' learning outcomes. Our resident course is conducted in an intense 5-week program during which students continuously interact in various leadership exercises. They receive a steady stream of feedback, formally and informally, from both peers and instructors. By contrast, our distance learning course is largely asynchronous. Students work independently and at their own pace. We asked if we cannot offer our distance learners an identical learning experience, what can we do to promote an equivalent learning outcome? For their capstone assignment, the distance learners develop a personal leadership development plan. This plan provides a rich data source for understanding how a student applies foundational leadership concepts in his or her professional life at his or her current level of personal development. If analyzed methodically in aggregate, student artifacts could be characterized to identify common perspectives shared among groups of students. Could this categorization of student artifacts to be used as a guide for developing additional instructional materials targeting these perspectives? We proposed the following research questions to guide our study:

1. How can qualitative analysis of student leadership development essays provide insight into how students process foundational leadership concepts? Can this insight be distilled into themes representing a diverse set of leadership perspectives?

2. How could these themes be used to develop course content that challenges or refines these perspectives?

Leadership perspectives are shaped by a variety of personal factors. Ernest Stech's (2013) psychodynamic approach to leadership study credits personality as a fundamental element of leadership. Demographics can also impact leadership style as discussed by Kao (2006). Models such as situational leadership II (Blanchard, Zigarmi, & Zigarmi, 2013) rely on the leader's personal experience. Rather than consider each of these factors: personality, demographics, and experience and their potential impact on our students' leadership perspectives, we examine them holistically. As an assignment requiring students' personal reflection on leadership style and preference in context with their current professional environment, we believe the relevant factors shaping these perspectives manifest themselves in the texts. Recommendations presented to enhance the curriculum will not target or even address individual elements that shape perspective, but the perspectives themselves and their implications for leadership development.

One hundred twenty-four student leadership development essays were collected between October 2014 and January 2015. All student artifacts were sanitized of personally identifiable information and also association with particular military units. Fifteen faculty research participants were each given a standardized data collection sheet for each artifact. The instructors read each paper and then focused separately on the “leadership vision” and “strengths and weaknesses” sections. They were asked to read the text and write a set of words or phrases characterizing how the student applied the course's foundational concepts in this application-level assignment. They were told to avoid using phrases describing the degrees of quality but rather to use words and phrases that described the paper's distinguishing characteristics. Faculty worked alone and in pairs each reviewing between five and 15 artifacts from various classes. The entries were consolidated and the lead researchers coded comments and observations summarizing the individual researcher's description. The lead researchers then read the original artifacts and made a second set of characterizations creating a pair of entries for each artifact. Each pair was examined and a third entry was made noting similarities and differences between the comment pairs.

To construct individual themes, entries were scrutinized using the “compare and contrast” approach as described in Bernard and Ryan (2010). From the first entry, an initial set of candidate themes were created. Each subsequent entry added to the existing set as applicable and new sets were formed for perspectives not yet identified. Once the complete data set was entered, similar groupings were consolidated.

Once the draft themes were assembled, narratives were created to describe each theme. Then, a set of previously unread artifacts was used to test whether the set of all themes was sufficient to characterize each unread paper. Once each of these final artifacts was placed into one or more themes, the set of themes was determined to be “saturated.” The concept of saturation is borrowed from grounded theory (Glasser & Straus, 1967). When additional artifacts examined fail to alter the current model, that model is said to be saturated (Charmaz, 2006). In this research, the “model” is the set of themes that describe common leadership perspectives held by students in our distance learning leadership course.

Analysis of student papers by subject area revealed distinct types of themes for each leadership subject. Themes associated with leadership vision were fairly homogeneous. Each theme was built in layers, with some artifacts rich in contextual detail and others simply stating a vision without additional elaboration. In aggregate, each theme formed a coherent idealized perspective. No one student belongs wholly to one theme, nor does one theme define any student completely. Our recommendations for curricular improvement in this area are likewise homogeneous. By contrast, themes associated with introspection varied both in structure and in our approach to pursuing equivalent learning outcomes. Some introspective themes were simple, while others were nuanced. Our approach to integrating the themes into recommendations for curricular improvement varied with the level of complexity of each theme. In both subject areas, we consider novel ways to fill the equivalency gap, to find alternate means of achieving equivalent learning outcomes. In the resident course, students have their perspectives reinforced when they observe like-minded peers approach problems in ways that are congruent with their own methods. Likewise, their perspectives are challenged when they interact with students whose methods and ideas are very different from their own. If instructional designers can create distance learning content in the form of readings, case studies, or vignettes, deliberately crafted either to reinforce or to challenge existing perspectives, then the learning outcomes will be more equivalent to those of the residence course where continuous student interaction and feedback provides this natural source of leadership growth.

Students are asked to create a personal vision for their leadership. This portion of the assignment provides an application-level assessment of foundational material concerning the need for vision in an organization and methods for creating a vision. Analysis of the vision presented in the student essays reveals the diversity of values and opinions concerning the manner in which students wish to develop their leadership and priorities they place on various actions, relationships, and behaviors. The following themes represent overlapping perspectives shared by elements of the student population. Each theme is first described by the researcher's interpretations and then supported by representative quotes from student artifacts. Two considerations for instructional design are then presented for each theme, one to reinforce the perspective and one to challenge it.

Artifacts contributing to this theme present evidence that the student sees the workplace as an extension of his or her family. So much time is spent at work and mission success depends on healthy relationships with others. By regarding coworkers as family members, students in this category stress the importance of taking care of one another as a necessary step toward mission accomplishment. Leadership is seen as a calling or vocation, often with spiritual underpinnings. Mentorship is seen as a responsibility to raise up juniors. Representative perspectives:

• create a sense of family and friendship between coworkers who all feel they have a personal stake in the organization's direction;

• helping shape the air force from the lower levels up;

• to be an understanding, God-inspired driven leader; and

• [my vision] strives to help [others] realize their full potential.

For instructional designers, a vignette or reading to reinforce the perspective might give an example of a work center where a leader assigns tasks that play to the individual strengths of the members while considering their development needs. A counterperspective may involve a leader having to make tough choices regarding putting people in harm's way. Here the “family” metaphor is challenged. A parent would be expected to always put themselves between their children and danger. A leader cannot always afford this luxury.

Artifacts contributing to this theme describe a leader's role as creating favorable conditions to enable success. These conditions have predominantly benevolent properties: fairness, honesty, equality, respect, trust, et cetera. These properties enable mission accomplishment by addressing basic human needs. Representative perspectives:

• “I intend to create an atmosphere of hardworking, dedicated individuals united by the overall goal of achieving excellence that leads to the effective accomplishment of our stated mission”;

• “I will accomplish this by fostering an atmosphere of trust”;

• “adopt the mindset of never giving up in a society where trust, fairness, dedication and honesty are lived every day”;

• “concepts such as honesty … allow us to work together [so] cohesively”;

• “create an organization today that has a solid foundation rooted in core values”; and

• “know that their work environment will stay productive, happy, and positive under my direction.”

Instructional designers may reinforce this perspective by incorporating readings or vignettes that emphasize organizational transformation. Alternative perspectives could be examined by vignettes that illustrate how teams and individuals have a voice in shaping their organizations and that leaders may need to cede some of their personal vision in order to integrate the diverse contributions of those they lead. Additionally, organizations have pasts, and leaders need to take this into account when seeking to change the culture.

Artifacts contributing to this theme show evidence the student envisions conditions where they are supremely dedicated to the mission and the welfare of others. They desire always to place themselves second in priority. For them, the best way to lead is by setting the example. They believe leaders can expect results from others only once they have demonstrated a willingness to do the heavy lifting themselves. Representative perspectives:

• “I want to be reliable, approachable, helpful, knowledgeable, friendly, professional, and adaptable”;

• “be a living model of excellence within [organization]”;

• “Armed with an untiring work ethic, strengthened by competence and self-disci-pline, I will lead from the front and by example”;

• “Be a supervisor that people will remember and try to emulate”;

• “by my example of hard work, responsibility, integrity, charity, obedience, and courage”;

• “As a leader, you cannot ask your troops to do something that you have not shown that you are capable of yourself”;

• “I envision a leader always leading by example, regardless of the situation”; and

• “A leader is one that motivates their subordinates through personal example, compassion, and selflessness.”

Instructional designers can reinforce this perspective with content illustrating positive outcomes, such as increased loyalty, following selfless leadership acts. Alternative perspectives can demonstrate how, in organizations requiring a good deal of technical specialization, a leader cannot know all or do all. Willingness of a leader to roll up her sleeves and get dirty must be balanced with the need to trust in the skills of her people and even defer to their professional opinion from time to time.

Artifacts contributing to this theme contain elements envisioning personal excellence. By being the best performers in their job or profession, their unit is more successful and those around them may be inspired. Improvement efforts tend to focus on increasing personal credentials and professional knowledge. Representative perspectives:

• “to become a top-notch Medical Service Corps Officer who directly contributes to the success of the AF mission”;

• “to always lead with integrity and strive for excellence in myself and others”;

• “Strengthened by competence and self-discipline, I will lead …”;

• “There are certain things I can do to help improve my efficacy and value to the USAF”;

• “shape the people around me through excellence in care, professional knowledge, and leadership”;

• “My vision is to have … competent operational skills and the influence to improve my unit”;

• “It shows direct application of my strengths in being a physicist and indirectly in leadership”; and

• “individuals united by the overall goal of achieving excellence that leads to the effective accomplishment of our stated mission.”

Instructional designers can reinforce this perspective with content that demonstrates the value of leaders learning not only their job but how their contribution supports the larger efforts of their organization. Alternate perspectives might provide a critical look at leaders who become so task focused they fail to see opportunities for process improvement or improved relations with others.

Students in this category focus on understanding the individual motivations and needs of subordinates as a means of tapping sources of motivation. They recognize that what may inspire one person (or themselves) may not inspire others. Their vision contains a deliberate effort to know their people and understand what they value. Representative perspectives:

• “by honing in on the emotional side by encouraging esprit de corps and to go above and beyond”;

• “inspire and empower others”;

• “utilize individual consideration … to assist in motivating the technicians I manage”;

• “inspire in others the ability to improve their personal and professional lives”;

• “motivating those around me, mentoring others to help them achieve their full potential”;

• “providing an individualized finely tuned leadership style when needed to further develop and transform those he leads”;

• “fully understand the environment, situation and individuals, so that I may inspire, motivate and encourage other to excellence”; and

• “identifying individual needs to enhance motivation, morale, and job performance.”

Instructional designers could reinforce this perspective with content demonstrating the transformative power of individual consideration (Sosik & Jung, 2010). An alternative perspective could be provided by content illustrating potential risks from leadership that is too deferential and too focused on pleasing others.

Artifacts contributing to this theme emphasize a focus on change. Students seek improvement in organizational processes and welcome new ideas. Representative perspectives:

• “to adapt to constant change while empowering others to adopt the mindset of never giving up”;

• “A leader who adapts to the situation at hand and inspires others to do so they may achieve success consistently”;

• “I need to be capable and ready to lead the organization's highly technical and socially evolved personnel under dynamic circumstances”;

• “willing to approach me with new best practices for improving long-standing finance processes”; and

• “challenging followers to be innovative and creative.”

Instructional designers could reinforce this perspective with content demonstrating a leader's success in overcoming resistance to change resulting in better processes or working conditions. An alternative perspective may highlight the unsettling effects of constant change in the workplace and of the tendency for new leaders to want to reinvent the wheel.

Students are asked to create a personal inventory by listing their strengths and weaknesses and discussing how they help or hinder their success as leaders. They are also asked to define for themselves a set of personal core values. This portion of the assignment provides application-level assessment of foundational materials concerning the need for introspection, methods of introspection, and understanding importance of personal development. The themes identified describe a set of similar perspectives observed among the student population. Analysis of the students stated strengths and weakness along with their perceived impacts reveal diverse ways in which students consider and value self-knowledge.

Three types of themes related to students' self-identified strengths and weaknesses emerged. We distinguish them based on their level of complexity and their relation to specific course content. We label the first type as “concrete.” We define a concrete strength or weakness as easy to describe, easy to see how its presence impacts work relationships or processes, and easy to imagine that others would reasonably be able to detect its presence based on demonstrated behavior. For themes in this category, we generally suggest that instructional designers can improve learning outcomes by adding content illustrating how leaders leveraged their particular strengths or mitigated their weaknesses in the pursuit of positive leadership outcomes. A trivial example would be an impatient individual with a bad temper. Behavior associated with ill-tempered individuals is easy to detect. If there's an ill-tempered person in the workplace, everyone is aware of it and the effects on the workplace environment are easy to observe.

By contrast, we label the second type of strength or weakness as “complex.” These themes are more ambiguous. Students have greater difficulty describing their impacts and their ambiguity opens the possibility for wide disagreement between the student's perception and the perception of coworkers. For themes of this type, we recommend instructional designers create content to challenge the student's initial perception and encourage deeper introspection. For example, a student who lists perfectionism as his biggest weakness may prematurely accept this quick self-diagnosis rather than expose himself to a potentially more unsettling self-examination. Content designed to challenge initial conclusions about complex patterns of behavior would enable development. The third type of theme we label as “referential” partially overlaps the first two. In a referential theme, the student-reported strengths and weaknesses are described in terms of skills, attributes, and behavior taught elsewhere in the curriculum. Examples of this include emotional intelligence and active listening. For students who define their weakness in terms of course content, we are provided with an unanticipated opportunity to target curricular improvement in these areas of the course. For example, for students who identify weaknesses with particular emotional intelligence skills, we can offer content designed to foster improvement in these areas.

While the first and third types of themes may inform course content improvements, the complex themes offer a specific opportunity to promote equivalency of learning outcomes with the resident course. Students in the resident course give and receive frequent peer feedback, both formally and informally. They are also required to engage in journal writing to reflect on how their leadership evolves throughout the course. The resident course's cycle of leadership and problem solving events, followed by journaling followed by feedback fosters self-assessment skills vital to leadership development. In pursuit of similar outcomes for distance learning students, instructional designers can leverage insights from the complex themes, to create content to encourage deeper levels of introspection. We believe that if content is created to explore some of these complex themes, students will make more personal connections with the material, creating the opportunity for greater personal reflection. Each introspection theme is described below and is classified by type. Representative quotes from student artifacts are given to support the theme's construction, and one or more instructional design considerations are presented.

Artifacts contributing to this theme featured students who reported difficulty with time management, juggling tasks and deadlines. Admissions of procrastination are common along with reports of being overwhelmed with work. Representative perspectives:

• “[I have] issues with splitting time and or delegating responsibilities across [multiple projects].”

• “One of my biggest weaknesses I see in myself is that I can become overwhelmed when my workload has reached a point which I think is insurmountable.”

• “Finally, my dedication to a task can often lead to tunnel vision, in that I will stick with a task until it is completed, which can sometimes affect other tasks.”

This theme is among the most concrete. Instructional designers could consider content to foster better time management. Students often cite a lack of time as a barrier to introspective practices. Discussions of time management would, therefore, be an appropriate enhancement in one of the course's introspection lessons.

Artifacts contributing to this theme reveal tendencies of micromanagement. Students claim technical proficiency and value quick, decisive action. Speed of task completion is often given priority over work quality. This weakness is often considered a mixed blessing, with several students stating that their decisiveness often benefits their productivity but can also hinder collaboration. Structure is often seen as an enabler of a productive environment rather than as a hindrance to creativity. Clarity resulting in task efficiency is generally valued over ambiguity, which increases task complexity. Control of a task or situation often replaces the need to trust others. Representative perspectives:

• “being extremely structured, lacking flexibility and micromanaging [are] my weaknesses”;

• “[My] most consuming weakness is my micromanagement tendencies”;

• “My hastiness to complete projects ahead of schedule and accomplish tasks quickly is often viewed as efficiency, but on a personal level I desire to put forth more care into my work”;

• “I tend to set clear expectations”;

• “Unfortunately, it is too easy for me to default to [methods I am used to] in order to bring a quick resolution to the discussion”;

• “I have a tendency to feel like my ideas are the generally the best way to reach a goal”;

• “My boldness to assume a leadership position [is] often followed by the steamrolling of others”;

• “Allowing myself to gain trust in others will help me minimize micromanaging”; and

• “I have a tendency to take on projects by myself in order to control the narrative.”

The tendency to micromanage is easy to detect, and the impacts are easy to identify. Students who hold this perspective generally admit to a desire for control and difficulty trusting subordinates. Instructional designers could include content offering strategies to improve students' ability to delegate, create greater trust within their organization, or learn how to extend trust themselves.

Artifacts contributing to this theme cite a tendency toward directness, which can be off-putting and create an adversarial environment. Some cite a lack of empathy that prevents them from anticipating potential consequences from their bluntness. Representative perspectives:

• “[My weaknesses include] a lack of empathy for others”;

• “[I have a problem] being too quick and blunt in responding to others”;

• “My biggest weakness that I have is that I am very direct. This can be considered positive because individuals know what I am thinking, but at the same time many people take offense to my method of delivery”; and

• “[People] admire my passion, but not the way that I deliver my thoughts and reactions.”

This theme is concrete, as the problem and the related consequences are easy to identify. It is also referential because a lack of empathy is described elsewhere in the curriculum as a deficiency of emotional intelligence. To better serve students who identify a weakness in this area, instructional designers could integrate content that helps students explore methods for increasing emotional intelligence skills related to empathy.

Artifacts contributing to this theme contained emphasis on confrontation avoidance. Students cited a tendency to be deferential to others and as leaders. There is often a stated conflict between the desire to be liked and accepted and accomplishing the mission. While some acknowledge that leadership sometimes requires making unpopular choices, they still struggle with exacting discipline and holding people accountable. Representative perspectives:

• “One of my weaknesses is carrying out discipline because I value the approval of others, and I dislike being the bad guy”;

• “Some of my weaknesses are … and effective communication in confrontations”;

• “My self-identified weaknesses are dislike of confrontation …”;

• “However, at times I can be … and nonconfrontational, which can undermine [my] vision of future leadership”;

• “I want to find the right balance of taking care of people but still be an effective leader”;

• “At times I have found myself passively managing by exception when I should be actively managing by exception under circumstances where deviations from standards are intolerable”; and

• “Worrying about people liking me will be the weaknesses that I need to overcome.”

This theme is fairly straightforward, though clearly deeper introspection may reveal a more nuanced understanding of the problem. Are students who hold this perspective genuinely afraid to speak out? Is confidence in the leader's knowledge and skills a factor? In either case, deferential behavior is discussed in the course's emotional intelligence unit. Instructional designers can improve learning outcomes with by developing content promoting greater introspection into the nature of a leader's deferential behavior or by enhancing the course's emotional intelligence unit by adding strategies to improving related emotional intelligence skill sets.

Artifacts contributing to this theme contain statements the students see themselves as personal and approachable. The students describe themselves as empathetic and trustworthy in interpersonal matters. They value open communication and are able to accept criticism. Representative perspectives:

• “One of my biggest strengths is that I am approachable and personable, and great at giving my team individual consideration and mentorship”;

• “I have been told I have a strong ability to relate and empathize with others”;

• “My approachable demeanor is not only beneficial to my subordinates, but it also ensures that the communication channels are always open”;

• “I have the ability to work well with a broad range of personalities”;

• “I believe that I must be easily approachable for all types of people so team members won't keep problems and issues to themselves”; and

• “I listen to people, and [have] an ability to take criticism and try to change myself for the better.”

This theme is less concrete. The leader may believe he is approachable and personable, but this does not guarantee he is perceived to be. Instructional designers can develop content to promote greater self-exploration into how leader behaviors affect others' perceptions. Additionally, content could be added to promote thinking about opening lines of communication to enhance mission accomplishment and foster trust.

Artifacts contributing to this theme contain evidence of students who pride themselves and in large measure define themselves, through their work ethic. Hard work is the answer to any question and engaging in tasks with total dedication is a moral imperative. Representative perspectives:

• “My strengths are … commitment, purpose, and [mission orientation]. These strengths drive me in my everyday tasks and support me when I doubt my leadership abilities.”

• “[My strength] is my extreme passion and drive for everything I do. When I am charged with a duty, project, or job I hold it with such high importance and tend to become emotionally involved, so that failure is never an option.”

• “My strengths include a strong work ethic and determination to excel wherever I can.”

• “My strengths include a good work ethic, a drive to succeed fueled by my family responsibilities.”

This theme has greater complexity than the previous ones. Although it is easy to recognize outward displays of hard work such as putting in long hours, creating voluminous documents, serving on various committees, et cetera, the potential exists that one's efforts align poorly with organizational goals and priorities. Furthermore, a leader who is consumed by tasks may find his subordinates lacking in guidance and purpose. Instructional designers can promote greater introspection for students who identify with this theme by creating content illustrating negative outcomes that may result when leaders focus on tasks at the expense of building relationships or providing guidance.

Artifacts contributing to this theme cite an inability to motivate or inspire others. There is often recognition that other people have motivations different from their own but students struggle to apply this knowledge effectively. They cite difficulty in creating a vision for the future that others will embrace. Representative perspectives:

• “I will need to work harder [at] Inspirational Motivation. [This has] always been difficult for me, and failure to develop and grow these abilities may lead to the unintentional alienation of coworkers.”

• “I have always considered myself a realist and rarely an optimist.”

• “Because I do not need external motivation or praise, I don't always give as much personal attention as I should to my employees.”

• “My motivation comes from within, therefore, I need to focus on what others may need on an individual basis.”

• “I find it difficult to interact with someone who is unmotivated.”

• “My best option is to … [to try] to get to know them better in an effort to determine what motivates them.”

• “My weaknesses as a leader [include] being motivational and taking more initiative.”

• “My challenge to myself is being able to provide inspiration and guidance.”

This theme is complex because of the uncertainty of its assumption. What factors might influence a leader's inability to align people to her vision? Is it a simple matter of packaging and communicating the idea or is it more involved? Do others want the same outcomes? Does competition for power and influence in the workplace play a role? To help students who share this perspective, instructional designers could create content to inspire deeper introspection on the student's relationships within the workplace and the alignment of individual ambitions with organizational goals.

The use of case studies, vignettes, and other illustrative tools to enable students to make connections between foundational concepts and their personal and professional lives is nothing new. However, course managers have limits on the length and duration of reading assignments and activities based on a course's credit hours. Further, when building distance education courseware, instructional designers try to streamline their units of instruction to avoid overcrowding of content. The methods proposed in this research offer instructional designers a chance at improving both the effectiveness as well as the efficiency of their content design. By understanding commonly held student perspectives on relevant topics, designers can be more deliberate in the design and selection of course content.

In this research, the authors suggest that creating course material with the students' perspectives in mind will not only lead to efficiencies in course design but can also be used to promote equivalency between resident and distance learning courses. By creating content to challenge or refine existing perspectives, designers have the opportunity to compensate for the lack of synchronous interaction inherent in our distance learning courses. If this research is used to modify the existing curriculum, follow-on research could compare essays of students who completed the redesigned course. Researchers could look for evidence of more nuanced themes than those used in this study. As an action research, a focused conversation with leadership faculty attempting to define or quantify levels of leadership sophistication by degree would be valuable as a faculty development exercise and informative in making further course improvements. The use of analytics connected with the course management system could also detect whether students engaged with the new materials longer or more frequently than the legacy materials. The methods employed in this research may be directly applicable to other programs of study for adult learners where significant personal and professional experience is expected of the student body. An executive course in business management may be a suitable choice, as such programs require the application of fundamental concepts in professional contexts and cater to experienced learners from diverse backgrounds. Once improvements are made, qualitative methods could again be used to detect evidence of change. Carver and Klein's (2013) qualitative approach for detecting student change offers one potential approach. The authors of this study believe that interaction among students is always beneficial in contextual courses of study such as leadership. They encourage course designers to seek opportunities for direct interaction whenever possible. However, when such interaction is impractical, the methods outlined in this research provide an alternate path to equivalency. When interaction is possible in distance education delivery, the methods in this study can still be employed to produce an evidence-based understanding of student populations and guide the design content based on this understanding.