Instructional Design Issues in a Distributed Collaborative Engineering Design (Ced) Instructional Environment Free

Like other professionals, today's engineers are working in distributed, interdisciplinary, problem-solving, technology-enhanced environments. New engineers must be prepared to be successful in these practice environments. Thus, engineering education is experiencing significant changes. Eschenbach et al. (2002) described the working style of future engineers as a highly technology-dependent team collaboration process requiring multidisciplinary expertise to solve engineering problems. It is necessary, therefore, to adapt current engineering instruction to better prepare engineering students to meet these future working environment requirements. Specifically, engineering students will need to develop a deeper understanding of basic engineering principles and procedures and the application of specific disciplinary perspectives to ill-structured problems. They will need to become better collaborative problem solvers and to be comfortable with, and able to use, advanced technologies for information retrieval and communications. They will also have to be effective at working within interdisciplinary and distributed teams. This study focused on understanding how engineering students evaluated the use of collaborative technologies during learning experiences that engaged them in technology-supported, distributed collaborative engineering design problem-solving activities.

The new trends in engineering work environments challenge engineering educators to design and create curriculum and learning experiences that better align engineering instruction with real-world engineering practice. Although there are many opportunities in the college environment to provide engineering students with activities encouraging collaborative learning and problem-solving, there is no one fully developed curriculum that helps a student appreciate and understand the differences that disciplinary perspective, learning and communication styles, values, and work ethics bring to the problem solving teams (Eschenbach et al., 2002). Therefore, given the diversity of currently practicing and up-and-coming engineers in both knowledge and personal characteristics, there is a strong need for engineering instruction that fosters deep understanding of basic and disciplinary engineering knowledge, problem-solving skills applicable to complex engineering problems, effective team-collaboration and communication, and skills to apply new technologies in both face-to-face and distributed work environments.

Designing and implementing a distributed collaborative engineering design learning environment that integrates multiple technologies is a complex task. It requires the use of sound pedagogical and instructional design foundations to engage engineering students in the activities of practicing engineers. Many academic engineering environments are moving away from teacher-centered expository instruction toward more learner- and problemcentered collaborative learning (Lord & Madsen Camacho, 2007). However, little is known about how successful such environments are with learners who have traditionally been schooled in presentation-practice-test environments and are now working in virtual collaborative teams on ill-structured design problems that do not have a single correct answer (Davidson et al., 2002; Koszalka, Wu, & Davidson, 2007, 2008).

Course facilitators are also challenged to change teaching practices from traditional lecture-based to more open-ended studentcontrolled activities (Cohen, 1995; Lord & Madsen Camacho, 2007; Turns, Adams, Linse, Martin, & Atman, 2004, Turns, Eliot, & Linse, 2003; Yager, 1991). Often such instruction includes activities, learning assessments, and engagement strategies that are complex and new to faculty. Yet, the high standards and high stakes for engineers in their first few years of practice cannot be lowered (Davidson, et al., 2002; Project Lead the Way, 2006). The shrinking pool of engineering graduates and growing number of engineering career openings demand high quality graduates who are ready to practice in these new collaborative environments immediately after graduation (Project Lead the Way, 2006). Most engineering education programs require students to participate in internships where they are exposed to the collaborative problem solving environments of practicing engineers. However students are not completely prepared with the skills and knowledge to be successful (Davidson et al. 2002; Lord & Madsen Camacho, 2007).

This article examines a senior engineering course that engaged engineering students in a distributed collaborative engineering design (CED) learning environment. The focus of the course was to prepare students with deep understanding of disciplinary engineering knowledge, strong abilities to solve complex engineering design problems, effective team-collaboration and communication, and solid skills to work effectively in both face-to-face and distributed (virtual) work environments.

To achieve the planned course goals, instructors from two universities collaboratively taught the CED course directly to the students from their home institute and synchronously at a distance for those at the partnering institute. Participating students from both universities simultaneously attended course lectures either in-person at their home institute or through distance education technologies, depending on which of the two professors was responsible for the session content. All students were assigned to distributed design teams, 50% from the local and 50% from the distance university. All students received instruction in the foundational engineering content and necessary technology skills to participate in the course activities. For several weeks, however, the students were split into one of two engineering content learning tracks. Thus students, for the sake of a culminating activity, had different engineering expertise from which to collaborate on a resolution for a given engineering design problem. Each distributed team, therefore, had a mix of students from both universities and each of the two engineering content tracks. These distributed engineering design teams thus had to bring together different types of engineering knowledge to collaboratively solve a design problem in an authentic engineering working environment; that is, as distributed interdisciplinary teams working to solve complex engineering problems. The course instructors acted as project managers and mentors to help the teams be successful at their tasks.

Formative and summative evaluation results for the third offering of this course are presented. The purposes of these evaluations were primarily to collect data from students on the learning environment and activities and their perceived ability to engage successfully and learn in this environment. Such data were used to determine course enhancements and gauge perceived success of students. Instructional design issues emerged from the data and are discussed based on course design feedback, results of team-collaboration activities, use of technology applications, student perceptions of their own learning, quality of interaction during classroom and teamwork activities, and overall instructional strategies and course structure design.

Two central New York universities collaboratively developed the Advanced Interactive Discovery Environment (AIDE) for Engineering Education. The AIDE integrates and advances the best features of virtual, collaborative engineering environments, state-of-the-art simulation tools, and advanced learning management systems (Davidson et al., 2002). The AIDE was integrated into a one-semester senior-level engineering design course that was delivered synchronously to students at both institutions. This version of the course was codesigned and coinstructed by two faculty, each of whom was from one of the two participating universities. Collaborative instructional strategies were emphasized that engaged students in multidisciplinary activities and engineering design problems in a distributed learning environment. The long-term goals of the project were to:

• Improve the AIDE through formative feedback and data collected on the nature of virtual interactions, and

• Evaluate the effectiveness of multifaceted instructional methods that leverage emerging information technology, enhance student learning on fundamental content and technologies, provide learning experiences in systems-level engineering, and expose students to the multidisciplinary nature of present and future engineering problems.

Given these goals, the instructional design foundation for this course was based on an integrated approach merging three instructional methods: generative learning, problem-based learning, and collaborative learning. This combination of instructional methods provided learners with instructional activities that prompted engagement in deep learning, development of problem-solving and collaboration skills, and application of new technologies into their learning processes. This instructional design approach also provided learners with experiences in the types of complex interdisciplinary collaborative design activities they will experience as working engineers (Davidson et al., 2002). Thus, during this course the students engaged in authentic engineering practices.

Generative learning suggests that learners are continuously active during the learning process. The focus of their activity is in building connections between existing and new knowledge. Generative learning suggests that the mind actively interprets new information and draws inferences from them as an individual interprets, organizes, and integrates new information into existing schema (Wittrock, 1990).

Wittrock (1990) claimed that there are two types of generative activities: activities generating or creating organizational relationships between different elements in the environment, and activities generating integrated relationships between the external stimuli and the memory components. Instructional strategies based organizational generative activities include writing summaries, developing relevant questions, and creating tables to summarize main ideas, for example. Instructional strategies based on integration types of activities include demonstrating new knowledge, writing examples, drawing pictures, interpreting phenomena, paraphrasing, predicting, or using inferences, all activities that engineers engage in regularly. By continuously engaging in and repeating these generative activities, students are able to practice newly-received knowledge in light of their prior knowledge level and to expand their cognitive schemas through building connections between the old knowledge and new information (Grabowski & Koszalka, 2002).

As technology plays a critical role in an online or blended learning environment, it is important to design and integrate technology tools into the instruction. It is even more important to apply the technology in a way through which students can be effectively engaged in activities that facilitate connections of new knowledge with students' existing knowledge structure (Talsma, 1999). For example, in the CED course learners take new knowledge from the lecture sessions and apply it during online synchronous design brainstorming sessions with team members, using drawing capabilities of provided tablet PCs to share and modify engineering component pictures, engineering design mathematical formulas, and conceptual environmental impacts. This hands-on-minds-on approach helps learners organize and integrate new knowledge into their existing schema for later application in problem-solving situations.

Problem solving is what engineers do, and it has long been regarded as a generative activity that can promote deep learning (Puntambekar, 2004). Problem-based learning (PBL) is an instructional method in which learners simultaneously develop both disciplinary knowledge and problem solving strategies. They learn by being placed in an active role of problem solvers confronted with an ill-structured problem that mirrors real-world problems (Finkle & Torp, 1995). Students construct knowledge by working with new information they identify while trying to solve problems. Thus, the learning is not in solving the problem; rather, it is in investigating the information necessary to solve a problem.

Problem-based learning also influences students' perception and motivation to learning. Using authentic problems in the learning situation motivates learners to take on the challenge of solving real-world problem rather than approaching learning as a content memorizing task. As PBL is moved into blended learning environments, extra thought and effort are required to design technology supports and tools that effectively facilitate learning and practice. This is because much of the learners' PBL learning experience relies on social negotiation; thus it is particularly important to consider how technology can facilitate distributed communication (Orrill, 2002). Cordeiro, Kraus, Hastings, & Binkowski (1997) assert that “in the group process of discussing and reflecting critically collaboratively, the problem-solving capacities of the group afford the individual learner opportunities to internalize these group understandings” (p. 7) and, thereby, improve each members' overall problem-solving skills. But as Orrill (2002) points out, collaborative PBL implementations can be weakened by poor use of tools and/or poor design of tool application during distributed or virtual instructional activities. Tools should be designed and applied in ways that support the process of PBL, particularly in regard to team-collaboration in a technology-based blended learning environment. For example, often engineering collaboration requires the use of hand-drawn designs augmented with calculations of structural or material strength for instance. These collaborations often are held in front of a drawing board where two or more engineers bring their expertise into design discussion of some structure. To have a successful collaborative design meeting in a virtual environment, engineers need tools that allow them to easily draw, talk, model, and edit. Technologies can facilitate or inhibit such activities based on their ease of use and features. Using complex 3-D engineering drawing package generally requires advanced technical skills and robust virtual environments thus can inhibit distributed collaborative problem solving, where as hand drawing capabilities of tablet PCs can facilitate such interactions very much like face-to-face sessions in front of a drawing board.

From a social constructivist perspective, learning is considered to be a social-constructive process by which knowledge emerges through a network of collaborative interactions and is distributed among humans and tools that interact (Lowyck & Poysa, 2001). Collaborative learning thus is an instructional method in which learners with various types and levels of knowledge work together toward a common goal (Gokhale, 1995). Each learner develops his or her own knowledge and skills as well as is responsible for contributing to team members' learning (Panitz, 1996). Thus learners develop content domain and interpersonal communication knowledge and skills during collaborative learning activities. Moreover, collaborative learning can also influence student motivation, increase self-efficacy, orientation toward a learning goal, and intrinsic value of the learning task (Lowyck & Poysa, 2001).

Together, collaborative learning, PBL, and generative learning activities provide a rich environment in which to learn and practice collaborative engineering design. In a recent study, Lord and Madsen Camacho (2007) found that engineering faculty believed in and practiced a variety of activities influenced by collaborative, problem-based, and generative learning in their classrooms. Activities such as engaging students in content discussions, inductive learning and inquiry, writing notes of key concepts, tying course material to relevant examples and student background knowledge, and incorporating the professor as guide and partner during learning. Thus, the foundations of generative learning, PBL, and collaborative learning informed the design of this learning environment. Generative learning theories specifically informed the design of: an active learning process of building cognitive connections between new and old knowledge, hands-on, minds-on instructional activities that supported learning, technology tools that support generative learning instructional strategies, and instructor roles that are facilitative of student active learning. Problem-based and collaborative learning foundations specifically informed the design of instructional activities that supported learners to engage in: solving ill-structured, real-world problems; approaching problems from multiple perspectives; learning while engaging in problem-solving processes; using technology tools to support their learning, communication, and problem-solving; and consulting with instructors as process facilitators rather than providers of knowledge (see Table 1).

The Collaborative Engineering Design course was designed as a one-semester capstone course for senior engineering students. There were 14 scheduled class sessions during the semester. The first two sessions were teleconferenced lectures providing general information about the course, course objectives, final collaborative design project, and the AIDE technology features. The students from the two participating universities also learned about each other during the first two sessions. The students from both universities were then divided into two short courses, discipline specific tracks (DST) in aerospace structure and finite element analysis. Lectures, discussions, and mentoring sessions were conducted synchronously to the students in each DST using both video-conferencing and IP-based web-conferencing technologies (IP-based web-conferencing technology is called AIDE SameTime [ST]). Through the DSTs training sessions each student developed a specific content knowledge and the skills to use provided analysis tools required to complete the collaborative project. Students in one DST track did not receive instruction from another DST track. As every student team was composed of students from different DSTs, it pro vided an opportunity for team members to work closely together in order to finish the final design project. Three to four teleconferenced lectures were presented between DSTs.

During the DST sessions a major part of the course was dedicated to the collaborative engineering design project. This project required students to create a preliminary design of a thermo-structural system for a specific location on a hypothetical second-generation reusable launch vehicle (RLV) for NASA space missions. As students worked on the design project, parallel DST lectures were delivered providing additional information on aerospace structure, materials, design and analysis of composites, thermal analysis, risk assessment, cost assessment, project management, optimization and probabilistic approaches to design. The DST lectures also helped students to operate the finite element software (ANSYS) for dynamic structural analysis. Necessary information and resources were provided to students to help them develop an understanding of the specific course content and prompt their application of their new knowledge in the design project. These resources include tables of structural and thermal loads, design examples, a pre-screened collection of NASA and other reports containing material properties, test results, and other information relevant to the design. All the materials, presentations, and resource information were uploaded onto the AIDE website for students to review.

Four student teams were created, each having a faculty coach. The coach helped the team members get organized, provided feedback on design concepts and progress reports, facilitated team interaction and communication, and aided with technology difficulties. Since the faculty and students were also at different locations, e-mail, phone, and IP-based web-conferencing were the usual ways for them to communicate.

The “not-so-hidden agenda” for the course was also to promote skills development in using cutting-edge technologies, working on distributed teams, writing project reports, and giving oral presentations. Teamwork was a critical aspect of the course design; thus, several team-building exercises were presented. Early in the semester, best practices labs were conducted to help students build productive teams using the collaboration technologies available in the AIDE. Assessments of student learning were based on individual work for DST assignments, team oral and written reports, and self- and peer-evaluations for work performed within each team.

The course infrastructure built up to support this engineering design course included four components: distance learning classrooms (DLC) equipped with video conferencing system for the full class instruction; IP-based web-conferencing satisfied needs of multiple simultaneous partial-class events or synchronous team collaboration; asynchronous online information management and communication system provided a platform to document course materials, exchange files and hold engineering discipline-related information and resources; and a design studio and individual tablet PC allowed students to have more opportunities and freedom to work with technologies.

Full class presentations and discussions took place in the distance education classrooms at both SU and CU classrooms. Figure 1 shows two photographs of the CU classroom from both students' view and faculty view. Both distance education classrooms at SU and CU sites contain two screens in front. When the instructor at SU site was doing the presentation, one screen at CU site would show the image of the instructor at SU site and the other displayed the image from a computer running a screen sharing application such as a slide presentation. The instructors were responsible to take control of all computer applications. While at the SU site, one of the two front screens showed the instructor's sharing application, such as a slide presentation, and another screen was a SmartBoard, an intelligent instructional device that allows the instructor to edit the presentation. The instructor would also write examples or draw graphics on the SmartBoard in a similar way as traditional chalk-to-talk. Using the SmartBoard provided instructors with more flexibility to interact with the instructional materials and distant students spontaneously rather than passively reading a presentation. In order for the instructor to observe students at another classroom site, both DLCs had a screen at the back of the room. At SU, two television monitors were used: one displayed the SU classroom images being sent out, while the other showed the students at the CU site. The similar technology was set up at CU: a single projected screen showed the students at the CU classroom. Sometimes, when the instructor finished the presentation and began discussion time, the front screen would change to focus from the instructor to the students at the remote site. In this way, students at both sites would see each other, which may enhance the social connectedness in the instructional settings.

The instructors used wireless, lapelmounted microphones for class presentation. Students at both SU and CU classrooms used push-to-talk microphones to speak. The push-to-talk system provided good audio effect with less background noise. In this manner, a virtual seminar environment was created wherein students and faculty at the two locations were able to be more interactive with each other (see Figure 1).

To satisfy the need of creating a multiple, simultaneous instructional environment, the AIDE synchronous one-to-many web-conferencing system was used. As it was mentioned above, the short courses delivered in this simultaneous instruction environment were called discipline specific tracks (DST).

Figure 2 shows the synchronous AIDE ST web-conferencing. The picture shows primary synchronous features in ST: video and audio, interactive shared whiteboard, sharing applications, chat box, instant messaging, posted-documents and presence-awareness. When an instructor was conducting a DST, only his image was visible at the right top of the screen until a student spoke into the microphone, at which time the image would change to the student speaking. The instructor showed the presentation on a shared whiteboard for all to see. The instructor could edit on the whiteboard or give control to any student to edit on the presentation slides. Students could speak to the entire class by clicking on a hand-raising icon at the right bottom of the screen. Students could also choose to type comments in a chatbox below the whiteboard. What students spoke or typed could be heard or seen by the whole online class. The AIDE ST web-conferencing was able to provide a simulated face-to-face learning experience to instructors and students in the traditional classroom and those at a participating at a distance.

Students also used ST web-conferencing for their team meetings. Every student was equipped with a tablet PC, a single camera, and a headset microphone with speakers. With all the technology facilities, students were able to connect to the meeting through the AIDE regardless of their location. For example, Figure 2 shows a student team meeting with eight people. Before a meeting started, a meeting moderator, who was usually one of the team members, would set up the meeting and make sure there were no technical problems or difficulties. The meeting moderator usually took charge of the whiteboard unless he or she gave the control to another member in the team. The AIDE ST conferencing was implemented to increase effectiveness and efficiency of students' online team meetings.

The asynchronous AIDE information management and communication system provided services to manage course information and support team communication. These services include shared documents, course announcement, team work spaces, student drop-boxes, threaded discussion board, and team project management tools. Team work spaces were also set up for both student and faculty teams. Within each team work spaces similar tools were provided to help students manage their teamwork, foster communication, and facilitate team efficiency. Figure 3 presents three different views of the asynchronous AIDE environment. The upper left image is the AIDE homepage with the navigation bar on the left and a current course announcement in the central content area. The image in the center is a faculty screen with a menu adapted to faculty's content needs. The foreground image is the homepage of a student team work space that had been customized by the team to suit their preferences.

Furthermore, to facilitate a computer-mediated working environment and to provide students with access to simulation software such as MatLab, Ansys, and Fluent, a design studio was built at each university site. The design studio contained workstations with large screens, web cameras, headsets, projectors, scanners, and SmartBoards. The establishment of the design studio supports local communication and collaboration among students located at the same site, as well as facilitates easier communication with remote team members. Thus, sufficient tools were provided at the institutions, both for in-person and virtual sessions, to support distributed learning and collaborative engineering design problem solving activities.

Designing a blended instructional environment can be more difficult than designing an entirely online course. The flexibility provided by the various components may be distracting to course facilitators and learners, causing difficulties in successfully integrating technology in meaningful ways (Koszalka & Ganesan, 2004). Integrating technology in a meaningful way suggests applying technology effectively into instructional and learning activities to enhance the learning process. If technology cannot be integrated effectively through well-planned instructional design, the course may fail. As Kearsley (2003) suggested, all components of an instructional system are interrelated and interdependent; designing a technology-rich collaborative educational environment requires careful consideration from an instructional systems perspective.

Koszalka and Ganesan (2004) advocated that online instructional environments must be designed to promote and scaffold learning. Well-designed online instruction must provide opportunities for learning that effectively: engages learners with multiple types of resources based on individual preferences; improves the flexibility of instruction by integrated multiple types of interactions; and integrates multiple forms of communications among instructors, learners, and others beyond what might normally occur in a classroom (Collis, 1999, cited in Koszalka & Ganesan, 2004). These suggestions were applied to this blended engineering education environment within an established theoretical foundation of generative, problem-based, and collaborative learning activities.

As Savery (2006) suggested, learners who are new to PBL require significant instructional scaffolding to support the development of problem-solving, self-directed learning, and teamwork/collaboration skills to a level of self-sufficiency. This course's structure was designed to specify particular instructional events and to emphasize how instructional activities should be systematically and reasonably sequenced to fulfill course objectives and better serve students' learning. Different instructional activities were designed that adopted unique technology features in the course. For example, distance learning classroom (DLC) presentations were conduced early in the semester to provide students with general information about the course content, instructional requirements, and technology application requirements. Discipline specific track (DST) sessions occurred both after the course introduction and during the final collaborative project period to help students learn discipline specific content. Team-based design projects allowed students to accumulate content knowledge and collaborative teamwork skills with new technologies throughout the course.

Savery (2006) stated that the critical to the success of PBL is the selection of ill-structured problem (often interdisciplinary) and a tutor who guides the learning process and conducts a thorough debriefing at the conclusion of the learning experience. In this course, each student team was required to develop a preliminary thermo-structural design for a portion of a hypothetical reusable launch vehicle, which may be thought of as the successor to the American space shuttle (Wang, Dannenhoffer, Davidson, & Spector, 2005). This real-world project provided a motivational learning environment that prompted students from multidisciplines to work together while focusing on specific areas of their own interested (e.g., structures, materials, or heat transfer). Each of the student teams was supported by a faculty coach who guided them in teamwork, engineering content application, and problem-solving activities.

The technical features integrated in AIDE can be categorized as a highly-interactive communication tool and highly-functional sharing applications (Davidson et al., 2002). These functional applications included interactive whiteboard and application sharing using interactive technologies to support the exchange ideas among team members facilitating their learning about the problem and their development of content knowledge. In addition, engaging them in drawing graphics, taking notes, or summarizing ideas on either the whiteboard or shared applications (generative activities) could facilitate their cognitive development (learning).

Students were required to participate and contribute to their team equally. Two best-practice lab sessions were held at the beginning of the course to prepare students to work collaboratively using AIDE technology. Guidelines of how to work as an effective team were provided to support a variety of communication needs.

As Silverman (1995) suggested, students in the information age must develop lifelong learning skills, such strategies to successfully choose and apply tools to support professional practices, employing problem-solving strategies, and building strong strategic thinking and meta-cognitive skills. This course was designed to provide a variety of instructional activities to support different learning styles and satisfy different learning needs. The bestpractices lab sessions familiarized students with the technology, the unit assignments fostered learning in DST sessions, the multiple course projects focused on solving authentic engineering problems, and the required oral presentations of solutions provided various ways for students to engage in content and strategy learning. The asynchronous and synchronous technologies provided multiple communication channels and options for students to interact within the class and with teammates and faculty advisors. Learning events and supporting technologies were designed to help students be successful beyond course work, into their profession. The activities and tools in this course provided opportunities to engage students in hands-on, minds-on practice to prompt the development of learning skills, rather than giving fixed body of static knowledge that is simply memorized. Through their engagement they were instructed in new engineering content and engaged in activities to learn how to collaboratively solve engineering design problems.

An evaluative study design was employed to gather descriptive data on student perceptions of course design and tools with the goal of identifying suggestions for course improvements. The study invoked a synchronous integrated evaluation approach that made full use of newly emerging tools to document and evaluate the quality of the digital learning activities, resources, and environment (Rieger & Sturgill, 1999). The aim was to identify how well instructional activities, resources, and provided technologies were used to facilitate learning. The results provide descriptions and suggestions instead of making value judgments as suggested by some evaluative research methodologies (Shadish, Cook, & Leviton, 1991). Thus, the questions investigated were:

1. Did the instructional design approach to the CED course facilitate students' perception of engineering content learning?

2. Did the instructional design approach facilitate students' perception of technology tool learning and use for learning activities?

3. Did the instructional design approach support the development of team-building, teamwork effectiveness, team management, and communication skills?

Questions were also raised to evaluate if the various technology tools within the AIDE helped or hindered collaborative learning, if the learners perceived their collaborative skills as important to the success of team collaboration, and which course components required enhancements.

Both quantitative and qualitative data were collected using multiple sources, including online-surveys, computer log information, team focus group debriefs, and student exit-interviews. Surveys and observations were used throughout the course to gather formative feedback on the design of instructional activities for learning engineering content and as support for team collaborations. The end of course summative evaluations (online survey, group debriefs, exit-interviews) focused on evaluating the overall effectiveness of course instruction and online collaborative tools on student learning, teamwork, and communication activities. Specifically, areas of the course evaluation focused on course objective achievement, course content and design satisfaction, effectiveness of course feedback system and faculty interactions, effectiveness of technology application, selfperception of student achievement and learning, effectiveness of team collaboration activities, effectiveness of communication and interaction among peers and faculty, and identification of course design enhancements.

Additional data analysis was conducted to identify if the instructional needs of this course, as identified in the context introduction, were met: deep understanding of disciplinary knowledge; problem-solving skills; team collaboration and communication; and use and value of technology and tools. This approach was inspired by the case study method that typically includes developing issues or themes (Stake, 1995, adopted in Creswell & Maietta, 2002) and can be a representative of a holistic analysis of the entire case (Yin, 1989, adopted in Creswell & Maietta, 2002). Table 2 summaries the data collection and analysis methods. The performance indicators corresponding to every research question, instruments used, frequency of every data collection, and informants were also included in the table.

Quantitative and qualitative data collected in this study serves the purpose of complementarity. According to Caracelli and Greene (1993), a complementarity purpose is demonstrated as results from one method type are intended to enhance, illustrate, or clarify results from the other. For instance, in this evaluation study, students' perception of their learning was first measured by using a unit outcome survey repeatedly four times throughout the semester. The unit outcome survey collected numerical information with regard to students' perception of individual learning, assessment of instructional activities, and self-evaluation of their individual study efforts. Due to the small study population, only descriptive statistics (means and standard deviation) were conducted. In order to extend the meaning of the numerical information from the surveys, students were asked specific questions about their learning during the course during exit interviews in order to enhance our understanding of students' overall learning process.

Seventeen senior level engineering students participated in the course, eight from CU and nine from SU. In summary, the students believed that the course was a good learning experience, felt the skills they developed from this course would be helpful to them in their careers, and developed a higher level of interest in their DST areas as a result of participating in this course.

Students had mixed feelings about the experience of lectures through the AIDE (distance group) versus in the classroom, admitted to getting lost in the course based on the volume of information and activities, and articulated feelings that meeting in-person would be more effective than participating virtually through the AIDE (see Table 3).

The combination of other quantitative questions, focus groups, and interviews suggested that students felt they learned much about engineering content, engineering problem solving and design practices, and new collaborative technologies. Furthermore, data suggested that students usually approached and solved learning problems during their collaborative (distributed group work) activities. Students generally felt very engaged in the class, both in classroom sessions and when they were working online with their collaborating team.

In response to the first question, students indicated that in general the instructional activities were clear, effective, and informative, supporting their learning needs. Students felt satisfied with DLC presentations; however, a few students suggested that some of full-class presentations were lengthy and could be condensed. Some students commented that SameTime is not an effective tool to deliver DST lectures. In particular, they noted that it was difficult to follow the instruction for the ANSYS program. Upon further prompting during focus groups and exit interviews, several students suggested that ANSYS notes should be immediately posted after the DST lectures. The students generally agreed that the PowerPoint slides used in DST were effective and helped them process and learn new engineering content covered during the lectures.

Some students suggested adjustments to the DST scheduling and reductions in the instructional content on learning theory were warranted. They suggested that additional training was needed on specific design skills that would help them improve their abilities to develop a successful deliverable for the design project.

In regards to technology support tools for learning, the students commented that having the DST replay function (all sessions were automatically recorded and posted in the AIDE for review) and the PowerPoint slides posted online helped them to review complex content for their own learning purposes, gave them the ability to review content to help them complete assignments, and provided access to content that helped support team discussion when engaged in designing their team project. One student specifically suggested that adding more tutorials and resources for buckling analysis would be helpful. Students also evaluated the best-practice labs as being helpful in learning how to use the provided tools in the AIDE.

Overall, interview data, supported by survey data, revealed that students felt they learned a lot of engineering content, particularly how to apply engineering concepts in problem-solving activities. They felt learning opportunities were strongest when: working on the final project; calculating, suggesting, or proposing answers that were consistent with those provided by the professors; completing assignments after the class sessions; reviewing course materials (using the recorded sessions and presentation materials); applying equations during the design projects; and attending classes. Some students also commented that they felt they had learned much about different collaborative technologies when typing in the chat windows, actively participating with the AIDE features during the DST lectures, and learning how to quickly set up a SameTime meetings. As an aggregate, the data suggested that the students were generally satisfied with the course design, instructional resources, and technology tutorials and features. Data suggested that the design of the course supported engineering learning and use of new technology resources to participate in classes, collaborate with teams and faculty, and complete project deliverables.

The quantitative data and supporting interview data together suggested there was no difference between the responses on effectiveness of team collaboration from students at the two universities. Students seemed to believe that collaborative skills were important to their learning and that their team members had used collaborative skills in effective ways during their learning and project activities.

Sketching concepts, defining problems, and solving the problem survey items were evaluated as the most valuable skills supporting collaborative learning. Data also suggested that students believed that their team was usually effective at reaching consensus after discussions, sketching concepts, and maintaining team member relationships. Overall, the data suggested that the design of the course activities, instructional supports, and technology tools supported team collaborations and enhanced the effectiveness of team interactions (see Table 4).

Upon analysis of the entire set of data, several themes emerged: technology use and value of both synchronous and asynchronous technologies, team activities, and disciplinary knowledge and skills. Each is further described below with example of data that represent clusters of responses from multiple resources.

Several responses in the survey data suggested that students required additional technical knowledge to use technology resources and resolved technical difficulties in the AIDE. An opposing theme emerged suggesting that the technology training should be reduced based on condensed scheduling. These conflicting clusters of themes may suggest that more effective and efficient instruction is needed to support and encourage students' application of AIDE technology in class and group activities. The data also suggested that students did not necessarily use all the technologies provided in the AIDE environment. The focus group and interview data suggested that rather than using the communication tools inside the AIDE, students tended to use technologies that were already integrated into their current life style outside of classes, such as e-mail, cell phones, or popular chat applications like AOL or MSN Messenger. Students seemed to prefer these outside technologies because they were familiar with them and frequently used them for their academic work. For example, e-mail was the most common and frequently used tool for scheduling meetings, exchanging documents, and sending alerts. Cellular phones were valuable to schedule team meeting and exchange immediately-needed information. Instant messenger was used both within and outside of ST when team members were working on their individual tasks and needed immediate feedback; in their words, for “loose” collaboration. These comments may suggest why students chose to use some technologies rather than the AIDE tools.

Both survey and interview data suggested that synchronous collaborative technologies, such as the interactive whiteboard, program sharing, and chat window were frequently used during teamwork. ST meetings were most favored by students because of the advantage ST video conferencing provided to synchronous engagement among team members at different locations. In addition, multiple data sources implied that students truly believed that the virtual collaborative experiences would be beneficial to their future careers. They believed there were well prepared to begin using such technologies in practices outside of academe.

It was noted that the AIDE asynchronous technologies (e.g., team discussion board, meeting agenda, calendar, and tasks) were rarely used or not used at all by some students. Responses suggested that the AIDE asynchronous technologies were not perceived as valuable. They did not seem to be necessary given the variety of other more familiar communication and planning resources available outside of course resources. Students also commented that they did not realize that some asynchronous tools were available, or they tended not to spend extra time exploring them when other more familiar tools (e.g., personal e-mail or instant messaging tools) were available. Designing more inclusive and effective instruction to promote use of these tools is suggested, if the faculty advocates or expects student use of such tools and if tools within the learning environment records interactions that the faculty monitor.

The data suggested that students liked and found value in the technologies used during the course. They came to believe that learning how to integrate different types of information and communication technologies into collaborative problem-solving activities was an important part of the preparation for their future career. They believed that engaging with these technologies during the course improved their technology competencies. Students specifically pointed out that the development of technical presentation skills and operational knowledge of Microsoft Excel were particularly beneficial. The tablet PC was also highlighted as one of the most valuable hard technologies in the course. It provided students with flexibility to work effectively on their projects and with their collaborative teams, especially in providing any-time access to each other through AIDE videoconferencing and online meetings capabilities.

Although data suggested that students found technologies in the course beneficial, students did not see value in all of the provided technologies. For example, they were unsure how the team bulletin board could help them be more efficient or effective in completing individual and team tasks. They also felt that the program/application sharing was more useful during DLC lectures, while the interactive whiteboard and AIDE chatroom was more effective in the ST meeting. Student data made it clear that additional incentives, guidance, and instructional activities were needed to help them better understand the value each tool provides to the learning and collaborative environments. They suggested that incentives and better training on the many different technologies may result in more effective and regular use of provided technologies.

Working in a team was perceived by the students as beneficial to their learning. As survey data suggested, students were positive about their teamwork and team collaborations. Students commented that the collaborations with students from off-site locations during class and team work were powerful learning experiences and beneficial in several ways. Teamwork provided opportunities to see how engineering was taught at different institutes, enriching their own knowledge and it provided opportunities to practice their own communication and technical skills in face-to-face and virtual environments. Students enjoyed communicating with teammates at a distance.

However, the data also suggested that students participated at different levels in team activities. On average, survey data showed that CU students had higher level of participation (Mean = 5.56, SD = 1.61) than SU students (Mean = 4.89, SD = 1.47). Although students in general thought that collaborative skills were valuable to both learning and team work, survey data inferred that their perceived effective use of AIDE collaboration tools to support team work was quite different among teams (means per team ranged from 3.8 to 4.96). In addition, the difficulty in connecting socially with distant team members and loose working schedules, lack of team leadership, and lack of team management plans seemed to reduce the effectiveness of some learning experiences. Together these data may suggest that instruction on and experience with collaborative tools as well as additional teambuilding and team management in such courses may help the team become more effective in learning and completing projects.

Students' comments in both survey results and focus-group interview suggested that the engineer knowledge and skills taught in the class were useful and the engineering content was quite deep and advanced. The design project was also evaluated as unique, complicated, and a valuable experience. Students commented, however, that they experienced difficulty when beginning the design project, as they did not feel well prepared in both analytical skills and skills on using ANSYS software, which inhibited their performance in the final design project.

Exit interview data suggested that the separate DST tracks brought a confusing dimension to the team collaboration and inhibited design project progress at times. Students from two tracks had difficulty explaining their newly developed disciplinary knowledge to team members during the collaborative problem-solving process. Some students suggested that DST content could be more in-depth and the learning of DSTs should be more administratively organized. Students also indicated that the time allocated to DSTs and assignments was insufficient. They thought these topics were quite intensive and it was very difficult to digest all learning content within one semester. The data suggested students did not feel confident to start the design project based on the density of content presented within a short time period, were not prepared to apply this new knowledge to a project, and did not feel prepared to teach others about the new content they had just learned.

Overall, the data suggest that this course design provided students in engaging learning activities and tools. It exposed them to new ways of learning and engaging with others in a distributed learning and practice environment. There were, however, problematic areas. These were primarily related to the technologies provided in the AIDE and processes in which the students were to engage in class session and group work.

Table 5 provides a brief summary of identified issues and general recommendations to enhance the design of the course and use of course resources.

Evaluation data suggested that this course provided students with a unique and challenging learning environment in which they appeared to have learned much beyond the course engineering content. The exposure to new ways of participating with faculty and peers (across a distributed learning environment) provided the students with opportunities to practice and develop collaborative design skills within a virtual community. Although several problematic areas were identified in the course, such issues may be resolved with instructional interventions that provide additional practice, demonstrate the value of different tools in a distributed collaborative environment, and advocate the use of tools and development of competencies in such tools.

Implications of these findings specifically relate to technology integration applications for engineering education. More research is required to determine specifically how these technologies and associated learning activities can help improve student learning in engineering and other content domains, scientific problem-solving competencies, and their performance and communication skills in collaborative activities.

The collaborative engineering design course described in this study has been conducted over three years with continuous revisions and modification since its inception. The successes or lessons learned from the evaluation of its third offering has provided some insights into the design of distributed learning environments for collaborative design learning and technology infrastructure recommendations that better support instruction and learning in complex domains such as engineering.

The support of the NASA Langley Research Center through Cooperative Agreement No. NCC-1-01004, the AT&T Foundation, the Microsoft Corporation, the State of New York, Syracuse and Cornell Universities, and the Cornell Theory Center are gratefully acknowledged.