Learners and the Learning Environment: Impact of Technology Use in K-12 Schools in Iowa

Evaluation has always been and continues to be a major component of the Iowa Star Schools Project. Results from evaluation activities conducted in 1998 indicated that technology was being integrated into the curriculum in Iowa’s schools. Teachers and students were using technology, at least at a very basic level. In addition, teachers were beginning to recognize technology as a tool that “allowed them to vary the instructional approach and to provide educational experiences that would be difficult to do otherwise.”

Evaluation in 1999 attempted to take a closer look at the integration of technology and focused on the impact of technology on the learner and the learning environment.

The purpose of this study was to examine the impact of the Iowa Star Schools Project activities on the learner and the learning environment. It is important to recognize that project activities have not been implemented in isolation, but have been carried out in conjunction with other technology initiatives at the local, state, and national level. This limits the interpretation of the data collected and prevents us from making a causal link.

Both models discussed below guided the data collection during the 1998-99 Project evaluation activities. This study, only one piece of the entire evaluation, served as a pilot for future data collection. Changes will be made to the 1999-2000 evaluation plan based on the results of this study. Instrumentation will be revised based on the emerging model supported by the results of 1998-99 evaluation data analysis.

The work of both the North Central Regional Educational Laboratory (NCREL) and the Milken Exchange on Education Technology provided guidance as well as a framework for evaluation and research activities. NCREL’s engaged learning model builds on the work of Barbara Means at SRI International. Means identified variables indicating the occurrence of effective teaching and learning. Researchers at NCREL reviewed other related research to expand this list and developed a framework of 26 indicators of engaged learning (Table 1).

The NCREL engaged learning model provides a starting point to look at the impact of technology on teaching and learning. The Educational Technology Seven Dimensions of Progress Framework developed by Milken Exchange on Education Technology provided a complementary model expanding the guidance for research in this area (Lemke & Coughlin, 1998). Table 2 describes the seven dimensions and the indicators provided for each dimension.

The purpose of this study was to serve as a pilot study to examine the impact of technology integration on learners and the learning environment, the first two dimensions of Milken’s Dimensions of Progress and to provide important formative evaluation data for decision making.

The instrument used in this study was adapted from a questionnaire used in a technology study in Virginia conducted by the Milken Exchange on Education Technology, NCREL, and SRI International (Report to the Commonwealth of Virginia, 1998). The original instrument addressed all seven dimensions for gauging progress in technology in schools. The instrument used in this study focused on only the dimensions of Learners and Learning Environment. The indicators included in these two dimensions and influenced by NCREL’s engaged learning model guided revisions.

The final instrument included four sections summarized in Table 3. As indicated, questions were grouped by the indicators associated with each of the two dimensions under study. The constructs from the learner dimension formed the basis of analysis of this study.

As indicated by Suen and Stevens (1993), the reliability of the subscales needs to be estimated when using subscale scores or constructs as the unit of analysis. Cronbach alpha is designed to estimate the reliability of a composite score from the responses to a number of items.

Cronbach alpha coefficients were determined for each of the five learner constructs. Table 4 shows the Cronbach alpha reliability estimate for each construct. These estimates indicated that the constructs were reliable, with coefficients ranging from .73 to 83. The reliability estimate determined for the overall Learner dimension was .94 indicating the reliability of this section.

Cronbach alpha coefficients were also determined for three of the learning environment constructs. These are presented in Table 5. These estimates indicate that the constructs are reliable, with coefficients ranging from .82 to 90.

Participants in this study were current and past winners of the Showcase on Technology award, part of the Iowa Star Schools Project. This award has been given out annually for the past four years. It recognizes K-12 and higher education teachers who are using technology in innovative ways. A summary of these projects can be found on Iowa’s Star Schools Project Web site (www.iptv.org/iowa_database/).

Showcase project leaders were contacted and provided a list of all past winners. Only K-12 teachers were included in this study. In spring 1999, 71 surveys were sent out and, on first analysis, 43 surveys were returned for a response rate of 61%. However, on closer examination, four responses were found to be unusable (one envelope returned empty, and three unmarked surveys returned) lowering the response rate to 55%. Though this is lower than desired, due to the time of year and the fact that this was a pilot study, it was felt that the response rate was sufficient.

Representatives from all grade levels, pre-K to 12th and ungraded, responded. Respondents were: predominately middle school (44%) and high school (31%) teachers (Table 6); represented all content areas with the highest response from English/Language Arts/ Reading (51%) and Science (46%) (Table 7); and mostly female (67%).

The majority (74%) indicated that they had access to a computer with modem at home; 67% used their computer at home for Internet access once or more a week or daily (Table 8). Internet access from school was higher with 80% indicating the used the Internet daily.

Data collected from Showcase winners were analyzed in four ways. First the frequencies were calculated. Second, the overall mean scores for the constructs allowed some determination of the impact of technology on the learner and teacher, and provided some indications of the overall use of technology in the classroom. Third, analysis of barriers and incentives provided guidance for future training; and last, analysis of open-ended responses provided further information of the impact of technology and indicators of change.

In general, it appeared that teachers feel that technology impacts students on a variety of levels. In assessing weak areas, a level of 25 percent indicating no change, negative change, or not sure was set by the researcher; items where one fourth or more of the teachers responded negatively were identified as areas of concern. For 16 of the 23 items, more than three-quarters of the teachers indicated a positive change. More than 75% indicated technology had impacted students through improvement in the following areas:

• acquisition of new technology skills (100%);

• competency in technology skills (97%);

• number of assignments produced with technology (97%);

• engagement in activities beyond “rote” learning (97%);

• engagement in inquiry-based learning activities (95%);

• use of technology in a variety of educational settings (92%);

• “real life” applications of subject (92%);

• ability to communicate results in a variety of ways (90%);

• breadth of students’ understanding of the subject (87%);

• independence as learners (87%);

• attitude toward subject (87%);

• ability to construct knowledge and solve problems (85%);

• depth of students’ understanding of the subject (82%);

• collaboration with others in school (80%);

• overall interest in school (77%); and

• attentiveness/engagement in class (77%).

There were seven areas where technology had no influence or a negative influence on learners. Areas where more than 25% of respondents indicated no change, negative change or not sure included:

• school attendance on days when technology is scheduled to be used (69%);

• school attendance in general (67%);

• development of work ethic (41%);

• understanding of the “basics” (28%);

• collaboration with others outside school (28%);

• attempts to go beyond minimal assignment (29%); and

• overall academic achievement (26%).

Results of student engagement in the learning environment dimension indicated students were using a limited variety of technologies. Analysis of items related to student use of technology identified only two areas where three-fourths or more of respondents indicated technology being used by students at least 30 minutes a week. These two areas were: use of computers for any educational purpose (95%); and use of word processing software (85%).

Over 50% indicated that students were using the two areas related to research at least 30 minutes a week. These two areas were: research using Internet (64%): and research using CD-ROM (62%).

Technologies where fewer than 25% of the respondents indicated students were using them at least 30 minutes a week included:

• spreadsheet software (10%);

• videodiscs (15%);

• database software (18%); and

• create/maintain Web pages (18%).

Analysis of items related to teacher use of technology identified three items where at least 75% of the respondents indicated they were using the technology a minimum of 30 minutes a week. These items included:

• use of computers for any work-related purpose (87.2%);

• use of word-processing software to create/update course materials (82.1%); and

• use of e-mail to communicate with colleagues inside and outside the school (82.0%).

Results indicated that teachers are not using e-mail to communicate with parents. In addition, there were four other technologies where less than 25% of the respondent indicated that they used the technology a minimum of 30 minutes per week included:

• Research using CD-ROM (23.1%);

• Use Internet to provide information to the community about your classroom/ school (20.6%);

• Create/manage/analyze spreadsheets (15.4%);

• Participate in an interactive video environment (e.g., ICN session) for professional development (15.4%); and

• Use e-mail to communicate with parents (5.2%).

It appears that not only does technology impact learners, it also influences teachers (Table 16). Similar to the analysis of learner dimension items, a level of 25% indicating no/ negative change was set by the researcher to identify items of concern. On seven items, three-fourths or more indicated a positive influence of technology on their teaching behaviors. These areas were:

• share ideas and skills with colleagues (92%);

• overall quality of instruction being delivered (90%);

• personal repertoire of instructional strategies (90%);

• amount of materials and resources being used in classes (90%);

• number of changes being made in the curriculum (85%);

• personal participation in instructional planning at either the department or school level (80%); and

• efficiency and/or effectiveness of classroom management (80%).

There were six areas where teachers indicated there was no/negative change or they were not sure. These areas included:

• relationships with parents and communities (51%);

• relationships with students (38%);

• personal participation in instructional planning at district/AEA or state level (36%);

• sense of empowerment to address school issues (31%);

• collaboration with colleagues to identify goals, make decisions, and solve problems (28%); and

• general morale (26%).

The 5-point Likert scale items related to the Learner dimension were grouped into five constructs and mean scores were calculated. As shown in Table 9, teachers appeared to feel that the integration of technology changed students in a positive way in all five areas. Not unexpectedly, the greatest improvement was in the area of fluency in the use of technology (mean = 4.55).

Means for three areas in the Learning Environment dimension were also calculated. The engagement constructs used the same scale and are shown below. The scale used for engagement constructs was in increments of time where 1 = none, 2 = less than 15 minutes, 3 = 15 to 30 minutes, 4 = 30 to 60 minutes, and 5 = more than 60 minutes. Teachers appeared to be using the technology more than students. Both groups were using technologies on average between 15 and 30 minutes a week.

Teacher change used a different scale: from 1 (much declined) to 5 (much improved). The construct mean for teacher change was 4.19. This indicated that teachers feel that their personal use of technology somewhat improved what they do and how they do things in the classroom, in communication and in curriculum preparation.

Respondents were asked to identify the ways they were encouraged to use technology. A full summary of responses is shown in Table 11. The most common incentives were use of school technology in evenings/weekends/ school holidays/or summer months (77%); and technology resources for media center and/or classroom (74%). Few teachers indicated that they had received salary incentives for the effective use of technology (3%).

Respondents were also asked to rate barriers to the increased use of educational technologies (Table 12). More than 50% indicated that time for training (69%) and lack of time for planning and implementing (64%) were significant barriers. Student’s current level of technology skills and community support did not appear to be significant barriers.

The constant-comparative method was used to examine the responses to the open-ended questions. Responses to noticeable change in academic achievement were categorized into six areas. These categories in order of frequency were fluency, motivation, learning environment, higher order thinking skills, relevancy, and basic skills. Fluency in the use of technology was the most frequent response. An improvement in basic skills was mentioned the least. Several indicated that no measurable change in achievement was apparent. Sample responses for each of the categories are presented below:

• “Student level of comfort using technology has risen tremendously.”

• “More computers and awareness of how to use by students and teachers.”

• “Students can use word processor at an earlier age.”

• “Students are able to use technology comfortably.”

• “Skills toward using the technology have improved.”

• “Student’s interest in and understanding of topics.”

• “Student’s interest in research projects.”

• “Student’s interest in subject matter and motivation to learn new skills.”

• “I think having access to so much information has made students and teachers much more eager to learn and explore. This can make for better learners.”

• “Our research resources have really mushroomed. Now, when students have a question they go to the computer and find out.”

• “Students are using a wider variety of sources, more likely to look up information, more likely to revise information and presentations.”

• “We now have a computer in every classroom.”

• “Increased access to current information.”

• “We have expanded our curriculum and course offerings.”

• “Critical thinking skills have improved.”

• “Higher order thinking skills have been further developed and enhanced.”

• “The use of technology has made students better independent achievers.”

• “More use of real world situations.”

• “Greater awareness of the world outside our rural community.”

• “My upper-level science classes have shown a clearer understanding of concepts through the use of technology.”

• “Some improvement in math and reading.”

While a few respondents indicated a rise in assessment or test scores, most identified other sources of evidence of academic change or improvement. Repeatedly, teachers identified an improved quality of projects and presentations as evidence of academic achievement.

They also indicated that communication skills had improved based on their observations of students in the classroom. As one respondent noted,

Results from this study indicated that technology integration does have an impact on both teachers and students. The learning environment is adapting to allow for greater access to the wealth of information that is currently available and the even greater explosion of information in the future. This requires changes in both the role of the teacher and the student.

No longer are teachers the only source of information in the classroom. Because of this, students need to become adept in accessing and, perhaps more importantly, evaluating information from a variety of sources. They increasingly need to have skills in communication with technology. Already, they are sharing their work via e-mail, PowerPoint presentations and Web pages. These skills closely mirror those needed in the business arena.

Teachers are slowly recognizing that their role in the classroom is changing from leader to facilitator. This may be a difficult shift to make and the support needed for this change to occur is not always available. Teachers indicated that student achievement is greater when using technology in the curriculum. However, they caution that this achievement is not measured by the usual standardized test. This change is more frequently shown in the quality of projects and presentations and in the use of higher order thinking skills. It would seem that these are two areas where we would want students to excel.

The results of this study clearly identify the need for different measures. The types of changes that occur in student learning are not reflected in the numeric scores that have been accepted in the past. Alternative ways of student assessment need to be developed and those already available need to be accepted as indicative of achievement.

In addition to new measures, there is a need for innovative/quality models that take technology use beyond simply doing what “we have always done” in the classroom. We need to move into the realm of learning with technology and allowing the technology to become tools to help us manage, organize, manipulate, hypothesize, analyze, and synthesize information. In this way, technology moves from merely making us more efficient to allowing us to think and process in ways not possible before.