Strengthening Academic Vocabulary With Word Generation Helps Sixth-Grade Students Improve Reading Comprehension Free

A prominent yet veiled problem among students in upper elementary, middle school, and high schools is that while they can read (i.e., they can decode words accurately and fluently), many do not understand what they read. It is well established in reading theory (e.g., Anderson & Pearson, 1984; Cain, 2010; RAND Reading Study Group, 2002) as well as research (e.g., Cain & Oakhill, 2007; National Institute of Child Health and Human Development, 2000; Snow, Burns, & Griffin, 1998) that by the time normally developing readers complete third grade, they are expected to be able to read with a reasonable degree of fluency. However, the problem is that a substantial number of these students have difficulties reading with comprehension, notwithstanding their ability to decode words (e.g., Biancarosa & Snow, 2006; RAND Reading Study Group, 2002).

The reading achievement of eighth-grade students in the United States has improved in the last 2 years. The results of the 2013 National Assessment of Educational Progress (NAEP), known as the nation’s report card, show that eighth graders’ average score in reading rose three points on NAEP’s 500-point scale since 2011, the last time the test was given. Although proficiency levels have increased from 2011 to 2013, in Grade 8, only 36% were found to be proficient in reading (National Center for Education Statistics, 2013).

However, the performance rates among students varying by race, ethnicity, gender, and income level remained persistently flat, with a lack of progress closing racial and ethnic disparities in the test results. Hispanic and African American students improved their scores, but gaps between their performance and those of their White and Asian peers have not improved since 2011. Disparities between the achievement of girls and boys persisted as well. In 2013, 42% of girls were reading at or above the proficient level in eighth grade, while only 31% of boys were doing so (National Center for Education Statistics, 2013).

Literacy researchers, policymakers, and practitioners know that there are various sources of difficulty that prevent students from comprehending what they read. These sources of difficulty may include one or more of the following: a lack of awareness of how certain aspects of academic language work, such as how words and sentences are formed (e.g., Carlisle, 1995; Demont & Gombert, 1996; Scott, 2004); a lack of understanding of how information is organized in different types of texts, such as narrative and informational texts (e.g., Armbruster, Anderson, & Ostertag, 1989; Meyer, 2003); a lack of awareness and use of metacognitive reading strategies, such as having a purpose for reading, knowing when and how to use certain strategies when reading comprehension fails, and evaluating one’s understanding (e.g., Baker & Brown, 1984; Flavell, 1979; Mokhtari & Reichard, 2002; Pressley, 2000; Pressley & Afflerbach, 1995); a lack of motivation to read (e.g., Guthrie, Wigfield, Metsala, & Cox, 1999; Taboada, Tonks, Wigfield, & Guthrie, 2009); a lack of vocabulary knowledge (e.g., Graves, 2006; Nagy & Scott, 2000); and weak prior knowledge and experiences (e.g., Kendeou, & van den Broek, 2007; Kominsky & Kominsky, 2001; Ozuru, Dempsey, & McNamara, 2009).

It is worth noting that a fairly consistent finding in reading research over the past seven decades is that vocabulary knowledge contributes significantly to students’ reading comprehension performance (e.g., Baumann & Kame‘enui, 2004; Davis, 1942, 1944; Ruddell & Unrau, 1994; Whipple, 1925; Yildirim, Yildiz, & Ates, 2011). This research has shown that vocabulary knowledge and reading comprehension are correlated, making vocabulary a leading predictor of reading comprehension among children and young adults. Results from the 2009 and 2011 NAEP reports indicate that students who scored higher on NAEP vocabulary questions also scored higher in reading comprehension (National Center for Education Statistics, 2012). The role of academic vocabulary in the teaching of reading is undeniable, particularly when considering its close association with reading comprehension performance for all students. Because of its revitalized importance, vocabulary has recently taken center stage in state and national standards for the English language arts (National Governors Association and the Council of Chief State School Officers, 2010).

When teaching students to develop adequate word decoding and comprehension skills, there is good news and bad news. The good news is that as a literacy community, we know what works to increase students’ levels of vocabulary knowledge that will help support their reading comprehension ability, which will in turn improve their learning and academic achievement outcomes. For instance, research indicates that students’ vocabulary knowledge determines how well they understand texts (e.g., Baumann, 2005; Stahl, 2003). We also know that effective vocabulary instruction incorporates direct (or explicit) and indirect (or incidental) approaches, which are outlined in a rich set of research-based and practice-based materials and resources (e.g., Baker, Simmons, & Kame‘enui, 1998; Baumann & Kame‘enui, 2004; Beck, McKeown, & Kucan, 2002; Biemiller, 2005; Graves, 2006; Lehr, Osborn, & Hiebert, 2004; Marzano & Pickering, 2005; Nagy, 2005; Snow, Lawrence, & White, 2009; Stahl & Kapinus, 2001). These resources, and many others, are very helpful in designing and implementing strong instructional programs aimed at helping students develop deep levels of word and language knowledge, which are critically important for reading and text understanding. The bad news is that as a literacy community, we do not always do what works when teaching children to read and do so with comprehension.

Although there are different approaches, methods, and programs that are designed to help advance students’ vocabulary knowledge and skills, for purposes of this study, we focus on Word Generation (http://wg.serpmedia.org), a research-based program that integrates the teaching of academic vocabulary, which consists of important words students often encounter when reading to learn across the disciplines.

Researchers and educators affiliated with the Strategic Education Research Partnership at Harvard University developed Word Generation (Snow et al., 2009). This innovative academic vocabulary curriculum was developed under the direction of Professor Catherine Snow, expert in language and literacy development within the Harvard Graduate School of Education. The curriculum includes weekly units about interesting and engaging topics with brief lessons for middle school teachers of all academic subjects.

The basic premise behind the Word Generation program is that teachers across disciplines such as language arts, science, mathematics, and social studies come to a consensus of the vocabulary words they believe are most important for their students to learn. Teachers then include a daily 15 to 20 minute instruction unit in language and vocabulary in their classrooms by connecting and using words learned across their discipline-specific texts.

It is important to note that the Word Generation program was designed to supplement, not to supplant, existing content curricula. The program introduces students to a set of academic vocabulary words by embedding them in brief texts about controversial issues of interest to many adolescents, such as steroid use among athletes, legalization of euthanasia, and censorship of libraries and popular music. The curriculum then provides opportunities for students to use the new words in classroom discussion, debate, and writing. Beyond teaching vocabulary, the program is designed to support students’ oral language, argumentation, and writing skills, while increasing their prior knowledge about various issues and questions of interest.

As a supplemental curriculum, Word Generation has been implemented effectively in various schools in the United States, including Boston public schools, with a great deal of success (Snow & Lawrence, 2011). Using the Massachusetts Comprehensive Assessment System - English Language Arts (MCAS- ELA) and Group Reading and Diagnostic Evaluation (GRADE) as pre- and posttest reading comprehension assessments, researchers targeted increased vocabulary treatment in the areas of science, math, and social studies. Study results indicated only a midrange vocabulary improvement for English-only students, but a greater vocabulary increase in languageminority students (Lawrence, 2012; Lawrence, Capostosto, Branum-Martin, White, & Snow, 2012; Lesaux, Kieffer, Faller, & Kelley, 2010). If literacy achievement outcomes are increased via vocabulary, it makes sense that all students, including English learners, would stand to benefit from an educational program devoted to increasing academic vocabulary, what Stanovich (1986) refers to as one of the most “significant independent” contributors to literacy (p. 362).

Results of various studies that have integrated the teaching of academic vocabulary instruction across the disciplines such as Word Generation have been shown to increase students’ vocabulary knowledge more than traditional approaches to teaching vocabulary (Hwang, Lawrence, Mo, & Snow, 2014). Furthermore, for English learners, this increase in vocabulary knowledge seems to vary depending on these students’ English proficiency levels (Hwang et al., 2014). These results are significant to our understanding of the role of academic vocabulary in reading comprehension, and the need to provide students with instructional opportunities that help them strengthen their language development by increasing their levels of word knowledge.

However, while the Word Generation program has been shown to be quite effective with middle grade native and nonnative English students, its developers have noted challenges to the success of the program, with one of the greatest challenges relating to the complexities involved in implementing the program within real-world classroom and school settings. Some of these challenges often pertain to teacher and student participation, attendance, and scheduling.

In light of the successes and challenges of using the Word Generation program in regular classroom settings, we wanted to find out whether this program would be equally effective when adapted for small group instruction in an after school setting. Specifically, we sought to examine whether a school-identified group of struggling sixth-grade readers who receive small-group supplemental instruction using the Word Generation curriculum show significant improvements in vocabulary knowledge and reading comprehension performance than a comparable group of struggling sixth-grade peers who did not participate in the intervention.

A number of schools in the United States have integrated academic vocabulary instruction using curricula such as Word Generation. A common reason for doing so among many schools, like the middle school that is a part of our study, is that they all have students who face important challenges with reading comprehension. One key issue shared among these schools is that students have limited knowledge of academic vocabulary. In fact, many of the students selected to participate in our study had very limited vocabularies as indicated by their performance on baseline tests used at the school.

For purposes of this study, we partnered up with members of the school administration in developing a shared vision and an action plan aimed at significantly improving students’ academic vocabulary and overall academic achievement outcomes. We integrated Word Generation curriculum as a supplement to an existing after-school program in which most middle school students voluntarily participate. Our main goal was to determine whether the implementation of the Word Generation program enhances students’ academic vocabulary knowledge, and whether that, in turn, mediates improvements in their reading comprehension and eventually academic achievement in subject areas such as science, mathematics, and social studies.

The study took place in one middle school located in a socioeconomically and ethnically diverse community (population: 105,000) in the Southwestern United States. The school has an enrollment of approximately 500 students in Grades 6 through 8 with a 25:1 student to teacher ratio. The percentage of students eligible for a free or reduced price lunch is approximately 93%. School ethnicity records indicate an overall enrollment of 3% White, 18% Black, and 79% Hispanic students. Approximately 39% of the students are designated as limited English proficient.

Study participants consisted of 72 sixthgrade students who were identified by the school leadership team from a total student population of approximately 200. Half of the study participants (n = 36) served as the intervention group and received small-group supplemental instruction using the Word Generation curriculum. These students were identified for participation in the study based on two key factors: teacher recommendations for reading assistance and underperformance (i.e., performing two or more grades below actual grade level) on the State of Texas Assessments of Academic Readiness (STAAR) test (Texas Education Agency, 2015), a state-mandated reading achievement test that all students take at the end of their fifth and sixth grades.

The other half of the students (n = 36) served as the comparison group and did not participate in the intervention. Instead, they participated in self-selected after-school instructional programs focused on developing students’ social skills such as teamwork and collaboration. These students were randomly selected from the remaining 160 sixth-grade students. While the two groups of students were matched with regard to demographic characteristics such as gender, ethnicity, native language, and socioeconomic status, the comparison group consisted of students who were not necessarily selected because of poor performance on reading tests. Unlike students in the intervention group, comparison students had variable reading abilities ranging from one or two levels below grade, at grade, or above grade level performance on the STAAR test.

Trained paraprofessionals served as small- group instruction tutors. Our approach to preparing effective tutors for small-group guided reading instruction was directed by the ecological context of our at-risk middle grade readers. Our tutor training consisted of the following three integrated components:

1. An initial 5 hours of training (2 and a half hours per day for 2 days). Training during the first day consisted of an overview of the Word Generation program, a review and discussion of its curriculum components, and a discussion of its academic vocabulary assessments. The second day focused on the implementation of an instructional framework and a set of reading strategies (see  Appendix 1) aimed at helping tutors organize instruction. We used this framework to monitor the degree to which the program was implemented as intended. This session also included an overview of the data to be collected, including pre- and postacademic vocabulary tests, student vocabulary self-assessments, and pre and posttests using the GRADE test.

2. A weekly, 1-hour debriefing session with tutors, which gradually changed to biweekly sessions after the first 6 weeks of instruction. During these debriefing sessions, members of the research team met with tutors to review progress made, discuss and resolve issues and problems emerging from small group instruction sessions, and provide assistance with issues pertaining to data collection, as well as instruction and assessment of student learning.

3. Individual tutor consultation with members of the research team who served as mentors for tutors. During individual consultation sessions, members of the team provided assistance, guidance, and moral support in preparing weekly lessons, addressing specific issues or problems surfacing in small group instruction, and helping ensure that instruction was implemented as intended. Individual consultation was initiated either by the mentor or by the tutor depending on perceived needs. Tutors kept a reflective journal, which served as a source for identifying challenges, questions, or issues for discussion.

We used an adapted version of the Word Generation program for purposes of this study. Word Generation (Snow et al., 2009) is a research-based vocabulary program for middle school students designed to teach academic vocabulary across the disciplines of language arts, math, science, and social studies. As indicated above, the Word Generation curriculum focuses on general academic vocabulary words that students are likely to encounter when reading academic texts across disciplines. A particular emphasis of the program is to introduce these words and their multiple meanings across these disciplines. The target words are embedded in relatively short passages addressing controversial topics, issues or questions of interest to adolescents. These topics are designed to encourage students to use newly learned words in discussion, debate, and writing.

The Word Generation curriculum is made up of brief, 1-week study units designed to help students develop academic vocabulary and reading comprehension skills. The units consist of a series of topics of interest to students and lend themselves to discussion and debate. The units include 15-20 minute daily activities organized in a 5-day cycle, giving the teachers of language arts, mathematics, science, and social studies unique opportunities for collaboration and teamwork with the shared goal of helping strengthen students’ academic language skills. For example, on Monday the language arts teacher introduces the five words to students so that they get a general sense of the meaning of the new words. On Tuesday, the mathematics teacher discusses how these words are used in the context of solving math problems. On Wednesday, the science teacher uses the words within the context of science experiments. On Thursday students use these words to debate issues in social studies. Finally, students have an opportunity to engage in writing activities using the words learned throughout the week.

For purposes of our study, we adapted the Word Generation program for after-school instruction in five ways so as to accommodate the after-school instructional setting and the student participants. First, we adapted the curriculum for use in small group settings rather than regular classroom settings. Wheelan (2009) discovered that groups working with three to eight peers perform better than groups of nine or more. Therefore, participating students were randomly assigned to one of six tutoring groups of six to enhance the learning environment and minimize negative effects of short-term group heterogeneity created via self-selection (Cooper, 1990; Wheelan, 2009).

Second, we recruited and trained college students who served as small-group guided instruction tutors. These paraprofessional tutors received training in understanding the Word Generation curriculum and its components, designing a lesson framework (see below) that helped organize instruction, and managing instruction in small group settings. We also provided close supervision of the small group instruction sessions to help ensure the Word Generation curriculum was implemented as intended. A key to success in tutorial programs such as this one, according to Allington (2011), appears to be related to “providing noncertified personnel with strong training, structured tutorials, and ongoing supervision” (p. 181).

Third, we extended the 15-minute per day time-allocation to 30 minutes. Due to afterschool time schedule constraints, we missed several sessions throughout the duration of the program, which only allowed us to teach 17 of the 24 units in the first two series of the Word Generation curriculum. However, with our extended daily tutoring session, we were able to teach a total of 85 of the 120 words available in the first two series, resulting in a total of approximately 34 hours of instruction. We extended the time allocation to 30 minutes per day rather than the recommended 15-20 minutes since our intervention students were markedly below grade level and needed extra time to develop and/or expand their academic language and vocabulary knowledge.

Fourth, we incorporated a student vocabulary self-assessment, which we used at the beginning and at the end of each week of instruction (see  Appendix 2), as well as an instructional framework, combining insights from Word Generation and Marzano’s six steps to vocabulary development (Marzano & Pickering, 2005). The instructional framework helped tutors, most of whom were not experienced teachers, implement the Word Generation daily lessons. The framework was also helpful for us as it enabled us to monitor and supervise the tutoring sessions and determine the extent to which lessons were implemented as intended.

Finally, we implemented the program in 4 (Monday-Thursday) rather than 5 days per week. We did so because school-sponsored activities were only scheduled during the first four days of the week within the after-school program at the school. In lieu of the fifth day involving writing, we incorporated writing activities into the last 5 to 10 minutes of each session across the 4 days of the week. Table 2 displays an outline of what our adapted Word Generation weekly lesson framework looked like during implementation.

It is worth noting that we designed our lesson framework to be flexible to accommodate teaching styles as well as student needs. However, throughout the week tutors collaborated in ways that engaged students in activities, which enabled them to learn new vocabulary words and to demonstrate that knowledge in various contexts.

For purposes of this study, we collected two types of data which enabled us to determine whether the implementation of the supplemental Word Generation curriculum influenced students’ vocabulary knowledge, and ultimately their overall reading achievement outcomes. These data included a measure of academic vocabulary knowledge using the Word Generation tests, developed by members of the Strategic Education Research Program (Snow et al., 2009), and an assessment of reading achievement using a norm-referenced test, the Group Reading Assessment and Diagnostic Evaluation [GRADE] (Williams, 2001).

Vocabulary Test (Snow et al., 2009). We assessed students’ academic vocabulary knowledge using the Word Generation vocabulary test, a 50-item multiple-choice test that was developed by the Strategic Education Research Program team. This pre and posttest, which is a part of the Word Generation curriculum, provides baseline data about students’ initial academic vocabulary knowledge. We used the same version of the Word Generation test to assess students’ vocabulary knowledge before the intervention (October, 2013) and after the completion of the vocabulary instruction program (April, 2014). We used scores on the posttest as an outcome variable to determine growth in vocabulary knowledge following the intervention. The pre- and postvocabulary tests have excellent psychometric properties (Cronbach’s α = .876, see Snow et al., 2009), and are available free of charge to teachers and other interested individuals through the Word Generation website http://wg.serpmedia.org.

Group Reading Assessment and Diagnostic Evaluation[GRADE] (Williams, 2001). The GRADE test is a standardized, groupadministered test of overall reading ability. It is organized by grade levels and divided into subtests reflecting the literacy skills at each grade level. The subtests that make up the total test score for the sixth- to 12th-grade levels of the GRADE test include sentence comprehension, passage comprehension, and vocabulary. The sentence comprehension subtest is used to determine whether students can construct the meaning of sentences as a complete idea unit. Students are presented sentences with missing words and are expected to determine the single word that is missing from a list of four or five choices. The passage comprehension subtest measures students’ ability to use metacognitive comprehension strategies while reading, questioning, clarifying, summarizing, and predicting. Students are presented with short, medium and long passages followed by multiple-choice questions. The vocabulary subtest assesses students’ knowledge of words appropriate to their respective grade levels. Students are presented with test items containing a phrase or short sentence with one boldfaced word. They are expected to determine the correct meaning of the boldfaced word from a list of four possible answers.

The GRADE test is intended to be a test of ability rather than speed, and thus it is not timed, but the assessment is typically completed in less than 90 minutes. The test publisher, American Guidance Service, reports good psychometric properties for the test (Williams, 2001). The ninth grade reported reliabilities ranged from .83 for sentence comprehension to .96 for the total test, indicating a high degree of homogeneity among test items within the grade. American Guidance Service reports high concurrent validity of the total test with other established instruments, such as the Gates-MacGinitie Reading Test in a sample of ninth-12th-grade students.

We administered all assessment measures at the beginning and at the end of the afterschool program in an attempt to examine the impact of the program on students’ vocabulary knowledge and reading achievement outcomes. We analyzed the data obtained using ANCOVA with group as an independent variable, scores on the vocabulary and the GRADE posttests as dependent variables, and students’ performance on the GRADE pretest as a covariate.

In this study, we sought to examine the impact of the Word Generation program activities on improving students’ academic vocabulary knowledge and reading achievement outcomes. Our research focused on whether a school-identified group of struggling sixthgrade readers who received small-group supplemental instruction using the Word Generation curriculum showed significant improvements in vocabulary knowledge and reading comprehension performance when compared to a group of struggling sixth-grade peers who did not participate in the intervention. To answer this question, we present our findings in three ways. First, we tested whether intervention students’ vocabulary knowledge improved significantly, keeping in mind the dose of vocabulary instruction they received throughout the intervention. Second, we tested whether the average change in the reading achievement outcome (Total GRADE test score) from pre- to posttest differed between the intervention and comparison students. Finally, we tested whether the reading achievement posttest mean scores (Total GRADE test scores), adjusted for pretest scores, differed between groups. The results are presented in Tables 3 and 4 and illustrated in Figure 1.

Preliminary analyses using nationally normed standardized measures of vocabulary knowledge and reading comprehension performance indicate that there was an overall positive effect of the Word Generation program taught in a small group instructional setting on the reading achievement outcomes of the intervention students. Preliminary analyses revealed three findings.

First, the academic vocabulary knowledge of our intervention students improved significantly as a result of completing 34 hours of vocabulary instruction when measured by the Word Generation vocabulary test (t[29] = 5.51, p <.001). Note that the average amount of time devoted to small group academic vocabulary instruction our intervention students received falls below the recommended 44-80 hour range of instruction needed to substantially reduce the incidence of reading failure in a school system by accelerating at-risk students’ reading proficiency to average levels of performance (Allington, 2011; Clay, 2005).

Second, our analyses show that students who received small group instruction in academic vocabulary read more proficiently at the end of the intervention than they did at the beginning (F[1, 57] = 84.95, p = .001) when measured by the average change in the desired reading achievement outcome (Total GRADE test score) from pre- to posttests (see Tables 4 and 5). We believe that this positive effect was mediated by improvements in students’ academic vocabulary knowledge. We further believe that students’ expanded vocabulary knowledge was mediated, in turn, through multiple exposures to words across the disciplines, deep reading of texts, active discussion of interesting topics, and writing and reflecting about interesting topical issues (Lawrence, Crosson, Pare-Blagoev, & Snow, 2015).

We reviewed pre- and posttest data so as to estimate the effects of vocabulary instruction on advancing intervention students’ overall reading achievement outcomes. First, in reviewing the proportion of intervention students who made sufficient progress in reading proficiency, we found that 90% of the intervention students made sufficient progress in reading proficiency after 34 hours of small group vocabulary instruction. These data show that nine out of 10 of these students achieved higher extended scale scores on the GRADE test in April 2014 when compared to their performance on the same test in October 2013. These data are consistent with literacy tutoring research suggesting that when taught by well- trained tutors, the average at-risk reader should be expected to read more proficiently than approximately 75% of the untutored students in the control group (Institute of Education Sciences, 2003).

Third, our findings show that students who received small group Word Generation instruction in academic vocabulary received nearly identical extended scale scores (adjusted means 94.40 versus 94.02) on the GRADE test total scores at the end of the intervention when compared to students in the comparison group after adjusting for initial mean score differences (see Table 5). Using a test of Between-Subject effects, we found no significant differences between the intervention and comparison groups (F[1, 57] = .032, p = .858), and no significant interactions between intervention time and group (F[1, 58] = .043, p =.836). As Table 5 shows, the adjusted posttest means for the intervention students (M = 94.4) are nearly identical to the means of students in the comparison group (M = 94.02). Following Field (2009), we used these means rather than the original means to more accurately interpret the group differences reflected in our ANCOVA analysis. These results indicate that systematic vocabulary instruction in cross-disciplinary vocabulary instruction can help close the reading achievement gap among skilled and less skilled sixthgrade readers (see Figure 1).

To compare whether students in the intervention and comparison groups made appreciable gains in reading achievement outcomes relative to their grade level (sixth grade), we reviewed their National Curve Equivalents (NCEs), which are normalized standard scores with a mean of 50 and a standard deviation of 21.06. NCEs measure progress in reading by describing a student’s position within the norming group at successive times during the year or grade levels. As a general rule of thumb, assessment experts agree that a student who maintains about the same NCE from fall to spring or earns a total score on a test level of less than seven NCEs has not changed relative to the achievement of students in the norming group. A student with an NCE score of 50 is considered at grade level. Table 4 displays the average NCE scores of students in our intervention and comparison groups. An examination of these data indicates that intervention students made gains of 11 NCEs between October 2013 and April 2014 when compared to a gain of 6.15 NCEs for students in the comparison group. Normally developing readers in similar grades should gain roughly 18 NCE points from fall to spring. While the NCE growth scores of our intervention students were lower than normally developing peers, they were higher than students in the comparison group, indicating that the small group vocabulary instruction significantly increased their vocabulary knowledge, which has in turn helped improve their overall reading achievement outcomes.

The results of our quasi-experimental study are encouraging, especially considering the complexities and challenges we encountered in planning for, implementing, and evaluating the effectiveness of our adapted version of the Word Generation vocabulary program on the reading achievement of our intervention students in an after-school extended-time setting. Our experience using the program is consistent with prior research documenting the effectiveness of the Word Generation program on middle grade students’ reading achievement outcomes (e.g., Lawrence et al., 2012, 2015; Moje & Tysvaer, 2010; Townsend & Collins, 2009). Indeed, the program provided valuable vocabulary learning experiences for our students.

The Word Generation program did help our intervention students increase their academic vocabulary knowledge after completing 34 hours of vocabulary instruction when measured by the Word Generation vocabulary test. We were encouraged by the modest yet significant gains in vocabulary knowledge (an average of 7.36 words out of 50 tested). Had we taught all 24 units (five words per unit), this number translates to a ratio of about 14 words of the 100 words taught. Note as we indicated earlier, the average amount of time devoted to small-group vocabulary instruction our intervention students received (34 hours) falls slightly below the recommended 40 to 80-hour range of instruction needed to substantially reduce the incidence of reading failure in a school system by accelerating at-risk students’ reading proficiency to average levels of performance (Allington, 2011; Allington & Gabriel, 2012; Clay, 2005).

Our results also show that intervention students read more proficiently at the end of the intervention than they did at the beginning when measured by the average change in the desired reading achievement outcome from pre to posttests. Our hypothesis was that students’ gains in vocabulary knowledge helped boost their reading achievement gains as well. This hypothesis is supported by a positive and significant correlation between students’ scores on the Word Generation posttest and their performance on the extended scale scores of the GRADE test (r = .600, p <.001). The extended scale score gains suggest that our intervention students had an estimated 6-month increase in their reading scores relative to comparison students. Our interpretation of this association is that while vocabulary contributes to reading achievement, it is difficult to determine the precise nature of this relationship. We do not suggest that increasing vocabulary alone translates into (or causes changes in) improvements in reading achievement. However, we believe that a suite of research-based components (e.g., teaching vocabulary through multiple exposures, deep reading of texts, active discussion of interesting topics, and writing and reflecting about topical issues) helped students perform much better on reading achievement test items.

Finally, our findings show that intervention students received nearly identical posttest extended scale scores on the GRADE test at the end of the intervention when compared to students in the comparison group, after adjusting for initial mean differences. Intervention students’ rate of reading achievement growth resulted in decreasing the difference in the achievement gap that existed at the start of the study between intervention and comparison students. These findings are also encouraging in light of the fact that students in the comparison group performed significantly better than intervention students at the start of the study as shown by mean disparities in performance on the pretest. These findings indicate that even a modest dose of instruction (34 hours) in academic vocabulary instruction can help narrow the reading achievement gap among skilled and less skilled sixth-grade readers. These findings also underscore the educational benefits of integrating supplemental academic vocabulary instruction, such as that provided through Word Generation, on the reading and academic achievement outcomes of striving middle school readers.

We encountered three major challenges during the planning, implementation, and evaluation of the study, which have implications for the validity of its findings. Snow and Lawrence (2011) who have had extensive experience implementing Word Generation across schools in the United States and internationally, point out there are unavoidable challenges that arise when employing a literacy training program into operating schools, and our execution of an adapted Word Generation program within a local school was no exception. Our challenges pertained primarily to adapting the Word Generation program to fit the needs of our target students in an afterschool program, implementing the program with a desired degree of fidelity given the diversity of backgrounds and experiences of our part-time small group tutors, and implementing the program in an after-school setting that was not conducive to student learning and engagement.

Our first challenge pertains to adapting the Word Generation vocabulary instruction program to fit the needs of our school-selected intervention and comparison students who were enrolled in an after-school program that was already in place. This meant that we had to modify the program structure so as to implement the curriculum within a 4-day rather than a 5-day schedule as originally designed. In addition, because the program was originally designed for normally developing middle school readers, we added more time for teaching the target words from 15 to 30 minutes per day for our intervention students, 60% of whom were two to three grade levels behind in reading. Adding more time proved to be helpful in that it enabled tutors to engage all students in active and productive discussions of issues and questions using the target words learned during the week. Selection of students for participation in the program must be taken into account when interpreting the study’s findings. As we indicated above, comparison students would not be considered a pure control group since, unlike our intervention students who were selected because they were two or three grade levels behind in reading, represented a mix of poor, average, and good readers, and as a group their performance on pre tests was significantly higher than that of the intervention students.

Our second challenge pertains to implementing the Word Generation program with a desired degree of fidelity given the diversity of backgrounds and experiences of our part-time small group tutors. Although we provided training in understanding and using Word Generation, supervised tutor-implementation throughout the program, and provided guidance via biweekly team meetings, there was variability in terms of how well individual tutors implemented the program as indicated by fidelity of implementation data and informal observations of small group instruction. In addition, due to scheduling constraints, we replaced two of the tutors half way through the program. Tutor effectiveness and fidelity of program implementation have important implications for the validity of the data presented in this study.

Our third challenge relates to implementing the program in an after-school setting that was less than optimally conducive to student learning and engagement. During the first 2 weeks of the program, we had some difficulties getting students to participate in program sessions at the right times and the right locations. Some of the students were initially quite reluctant to participate in the program and viewed the program as a burden, especially when some of their peers were participating in other activities. In addition, while all program sessions were scheduled to begin and end at the same time, our small group sessions were scheduled in different locations within the school. Three groups of six students and their tutors met in the school library, which proved to be ideal for small group instruction; two groups met in the school cafeteria, which was open and rather noisy; and one group met in the computer lab that was occasionally scheduled for other school activities. Furthermore, because students were occasionally called upon to perform school-related tasks such as test makeups or participation in other activities, we missed sessions, which ultimately resulted in teaching only 17 of the 24 units of the program. This means that tutors only taught 85 words of the 120 words covered in the first two series of the program. Again this is a limitation that needs to be taken into consideration when interpreting the results of the study.

However, despite the above challenges, our quasi-experiment generally shows that it is possible to significantly advance the reading achievement outcomes of struggling middle school students when we provide them with supplemental cross-disciplinary vocabulary instruction using Word Generation. We attribute the positive results observed in our study to three important factors: The use of an effective research-based program such as Word Generation, implementation of the program with as much fidelity as possible, and close monitoring and supervision of program activities.

While Word Generation helped our students increase their academic vocabulary knowledge and reading comprehension scores, it is unclear how this knowledge moderates the improvement in their reading comprehension performance. Improvement in reading comprehension may be due to a combination of factors including, but not limited to: exposure to and use of academic words across the disciplines, which helps expand understanding of word meanings; using these words in discussions and debates, which helps students develop background knowledge of specific topics and subjects; and using these words in reading and writing assignments, thus helping them refine oral and written communication skills.

Despite positive results, an achievement gap remains when comparing the reading proficiency of our intervention students to that of their normally developing sixth-grade peers. This is not too surprising, since over 60% of our students entered sixth grade with a significantly larger gap in literacy achievement than did normally developing peers. Closing this reading achievement gap will take additional instructional time in the form of one-on-one and/or small group instruction. Additional instructional time will help accelerate to average levels of performance the progress of these students, especially for those who show early signs of reading difficulty. Some of the students in the intervention group were reading at two to four grade levels below their actual grade level when we started working with them. They were identified for participation in this after-school intervention because they were underachieving in their reading as well as in other academic subjects such as science, mathematics, and social studies. We suspect that additional supplemental instruction in academic vocabulary may help raise their vocabulary knowledge, and ultimately their overall reading, writing, and learning achievement outcomes.

In an effort to help maintain the progress students make during the school year, they should be provided opportunities to read and write during the summer months. Research has shown that students in primary, elementary, and middle grades regress in their reading ability by as much as three to four months when they do not read during the summer when school is not in session. This reading loss has been shown to affect students’ reading performance when they return to school in the fall. Research has shown that students from lower socioeconomic backgrounds suffer greater summer reading loss than do students from upper socioeconomic levels (Allington & McGill-Franzen, 2003; Kim, 2004; Kim & White, 2008; Lawrence, 2009). Therefore, the likelihood of summer reading loss is more real for students who are poor and who have poorly developed language and literacy skills.

The Anderson-Vukelja Wright Endowment Fund provided support for the design and implementation of this project.

Please take a few minutes following each lesson to share insights regarding its implementation.

Tutor: _______ Start Time: _____ P.M.

Tutees: _______ End Time: _____ P.M.

Open-Ended Comments:

1. Describe aspects of this lesson that worked particularly well.

2. Describe aspects of this lesson that did not work well.

3. Describe how you plan to address the aspects of the lesson that did not work as intended.

Student Name: _____ Tutor: ____ Date: _____

Unit # & Title: (e.g., Unit 1.07: Censorship: Who should decide what young people read?)

Directions for Students: Write in the five new words for the week and rate your level of knowledge using the scale below.

Beginning of the Week (Monday)

End of the Week (Thursday)