Using the annotating strategy to improve students' academic achievement in social studies Open Access

The idea that “Every teacher is a reading teacher” is commonly accepted in the education world. In elementary and middle-school classrooms, it would not be easy to find teachers who do not believe this statement to be true (Alvermann and Moje, 2019). Convincing educators to use various reading comprehension strategies across other content areas to increase learning is not an issue. Rather, the challenge for teachers is to understand how to utilize purposeful reading comprehension strategies in instructional settings (Vaughn and Massey, 2019).

It is important to teach reading comprehension strategies to recognize words and comprehend what the words being read actually mean; learning must go deeper than a connection between written words and the eyes identifying those words. A reader may not possess this ability naturally and must be taught to cognitively process the words read and then take what was processed and draw a meaning based upon the reader's prior knowledge and personal experience. Teachers must provide students with reading comprehension strategies that can be applied repeatedly in all learning settings. Macceca (2014) states, “Comprehension strategies best serve students when they are employed across the curricula and in the context of their actual meaning” (p. 4).

Students are expected to have already acquired foundational reading skills by the time they reach middle school. In middle school, these basics are the foundation on which students learn to apply literacy skills to foster understanding in specified content areas such as in social studies. Social studies curricula at the middle-school level place great importance on the reading comprehension of informational texts. Not only are students required to learn the basics of the social studies content, but they must also work toward literacy mastery of the social studies texts. Social studies content and standards require students to be dual learners: they must master the actual content knowledge as well as learn how to comprehend social studies texts (McCulley and Osman, 2015). However, social studies is often the last content to be addressed with new interventions and changes, so it is important that teachers of this subject be provided with effective learning tools, strategies and methods that can better their instruction and in turn better the students' understanding of the instruction (Dobbs, 2003; Evans and Clark, 2015). Students must be equipped with learning strategies they can apply to think critically and master the social studies curriculum (Claravall, 2017).

Annotating comprises structured markings in the text (e.g. highlighting, circling and underlining) and added visual notes (e.g. drawing and writing) for various purposes, including explanation and comment on key concepts in the text. It is a unique learning tool because it applies to different content areas, as well as outside the conventional content areas (Porter-O'Donnell, 2004). In this study, we explore how annotating can be a reading comprehension strategy to assist students in understanding complex historical texts in the social studies classroom. The study is guided by the following research questions:

1. Does annotating a historical text improve eighth-grade students' academic achievement in social studies classes?

2. What are students' and teachers' perceptions regarding using the annotating strategy to increase student achievement?

This study adopted the cognitive approach to explore how teachers and students perceive the reading comprehension strategy of annotating to increase student academic achievement in social studies. This cognitive approach is based on Piaget's work, focusing on the internal constructions of knowledge within the individual learner (Eggen and Kauchak, 2004). The individual applies their own environment and past experiences to a current learning situation to construct their own interpretation and meaning. This approach supports the idea that a teacher's role in a student's learning experience is to inspire students to take possession of their own learning experience by teaching, instructing, directing and overseeing learning strategies (Schreiber and Valle, 2013).

Snow (2002) defined reading comprehension as “the process of simultaneously extracting and constructing meaning through interaction and involvement with written language” (p. 11). The three foundational components of comprehension are the reader, text and activity. Each function in a specific capacity in the overall process of reading comprehension. The reader brings his or her own prior knowledge/experiences and previously acquired abilities/skills to the text and activity. The text simply provides the material to be read and interacted with. The activity itself is the interaction by the reader, using the text to identify purpose, meaning and consequences. Reading comprehension occurs when the reader can take a piece of text, identify the main idea and other important information in the text and then categorize/associate it with prior knowledge regarding the overall meaning of the text (Ness, 2011).

Students' understanding and comprehension in social studies increase when they make connections between prior knowledge and new knowledge being studied (McCulley and Osman, 2015). However, teaching students to connect material and knowledge they have already been taught combined with new material and content from texts is easier when they already have strong reading comprehension skills. In addition, students must possess and be able to use critical thinking skills to gain a better and fuller understanding of the content of their social studies texts (Reidel and Draper, 2011). The use of more complex texts could help develop a learner's ability to think more critically (Evans and Clark, 2015). Since the need for students to be able to think critically has intensified with the Common Core curriculum, there has been a continual demand for increasing text complexity. This has been a recurring theme in mastering the Common Core curriculum (Fisher and Frey, 2015). The embedded literacy standards address text complexity. Evans and Clark (2015) stated that text complexity increased with new standards and curriculum and concluded that if content texts in social studies were more complex than in the past, then using reading strategies in social studies could increase comprehension.

However, comprehension of complex expository texts is especially difficult for students who struggle with reading because it requires them to utilize reading comprehension skills, which go beyond the foundational understandings. Instead, students are required to dig into a text to identify major and minor points while connecting those points to prior knowledge and patterns within the text. For students to be able to perform these tasks while reading expository texts in social studies, they must be taught reading comprehension skills, which can aid them in learning the content. Barber et al. (2015) noted that because social studies content is often composed of a lot of reading of complex texts, it is vital to the success of learners that they become engaged with the content. They noted that comprehension increases when literacy intervention strategies are used to increase engagement in social studies content. Stimulating student interest through reading strategies that involve the student with the content could increase engagement. Key et al. (2010) found that when reading strategies were used to facilitate engagement in social studies, learning improved.

Many studies have been conducted on reading comprehension strategies and how to improve knowledge, understanding and overall learning using these strategies (Alexander-Shea, 2011; Angulo, 2019; Chiariello, 2018; Dobler, 2013; Fisher and Frey, 2015; Hattie, 2019; Key et al., 2010; Neugebauer and Lia, 2018; Turner and Himmel, 2019; Vaughn and Massey, 2019). However, researchers have rarely mentioned annotating as a standalone reading strategy. The importance and significance of annotating have been presented through its inclusion in the close reading strategy (Fisher and Frey, 2014a; Zywica and Gomez, 2008). Much of the research conducted on annotating has focused on its role in the close reading strategy process, making it important to include literature on close reading as part of the foundation for the importance of annotating as an effective reading strategy. For example, in Fisher and Frey's (2014a) study, annotating was a part of the overall close reading intervention along with re-reading, think-alouds, questioning and discussions. They concluded that students who received the intervention of close reading showed significant increases in their academic achievement. It also instilled motivation and engagement in the active learner when these strategies, including close reading, were utilized, increasing comprehension and learning (Cole, 2014). Annotating required the learners to become engaged in their reading by dissecting the text, which, according to Barber et al. (2015), increased their understanding and comprehension.

Boyles (2013) expressed the importance of teaching students what annotating is and how to use it earlier in their education. This early instruction in annotating would better prepare students for the complex texts they are required to master in middle school. Obtaining this skill would equip the students early on with the tools they needed to be successful when they reach higher grades. Furthermore, students would encounter more complex texts, which would mandate more skills to comprehend the text successfully.

We adopted a mixed-methods design (Creswell and Plano Clark, 2011) to sequentially collect quantitative and qualitative data. First, we collected quantitative data with a series of pre- and post-tests from all student participants during a six-week instructional time frame. We then collected quantitative and qualitative data with a survey from teacher and intervention group student participants. Quantitative data were analyzed to evaluate the mean differences in participants' test scores and survey responses, and qualitative data from open-ended survey questions were transcribed and analyzed using an inductive approach (Miles et al., 2014) to supplement the quantitative findings and develop a holistic picture of the students' learning experiences (Kim et al., 2013, 2017).

The participants for this study included 256 students and five teachers in eighth-grade social studies classes at a middle school in the southern USA. Over 90% of the students at the school were economically disadvantaged. The classes included regular education students, students with disabilities, advanced students and gifted students. Each teacher participant had two separate sections of eighth-grade social studies classes. Each section was randomly assigned to either the intervention or the control group. Thus, a total of ten sections from five classes were allocated in Group A (intervention group, 129 students) and Group B (control group, 127 students).

Two series of pre- and post-tests (pre-test-1, post-test-1, pre-test-2 and post-test-2) were created using the Illuminate online instructional resource (Illuminate Education, 2018). Though teachers could use various daily assessment strategies, the school administrators expected them to utilize Illuminate for the formative and summative assessments. Illuminate offers school districts a plan for creating and implementing a comprehensive, balanced assessment system that is aligned with national and state standards. Due to the length of time needed to teach the unit (six weeks), the unit was divided as evenly as possible based on its number of standards/elements. Each series (e.g. pre-test-1 and post-test-1) test had the same 20 multiple-choice questions with four possible answer choices for each question.

We developed a survey for students and teachers that used a similar overall format, including demographic, Likert-scale and open-ended questions Table 4-1. Overall, the Likert scale questions were used to elicit participants' perceptions or experiences of annotating historical texts for teaching and learning. Open-ended questions gave participants the opportunity to provide additional reasoning and/or explanations to buttress their responses. A pilot test was conducted with one sixth-grade ELA/SS teacher with 28 students to help establish the content validity of the survey instrument used in the study (Creswell and Plano Clark, 2011). We ensured that the quantitative and qualitative questions on the survey were easy to understand and appropriate for acquiring respondents' perceptions about using the strategy of annotating.

First, we provided two different 1-h training sessions on annotating as a reading comprehension strategy to all five teachers. The first training session was an overall introduction to the study focused on annotating texts. The second training session taught the teachers how to annotate using different symbols, highlighters or sticky notes. After training, the teachers were able to instruct their own students on how to annotate. We also provided teachers with articles/passages from ReadWorks and Newsela to use during instruction in addition to texts from the traditional eighth-grade social studies textbook. Finally, to keep the consistency of implementation from classroom to classroom, we conducted short classroom observations to ensure that teachers and students correctly followed the predetermined guidelines of the annotating strategy.

The teachers' role in the study was to instruct and model how to annotate and then monitor the students' use of annotating in their classrooms. The students' role was to use the intervention for six weeks, during which they applied annotating while they were learning in the social studies content area. The intervention was used a minimum of two times a week, extending throughout the six weeks. It provided a minimum of ten class experiences with the annotating strategy as a reading comprehension strategy. Each time the strategy was used, it was employed for no less than 10 min, providing a minimum of 100 min with the strategy. The strategy was used mostly to introduce new content material, so it might have been used more than the minimum 100-min intervention time. Figure 1 is a sample of one student's annotations on a provided ReadWorks document.

Students in the control classes were given reading texts independently and comprehension questions to complete. Teachers were asked to continue their traditional forms of instruction with each text. All students/classes were required to read the same texts and complete the same questions.

All test data were analyzed using IBM SPSS Statistics 26 software to determine the significance of the results. All qualitative data from the survey were analyzed using MAXQDA. The qualitative data were used for further explanation and interpretation to support the quantitative data to help triangulate the overall findings.

Students in Group A (intervention) and Group B (control) took two different series of pre- and post-tests throughout a six-week unit of study. Each of the five teachers (Teachers Z, Y, X, W and V) who participated in this study taught two sections of eighth-grade social studies classes. The results of the pre- and post-tests are shown in Table 1.

The gain score from the pre-test to post-test was computed for each participant (e.g. post-test score–pre-test score). As shown in Table 2, all mean scores increased between the pre- and post-test in both Series 1 and 2. It appears that despite the difference in starting scores, all gain scores showed similar improvement within their groups. An interesting finding was that students in Group B (M = 38.7) had a higher mean than Group A (M = 36.51) on the pre-test, but the students in Group A (M = 80) scored higher than Group B (M = 79.09) on the post-test in Series 2.

Independent samples t-tests were conducted to analyze the differences in gain scores between Groups A and B, as shown in Table 3. The results indicated a statistically significant difference between students who were in the intervention group classes (Group A) that used annotating and the control group classes (Group B) that used traditional instructional strategies. These results support that the students who used the strategy of annotating texts for reading comprehension performed better than other students on the tests.

In addition, a two-way ANOVA was conducted to examine the effect of gender and student achievement from Series 1 and 2 pre- and post-tests. However, the main impact of the gender variable (male or female) was not significant.

Group A students and all five teachers completed the survey after the six-week unit of study. The survey results provide both quantitative and qualitative data about their perceptions and experiences regarding using the annotating strategy to increase the learning outcomes on the tests. The demographic data collected from the survey are presented in  Appendix. The Likert scale questions (1a–5a) in the survey provided the quantitative data, while open-ended questions (1b–5b) provided the qualitative data.

The responses on the five Likert scale questions indicate that both students and teachers had similar positive views (Student M = 4.54 and Teacher M = 4.92) about the impact of annotating on teaching and learning, as shown in Table 4. For example, both teachers and students strongly agreed that annotating as a reading comprehension strategy was easy to implement in the social studies classroom, and that they would likely implement it in future social studies instruction. Furthermore, there was strong agreement that annotating increased students' engagement while reading and working to comprehend historical texts and ultimately learning in the social studies classroom.

The open-ended questions provided the students an opportunity to offer further explanation of their responses to the Likert scale questions. However, answering these questions was not mandatory, and the actual number of participant responses varied by question.

The qualitative findings were derived from coding and theme generation. Repeated close readings of the open-ended responses were conducted. Each question's initial findings generated categories and codes so that keywords/statements could be identified and labeled. Themes were then generated based on central patterns among the data, as shown in Table 5.

Our achievement test results indicate statistically significant differences in the average growth of Group A (M = 44.19, SD = 7.23) and Group B (M = 40.95, SD = 6.90), t(254) = 3.668, p = 0.000. Cohen's d produced a medium effect size (d = 0.458). These preliminary findings support that Group A students who used annotating to improve their reading comprehension performed better than Group B students on the tests, as shown in Figure 2.

Fisher and Frey's findings (2012, 2013, 2014a, b, c, 2015, 2018) support that close reading, including annotating, can deepen understanding and increase overall learning when used in content area texts. Our study presented how annotating as a reading comprehension strategy improves middle-school students' learning, specifically of history content. The findings showed that annotating increased students' academic achievement, and there were no significant differences between male and female students.

Most students and all five teachers agreed that the annotating strategy was easy to use (M = 4.69 and M = 5.0) in their classrooms. In fact, they consistently responded that annotating helped increase students' engagement (M = 4.36 and M = 4.8) and learning of the content (M = 4.63 and M = 4.8). The students described how annotating increased their engagement and learning, as can be seen in the following sample student responses:

What happened when students were required to use annotation in the classroom? The teachers noted that students actively used at least one of the visualization tools (e.g. highlighters and sticky notes) to organize and summarize key points or to break down complex texts. One teacher explained:

Students could successfully annotate historical texts to improve their reading comprehension. Thus, the students became more willing to use it in future social studies classes (M = 4.29 and M = 5.0) and even in other classes (M = 4.71 and M = 5.0) because of their positive perceptions and experiences.

The summary of participants' responses to five Likert scale questions in the surveys revealed the benefits of learning by annotating historical texts in social studies classes. Both students and teachers agreed that annotating was easy to use in class, increased engagement and even improved their learning. Therefore, they were willing to continue to use annotating as a reading comprehension strategy in future social studies classes and other content area classes.

Why was annotating effective? The qualitative data analysis of the first open-ended question (1b) in the student survey indicates that students had prerequisite skills and prior experience with annotating. Many students already knew how to annotate with highlighters and sticky notes for reading compression. One teacher stated:

Thus, students had applicable tools that helped immerse them into what they were reading. This finding supports that learning activities can be very effective if they are designed to be easy to use for learning, as Kim and his colleagues suggested.

The teachers consistently described that students were actively engaged in their learning when using annotating, as illustrated by one teacher statement:

This comment supports Newkirk's (2012) finding that the reading strategy of close reading including annotating could help to increase engagement.

So how did annotating help students? The findings from the other open-end questions revealed the practical and pedagogical benefits of learning with annotating. Specifically, this strategy helped students to visualize key points, break down complex texts, slow down their reading, increase focus, encourage critical thinking and increase discourse. Table 6 presents some illustrative comments on the generated themes shown in Table 5. These benefits support students in increasing engagement and reading comprehension on complex texts and, most importantly, increasing their academic achievements.

Thus, it helped students focus, think critically and discourse to understand complex content. Annotating can be a powerful tool for both teachers and students to keep students engaged and improve their understanding and comprehension of what they have read.

Social studies content can no longer be mastered simply by memorizing facts or identifying locations on a map (Gilles et al., 2013; National Council for the Social Studies, 2017; Tanner, 2008). It now includes not only acquiring understandings of history, geography, government and economics, but also being able to analyze issues, think critically, solve problems and become informed citizens (Georgia Department of Education, 2017; Macceca, 2014; Reidel and Draper, 2011). Annotating could help students to do all of these things as they form their own connections and understandings of complex texts while providing hands-on and visual learning experiences. As a result, students learn to read more effectively and learn content area topics more deeply (Zywica and Gomez, 2008).

This study adds to the current body of research and undergirds reading comprehension strategies used to improve the learning outcome in content other than reading. Furthermore, the findings from this study provide evidence that the reading comprehension strategy of annotating is a valuable teaching and learning strategy for daily use in the context of historical content in the social studies classroom. However, adding new annotating activities can create a high extraneous cognitive load with negative experiences. This challenge can be reduced by (1) adopting appropriate instructional strategies to design and implement practical learning activities with annotating and (2) offering training sessions or professional development workshops with resources (e.g. annotating guide, highlighters and sticky notes). In addition, classroom teachers should understand their students' pedagogical needs to ensure that students learn with the benefits of annotating strategy.

Limitations occur in research when shortcomings appear in the research design or methodology that can impact or influence the interpretation of the findings (Patton, 2002). Although the authors took precautions when designing the study, the study contained limitations that may have adversely influenced the overall findings, including lack of generalizability due to the limited number and location of participants, possible inadequacies of the instruments, bias based on personal experiences with annotating and issues related to self-reporting measures on the surveys.

The overall purpose of this study was to examine the effects of annotating a historical text as a reading comprehension strategy on student academic achievement in an eighth-grade social studies class. The quantitative and qualitative data results showed that annotating increases student engagement, reading comprehension and thus academic achievement. Annotating helped students visualize key points, break down complex texts and slow down when reading complex historical texts. As a result, it helped students focus, think critically and discourse to understand complex content. Annotating can be a powerful tool for both teachers and students to keep students engaged and improve their understanding and comprehension of what they have read.

Competency in reading comprehension is not natural for all students (Mahdavi and Tensfeldt, 2013; Ness, 2018). When learners struggle with understanding and comprehending what they have read, it can be due to a lack of knowledge of appropriate comprehension strategies or uncertainty about how to use the reading comprehension strategies they have been taught (Mariage et al., 2020). Therefore, educators must prepare students to use reading comprehension strategies such as annotating in all content areas and not only in a traditional academic setting.

¹