Computer-based literacy support in Trinidad and Tobago: evidence from school interventions

Tremendous progress has been made globally to ensure that most children have access to primary education. The growth in access to schooling has been notably impressive in low- and middle-income countries (LMICs) since the international community has rallied around targets such as the sustainable development goals that aim to ensure “that all girls and boys complete free, equitable and quality primary and secondary education” by 2030. Despite this progress, however, important inequities remain within and across countries, and the quality of educational opportunities varies drastically. Prior to the pandemic that started in 2020, international assessments of student learning pointed to a global learning crisis (UNESCO, 2013; World Bank, 2018). In the last few years, school closures and disruption have deepened the need to focus on student learning and in some cases have increased inequities in learning outcomes (Donnelly & Patrinos, 2022).

Addressing the growing inequality in educational opportunities globally and within countries is the focus on many policies and interventions. In Uruguay, a program lengthened the school day in disadvantaged urban schools and led to a modest 0.07 and 0.04 SD increase in student math and language scores, respectively (Cerdan-Infantes & Vermeersch, 2007). Many countries have implemented some form of financial incentives for teachers based on student outcomes. In LMICs, the evidence of such performance-based incentives is generally more favorable than high-income countries (Ganimian & Murnane, 2016) but sometimes exacerbates inequalities with larger gains in better-performing schools (e.g. Filmer, Habyarimana, & Sabarwal, 2020). Incentives tied to teacher outcomes (rather than student performance) are generally more positive and reduce unintended consequences such as “teaching to the test” or cheating (Biswas, de Galbert, Sabarwal, Glave, & Asaduzzaman, 2025). Importantly, evidence suggests that programs aimed at improving the average performance of foundational skills in low-income communities also reduce inequality (Rodriguez-Segura, Campton, Crouch, & Slade, 2021). Similarly, Evans and Yuan (2022b) find that programs that specifically aim at improving girls education have similar beneficial impacts for girls as programs that target all children. Taken together, these findings suggest that reducing inequalities in educational outcomes may be best achieved by targeting all students in the most vulnerable schools and communities.

One promising avenue to improve learning outcomes and reduce inequities is the integration of new technologies. The use of technology in education is ubiquitous and has increased in many parts of the world in recent years as the drop in hardware and internet access costs has reduced the barrier to access for many. Investment in technology continues to be an important part of educational budgets. In the United States, more than $13 billion was spent on education technology in 2015, nearly two-thirds of which was for software (Molnar, 2017). Governments in LMICs have also invested in important resources to increase the use of technology in their classrooms. Unfortunately, these investments do not always result in positive impacts on student learning. A comprehensive review of 126 impact evaluations of technology interventions in education found that computer-adapted learning and behavioral interventions were two promising areas but that increasing access to computers or internet alone has little or no impact (Escueta, Nickow, Oreopoulos, & Quan, 2020). Most of the successful interventions identified, however, were programs focused on improving mathematics. More evidence for computer-based literacy is needed, as evidence of these programs was limited and showed mixed results. The findings from this review point to the importance of focusing on the mechanisms that could improve learning and understanding subject-specific outcomes rather than simply promote general investments in technology.

Supporting literacy with technology in the classroom can be done through different approaches. Access to electronic books and other media can increase the variety of reading materials. Assistive technology can support students with specific needs not readily available in typical classrooms. Computer-based programs can offer new approaches to teaching literacy skills or complement the curriculum used by teachers. Ganimian, Vegas, and Hess (2020) categorize the ways in which technology can be used as a tool to facilitate learning in four ways: (i) scaling up standardized instruction, (ii) facilitating differentiated instruction, (iii) expanding opportunities for practice and (iv) increasing learner engagement. The program evaluated in this study has the potential to improve student literacy using specialized software as a tool through two of these: students had additional instruction after school hours and using an engaging software designed to support reading instruction and targeted to their initial reading level. Importantly, teachers need support to develop specific knowledge and skills to integrate this technology in their classrooms (Belo, McKenney, Voogt, & Bradley, 2016), and teacher beliefs and attitudes toward the specific technology can present a barrier towards its successful implementation (Ertmer, Ottenbreit-Leftwich, Sadik, Sendurur, & Sendurur, 2012).

This study presents the evaluation of two computer-based literacy interventions conducted in Trinidad and Tobago in 2021 and 2022. These programs were implemented by a non-governmental organization – the ARROW (Aural, Read, Response, Oral, Write) foundation – in partnership with the Ministry of Education in two low-income urban communities near the capital Port of Spain. Phase 1 supported 183 students in the second half of the 2021–2022 school year. The first evaluation (Phase 1) uses a pre–post evaluation design and finds an increase in reading and spelling of 0.30–0.40 SD, equivalent to six to nine months of progress. The second evaluation supported 201 students between September 2022 and March 2023 and used a phased-in randomized control trial. We find smaller gains of 0.12 SD, equivalent to two to three months on the outcome measure. We also find that student behavioral measures reported by teachers improve after students participated in the intervention. The findings from these two studies add to the limited body of evidence on computer-based literacy interventions in LMICs.

The rest of the study is structured as follows: Section 2 describes the education and social context in which the interventions took place and presents research on technology’s use in literacy programs, as well as a description of the program. Section 3 includes the methods and analytical approach and Section 4 presents the results. The study ends with a discussion of the implications of the evaluations and a conclusion that summarizes our findings.

Education in Trinidad and Tobago is compulsory for seven years beginning at the age of six, and public primary and secondary schools are free. The education system is based on the British System as a result of its colonial ties. Primary school begins at the age of 6 and lasts for 6 years. At the end of primary school, students take the Common Entrance Examination to determine their placement in secondary school. Secondary school is five years long. Prior to the establishment of the Caribbean Examination Council (CXC) in 1972, at the end of secondary school (Form V), students were assessed using the General Certificate of Education Ordinary Level (GCE O-Level) which was administered by examination authorities in the United Kingdom. The CXC Caribbean Secondary Education Certificate, administered by the Caribbean Examination Council, has replaced the GCE O-Level exam. There are currently 481 public primary and 64 private primary schools and 133 public secondary and 63 private secondary schools.

Student performance in Trinidad and Tobago is good overall, but important inequities exist. The country has taken part in a few international assessments over the last two decades, allowing comparisons with its neighbors. Between 2006 and 2011, the country improved significantly in the Progress in International Reading Literacy Study (PIRLS) reading assessment, with average score of elementary students increasing from 436 to 471, still below the 500 international mean (Mullis, Martin, Foy, & Drucker, 2012). However, inequality across gender was very large with boys scoring much lower than girls. In 2009 and 2015, the country was on par with the countries in the Latin America and Caribbean countries on the PISA reading assessment for 15-year olds and was third out of nine in 2015, behind Chile and Uruguay. However, students in the lowest quintile of wealth scored close to three years behind their peers in the top quintile, highlighting a large inequality in the system (World Bank, n.d.).

Like many education systems across the world, the COVID-19 pandemic created additional challenges and exacerbated inequities. Between March 2020 and March 2022, schools were fully closed in the country for 26 weeks and only partially opened for the 47 remaining weeks of the academic calendar (UNESCO Institute for Statistics, 2022). Alternative modes of instruction were offered through television and online learning. However, at the start of the outbreak, in March 2020, approximately 20% of the student population did not have access to necessary hardware to access online education (Kalloo, Mitchell, & Kamalodeen, 2020).

With a population of 21,000, the Laventille and Morvant communities in the eastern part of Port of Spain are some of the most marginalized of the country. They lie within the San Juan/Laventille Regional Corporation, which has the second highest poverty rate of the country’s fifteen administrative regions (De Lisle, Smith, & Jules, 2010). Morvant and Laventille have high crime rates, and Laventille especially is particularly known for its gangs (Wallace, 2018). As a result, these communities are often synonymous with crime and youth originating from there are stigmatized (Editorial, 2020). Further, the communities are identified “at-risk” by the government, leading perception that all students from these schools are “bad”, as noted by elected officials (Republic of Trinidad and Tobago, 2021). These negative stigmas lay in the historic neglect and marginalization of this area and contribute in part to the lack of trust in its schools.

Trinidad’s Ministry of Education oversees 11 primary schools in Morvant, Laventille and Upper Laventille, all of which are government and “government-assisted” religious schools. In all 11 schools, student performance on the 2020 Secondary Entrance Assessment was below the national average. The average score of students in Morvant and Laventille was 182.49, significantly below the national average of 200.06 (Republic of Trinidad and Tobago, 2021). The government is now supporting the Laventille/Morvant School Improvement Project, which includes four components: parenting in education, promoting discipline and reducing violence, literacy/numeracy, teacher training and development, and infrastructure and esthetics. The third component is directly tied to the programs evaluated in this study. The ARROW foundation has worked in these schools since 2010 and the Ministry of Education has called on the organization to support vulnerable youth who have not been responding to traditional methods of teaching and who may need a different approach to their learning.

Research on computer-based literacy programs points to compelling evidence of their positive impacts. Experimental studies, mainly in high-income countries, suggest that interactive, multimedia-driven and personalized approaches significantly enhance both motivation and cognitive engagement, leading to improvements in reading comprehension, fluency and vocabulary acquisition (Escueta et al., 2020). For instance, a reading comprehension computer-assisted learning (CAL) program for middle school students in the Unites States found reading gains ranging from 0.20 to 0.53 standard deviations (Wijekumar et al., 2014; Wijekumar, Meyer, & Lei, 2012). A CAL program conducted in India found effects of 0.22 standard deviations on Hindi language scores (Muralidharan, Singh, & Ganimian, 2019).

Beyond the average impact of computer-based literacy programs, research has suggested specific skills that are important to focus on. Reviews in primary and secondary schools highlight that the most effective programs include structured schemes for improved spelling, work on phonological skills embedded within an approach of improved reading, directly targeted practice for comprehension skills and targeted use of technology (Brooks, 2007; Raspin, Smallwood, Hatfield, & Boesley, 2019). In addition, CAL tools, such as programs with speech-to-text, voice input and speech feedback, can have large positive impact to support teaching students with dyslexia (Singleton, 2009).

Within the large range of uses of technology in literacy interventions, the use of self-voice feedback is particularly beneficial. Self-voice programs, such as ARROW, IDL (indirect learning) and own-voice intensive phonics, are used to improve reading and writing skills, mainly for children who lag behind their peers. By recording and replaying their own voices, learners develop phonological awareness and decoding skills in a positive, self-directed environment (Gwernan-Jones et al., 2018; Macleod, Macmillan, & Norwich, 2007). Hearing their own voices helps learners internalize the sounds of language, reinforcing phonics and reducing anxiety associated with prior literacy struggles (Gwernan-Jones et al., 2018).

Most of the evidence of the efficacy of technology-based literacy interventions comes from research in high-income countries. A review of interventions in LMICs suggests that their success is often tied to teacher professional development, integration of adaptive technologies and a focus on phonics-based literacy development (Norman, 2023). Explicit instruction in phonics and decoding skills is also crucial, as it enables students to master letter–sound relationships (Jamshidifarsani, Garbaya, Lim, Blazevic, & Ritchie, 2019; Norman, 2023). Another essential component is the use of multisensory learning approaches, as dynamic visuals, audio feedback and interactive interfaces contribute to enhanced vocabulary acquisition and reading comprehension (Norman, 2023; Oakley, Howitt, Garwood, & Durack, 2013).

Despite the promising evidence presented above, there are important gaps in literature. First, there is very limited research addressing computer-based literacy interventions within the Caribbean region. The unique socioeconomic and cultural context necessitates studies conducted locally to identify effective literacy strategies. Another critical gap identified is the methodological approaches used for evidence of self-voice techniques. Most evaluations have employed repeated measures with a pretest and posttest approach (Gwernan-Jones et al., 2018; Macleod et al., 2007; Raspin et al., 2019), and we have not found any randomized evaluation of this intervention. This study specifically adds to the body of knowledge by using a randomized controlled trial in the Caribbean.

The ARROW foundation is a nonprofit organization focused on developing literacy skills for children and adults, through the use of their literacy program. The organization has been operating in Trinidad and Tobago for over 16 years and has worked in more than 200 schools in Trinidad and Tobago with thousands of children. Programs can be implemented in schools, community centers, homes and from its offices in Port of Spain. Some programs are directly implemented by staff members, while others are run by teachers or tutors trained by the organization. The foundation also conducts parent and teacher workshops in collaboration with the multi-sensory program and has also worked in secondary schools.

ARROW is a program originally developed in the United Kingdom based on learning through self-voice techniques, while undertaking various activities covering reading, spelling, dictation, speech and listening skills. The method currently uses specialized software that has been modified since it was first developed several over 40 years ago. This program has been implemented in different contexts and produced important results in both children and adults who have learning difficulties as well as those who do not (e.g. Nugent, 2012; Raspin et al., 2019). In fact, ARROW is the most effective computer-based program included in a series focused on successful supports for literacy difficulties in the United Kingdom (Lavan & Talcott, 2020).

The two programs evaluated in this report are part of a larger effort started by the foundation in late 2021 in the East Port of Spain communities of Laventille and Morvant. The organization will work with 20 primary schools, supporting over 3,000 students in total. The sessions take place in schools, either in the library or a classroom. Each student is initially assessed using the Schonell Reading and Spelling Assessments. Students are then placed on their individual learning program based on their initial level. The program typically includes 8–10 hours of intervention and requires students to complete specific computer-based cognitive developmental exercises using a range of multisensory tools. One key tool is the self-voice of the student, which encourages both speaking and listening. These improvements cover environmental sound listening, consonant and vowel discrimination and sentence understanding.

The Personal Spelling Profile element of the Program also tests an individual’s spelling knowledge of all the word families in the English language and then designs a program tailor-made to each child’s specific need of improvement. Students work in groups of five or six with one tutor for one-hour sessions on their individual computers and then return to their regular class. During that hour, the program ensures each student is engaged in the software, operating at their optimum level and ability, allowing for self-managed, student-focused engagement and learning in an ideal environment, and conducive to learning.

This study presents the evaluation of two programs implemented by the ARROW foundation in partnership with the Ministry of Education. The foundation has maintained a longstanding relationship with the Ministry of Education in Trinidad and Tobago. The Ministry has sought literacy support from the foundation for “academic watch” schools across the country. The Ministry’s Curriculum Development Department has evaluated and approved the program for use in its schools. Additionally, the schools involved in these research projects chose to take part and provided written statements that parents were informed about the program their children would participate in, and that all parents provided consent for their children’s involvement ^[1]. Finally, all parents were invited to virtual meetings to understand the purpose of the intervention and research. These meetings also provided an opportunity for parents to ask questions and express any concerns they might have.

The two programs presented in this study were rolled out in different ways, which allow for different research designs. In Phase 1, there was no randomization of student participation, allowing for a pre–post comparison of student performances. In Phase 2, the program was gradually rolled out, with participating students randomly assigned to benefit in the first or second group, allowing for an experimental design. There are two specific research questions this study seeks to answer, one for each phase of the study:

1. How much did reading and vocabulary scores increase for students who participated in the ARROW program?

2. What is the impact of the ARROW program on student reading, vocabulary and literacy scores?

As explained in the introduction, this study presents the results of two programs – which we call Phase 1 and Phase 2 – that used the same pedagogical approach but required different evaluation designs. Phase 1 supported two cohorts of students (183 total) in the second half of the 2021–2022 school year. All of the students took part and participated in literacy assessments before and after the intervention, allowing a pre–post comparison. Students from 10 schools participated in the program in two separate periods: 85 students (Cohort 1) participated in the ARROW program from January to March 2022 and 98 students (Cohort 2) participated from April to June 2022.

Phase 2 consisted of a randomized phased-in approach. Teachers and principals from 6 schools selected students they thought would benefit the most from the program, resulting in a sample of 201 students ranging from grades Standard 1 to Standard 5. Students in this study were randomly assigned to one of two groups: a treatment group (T) of 100 students who were invited to participate in the ARROW intervention from September to December 2022 and a control group (C) of 101 students who could participate in the ARROW intervention from January to March 2023.

Data from students in Phase 1 were collected at two time points: pre-intervention and post-intervention. The pre-intervention assessment happened in January 2022 for Cohort 1 students and April 2022 for Cohort 2 students. Students were also assessed post-intervention, as soon as possible after they completed the 10-hour ARROW program. We finalized post-assessments for Cohort 1 students in March 2022 and June 2022 for Cohort 2.

Data from students in Phase 2 were collected at three time points: baseline in September 2022, midline up until December 2022 and endline up until March 2023. The baseline assessment happened before either group started the program, while the midline assessment was conducted after the treatment group finalized the 10-hour program and the control group hadn’t participated yet. Finally, endline assessments were conducted after control students participated in the program.

Assessments at each time point focused on measuring student literacy skills using the Schonell Graded Word Reading Test (SGWRT), Schonell Graded Word Spelling Test (SGWST) and the Annual Status of Education Report (ASER) reading assessment. SGWRT presents students with a sequence of words they are asked to read aloud. Younger children start at the beginning of the list, while older students can begin with a later group. The test stops when students make errors in ten consecutive words. Raw scores for this test range from 0 to 83, and the total number of correct words is transformed to Reading Age in years and months, with ages ranging from 6 years to 12 years and 6 months. Similarly, SGWST presents students with a sequence of words they are expected to spell. Words are dictated aloud on their own, used in a sentence or phrase, and then said aloud once again. The test stops when students misspell ten consecutive words. The words originate from a list ordered by difficulty level. Spelling age is estimated by formula (number of words correctly spelled/10 + 5), giving a decimal number that can be converted to years and months. The ASER reading assessment is complementary to the Schonell test as it can categorize students into five literacy levels based on their demonstrated skills: beginner, letters, words, paragraph and story (Pratham, 2017). Importantly, the ASER is an external assessment that was used by ARROW for the first time in this study. These three assessments were conducted at every data collection time for both Phases.

Additionally, student behavior was collected for the treatment group of Phase 2 at baseline and midline using teacher reports. The ARROW foundation has used this questionnaire in the past and includes six items where teachers report the extent to which they view student behavior and participation according to be on par with their peers and expected standards. These items are scored on a 5-point Likert scale and include scores on self-esteem and confidence, focus and concentration, level of participation, overall behavior and conduct, respect for authority and discipline. A score of 1 relates to teachers view this student “significantly below expected standard”; a 3 means teachers view their student as “average” on this attitude; a score of 5 means the teacher views their student as “significantly above expected standard”.

Table 1 presents the sample characteristics from Phase 1. Overall, girls represented 40% of the 183 participating students. Cohort 1 was comprised entirely of Standard 5 students, with an average age of 12.1 years. Cohort 2 students ranged from Standard 1 to Standard 5, with the most students in Standard 4 (62%) and an average age of 10.6 years. The overall age average is 11.3 years old. Before the intervention, the average raw score for SGWRT was 38.4, which translates to a reading age of 8 years and 7 months. The raw score for SGWST was 33 on average, which corresponds to a spelling age of 8.3 or 8 years and 4 months. Finally, 7% of students were classified to the letter level in the ASER assessment, while 11% reached word, 34% paragraph and 48% story.

Table 2 presents the sample characteristics from Phase 2, as well as the difference between the two groups. The causal claim about the program’s effects on literacy skills depends on the assumption that treatment and control groups are statistically the same in the absence of the intervention. The control column shows the means and standard deviation of each characteristic for the control group, while the treatment column shows the means and standard deviation for the treatment group. The overall column shows the means for the whole sample. Finally, the control–treatment column shows the difference in means between these groups. There are no statistically significant differences between groups in any of the specified variables. This demonstrates that the randomization was effective. Overall, 45% of the students were girls with an average age of 9.4 years. Most of them were in Standard 3 (35.8%), followed by Standard 4 (29.9%) and Standard 2 (22.9%). The average initial raw SGWRT score was 24.6 (7 years and 7 months). The average initial raw SGWST score was 22.2 (7 years and 2 months). Finally, 29.1% of students categorized to the letter level in the ASER assessment, while 21.1% reached word, 13.6% paragraph and 36.2% reached story.

One of the main concerns in maintaining internal validity in randomized-control evaluations is attrition of students, especially if it differs across groups. After they were initially assessed, some students from Phase 2 did not complete the 10-hour intervention or were not found to be evaluated either at midline or endline. In the control group, 85 of the 101 students were evaluated at midline and 84 at endline. In the treatment group, 93 of the 100 students were evaluated at midline and 83 at endline. Although this could be a concern if there is differential attrition between groups, there was still balance in characteristics of the remaining students ^[2].

For Phase 1, we first estimate the average change in assessment scores after the intervention period and evaluate whether this change is statistically significant. This is specified by Equation (1), where $Yi$ represents the literacy skill outcome (SGWRT, SGWST or ASER), $β0$ represents the average score before the program and $β1$ represents the change in score after the program. $Post$ is variable that takes the value of 1 if the score is measured after the program and 0 if it is measured before.

For Phase 2, we estimate the average treatment effect using the midline assessment. For each midline literacy outcome (SGWRT, SGWST and ASER), we estimate Equations (2) and (3). $Yi,s$ is the midline outcome for student $i$ in school $s$⁠, $Ti,s$ indicates whether the student was in the treatment or control group and $λs$ indicates which school each student is in. Equation (3) controls for student sex, age and their baseline score. $β1$ is our coefficient of interest as it estimates the average difference between treatment and control groups at midline.

We also include the analysis of endline scores to assess whether students in the control group “caught up” to students in the treatment group or if the treatment group’s learning gains faded over time. To do so, we estimate Equations (2) and (3), where $Yi,s$ is the endline score. If students in both groups learned the same amount, and if none of the gains were lost overtime, we expect no average difference at endline.

Finally, we use behavioral data from students in the treatment group to estimate the average change in behavioral outcomes after participating in the intervention and to assess whether these changes are statistically significant. We use a t-test to compare average behavior score for each of the six behavior outcomes before and after the intervention.

Table 3 shows the results for Phase 1 analysis, including the difference in average initial and post-scores and its statistical significance. As was shown in Table 1, the initial average reading score before intervention was 38.4 and the initial spelling score was 33, which correspond to 8 years 7 months and 8 years 4 months, respectively. The average initial ASER score was 3.2. After the intervention, the average reading raw score was 47, which corresponds to 9 years and 4 months reading age. In the case of spelling, the raw score after the intervention was 38, which corresponds to 8 years and 10 months reading age. Finally, ASER average scores increased to 3.7, yielding a difference of 0.4 points (0.5 SD). In all cases, the differences between post-scores and initial scores are statistically significant. This preliminary analysis shows that the average gain corresponds to 9 months for reading (0.40 SD) and 6 months for spelling (0.30 SD), respectively. However, there are other factors that could have contributed to this gain, such as regular school curriculum learning. Phase 2 provides a more rigorous evaluation due to the randomization of students and the phased-in approach used.

Table 4 shows the results of estimating Equations (2) and (3) for each literacy outcome. Panels I and II display the results for Equation (2) and (3), respectively. In Panel I, we see that the average midline reading score for the control group is 28, which corresponds to a reading age of 7 years and 9 months. The reading score of the treatment group is 28 + 3.9 = 31.9, which corresponds to a reading age of 8 years and 3 months. This difference of 4 months in reading age (0.15 SD) is not statistically significant. The spelling score of the control group at midline is 24.8, which corresponds to a spelling age of 7.5 or 7 years and 6 months. The spelling score of the treatment group is 24.8 + 2.9 = 27.7, which corresponds to a spelling age of 7.8 or 7 years and 10 months. This difference of 4 months (0.18) SD is also not statistically significant. Finally, the average ASER score for the control group was 2.87, compared to 2.92 for the treatment group. This 0.05 difference in points (0.05 SD) is not statistically significant.

In Panel II, we add control variables for student age, sex and baseline score. The baseline score is the most important predictor for all scores. Adding these controls adjusts the effect size and yields significant results for the SGWRT and SGWST. Specifically, the effect or the intervention on raw reading score is 2.9 points. This means that the average difference between treatment and control students with the same baseline score is 2.9 points. In terms of reading age, the conversion chart shows that a difference of 3 raw points translates to between 2 and 3 months, which is equivalent to a 0.12 standard deviation effect size. In the case of spelling, the effect of the intervention on raw spelling score is 2, which means that the average difference between treatment and control students with the same baseline score is 2 points. In terms of spelling age, this translates to 0.2 or a 2-month gain in spelling age, which is equivalent to 0.12 SD Finally, this specification yields no significant results for the ASER score, and the difference between control and treatment is 0.13 points (0.12 SD) after controlling for age, sex and baseline score.

Table 5 shows the results for Equations (2) and (3) using the endline scores as outcomes. Because all students have completed the program at that point, we would expect no differences between the groups, on average. However, students in the treatment group finished the intervention approximately 3 months prior to being assessed, while the control group students are assessed immediately after finishing, which could lead to differences if the benefit of the program is short-lived. Panel I shows that the raw reading score for the control group is 30.3, which translates to a reading age of 8 years. The treatment group shows a raw reading score of 30.3 + 4.7 = 35, which translates to a reading age of 8 years and 5 months. This difference of 5 months (0.19 SD) is not statistically significant. The raw spelling score for the control group at endline is 26.4, which translates to a spelling age of 7.6 or 7 years and 7 months. The raw spelling score for the treatment group is 26.4 + 2.4 = 28.8, which translates to a spelling age of 7.9 or 7 years and 11 months. This 4-month difference (0.13 SD) is not statistically significant. In the case of ASER, the average score for the control group is 3.1 compared to 3.0 for the treatment group, showing a −0.1 difference (−0.03 SD) which is not statistically significant.

Panel II includes controls of sex, age and baseline score. In this case, there is a statistically significant difference for reading in favor of the treatment group of 3.2 raw points or 0.13 SD. This means that the control group gained less as a result of the program than the treatment group, which is surprising. In terms of reading age, this represents 2 or 3 months. In the case of spelling and ASER scores, there is no significant difference between treatment and control groups, even after accounting for sex, age and baseline score. Figure 1 displays the results from midline and endline across the three literacy outcomes.

Finally, Figure 2 shows the pre–post comparison analysis of behavior outcomes for the treatment group from baseline to midline. Teacher reports of students’ self-esteem and confidence increased from an average of 3 (in a Likert scale of 5) to 3.5. A similar 0.5 difference is observed for focus and concentration, improving from 2.9 to 3.4. Reported level of participation improved from 3.2 to 3.6. Overall conduct increased from 3.5 to 3.8, while respect for authority increased from 3.8 to an average of 4. Finally, discipline score increased from 3.6 to 3.9. These differences are all statistically significant, but they cannot be attributed solely to the program because of the lack of data for the control group.

Despite important progress made toward universal access to education globally, important inequalities remain in the quality of services and student learning, both across and within countries. While many student assessments point to the difference in student outcomes between communities, or between demographic groups, it is important to reframe these inequities as a result of chronic under-investment and focus on the educational debt that is owed (Ladson-Billings, 2006). The most effective approaches to reducing these inequalities seem to be focused on supporting underserved communities and targeting all the students in these schools rather than focusing on a subset of children (e.g. only girls). In addition, it is crucial to focus instruction for struggling learners in heterogeneous classrooms at the right level, with individualized programs rather than homogenous lessons or programs. This is particularly challenging in resource-low contexts – such as in Laventille and Morvant – where teacher shortages are chronic (UNESCO & International Task Force on Teachers for Education, 2030, 2024). Technology can be an important tool in supporting educators.

Investment in technology continues to be an important part of educational budgets, without clear evidence of improvement for student outcomes. In the United States, more than $13 billion was spent on education technology in 2015, nearly two-thirds of which was for software (Molnar, 2017). While investment in hardware is a necessary prelude to the use of most edtech solutions, interventions limited to providing access are not usually effective beyond fostering computer literacy (Escueta et al., 2020). In contrast, supports that include computer-adapted learning or target student instruction with software are one of the two most promising uses of technology according to a review of more than 100 rigorous edtech evaluations conducted internationally (Escueta et al., 2020). However, more research needs to be done on literacy-based interventions, as well as in low- and middle-income settings.

The growing consensus from high-income countries, as well as the limited evidence from LMICs presented earlier in this study, is that computer-based literacy programs are a promising approach to support struggling readers. The specific use of self-voice as a tool for struggling readers is shown to help both students with dyslexia and more broadly students lagging behind their peers in literacy skills. This study supports the findings that students in the elementary years can significantly grow their literacy skills with as little as 10 hours using a computer program. Growing the infrastructure to put in place such programs, both computers and trained teachers, or aides to support them, is a promising avenue to improve literacy skills and reduce inequities.

The results presented and the inferences drawn from this study include some methodological limitations. Firstly, Phase 1 results and behavioral outcomes in Phase 2 only provide a pre–post analysis of changes. We cannot claim that the changes observed in these cases are caused by the program. Additionally, while we do find statistically significant effects for the instruments developed by ARROW (reading and spelling) in Phase 2, we found none on the ASER assessment. This may be due to the small sample size, which makes it harder to detect statistically significant differences. This could also be due to the measurement scale being too coarse to detect differences. Finally, it is also possible that progress on reading fluency and comprehension may need more time or more support than what the ARROW program offers. Future evaluations of ARROW would benefit from including some teacher-level and school-level characteristics that can influence literacy outcomes. Overall, however, these evaluations contribute important knowledge to the efficacy of the ARROW program overall and educational interventions that use computers to support literacy. This is the first rigorous evaluation of the ARROW program outside of the United Kingdom to our knowledge and adds to the limited amount of evidence on these types of programs in LMICs.

The findings from these two studies have important implications for theory, research and practice. The theoretical basis for a self-voice feedback approach used by the ARROW program is reinforced by these findings. Students’ literacy skills improved through this technique, and the second study points to the causal impact of the approach. In addition, the intervention supports two of the four areas where technology can facilitate learning: expanding opportunities for practice and increasing learner engagement (Ganimian et al., 2020). However, adapting the program to cater to specific needs of the learners could use a third approach through facilitating differentiated instruction. Additional research into how to adapt the ARROW program to improve its effectiveness is warranted. Similarly, the impact of the program beyond the types of assessments designed by the ARROW team is lacking. Further evaluations should investigate the extent to which literacy skills specific to the program transfer to reading and comprehension. Finally, this study points to important implications for practice. Opportunities to learn in the classroom are often limited or inadequate for students in under-resourced communities such as Laventille and Morvant. Providing support outside of traditional schooling hours, in after-school programs or community centers can create new spaces for students to reinforce their literacy skills. The government could lead partnerships to implement such programs to reduce structural inequities.

As Jhurree (2005) notes, integration of ICT-based educational initiatives in developing countries needs careful planning and should take into consideration the specificity of the local context. While the results of this evaluation are clearly positive, future research should include more external tools, focus on replicating the Phase 2 research design and include more robust data collection to help highlight more clearly the positive impact of the program. This evaluation shows that ARROW and similar interventions have their place in supporting student learning in schools and can be adopted in some of the most challenging settings in Trinidad and Tobago.

In this study, we present the results of two evaluations on a computer-based literacy program in Trinidad and Tobago. Results from the pre–post Phase 1 analysis show that the average gain in reading and spelling Schonell scores after participating in the ARROW intervention are nine and six months or 0.40 and 0.30 SD, respectively. Additionally, the average gain in ASER score after the program is 0.4 points or 0.50 SD. However, these results do not capture a causal effect of the program, as other factors such as regular classroom learning can contribute to this increase. Phase 2 allows us to identify a causal effect of the program, when we compare treatment and control groups at midline. This analysis shows that students in the intervention scored higher in reading and spelling than students in the control group. Controlling for gender, age and initial score, the intervention led to an increase of 2 to 3 months in reading age and an increase of 2 months in spelling age, both equivalent to 0.12 SD. However, there was no significant difference on the ASER average scores. Finally, we find that behavioral measures reported by teachers improve after students participated in the intervention, though these reports do not have a comparison group.

The magnitude of the effects found here is important but sits in the middle of the distribution of education interventions internationally. An effect size of 0.12 SD is slightly below the median effect size for reading Randomized Contorl Trials (RCTs), which Evans and Yuan (2022a, b) find to be 0.14 SD across a sample of 138 randomized trials in LMICs. Additionally, literature suggests that effect sizes tend to be larger when studies are conducted with smaller samples (Evans & Yuan, 2022a; Hall & Burns, 2018), where Evans and Yuan (2022a, b) find that the median RCT effect size on reading and math assessments with samples of less than 732 participants is 0.23 SD. The effect size in this study is smaller than other targeted small-group reading interventions. A meta-analysis of 26 of these programs found that the median effect size was 0.54 g, which is a weighted effect size measure that accounts for the sample size of each group (Hall & Burns, 2018). However, the ARROW program is short and the size of the effect for fewer than 10 hours of support is impressive.

In order to test differential attrition, we use baseline characteristics as outcomes in the following model:

$Atti$ is in indicator of whether students are attritors at timepoint i. We run this model under two different specifications: (1) student did not take part in the midline data collection and (2) student did not take part in endline data collection. $β3$ provides an estimate of whether there are average differences between attritors from treatment and control groups. To maintain internal validity, we expect to see no significant differences in these groups. Table A1 provides estimates of $β3$ for the two specifications with each of the baseline characteristics presented in Tables 1 and 2 as outcomes. Besides a higher percentage of attrition from girls in the treatment group, there does not appear to be systematic differential attrition across the different baseline characteristics.