Improving Reading Skills in Middle School Students Using Educational Technology

Framework

Unfortunately, far too many middle school students in the United States are not proficient readers. According to standards set by the US Department of Education and Institute of Education Sciences, approximately 65% of eighth graders fail to meet reading proficiency standards (National Assessment of Educational Progress [NAEP], 2019). These gaps in English Language Arts (ELA) can have negative downstream effects across content areas (e.g., social studies, science), which require students to read advanced, informational texts (Schiefele et al., 2012).

The Simple View of Reading (Gough & Tunmer, 1986) served as a theoretical framework for this study. According to this theory, proficient reading is based on strong word identification skills coupled with solid language processing skills. Word identification requires the ability to apply decoding rules to quickly and accurately recognize words in print (Snow, Burns & Griffin 1998). The role of language processing in reading is multi-faceted (Nippold, 2017). The reader must apply rules of grammar to understand morphologically complex words and identify syntactic forms. Reading comprehension makes use of word identification skills and aspects of language processing, and requires some degree of background knowledge, verbal reasoning, and use of strategies such as inference making.

PowerUp, the instructional program used in this study, was built to meet the needs of struggling and non-proficient readers in accordance with the Simple View of Reading. Thus, it contains three instructional strands. The Word Study stand emphasizes decoding rules to foster efficient word identification skills and provides instruction in how to process morphologically-complex words. The Grammar strand addresses aspects of language processing such as identifying parts of speech, recognizing various syntactic forms, and learning about connective words in sentences. The Comprehension strand supports advancing background knowledge and employing comprehension strategies.

A review of the literature shows that well-designed digital interventions can be used to support non-proficient readers in middle and high school (Lenhard et al., 2013; Kim et al., 2011; see Cheung & Slavin, 2012). Moreover, such interventions can be quite appealing to students (Chen et al., 1980) and teachers (Kim et al., 2006) alike. However, not all digital programs succeed in helping non-proficient readers (e.g., Strong et al., 2011). To be effective, such programs need to be engaging and, in addition, built off established theories (Hirsh-Pasek et al., 2015) and implemented with fidelity (Pace & Mellard, 2016).

In the case of PowerUp, early studies have shown that its use can lead to gains on general assessments of reading ability in middle school students (Authors, 2018, 2020). Yet, it remains unclear how, more specifically, PowerUp contributes to these gains. Here we asked to what extent does PowerUp contribute to reading gains on tasks that require discrete or multiple skills in comparison to an alternative program.

Methods

Sample

Students were enrolled in a mid-sized school district in the greater Boston area. In the year prior to this study, only 42% of sixth grade students in the district met or exceeded state ELA proficiency standards. This study focuses on supplemental classes for sixth grade students in two middle schools. These classes met 2-3 days per week, providing students extra time to work on literacy skills. Initially there were 135 students in the supplemental classes. Thirteen were removed from analyses because they did not complete both pre- and post-test assessments, or they presented with extreme outlier scores across assessments.

The final sample consisted of 122 students. Most were Latinx (69%) followed by White (25%), Asian (4%) and Black (2%). Ten percent were English Learners (ELs) and an additional 33% were former ELs.

Procedure

This is a cluster randomized control study. There were six supplemental reading classes, two in one school and four in the other school. Classes within a school were taught by the same teacher. Half of the classes in each school were randomly assigned to use PowerUp and the other half to a control group.

PowerUp. Students begin PowerUp with an auto placement tool, which determines an appropriate start level in each strand: Word Study, Grammar and Comprehension. Students then work through activities in each strand and the program adapts to provide explicit instruction when necessary. As such, PowerUp provides a form of differentiated instruction. Based on their placement levels, the program provides recommendations for how many minutes students should use the program. The program also contains game-like elements (such as winning “streaks” and polls) to help motivate and engage students, and offers offline lessons for teachers to deliver face-to-face.
Control. Control classes in both schools used the same alternate educational technology program. This program focused on building reading comprehension skills. Students were presented with passages to read at their reading level and were provided with activities designed to improve their ability to comprehend the passages.

Reading Assessment

Students’ reading ability was assessed with three standardized tests. Each consisted of a fluency task designed to address different reading skills. There was a 3-minute time limit to complete each task. Classes were administered each test in the Fall (pretest) and the Spring (posttest).
TOSWRF2. The Test of Silent Word Fluency, Second Edition (Mather et al., 2014) was used to assess word identification skills. This test presents strings of unrelated words without spaces (e.g., strictdepthmuzzlefudgefickle) and students mark off as many distinct words.
TOSREC. The 6th grade version of the Test of Silent Reading Efficiency and Comprehension (Wagner et al., 2010) was used to assess basic reading comprehension and word identification skills. This test consists of true or false sentences (e.g., “If you cannot hear you may need to wear goggles on your forehead.”). Students mark off “Yes” or “No” to indicate whether each sentence is true.
TOSCRF2. The Test of Silent Contextual Reading Fluency, Second Edition (Hammill et al., 2014) was the most complex task and was used to assess multiple skills (word identification, syntactic processing, and basic reading comprehension). The TOSCRF2 includes 17 passages, each containing a series of words presented without spaces. For example:
THEBOYSWENTINTOASTOREANDLOOKEDATBOOKSABOUT ANIMALSONEBOYSAWABOOKABOUTHORSES
Students separate the string into distinct words that render a coherent reading of the passage.

Results

An ANCOVA showed that there was no significant difference in gains (posttest – pretest) on the TOSWRF2. The mean gains were 11.1 and 8.1 points for the treatment and control groups, respectively.

Likewise, there was no significant difference in gains on the TOSREC. The mean gains were -1.0 and 2.3 points for the treatment and control groups, respectively.
However, an ANCOVA revealed a significant difference on the TOSCRF2 (F(1,101) = 12.311, p = .001, partial η2 = .109, Cohen’s d = 0.69). The treatment group showed a mean gain of 26.7 compared to -2.5 for the control group. Thus, on the most complex reading fluency task, students using PowerUp displayed larger gains than the control group.

Importance

In this cluster randomized control study we found that non-proficient readers who used PowerUp in middle school showed greater gains on the TOSCRF2 – a fluency task that requires multiple reading skills – than control students using an alternative program. The obtained effect size (Cohen’s d = 0.69) is more than 2.5 times larger than estimates of the median effect size (.26) on assessments of discrete skills typically found in interventions applied in middle school (Lipsey et al., 2012).

The finding that treatment students showed the greatest benefits on the most complex reading fluency task dovetails well with use of PowerUp. Success on the TOSCRF2 requires strong word identification skills, coupled with syntactic processing and basic reading comprehension skills. These are, in fact, the areas addressed in PowerUp.

Unlike outcomes on the TOSCRF2, gains on the TOSWRF2 and TOSREC did not differ across groups, which is somewhat unexpected given that PowerUp also provided instruction in the skill areas covered by these tasks, word identification and basic reading comprehension. These latter tasks are less complex and tap into fewer skill areas than the TOSCRF2. The alternative program used in the control classes turned out to be indistinguishable from PowerUp in terms of its ability to support word identification and basic reading comprehension. The finding on the TOSREC is easier to interpret: Both PowerUp and the alternative program had an emphasis on reading comprehension, thus both programs may have equally prepared students for this test of basic reading comprehension. However, the finding on the TOSWRF2 is more surprising, given that – unlike PowerUp – the control program did not provide explicit instruction in word identification skills. Control students may have gained enhanced word identification skills implicitly through reading texts, even in the absence of explicit instruction.

Despite the null outcomes, it is nonetheless remarkable that PowerUp had a positive impact on the TOSCRF2, the most complex reading measure used in this study. This finding is consistent with other studies showing that educational technology can be beneficial for non-proficient readers in middle school (e.g., Kim et al., 2011; Lenhard et al., 2013). To achieve academic success, students in middle school must be able to master complex texts, and deficiencies in any of the areas tied to word identification and language processing need to be addressed (Nippold, 2017). A comprehensive program like PowerUp built on established reading science and pedagogy has the potential to be part of the solution.

References

Chen, M. (1980). Television, science, and children: Formative evaluation for 3-2-1 Contact. Journal of Educational Technology Systems, 9, 261-276.

Cheung, A. C., & Slavin, R. E. (2011). How features of educational technology applications affect student reading outcomes: A meta-analysis. Educational Research Review, 7, 198-215.

Gough, P. B. & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7, 6-10.

Hammill, D. D., Wiederholt, L. J., & Allen, E. A. (2014). Test of Silent Contextual Reading Fluency (TOSCRF-2) (Second ed.). Austin, TX: PRO-ED.

Hirsh-Pasek, K., Zosh, J. M., Golinkoff, R. M., Gray, J. H., Robb, M. B., & Kaufman, J. (2015). Putting education in “educational” apps: Lessons from the science of learning. Psychological Science in the Public Interest, 16, 3-34. doi:10.1177/1529100615569721

Kim, A. H., Vaughn, S., Klingner, J. K., Woodruff, A. L., Klein Reutebuch, C., & Kouzekanani, K. (2006). Improving the reading comprehension of middle school students with disabilities through computer-assisted collaborative strategic reading. Remedial and special education, 27, 235-249.

Kim, J. S., Capotosto, L., Hartry, A., & Fitzgerald, R. (2011). Can a mixed-method literacy intervention improve the reading achievement of low-performing elementary school students in an after-school program?: Results from a randomized controlled trial of READ 180 enterprise. Educational Evaluation and Policy Analysis, 33, 183-201.

Lenhard, W., Baier, H., Endlich, D., Schneider, W., & Hoffmann, J. (2013). Rethinking strategy instruction: Direct reading strategy instruction versus computer-based guided practice. Journal of Research in Reading, 36, 223-240.

Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M.
W., . . . Busick, M. D. (2012). Translating the statistical representation of
the effects of education interventions into more readily interpretable forms
(NCSER 2013-3000). Washington, DC: National Center for Special
Education Research, Institute of Educational Sciences, US Department of
Education.

Mather, N., Hammill, D. D., Allen, E. A., & Roberts, R. (2014). Test of Silent Word Reading Fluency (TOSWRF-2) (Second ed.). Austin, TX: PRO-ED.

National Assessment of Educational Progress (NAEP). (2019). NAEP
report card: Reading. Retrieved from
https://www.nationsreportcard.gov/reading/nation/achievement/?grade=8

Nippold, M. A. (2017). Reading comprehension deficits in adolescents:
Addressing underlying language abilities. Language, Speech, and Hearing
Services in Schools, 48, 125-131. doi:10.1044/2016_LSHSS-16-0048

Pace, J. R., & Mellard, D. F. (2016). Reading achievement and reading
efficacy changes for middle school students with disabilities through
blended learning instruction. Journal of Special Education Technology, 31,
156-169. doi:10.1177/0162643416660837

Schiefele, U., Schaffner, E., Möller, J., & Wigfield, A. (2012). Dimensions of reading motivation and their relation to reading behavior and competence. Reading Research Quarterly, 47, 427-463. doi:10.1002/RRQ.030

Snow, C., Burns, S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in young children. Washington, DC: National Academy Press.

Strong, G. K., Torgerson, C. J., Torgerson, D., & Hulme, C. (2011). A systematic meta‐analytic review of evidence for the effectiveness of the Fast ForWord language intervention program. Journal of Child Psychology and Psychiatry, 52, 224-235.

Wagner, R. K., Torgesen, J. k., Rashotte, C. A., & Pearson, N. A. (2010). Test of Silent Reading Efficiency and Comprehension (TOSREC) (Second ed.). Austin, TX: PRO-ED.