Table5: Literacy App Evaluation Tool for Teachers
Location: W179a, Table 5
Listen and learn : Research paper
Monday, June 25, 10:00–11:00 am
Location: W179a, Table 5
Dr. Peggy Lisenbee
This session provides an overview of a site for teachers to collaborate assessing FREE literacy apps. Apps are constantly changing making it difficult to remain current on authentic, effective literacy apps for use in PreK-2nd grade classrooms. An evaluation tool will be shared for collaborative use beyond the conference.
|Attendee devices:||Devices useful|
|Attendee device specification:||Smartphone: Windows, Android, iOS
Laptop: Chromebook, Mac, PC
Tablet: Android, iOS, Windows
|Focus:||Digital age teaching & learning|
|Topic:||Online tools, apps and resources|
|Subject area:||Language arts|
|ISTE Standards:||For Coaches:
Teaching, Learning and Assessments
Digital Age Learning Culture
Changes in technology in education and society in the 21st Century have been vast. IPads were the first tablet unveiled for use in January 2010. It was described as having “most of the capabilities of a desktop or laptop computer, but with the additional unique affordances, such as a multi-touch screen….” (p. 15), making the iPad distinctive from other technological devices at that time (Hutchison et al., 2012). Since 2010 there has been an exponential proliferation of tablets and apps. Tablet use in classrooms has the potential to emerge as the foremost technological tool used in teaching and learning (Cahill & McGill-Franzen, 2013; Hutchison et al., 2012; Karchmer-Klein & Harlow-Shinas, 2012; Mallette & Barone, 2014; Neumann & Neumann, 2014; Shuler, Levin & Ree, 2012; Smith, 2012; Thoermer & Williams, 2012; Yokota & Teale, 2014). Tablets continue to be the prominent mobile device preferred for use in education due to the multitude of apps specifically designed for educational applications (Hutchison et al., 2012; Pressman & Pietrzyk, 2014; Smith, 2012).
The Secretary’s Conference on Educational Technology (2000) supports the notion that curricular integration with technology increases personal motivational outcomes and student achievement as well as prepares students for the 21st Century (McComb, B. L). Although the National Reading Panel Report (2000) is over a decade old, it still provides the basis for best practices in literacy instruction. The National Reading Panel (2000) reported technology “can be used to deliver a variety of types of reading instruction successfully” (p. 6-9). In 2009, the International Reading Association’s (IRA) position statement specified for teachers to remain abreast of “best practices for using technology in instruction to enhance students’ literacy learning” (p. 4). Additionally, The International Society of Technology in Education (ISTE) Standards for Teachers (2008) provide a framework for effectively using digital tools in classrooms as a way to remain connected to an increasingly technological society. All of these specialized educational organizations are promoting an effort at preparing students with 21st century skills balanced with pedagogical practices. Apps identified and evaluated by teachers for targeted literacy practice by students creates a rich and varied learning environment supporting pedagogical practices in the classroom.
I conducted reviews of current methods for designing apps and teaching literacy skills. A review of app design features offered a framework for evaluating quality literacy apps with effective learning processes (Neumann and Neumann, p. 234). Neumann and Neumann (2014) recommended nine app design features. When these nine app design features were combined with pedagogical best practices, they provided a framework for identifying and evaluating quality literacy apps.
In reviewing pedagogical best practices for scaffolding students’ understanding of literacy skills, I reviewed the National Reading Panel Report (2000) and the Phonics Developmental Continuum from the Literacy First Framework (n.d.). The National Reading Panel Report is well recognized for sharing research stating the need to practice phonemic awareness skills prior to phonics instruction and to offer immediate and corrective feedback to improve literacy skills during independent practice. The Phonics Developmental Continuum from the Literacy First Framework (n.d.) is used by the school district of the second grade teachers cooperating in my research. I created a literacy skills checklist to use on my evaluation tool to merge the phonemic awareness guidelines suggested by the National Reading Panel Report (2000) with the Phonics Developmental Continuum of literacy skills (Literacy First, n.d.). Merging the two continuums into one checklist provides a greater depth and breadth of literacy skills needed for reading instruction. The phonological skills checklist is included in the evaluation tool I created to offer teachers the ability to not only rate each apps’ pedagogical skills and app design features, but more importantly, to specify which phonological skills are offered for practice within each app.
The evaluation tool created used in my research was titled, Phonemic Awareness & Phonics Evaluation Rubric (PAPER). The framework for PAPER was created by aligning criteria derived from common terms, foci and elements of effective pedagogy and app design into six criteria to use for evaluating quality apps. The checklist of phonological skills, effective pedagogical practices and app design features were aligned to merge together into six descriptive criterions evaluated in PAPER. The definitions of the six criterions, Literacy Skills, Engaging, Immediate Feedback, Flexibility & Adaptations, Independence with Technology and Remained Focused on Literacy, are provided below:
1. Literacy Skills—Phonemic awareness and/or phonics skills offered for practice in apps are listed as specific skills so literacy practice can be selected based on student need. The phonemic awareness and phonics skills listed are requisite for becoming a fluent reader.
2. Engaging—Literacy skills practiced in apps are offered in an authentic manner for the context of the learning environment in each app. Tasks are appropriate for the learning environment and are able to keep students engaged in practicing the literacy skills.
3. Immediate Feedback—Feedback provided in apps is specific enough for students to confirm their understanding of the literacy skill(s) or respond again before moving on in order to correct any mistake in their understanding of the literacy skill(s).
4. Flexibility and Adaptations—Students interact within apps by making their own choices to discover literacy skills instead being forced to use prescribed options. An option for collaborating with peers is customizable in the settings of apps to provide adaptations needed for individual students.
5. Independence with Technology—Students launch and navigate through apps independently without assistance or frustration.
6. Remained Focused on Literacy—Students remain focused on practicing literacy skill(s). They did not have to avoid multiple ads/links requesting them to consider upgrading to a more complete version of the game.
PAPER includes germane information for teachers such as the date of evaluating each app and the initial cost of each app. PAPER provides a method for teachers to efficiently and effectively identify, select and evaluate quality literacy apps for use on a tablet in classrooms. Lastly, PAPER provides an opportunity for teachers to collaborate as they identify and evaluate apps to create and share a master list of quality apps.
Finally, utilizing PAPER, a systematic content analysis of literacy apps available in the iTunes App Store was conducted. The initial search was conducted by searching the Education category using the terms, Phonics and Phonemic Awareness. An advanced search of the identified apps was completed by searching for free apps. Some of these apps only offered limited literacy practice before distracting students with ads or requests to upgrade so they were thrown out of the research pool of apps for further evaluation. Content analysis continued until 10 free apps marketed in the iTunes App Store as a phonemic awareness app and 10 free apps marketed in the iTunes App Store as a phonics app was selected for evaluation using PAPER. The 20 literacy apps analyzed were: Professor Phonics, Hear 2 Read Lite, ABC Magic, Fun Rhyming, Letter Sounds, ABC Magic 6, Phonics Island, Literacy, Reading 1, Rhyming Bee, Reading 2, Phonics Genius, Phonics Awareness, Phonics Vowels, Literacy, Sky Fish, Express, Tic Tac Toe, ABC Genius and Little Speller. A screen shot of the app icons separated into groups of 10 phonemic awareness apps and 10 phonics apps will be provided.
Apps were evaluated using a four point Likert scale. The six criterions evaluated using the four point Likert scale in PAPER are Literacy Skills, Engaging, Immediate Feedback, Flexibility & Adaptations, Independence with Technology and Remained Focused on Literacy. These six criterions evaluate pedagogical best practices and app design features to determine the quality of each literacy app. The possible scores using PAPER ranged from six to 24. Initially, the author completed evaluations on each of the 20 apps using PAPER. The seven second grade teachers involved in this research were asked to rate apps using PAPER to determine inter-rater reliability. Inter-rater reliability for evaluating apps among the seven teachers and the author was 77%.
Analysis of PAPER scores was based on percentages associated with 24 total possible points. Exemplary literacy apps were evaluated in the 24-22 range signifying scores in the 90-100% range. Excellent literacy app scores were evaluated in the 21-20 range exemplifying scores in the 80-90% range. Finally, literacy apps scoring in the 19-17 range were evaluated as adequate literacy apps since they are equal to 70-80% of the total points possible. Any literacy app scoring at 70% or above which includes adequate, excellent and exemplary apps according to PAPER would be considered quality literacy apps for this researcher’s purpose. All apps scoring below 16 on PAPER were considered as needing improvement or not considered a quality app since they fall under 70% of the 24 points possible on PAPER.
PAPER evaluations were conducted on all 20 apps identified from the iTunes App Store search. Two phonemic awareness apps and one phonics app scored in the exemplary range on PAPER which included scores ranging from 24-22. The two exemplary phonemic awareness apps were Reading 1 and Professor Phonics. The one phonics app evaluated as exemplary was Sky Fish.
Rhyming Bee and Literacy apps were evaluated as excellent phonemic awareness apps. The one phonics app evaluated as excellent using PAPER was Build A Word Express. These three excellent literacy apps, according to PAPER, each earned a score of 21.
The four phonemic awareness apps evaluated as adequate literacy apps were ABC Magic 6, Phonics Island, Hear 2 Read Lite and Fun Rhyming. Reading Magic 2, ABC Genius and Little Speller apps were scored as adequate phonics apps. These seven apps were evaluated as adequate literacy apps with scores ranging from 19-17.
The final two phonemic awareness apps and five phonics apps evaluated as needing improvement or not a quality app were literacy apps scoring below 16 on PAPER. The two phonemic awareness apps were ABC Magic and Letter Sounds. The five phonics apps were Phonics Vowels, Tic Tac Toe, Phonics Genius, Phonics Awareness and Literacy! with scores ranging from 16-14.
One hundred and thirty second grade students were offered time to play any of the 20 literacy apps on tablets in their classrooms prior to being assigned apps based on their specific reading skill needs. In a free choice scenario, the second grade students were asked to share their favorite literacy apps. The favorite phonemic awareness apps shared by students were Professor Phonics, Hear 2 Read Lite, ABC Magic 6, Phonics Island, Literacy, Reading 1 and Rhyming Bee.
Seven of the eight phonemic awareness apps evaluated by PAPER as a quality app, Reading 1, Professor Phonics, Rhyming Bee, Literacy, Phonics Island, ABC Magic 6 and Hear 2 Read Lite were rated by students as a favorite app also. Two of five phonics apps, Sky Fish and Little Speller, were chosen by both PAPER and students as quality apps. Some of the descriptive comments made by students about their favorite apps were “I can make the train move”, “It’s fun”, “It has stickers”, “You get to change his clothes” and “We get to choose our own character”. The student comments seem to align with four criterions in PAPER: Engaging, Flexibility & Adaptations, Independence with Technology and Remained Focused on Literacy.
The favorite phonics apps shared by second grade students were Phonics Genius, Sky Fish, Tic Tac Toe and Little Speller. The PAPER scores for these favorite phonics apps ranged from 15-24 which varied in their evaluations from apps needing improvement to exemplary apps. Sky Fish earned the maximum score on PAPER of 24 as well as being voted as a favorite app by students. These four phonics apps all scored the highest in the Engaging criteria on PAPER which evaluates engagement in apps. It seems that for phonics apps, the versatility in favorite apps chosen by students might be related to a student’s ability to decode and apply phoneme grapheme relationships to the activities embedded in the phonics apps even if they are still learning to read well. Whereas, phonemic awareness apps considered as favorites by students align with PAPER ratings for quality apps. Students still learning phoneme grapheme relationships need focused reading practice and immediate feedback to effectively engage in the activities within phonemic awareness apps. Students with a firm foundation of phonemic awareness skills seem to allow students the ability to enjoy any phonics app even if the app isn’t rated by PAPER as a quality app.
A teacher offered her perspective after letting students use tablets for literacy practice, “I learned how valuable technology can be!” and “…I realized how important it is to take advantage of technology in my own classroom to allow students more choices....” Another teacher commented on using PAPER stating, “I think the rubric looks really good.” and “…it will be a useful tool moving forward”. Teachers shared their pleasure in observing students focused on practicing literacy skills while having “fun (and learning)!” Appreciative comments regarding the list of 20 phonemic awareness and phonics apps were offered especially since these teachers plan to use PAPER to continue identifying and evaluating more apps for use in their classrooms next year.
Analysis of literacy content within the 20 apps revealed most apps focused on a range of literacy skills for practice instead of offering practice on isolated literacy skills. Isolated (I) literacy skills were defined as apps offering one or two literacy skills for practice in an app. Apps which provided practice on three or more literacy skills were considered as offering a range (R) of literacy skills. “It is important to select electronic programs that focus on one or two types of phoneme manipulation at a time. Young children are confused if they are exposed to more…” (Anderson et al., 2008, p. 63). Analysis revealed phonemic awareness apps were focused on isolated, specific, literacy skills in four out of 10 apps while phonics apps only focused on isolated, specific, literacy skills in one app.
Stacked bar graphs offer a visual display of the literacy skills available for practice throughout all 20 apps. Two phonemic awareness skills targeted for practice 90% of the time in all 20 apps were initial sounds and consonants. Short vowels was the only other literacy skill commonly practiced in five of the 10 phonemic awareness apps and nine of the 10 phonics apps. These letter knowledge and phoneme isolation skills are beginning phonological skills.
Literacy skills analyzed as being offered for practice more than 50% of the time in all 20 apps include these seven phonological skills: initial sounds, final sounds, consonants, short vowels, long vowels, CVC patterns and onset & rime. These seven literacy skills represent common phonological skills typically projected for mastery in kindergarten and first grade. (Fountas & Pinnell, 2003; Literacy First, n.d.).
In contrast, literacy skills analyzed as being offered only 15-45% of the time in all 20 apps were literacy skills mastered typically by the end of 1st grade or the beginning of 2nd grade. These phonics skills included digraphs, blends, diphthongs, r & l-controlled vowels and syllabication which assist students in becoming proficient readers by the end of 3rd grade (Fountas & Pinnell, 2003; Literacy First, n.d.). Although opportunities to practice these higher level phonics skills in apps are valued, the focus in apps appears to be on kindergarten and first grade level phonemic awareness and phonics skills.
A cursory examination of the names of phonemic awareness apps illustrates a misleading characteristic of literacy apps. The names of some phonemic awareness apps such as Reading 1, Professor Phonics, and Literacy imply that these might be apps offering phonics skill practice instead of practice on phonemic awareness skills. Additionally, there is a phonemic awareness and a phonics app found for this research marketed under the name of Literacy. The name of an app does not appear to be helpful identifying quality literacy apps, but the evaluations by PAPER seem to provide helpful data when focusing on literacy skills and students ability to utilize tablets independently.
The User Friendliness criteria scored the highest overall as evaluated by PAPER for all literacy apps. This makes sense since students need to be confident navigating tablets even if the apps do not offer options for students. It would be ideal for students to make individual choices as they construct their understanding of a literacy skill, but more importantly, students need to be able to launch and navigate through an app without frustration in order to learn from the experience. For all phonemic awareness and phonics apps evaluated by PAPER, the one criterion which scored the lowest was Flexibility and Adaptations. Apps offering the ability to individualize for each student beyond focusing on a specific, discrete, literacy skill was not found in the 20 apps identified for this research. Yet, creating apps offering the ability to meet individual student needs was considered a quality app design feature by Neumann and Neumann (2014).
When assessing the reliability of PAPER to evaluate quality apps, the results found similarities between student’s choice of favorite apps and the apps identified as quality apps from the PAPER evaluation scores. Student comments on their favorite literacy apps related to the apps evaluated with excellent or higher scores from PAPER. Finally, triangulation of data among teacher comments of apps used in their classrooms, scores on PAPER and student favorite apps suggests reliability among the evaluations to identify quality literacy apps. This research supports the need for teachers to have an evaluation rubric to identify quality apps focused on specific phonemic awareness or phonics skills instead of relying on the name of an app to select literacy apps for use by students in a classroom.
Overall, it seems that searching for literacy apps using the term “Phonics” selects apps with a broader range of literacy skills for practice while searching for apps using the term “Phonemic Awareness” identifies apps with more isolated literacy skills. Generally, it seemed that these two terms were effective in searching for literacy apps to use on tablets in a classroom to support students practicing literacy skills. Tablets are in their infancy for use in educational settings, but “apps are rapidly emerging as a new medium for providing educational content to children nationwide, both in terms of their availability and popularity” (Schuler et al., 2012, p. 26). Students need to practice computer skills daily so their foundation for successfully navigating a tablet is not impeded by their lack of computer skills. Even though most teachers don’t have enough tablets in their classrooms for 1:1 use by students on a daily basis, teachers can use document cameras connected to interactive whiteboards to display the tablet screen. This type of visual display can be utilized each time a new app is added to the classroom tablet for literacy skill practice in order to model navigating within each app prior to offering students independent use on the tablet.
Embedded ads encouraging in-app purchases, links taking students to unknown sites on the Internet, and time to evaluate the breadth of literacy apps are other challenges causing teachers difficulty integrating tablet use in their classroom (Schuler et al., 2012; Yokota & Teale, 2014). The cost of apps can be a challenge for selecting a variety of apps for use in a classroom setting. In 2012, the average price of children’s apps ranged from $0.99 to $1.99 (Schuler et al., 2012, p. 16). Yet, there were numerous free apps, and free for a trial period apps, aligned with best pedagogical practices as evaluated by PAPER in this research.
Since teachers are, inherently, constrained by time it is a double edged sword as teachers attempt to embrace using tablets in their classrooms. Time constraints overwhelm many teachers causing them to choose a limited number of apps for all students to use when practicing literacy skills on a tablet. PAPER provides a method for teachers to identify and evaluate quality apps and provide differentiation for students’ literacy skill needs. Apps proliferate so quickly which is why PAPER includes the date of each evaluation as well as the cost of each app offering the option to re-evaluate an app if the data is not current. All criteria evaluated in PAPER provide valuable information to effectively evaluate quality literacy apps for use in a classroom.
Teachers need to embrace the rapidly changing educational focus to use technology in their classrooms as a way to incorporate digital literacies into their pedagogy. “…teachers must recognize the new literacy demands of the 21st Century and, most important, must transform their programs to meet these demands with timely literacy instruction” (Karchmer-Klein & Harlow Shinas, 2012, p. 289). Teachers finding and utilizing literacy apps to practice literacy skills ensures students become proficient in skills and practices needed for the 21st Century. Research on tablets suggests high quality apps can improve literacy skills (Hutchinson et al., 2012; Neumann & Neumann, 2014; Shuler et al., 2012).
These implications support the use of an evaluation tool, such as PAPER, to focus simultaneously on app design, literacy, and pedagogy as teachers identify and evaluate literacy apps. PAPER provides a method to assess literacy apps beyond cost and marketing, but by evaluating app design features and developmentally appropriate practices which support the use of tablets as a pedagogical tool used in classrooms.
Lastly, teachers need to find a way to decrease the duplication of effort while maximizing time evaluating a wider variety of literacy apps for use in a classroom. Using the PAPER, teachers can synergistically evaluate apps to create a master list describing the specific literacy skills practiced within each app. Evaluating apps and disseminating a list of quality apps focused on isolated literacy skills not only strengthens support for students using tablets in a classroom, but can be powerful in strengthening the student’s literacy skills as well. PAPER provides support for selecting high-quality, interactive, differentiated literacy experiences for students while using an engaging technological tool.
Anderson, R. S., Grant, M. M. & Speck, B. W. (2008). Technology to teach literacy: A resource for k-8 teachers. Upper Saddle River, NJ: Pearson/Prentice Hall.
Cahill, M. & McGill-Franzen, A. (2013). Selecting “app”ealing and “app”ropriate book apps for beginning readers. The Reading Teacher, 67(1), 30-39. doi:10.1002/TRTR.1190
Cavanaugh, T. W. (2007). Technology and phonological awareness. Florida Reading Quarterly, 43(3), 26-30.
Darrow, D. (2011, Sept 22). K-5 iPad apps for evaluating (bloom’s revised taxonomy, part 5). Retrieved from http://www.edutopia.org/blog/ipad-apps-elementary-blooms-taxomony-evaluating-evaluation-diane-darrow
Fountas, I. & Pinnell, G. S. (2003). The word study continuum. Retrieved from http://assets.pearsonschool.com/asset_mgr/current/201228/word_study_continuum.pdf
Hutchison, A., Beschorner, B., & Schmidt-Crawford, D. (2012). Exploring the use of the ipad for literacy learning, The Reading Teacher, 66(1), 15-23. doi:10.1002/TRTR.01090
International Reading Association Position Statement. (2009). New Literacies and 21st-Century Technologies. Newark, Delaware: International Reading Association.
International Society for Technology in Education. (2008). ISTE standards teacher. Retrieved from http://www.iste.org/docs/pdfs/20-14_ISTE_Standards-T_PDF.pdf
Karchmer-Klein, R. & Harlow Shinas, V. (2012). Guiding principles for supporting new literacies in your classroom. The Reading Teacher, 65(5), 288-293. doi:10.1002/TRTR.01044
Literacy First (n.d.). Literacy first phonological awareness skills continuum and phonics skills continuum. Retrieved from
Mallette, M.H. & Barone, D. (2014). Interesting ways to use ipads in the classroom. The Reading Teacher, 67(8), 621-625. doi:10.1002/trtr.1264
Marsh, J. (2009). Digital beginnings: Young children’s use of popular culture, media and new technologies in homes and early years settings. In A.G. Bus & S.B. Neuman (Eds.), Multimedia and literacy development: Improving achievement for young learners. (pp. 28-43), New York, NY: Routledge.
McCombs, B. L. (2000). Assessing the role of educational technology in the teaching and learning process: A learner-centered perspective. Paper presented at the Secretary’s Conference on Educational Technology 2000, Washington, DC. Abstract retrieved from http://www2.ed.gov/rschstat/eval/tech/techconf00/mccombs_paper.html
Monaghan, C. (2015, Jan 8). App store rings in 2015 with new records. Retrieved from http://www.apple.com/pr/library/2015/01/08App-Store-Rings-in-2015-with-New-Records.html
National Reading Panel Report (2000). Teaching children to read. Washington, DC: National Institute of Child Health and Development.
Neumann, M. M. & Neumann, D. L. (2014). Touch screen tablets and emergent literacy. Early Childhood Education Journal, 42, 231-239.
Pressman, H. & Pietrzyk, A. (2014). Literacy apps for struggling learners. Retrieved from www.education.nh.gov/instruction/special_ed/documents/apps_early_literacy.pdf
Ritchie, R. (2014, Oct 6). History of ipad (original): Apple makes the tablet magical and revolutionary. Retrieved from http:/www.imore.com/history-ipad-2010
Roskos, K., & Neuman, S. B. (2014). Best practices in reading: A 21st century skill update.
Schuler, C., Levine, Z. & Ree, J. (2012). iLearn II: An analysis of the education category of apples’ app store. New York: The Joan Ganz Cooney Center at Sesame Workshop.
Smith, J. (2012, March 25). iLit: Using ipads for multi-sensory literacy development intervention. Retrieved from http://cdn-media1.teachertube.com/doc603/5791.pdf
Statista, Inc. (2015). Most popular apple app store categories in january 2015, by share of available apps. Retrieved from http://www.statista.com/statistics/270291/popular-categories-in-the-app-store/
Thoermer, A. & Williams, L. (2012). Using digital texts to promote fluent reading. The Reading Teacher, 65(7), 441-445. doi:10.1002/TRTR.01065
Van Daal, V. H. P. (2009). Struggling readers and multimedia. In A. G. Bus & S. B. Neuman (Eds.), Multimedia and literacy development: Improving achievement for young learners. (pp. 124-134), New York, NY: Routledge.
Walker, H. (2011). Evaluating the effectiveness of apps for mobile devices. Journal of Special Education Technology, 26(4), 59-63.
Yokota, J. & Teale, W.H. (2014). Picture books and the digital world. The Reading Teacher, 67(8), 577-585. doi:10.1002/trtr.1262