Purpose & objective
The Digital Early Literacy (DEL) project provided exposure to English vocabulary and immersive literacy experiences to kindergarten through second grade students using content specific applications (apps) delivered on tablet technology (iPads). In the first and second years (2011-12 and 2012-13), participating teachers reported on how they used the iPads in their teaching practice, types of learning activities conducted, challenges to use, and observations of the impact on student learning and parental involvement. In the second year we explored the impact on parent. In the third year, the long term effect on literacy skills was evaluated. Over the course of this study, classrooms in Napa Valley Unified, St. Helena Unified School District and Calistoga Joint Unified School District participated. The purpose of this report is to provide a preliminary summary of data, teacher perceptions and student achievement during the 2011-2014 school years. The study will conclude in August 2015 when the final scores for the last group of students will be collected.
2015 Preliminary Data from Calistoga:
• 11 teachers implemented the program with 150 students’ performances analyzed in Kindergarten, First and Second grades.
• 44 students formed the control/comparison group drawn from the prior year 2010-11 Kindergarten students who had no access to iPads.
• 106 students comprised two treatment groups: 2011-12 who used iPads in a 1:1 format and 2012-13 who shared iPads between students.
• English language proficiency plus Reading, Spelling and Writing scores formed the data set used in comparisons.
• By the end of 1st grade, 1:1 iPad users English language skills grew at a rate that was 55% greater than the control group despite starting at a 33% lower English proficiency rate at the end of kindergarten
• By the end of 1st grade, shared iPad users English language grew at a rate that was 47% greater than the control group despite a 15% lower English proficiency at the end of kindergarten.
• By the end of 1st grade iPad users performed between 17 and 20% better on reading fluency tests, 11-19% better on spelling and 2-7% better on writing than the control group.
• By 2nd grade all student growth in literacy decelerated but the 1:1 iPad users were within .39 points of meeting the target score. Their performance was only 2% lower than the control group.
• a strong correlation was found between the use of iPads and growth in literacy skills – especially receptive language.
• English Learners demonstrated significant benefit from the digital early literacy program, however, the average of all participating students’ growth was greater than the traditionally taught control/comparison group.
• Participants’ comprehension grew by 29% over the year and they were exposed to an average of 25,000 words in the first year of implementation of the program.
• Through ipad use teachers were better able to differentiate instruction and provide remediation and individualized learning to students.
• Participating teachers believe the iPads have become an important part of the program and that they are better able to support their students’ literacy development.
• While there was no specific goal set for observing changes in collateral learning students were observed to possess noticeably improved self-management, confidence and communication abilities.
• Increased teacher collaboration was an unintended positive effect of the program.
Additional Conclusions and Recommendations
• The iPad use treatment had a significant effect on student growth in receptive language, English language proficiency and literacy performance..
• EL students demonstrated a significant improvement on the PPVT Growth Score Value measure of receptive language in 2011-12 data collection. Students will be re-tested at the end of 2nd grade in 2015.
• Teachers observed remarkable growth in iPad using students’ self-esteem, willingness to take risks, ability to self-monitor their needs and performance, and confidence in speaking to others.
• The project should be repeated with a larger treatment and control group.
• The methodology of the iPad use and duration of the study need to be more regulated.
• Teachers requested both online and face-to-face collaboration opportunities.
• Professional development for teachers should be expanded to support implementation and expansion of the program.
• Parents should be more involved in the program to learn how to support literacy skills.
• Technical support and infrastructure reliability needs improvement.
• Data collection and alignment would benefit from routinization and greater management oversight.
The Digital Early Literacy (DEL) program provided literacy enrichment services to kindergarten, first and second grade students in Napa County including low-income and English learners. To enable access to experiences with key literacy skills, the DEL program supported teachers’ and students’ use of iPads and applications (apps) intended for exploration and mastery of emergent literacy skills. In the initial year of evaluation of the program, it was determined that the most significant demographic factor for students who demonstrated accelerated performance was that of whether they were an English Learner. Evaluation measures tracked the effect of the DEL program on student performance (reading, spelling and writing) and English Language improvement. Changes in teachers’ practice and potential impact on parents were both tracked in the first two years of evaluation. Extrinsic factors to success, such as Internet access, were also evaluated.
The purpose of this report is to present preliminary findings from three years of program implementation including student performance from 2011 to 2014 along with program perceptions, and goal achievement. Evaluation data will be shared with project staff, stakeholders, and NapaLearns board members to identify successes, challenges, and lessons learned in order to inform planning efforts for future investment and support. First, this report describes the evaluation methodology, including the data collection instruments, procedures, and analyses utilized. Next, the report presents summative evaluation results. Lastly, the report presents conclusions and recommendations for the future.
The Table below summarizes the initial goals of the Digital Early Literacy program and the year in which the goals were met to provide history for this report
Goals and Objectives
1. Students: Date goal was met
A. Students learn to use iPad technology and appropriate apps in the classroom for learning. October 2012
B. Expose students to high frequency words known to contribute to academic success. October 2012 via Footsteps to Brilliance and teacher selected Apps
October 2013 via Teacher selected Apps
C. Provide exposure to the written word and evaluate comprehension. 2011-12 via Footsteps to Brilliance
2012-14 via Teacher selected Apps
D. Increase students’ receptive vocabulary. 2011-12 via Footsteps to Brilliance
2012-15 via Teacher selected Apps
A. Provide training to teachers on the use of iPad and appropriate apps in the kindergarten classroom October 2012
B. Determine effective practices and methodologies for using the iPad in the classroom. October 2012
C. Identify iPad applications appropriate for use in the classroom and observe perceived effect on student performance in literacy tasks. October 2012
D. Note student behaviors and attitudes in DEL participating students. October 2012
3. Impact of Work:
A. Increase parental involvement in literacy activities both at home and at school. October 2013
B. Increase the percentage of kindergarten teachers who use technology and the Internet to support their teaching practice. October 2013
C. Identify support needs for classroom use of iPads. October 2013
2015 Data Collection Instruments, Procedures, and Analyses
Calistoga Literacy Test uses English literacy assessments develop by the Sacramento County Office of Education (SCOE) to measure performance in the different English Language Arts (ELA) strands: reading, speaking, writing and spelling. The assessments were developed by SCOE to provide ongoing formative and summative data to teachers about their ELA instruction and student achievement. The assessments specifically target recently taught skills from the Houghton Mifflin Reading Program, Calistoga’s district adopted curriculum.
California English Language Development Test (CELDT): Students in kindergarten through grade twelve whose home language is not English are required by law to be assessed in English language proficiency (ELP). In California, the ELP assessment is the CELDT. The test allows schools to identify students who need to improve their skills in listening, speaking, reading, and writing in English. Schools also give the test each year to students who are still learning English.
This section presents information on student performance over three years of implementing the Digital Early Literacy project.
Description of the Calistoga Sample Population
Table 1 below indicates performance scores collected for comparison in each research (treatment) group during their kindergarten, first and second grade experiences. Note, there were two treatment groups: One began kindergarten in fall 2011 and are called Treatment1 and the second began kindergarten in fall 2012 and are called Treatment2. Treatment1 group has the use of an iPad for every student and Treatment2 shared iPads between 2-3 students. This allowed the initial purchase of iPads to be shared with the first grade classrooms.
Table 1: Use of iPads and Types of Data Collected to Measure Student Progress
iPad Use 1st Grade iPad Use 2nd Grade iPad Use English Language Proficiency Reading, Writing Speilling
Control None None Shared iPads CELDT Literacy Test
Treatment1 1:1 iPads Shared iPads No iPads
Treatment 2 Shared iPads Shared iPads 1:1 iPads
Table 2: Years for Which Data Were Analyzed
GROUP Kindergarten First Grade Second Grade
Control 2010-11 2011-12 2012-13
Treatment1 2011-12 2012-13 2013-14
Treatment 2 2012-13 2013-14 *
*This report is preliminary to collection of the final dataset: End of 2nd grade scores for the Treatment 2 group will occur in June-July 2015.
A total of eleven classrooms participated in the Calistoga DEL project. While the total population at each grade level ranged between 45 and 66 during the school year, due to transiency of the school population the data sets collected were often much smaller. Students may have been absent on the day of examination, but most often they had moved out of the district. For this reason, only those students who began in the group in kindergarten (control or treatment) and were still enrolled at the end of second grade are reported herein.
Table 3: Calistoga Sample Population Demographics
N Gender Ethnicity English Learners Free/Reduced Meals
Male Female HIspanic Other EL Non-EL Eligible Not Eligible
Control 45% 55% 98% 2% 98% 2% * *
Treatment1 48%% 52% 100% 0% 100% 0% * *
Treatment2 # # 86% 14% 86% 14% * *
*These data were not found to be influencing factors. # These data are still being collected.
It is important to note that in the performance scores below, students who were in the “other” category for ethnicity are reported separately in the reading, spelling and writing scores. CELDT scores do not apply to these students.
Goal: Increase Student English Language Proficiency
The California English Language Development Test (CELDT) is administered each year to measure English Language performance in four areas: listening, reading, speaking and writing. Scores are reported for each area, then combined to create an overall proficiency score.
Table 4: Mean Kindergarten CELDT Scores for Calistoga Students from 2011-2013
Group N Listening Reading Speaking Writing Overall
Control 2011 44 2.32 2.16 2.39 2.64 2.27
Treatment1 2012 56 1.66 1.73 1.63 2.04 1.52
Treatment2 2013 50 2.20 2.14 1.80 2.20 1.92
It is important to note that the control group students were more proficient in English than either of the treatment groups when they exited Kindergarten. It is likely that this group of students were higher performing when they entered kindergarten. The data show that at the end of kindergarten the 2012 treatment1 group has a 33% lower performance on the overall CELDT score and the 2013 treatment2 group had a 15% lower CELDT performance. This factor should be taken into account when reviewing the literacy scores that follow.
Table 5: Mean First Grade CELDT Scores for Calistoga Students from 2012-2014
Group N Listening Reading Speaking Writing Overall
Control 2012 42 3.18 2.20 3.45 2.25 3.20
Treatment1 2013 55 3.18 2.24 3.28 2.00 3.02
Treatment2 2014 42 3.21 2.17 3.50 2.17 3.29
By the end of first grade, both treatment groups demonstrate that they acquired English language proficiency at a higher rate than the control group. The treatment1 group which was 33% behind the control group in kindergarten demonstrate performance that is 94% of the control group score by the end of 1st grade. Similarly, the treatment2 group which was 15% behind the control group at the end of kindergarten, surpassed the control group with a 3% higher performance.
By examining the change in scores between years, the rate of growth becomes evident. CELDT scores are measured on a scale of 5. Subtracting the kindergarten overall CELDT score from the first grade score reveals the following:
Table 6: First Grade Growth in Overall CELDT Scores from Kindergarten
Control Growth 0.93 Treatment1 Growth 1.45 55% greater than control
Treatment2 Growth 1.37 47% greater than control
These data demonstrate the following from the end of 1st grade scores:
• Treatment1 (1:1 iPad users) English language skills grew at a rate that was 55% greater than the control group despite starting at a 33% lower English proficiency rate at the end of kindergarten
• Treatment2 (shared iPad users) English language grew at a rate that was 47% greater than the control group despite a 15% lower English proficiency at the end of kindergarten.
Table 7: Mean Second Grade CELDT Scores for Calistoga Students from 2013-2015
Group N Listening Reading Speaking Writing Overall
Control 2013 44 3.32 2.09 3.89 2.43 2.98
Treatment1 2014 50 3.18 2.24 3.28 2.00 3.02
Treatment2 2015 0 * * * * *
* Test has not been given yet. Will be measured in June-July 2015.
The end of 2nd grade CELDT scores indicate that the Treatment1 group outperformed the control group by 1%. Treatment2 scores cannot be collected until June 2015.
Goal: Increase Literacy Performance
The California English Language Art curriculum addresses literacy development. The ELA test scores were use to assess student progress in literacy skills of reading, writing and spelling.
Table 8: Mean Kindergarten Literacy Test Scores for Calistoga Students from 2011-2013
ELA Test N Phonemes CVC Words Rhyming Words High Frequency Words
Target Score 6 to 8 8 to 10 4 to 5 8 to 10
Control 2011 44 5.93 7.75 2.36 9.02
Treatment1 2012 55 5.79 7.66 2.59 8.48
Treatment2 2013 64 5.73 7.13 2.88 8.23
Despite lower English language proficiencies, both treatment groups score closely to the control group on end of year English Language Arts (ELA) literacy tasks. Treatment1 score is 94% of control score and Treatment2 is 91% of control score.
Table 9: Mean First Grade Literacy Test Scores for Calistoga Students from 2012-2014
ELA Test N Reading Fluency` Difference
from Control Spelling Difference
from Control Writing Difference
Target Score 55 8 to 10 3-4
Control 2012 44 45.50 - 6.36 - 2.47 -
Treatment 2013 46-47 52.74 17% greater 7.54 19% greater 2.40 2% lower
Treatment2 2014 33-34 54.40 20% greater 7.06 11% greater 2.61 7% greater
Table 9 illustrates that by the end of first grade both treatment groups out-performed the control group in all areas except writing. The Treatment1 group had the lowest English Language CELDT score at the end of kindergarten. This group performed 2% lower than the control group in writing. The highest performance for both treatment groups was in the area of reading fluency with 17-20% greater performance.
Table 10: Mean Second Grade Literacy Scores for Calistoga Students from 2013-2015
ELA Test N Reading Fluency` Difference
from Control Spelling Difference
from Control Writing Difference
Target Score 90 8 to 10 3 to 4
Control 2012 38 91.70 - 7.05 - 2.43 -
Treatment 2013 41-26 89.62 2% lower 6.58 7% lower 2.30 5% lower
Treatment2 2014 33-34 * TBD * TBD * TBD
* Test has not been given yet. Will be measured in June-July 2015.
All second grade performance scores are not yet available. In June 2015 the Treatment2 scores will be collected an analyzed. Preliminary data show that all students demonstrate a deceleration in literacy skills performance. However, the Treatment1 group was within 2% of the control group scores and just .38 points below the target for Reading Fluency.
Goal: Teacher & Student Use of Technology to Support Learning
• 100% of teachers become prepared to use iPads and literacy apps through training activities. Objective was met in all classrooms. All teachers reported having learned to include iPad use in their daily teaching practice.
• 100% of teachers noted improved student behaviors and attitudes in DEL participating students.
• 75% percent of her colleagues felt the program helped their students a lot or somewhat with communication.
• Collaboration, thinking creatively, and demonstrating critical thinking (21st century skills) garnered 87.5% of the votes to some degree (a lot/somewhat/a little).
• All teachers (100%) felt the program helped students learn essential literacy skills a lot and 100% indicated that the program helped students improve their vocabulary, improve English speaking skills, and enjoy school more.
Formative and summative evidence demonstrated that the DEL project was a success and all program goals were met to some degree: By the end of first grade, iPad using students outperformed the traditionally taught students on a nationally normed test of receptive language (PPVT), English Language Arts assessments and the California English Language Student engagement was high for all students and the greatest benefit was seen with students who needed intervention. All iPad using students were observed to be more self-confident, independent, willing to take risks, and able to monitor their own learning.
Teachers struggled early in the year with implementation and use of the iPads. This was primarily due to infrastructure and technical support issues at each site, but also due to a lack of a common and comprehensive training program prior to launching the project. Nonetheless, the teachers successfully learned how to use the wireless network and navigated through an iPad-based curriculum by collaborating with each other to investigate appropriate apps and their “best” use in the curriculum and regular discussions about pedagogy and appropriate methods for students. The teachers overwhelmingly endorsed the use of iPads in the curriculum and saw benefit to all students in the affective domain and 21st century workforce skills. For students needing remediation, intervention and English language experience, the iPads were especially supportive of learning. Many teachers found the ability for each student to be working at their own pace and in programs appropriate especially to them was the key to differentiating instruction. This feature of the program allowed teachers to spend more individual time with all students. They were also able to use the iPads for whole class instruction, centers, group work, cross-grade collaborations, and pair/partnerships.
Parent involvement in the program varied at each site and ranged from a series of formal trainings to informal before school interest based meetings to workshops followed by a parent-child homework assignment. It was noted that parents who are illiterate need greater support to learn how to help their student be successful in school; some felt that all parents require encouragement to become and stay engaged in the program. About half the teachers believe they are better able to support parents after having implemented the iPad program and more than half believe they are better able to support students in their literacy development after having participated in the DEL program.
The following recommendations are suggested based on longitudinal qualitative and quantitative data findings in this report and prior reporting:
• Provide scope and sequence for iPad use: The “treatment” needs to be more evenly applied across all iPad using teachers and students. Usage and types of activities should be established and documented for evaluation purposes.
• Refine student access to apps: Where possible, all students in the iPad using group should have equal access to the technology tools.
• Provide teachers with clear guidelines and support materials: Teachers and the administration in collaboration with NapaLearns should consider developing a set of “agreements” with regard to implementation of the program.
• Align assessment tool administration: While different districts may use different tools to assess ELA progress, each tool has common elements in their measurement. Creating an alignment of these tools will allow for cross-district comparison of scores.
• Develop Assessment measures
o Rubrics for assessing student work and competencies should be defined and adopted across the program
o Evidence (products, scores, reports) generated while using the apps must be identified, collected and scored.
Expand professional development to support teachers with implementation
• Teacher training should begin in the summer before school and should include both iPad training, app training and practical methods and pedagogy for iPad use
• A series of workshops could be run that are interest based. For example, teachers asked about learning to use media on the iPads to produce products; they also requested more PBL based training
• Regular collaborations between teachers and schools should be encouraged. Time will need to be allocated for this.
• An online PLC should be developed in which all teachers contribute to the content of the community.
Continue to promote parent involvement
• Teachers requested support with developing the parent component of the project. To this end, perhaps the online PLC would be a good location for collecting resources and sharing of guide materials.
• First Steps could be approached to provide training sessions for parents and to support the teachers.
• A parent coordinator would be helpful to increase parent participation.
• An evening parent program might make the workshops/training more accessible.
Continue to support infrastructure development
• Wireless nodes and adequate bandwidth are in the process of being updated.
• As one-to-one technology is expanded.
Expand Program Management and Oversight
• As this program grows, it needs a management team. Lead teachers/administrators from each district could be identified to coordinate with NapaLearns leadership.
• Reconstitute the DEL Task Force
o Hold monthly meetings to address methodology and implementation issues and oversee data collection.
o These could be held virtually to conserve time and travel expenses
Improve data management and oversight responsibilities
• All data should be held in a secure location for NapaLearns
• Personnel are needed to:
o Collect demographic data from the districts, PPVT data from assessors, organize scoring sessions for app artifacts.
o Data entry
o Alignment of all data for each individual student in an ANONYMOUS manner into a single SPSS ready spreadsheet.
o Data should be regularly checked for missing or incorrect input so that when reports are due, there is no further mining necessary.
• Institutional Review Board for the Protection of Human Subjects (IRB) approval should be obtained through a district or county level agency to meet Federal requirements for publication of this work and grant eligibility.
Content and Activities:
To begin - quick survey with poll everywhere or other quick easy tool
1. Overview of the project, goals, participants, demographics
presented graphically and with brief student centered videos
2. At preschool level - we'll share the Footsteps2Brilliance program that every student in the county may use for free and how we got this deal. it is a large download so we will not require participants to have the program in-hand We'llpresent results - clips from teachers, parents, legislators and what they've seen in the classroom
3. At the TK-3 grade level we'll provide app lists (QR codes) and will demonstrated projects that teachers used in their classroom and how the use and the apps chosen evolved as the teachers became comfortable, saw the power of the tools, and began reinventing how they teach - to include PBL and 21st Century skills. Examples, clips from real teachers.
Closure - qr code, Remind, and
References ISTE 2014 Proposal
Byrne, R. (n.d.) iPad Apps for School. Retrieved from iPad Apps for School http://ipadapps4school.com
Fisher, M. (n.d.). iPads in Schools . Retrieved April 2013, from iPads in Schools Livebinder: http://www.livebinders.com/play/play/26195#
Goldman, S. & Lucas, R. (2012). Issues in the transformation of teaching with
technology. In Resta, R. (Ed.), Proceedings of Society for Information Technology & Teacher Education International Conference 2012 (pp. 1792-1800). Retrieved from http://www.editlib.org/p/39848
iPads for Learning (2011, January) Department of Education, State of Victoria http://www.ipadsforeducation.vic.edu.au
Johnson, L. (2012, June 1). Climbing the Bloom’s Ladder with Hot Web Apps. Retrieved from http://www.techchef4u.com/ipad/climbing-the-blooms-ladder-with-hot-web-apps/
Kharbach, M. (n.d.). Teachers iPad Apps. Retrieved from Educational Technology and Mobile Learning: http://www.educatorstechnology.com/p/blog-page_9.html
Kolis, M. (n.d.) Connecting the Common Core to IOS Apps K-3. Retrieved from http://commoncoreapps.weebly.com
Lirenman, K. (2013, August 4). Using an iPad in Grade One- more ideas one year later. Retrieved from http://learningandsharingwithmsl.blogspot.ca/2013/08/using-ipad-in-grade-one-more-ideas-one.html
Matthews, R. (2012, June 23) Designing iPad lessons: Focus on the objective, not the app. Retrieved from http://www.slideshare.net/RebaGordonMatthews/designing-ipad-lessons
Melhuish, K. & Falloon, G. (2010). Looking to the future: M-learning with the iPad. Computers in New Zealand Schools: Learning, Leading, Technology , 22 (3), 1-16.
mikefisher821 (n.d.). iPads in Schools. Retrieved from http://www.livebinders.com/play/play/26195#
Oxnevad, S. (2013, June 21) Free Apps to Capture, Create and Connect to the Common Core. Retrieved from http://gettingsmart.com/2013/06/free-apps-to-capture-create-and-connect-to-the-common-core/
Puentedura, R. (2013, August 26). Rubin R. Puentedura's Weblog. Retrieved from Hippasus: www.hippasus.com/rrpweblog/
Robkmil4801 (n.d.) iPad¬_Lesson. Retrieved from http://www.scribd.com/doc/52812881/ipad-lesson
Schrock, K. (n.d.), iPads in the Classroom. Retrieved from Kathy Schrock’s Guide to Everything http://www.schrockguide.net/ipads-in-the-classroom.html
Tmccabe (LiveBinder). iPad Resources iCan Use. Retrieved from http://www.livebinders.com/play/play_or_edit?id=108430
Tolisano, S. (2011, August 21) Bloom’s Taxonomy and iPad Apps. Retrieved from http://langwitches.org/blog/2011/08/21/blooms-taxonomy-and-ipad-apps/
Tolisano, S. (2013, June 10) I have iPads in the Classroom. Now what? Retrieved from http://www.slideshare.net/langwitches/i-have-ipads-in-the-classroom-now-what
Tolisano, S. (2013, February 7) Transformative Use of iPads in Education: Pedagogy, Examples, Activities. Retrieved from http://www.slideshare.net/langwitches/i-pads-transformativepedagogyclassroomactivities