MORE EVENTS
Leadership
Exchange
Solutions
Summit
Change display time — Currently: Eastern Daylight Time (EDT) (Event time)

5 Ways Digital Books Can Empower Play: A Psychophysiological Design Study

,
Pennsylvania Convention Center, 121BC

Lecture presentation
Listen and learn: Research paper
Save to My Favorites

Research papers are a pairing of two 18 minute presentations followed by 18 minutes of Discussion led by a Discussant, with remaining time for Q & A.
This is presentation 2 of 2, scroll down to see more details.

Other presentations in this group:

Presenters

Photo
Elliott builds great experiences by helping organizations understand the emotions of their customers. He earned his PhD at the MIT Media Lab using psychophysiological sensors to measure emotions in a new, objective way. Previously, Elliott worked at IDEO as a design researcher, where he learned how to combine sensors, design, and emotions. Today, he runs mPath, a design consulting firm. Elliott has had the privilege of designing a vast range of emotions, from the excitement of using Google, to audience engagement of the Blue Man Group, to customers’ confidence at Lowe’s Home Improvement, to the anxiety of children with autism in therapy.

Session description

Play helps grow literary skills, but how can we encourage children to play in today’s schools? Using eye tracking and emotional sensors, we prototyped new ways for books to encourage play. Our most effective intervention was adding digital cameras to books — prompting children to actively take pictures while reading.

Framework

Play can be a strong way to learn. From cognitive processing and self-regulation (Vygotsky & Cole, 1978), to social and emotional development (Elkind, 2007), to story comprehension and narrative competence (Roskos et al., 2010). However, with a strong push towards core academic skills, schools are rapidly reducing time for play (Hirsh-Pasek et al., 2009; E. Miller & Almon, 2009).
Research suggests that apps or books that encourage learning through play should have five core traits: joyful, active, meaningful, social, and iterative (Hirsh-Pasek et al., 2015; Zosh et al., 2018). Specifically within socio-dramatic play, Smilansky (1968) suggests children should be taught how to play with modelling, verbal guidance, thematic-fantasy training, and imaginative play training. Using these guidelines, is it possible for a book to empower children to learn through play?
We designed our books to encourage socio-dramatic play where children role play their own imaginative story, (Fleer, 2016; Smilansky, 1968). Children were identified as playing when they interacted with their toys and verbally described a story that took place.

More [+]

Methods

Over 10 months, we worked with 60 children ages 6 to 9 in afterschool programs and in their homes in Denver, Colorado. We used a co-creation and iterative design (Cobb et al., 2003) to fully understand what young readers need to be encouraged to play.

When willing, children wore video recording eye tracking glasses (Tobii Technology AB, Sweden) during each session to help reveal how the story was read and whether the children played unprompted and unobserved. The eye tracking cameras allowed the researchers to be in a separate room and observe remotely, reducing the influence an observing scientist would have on the play session: our previous research showed that children are highly influenced to “play correctly” with an observer watching them (AUTHOR, 2008). At other times, a camera was set up on a stand to record the session.
When willing, some children wore skin conductance sensors ( AUTHOR, 2008) to track their emotional responses during reading and play as we have done in other studies ( AUTHOR, 2010, 2018, 2011). Skin Conductance is a marker of sympathetic arousal and can help show how much a child is emotionally or cognitively engaged (Boucsein, 2011).
Additionally, we conducted open ended interviews after each intervention. Children were asked to describe how they used the books and toys. Based on our remote observations, we would prompt children to tell us about certain behaviors, “Can you tell us what happened on this page?” We would then ask children about their preferences for different designs and ask what they would add to the books to better encourage learning through play.

In each research session we gave children a toy (plastic dinosaurs, a fidget cube, Play Doh, a Slime Making Kit, a LEGO Set, or Pokemon action figures). We also gave children a book that referenced their toy. Each week, based off of children’s feedback we modified the book to better encourage learning through play. Over the course of the study 24 different books were developed and tested, all of which can be reviewed here www.wonderstories.app/empowerplay. Books were 18 - 64 pages long with typically 1 to 2 sentences per a page. Most pages included pictures of the toy and comic bubbles were used extensively. Stories directly referenced the toys children were given and prompted children to play with the toys. Text was written at a second grade reading level. Parents or researchers read with children unable to read the text independently.

In our later interventions, children were given the book 5 minutes after the toy to better understand how children naturally play. We ended recording when the child told the researcher they were done. This allowed us to better observe children’s natural play after reading as well.

More [+]

Results

We developed 5 design principles for how digital books could encourage actively encourage children to play.
1. Provide open-ended prompts for storytelling (Joy/Curiosity)
In play tutoring, a teacher provides prompts for children to model play: “Pretend this is a phone and you are calling me” (Smilansky, 1968). In our prototype books, we added open-ended prompts as well.
Children reported that they liked these prompts, and when they engaged in play, it was often in direct response to the prompts. For example, “What kind of home should Layla and Emma build for Turbo and his mom?” prompted Cece to build the home. “Can you help Zane find a way to overcome his fear and defeat the radical reptiles?” prompted Elijah to put a group of snakes inside a LEGO robot. In response to, “Help tell a story of Zane bringing the chest down Fridge Mountain back to the Dojo,” Coulomb put the treasure chest in the robot and had the treasure chest drive itself home.

2. Prompt the Children to Take Digital Photos (Active)
Digital cameras can help children self-reflect and retell stories without having to write (Byrnes & Wasik, 2009; Yoon, 2014). We asked some readers to both read a physical and digital book with a camera which can be accessed here https://wonder.io/book/902/1.
In the digital book, we include interactive questions that previous research showed could engage readers in a playful manner (Byrnes & Wasik, 2009; Yoon, 2014). Reconfirming our previous findings, our emotional sensors showed the same expected playful engagement with the digital book. Specifically, we found that children emotionally engaged with answering questions during the story, making the reading experience more playful.
The last question in our digital book asked the child to “Pretend your phone is a drone and take a picture of the monkeys being silly.” All tested children played with the LEGO toy during this prompt. Quentin, placed the monkeys in the tree lying down and eating a banana. Nolan pretended the monkeys were taking a selfie. Lucy and Ella had the monkeys take a picture of the monkeys juggling bananas as they made sound effects.
Upon seeing Emma and Lucy’s success with the camera, we asked them to make a movie with the camera. Their impromptu movie lasted for four minutes and included additional details from the books:
[Lucy]: “What are you doing here?”
 “I’m taking pictures of the turtles.”
“You shouldn’t be here, that’s wildlife rules.”
When the children read our physical book, none of them played with their model, but the digital camera engaged all of them to play and story tell with their toys. Integrating a digital camera into a book is a strong way to facilitate the transition from reading to play and encourage children to story tell.

3. Include Adults and Siblings in Play (Social)
A key pillar of learning through play is having a social component. While many of the children we worked with read and played alone, which is often the case when practicing reading, at other times they read and played with an adult or sibling. These peer interactions with siblings, parents, or the researcher were often much more emotionally engaging than the reading itself.
When Niki was storytelling about a LEGO mech stomping a figurine in half (Figure 15), the sensors recorded a small emotional reaction. When his younger brother asked what was happening, Niki produced a larger response. An even larger response occured when the researcher asks about Niki’s play.
When Evan played with his mother, she pretended the roasted chicken was fighting the mech, and the roasted chicken was going to throw drumsticks at it. One of Evan’s largest emotional responses during play was to his mother’s provocation (Figure 15).
Similarly, Cece only began building Turbo’s home when her mother emphasized the prompt in the text, “Do you want to build that home?”
After Gracie and Kaya finished reading, Kaya sat still, staring out into space while Gracie, the younger sister, played. Gracie made a joke with her toy, which had the effect of inviting Kaya to join in and tell a story about the toys brand new outfit (an important part of the story).
Gracie, moving a small bunny LEGO character: “Looks like I’ve got to go to the bathroom. Pssss. Aaahh”
Laughter from Gracie and Kaya.
Gracie: “The bunny peed on the shoe!”
Kaya: “My brand new outfit, my brand new shoes, my brand new everything!”

Digital books that encourage reading and playing together with parents or peers are more likely to engage children in play by bringing the social aspect of play to the forefront.

4. Provide a background story for real world toys (Meaning)
Smilansky (1968) suggests teaching children background information about their play theme can enhance play. Children naturally tell the stories of Batman and Cinderella because they know their stories: they know the character names, how the characters behave, and the stories that revolve around those characters (Paley, 2005).
To provide children with background knowledge, our books included character profiles for each character detailing information about the toy to help inspire play.
In addition to character background, the books also told a short story about the characters that children could imitate in play afterwards.
The final version of these backgrounds were positively received. For example, Emily repeatedly called her brother over to listen to her describe each new character, “Kyle, look, it says afraid of, she is afraid of crabs.” Noah, upon reading about how to remove the robot’s battery, commented, “This will help me figure out how the battle will go.”
The children who engaged in play repeatedly referenced the newly-learned character traits. Cece received a LEGO set that included a turtle named Turbo and a LEGO figure named Emma. She also received a prototype book that described Turbo and Emma in detail. After reading the book, Cece built a home and explained that it was for Turbo, the turtle, and the turtle’s mother. She added a hollow rock full of ice cream, Turbo’s favorite food. In addition, Cece gave the LEGO figurine, Emma, a life jacket in case she gets sucked away by the ocean, which was another theme in the book. Then, Cece acted out how Emma used the Turtle Protection Vehicle to scare away the crabs, which happened in the book. Cece also gave Emma a trash can to clean up the ocean, which was an extension of the story -- Emma’s original job was to protect turtles.

More [+]

Importance

Software like Prodigy and ABCMouse are entirely digital, play and learning all take place on the screen. Or, alternatively, physical books lack a means to motivate children to play and think deeply about the story. In our attempt to create a mixed medium where children read a digital book and then played in the real world, they often failed at transitioning to that real world play, despite play being an often desirable activity.
Our research shows that edtech designers should carefully consider how they design digital experiences to connect to the real world. The most promising example we uncovered was the integration of a digital camera combined with story prompts. The digital camera acted as a conduit, connecting the child and screen to the real world. We also found that adults could help readers transition from books to real world play. Typically educational software is designed for the individual, discouraging real-world conversations and interactions, but we hope to see educational technology that encourages real world connections and real world play, initiated from a digital interface.

More [+]

References

Boucsein, W. (2011). Electrodermal activity. In Techniques in psychophysiology (2nd Ed). Springer-Verlag New York.
Bubikova-Moan, J., Næss Hjetland, H., & Wollscheid, S. (2019). ECE Teachers’ views on play-based learning: a systematic review. European Early Childhood Education Research Journal, 27(6), 776–800.
Byrnes, J. P., & Wasik, B. A. (2009). Factors predictive of mathematics achievement in kindergarten, first and third grades: An opportunity–propensity analysis. Contemporary Educational Psychology, 34(2), 167–183.
Cepeda, N. J., Kramer, A. F., & Gonzalez de Sather, J. (2001). Changes in executive control across the life span: examination of task-switching performance. Developmental Psychology, 37(5), 715.
Cobb, P., Confrey, J., DiSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13.
Elkind, D. (2007). The power of play: Learning what comes naturally. Da Capo Lifelong Books.
Fleer, M. (2016). Theorising Digital Play: A Cultural-Historical Conceptualisation of Children’s Engagement in Imaginary Digital Situations. International Research in Early Childhood Education, 7(2), 75–90.
Hedman, E. (2010). In-situ Measurement of Electrodermal Activity During Occupational Therapy: Vol. Masters. Massachusetts Institute of Technology.
Hedman, E. (2018, March). 7 Ways to Help Kids Feel they’re Learning. Medium, 1. https://medium.com/@elliotthedman/7-ways-to-help-kids-feel-their-learning-80f4abbeefb0
Hedman, E. (2021). Motivating Struggling Readers to Mentally “Show Up” with Wonder Stories. ELO 2021 Conference and Festival: Platform (Post?) Pandemic.
Hedman, E. (2011). The Frustration of Learning Monopoly: The Emotional Tension of Entering a New Game Encounter. The Ethnographic Praxis in Industry Conference, September 18-21, Boulder.
Hedman, E. B. (2008). Thick Psychophysiology for Empathic Design. Massachusetts Institute of Technology, 2008. https://dspace.mit.edu/handle/1721.1/91436
Hirsh-Pasek, K., Golinkoff, R. M., Berk, L. E., & Singer, D. (2009). A mandate for playful learning in preschool: Applying the scientific evidence.
Hirsh-Pasek, K., Zosh, J. M., Golinkoff, R. M., Gray, J. H., Robb, M. B., & Kaufman, J. (2015). Putting education in “educational” apps: Lessons from the science of learning. Psychological Science in the Public Interest, 16(1), 3–34.
Miller, E., & Almon, J. (2009). Crisis in the kindergarten: Why children need to play in school. Alliance for Childhood (NJ3a).
Miller, N. E., & Dollard, J. (1941). Social learning and imitation.
Paley, V. G. (2005). A Child’s Work: The Importance Of Fantasy Play (Google eBook) (Vol. 1). University of Chicago Press. http://books.google.com/books?id=hpO72LmvC1YC&pgis=1
Roberts, A. (1980). Out to play: the middle years of childhood. Out to Play: The Middle Years of Childhood.
Roskos, K. A., Christie, J. F., Widman, S., & Holding, A. (2010). Three decades in: Priming for meta-analysis in play-literacy research. Journal of Early Childhood Literacy, 10(1), 55–96.
Smilansky, S. (1968). The effects of sociodramatic play on disadvantaged preschool children.
Vygotsky, L. S., & Cole, M. (1978). Mind in society: Development of higher psychological processes. Harvard university press.
Yoon, H. S. (2014). Can i play with you? The intersection of play and writing in a kindergarten classroom. Contemporary Issues in Early Childhood, 15(2), 109–121. https://doi.org/10.2304/ciec.2014.15.2.109
Zosh, J. M., Hirsh-Pasek, K., Hopkins, E. J., Jensen, H., Liu, C., Neale, D., Solis, S. L., & Whitebread, D. (2018). Accessing the inaccessible: Redefining play as a spectrum. Frontiers in Psychology, 9, 1124.

More [+]

Session specifications

Topic:
Storytelling/multimedia
Grade level:
PK-5
Audience:
Curriculum/district specialists, Library media specialists, Teachers
Attendee devices:
Devices useful
Attendee device specification:
Smartphone: Android, iOS, Windows
Laptop: Chromebook, Mac, PC
Tablet: Android, iOS, Windows
Participant accounts, software and other materials:
The interventions we will share are web based and do not require login. The digital camera we used in our prototype does not work on iOS smart phone and tablets due to security restrictions. You can try and run our example book here:

https://wonder.io/book/902/1

Subject area:
Language arts, Performing/visual arts
ISTE Standards:
For Educators:
Designer
  • Design authentic learning activities that align with content area standards and use digital tools and resources to maximize active, deep learning.
Facilitator
  • Model and nurture creativity and creative expression to communicate ideas, knowledge or connections.
For Students:
Creative Communicator
  • Students communicate complex ideas clearly and effectively by creating or using a variety of digital objects such as visualizations, models or simulations.