Simple, Curricular-Aligned Games in the Wild: Study Results (Paper table 5)

Location: Room 217A, Table 5

Listen and learn

[Listen and learn : Research paper]

Monday, June 26, 11:30 am–12:30 pm
Location: Room 217A, Table 5

Vadim Polikov  
A new published controlled study shows building games off of the curriculum teachers use and offering a variety of games for each learning objective increases lesson retention and student engagement, even with no training or support for teachers using the games.

Skill level: Beginner
Attendee devices: Devices not needed
Focus: Digital age teaching & learning
Topic: Games and simulations
Grade level: 6-12
Subject area: STEM/STEAM
ISTE Standards: Teachers : Design and develop digital age learning experiences and assessments
Teachers : Facilitate and inspire student learning and creativity
Teachers : Model digital age work and learning

Digital tote resources

Proposal summary


The research raises three distinctions that are addressed in the current study. First, there is an important distinction between studies that examine what might be possible with specialized or advanced game designs and curricula versus what is probable in terms of typical current educational games and curricula. This study focuses on the latter in terms of what is currently probable for typical educational games and curricula.

Second, a challenge in any research on games for learning involves representing a subgenre of games broadly enough to be generalizable beyond the design of single game or developer while including a sample of games homogenous enough to arguably represent something more specific than the omnibus term “digital game.” A digital game by definition can span from multi-million dollar complex titles to “games” that involve only rudimentary gamification of a typical school worksheet. This study focuses on a subgenre that we define as “typical educational games” to constrain and define a subset of games specific enough to address our focus on probable/typical and begin to address our distinction of specificity in terms of a subgenre of games that might be envisioned for classrooms.

Third, recent meta-analyses by Wouters et al. (2013) and Clark, Tanner-Smith, and Killingsworth (2015) suggest that extended interaction with digital games for learning is central to their efficacy. This study thus focuses on curricula designed and implemented by each teacher with durations of approximately three weeks.


This study employed a quasi-experimental comparison group design to evaluate the efficacy of using digital history games for teaching historical concepts to middle- and high-school students. This non-equivalent posttest-only comparison group design (Shadish, Cook, & Campbell, 2002) was used to contrast the means of posttest outcome measures for the game and non-game conditions. Classrooms were the unit of assignment to conditions, thus this design can also be referred to as a cluster-quasi-experimental design. Baseline differences between intervention and comparison groups are one of the primary threats to interval validity in a quasi-experimental design (i.e., due to potential selection bias in assignment to conditions). Therefore, and as described in greater detail below, we matched students in the game and non-game conditions on prior history grades and ELL, IEP, and Special Education status to ensure baseline equivalence between the groups. This quantitative design was supplemented with analysis of qualitative implementation data from teacher interviews.


The original sample included 1,080 students enrolled in 46 classrooms taught by 15 teachers located in 10 public, private, or charter schools in the United States (2 in Alabama, 2 in the District of Columbia, 1 in Kentucky, 2 in Maryland, 1 in Missouri, 1 in New Jersey, 1 in Rhode Island). The schools are located in urban (54%), suburban (23%), and rural (23%) contexts. Teachers were instructed to teach at least one of their history classes using the digital games, and at least one other class using (non-game) practice-as-usual teaching techniques. Teachers did not randomly allocate classrooms to conditions. Instead, teachers were encouraged to balance between the game and non-game classes to the degree possible when selecting which classrooms received the game conditions.


The main findings from the quantitative analysis can be interpreted in two ways. If the one teacher who reported a failed implementation is dropped, the results demonstrate significantly higher gains for the game condition in terms of multiple-choice, open-response factual outcomes, evidentiary depth, and student engagement outcomes. If the failed implementation case is included in the analyses, all learning outcomes were higher for students in the game condition, but only student engagement and evidentiary depth were significantly higher. Moderator analyses further highlighted the role of teacher experience and student engagement in the efficacy of the game condition. Although the beneficial effects of game instruction on students’ outcomes were remarkably consistent for students across levels of prior achievement in history, the results indicated that the game instruction may be particularly beneficial for Special Education students.

All of these results from the quantitative analyses must be interpreted cautiously given that the quasi-experimental design precludes any strong causal inferences. Nonetheless, these findings highlight the potential of typical games for enhancing instruction, particularly when combined with the data from the teacher surveys. In terms of student engagement, the teachers rated students’ engagement in the game classrooms as significantly higher than students’ engagement in the non-game classrooms. In terms of future usage, all of the teachers except Ann “strongly agreed” or “agreed” that they would like to use games like these in the future (and Ann marked “unsure” rather than “disagree” or “strongly disagree”).

Teachers on average expressed that they would want to use games like these in the future for 32% of their instructional time. Thus the teachers viewed these games as a highly beneficial curricular activity.

The success of the implementation of games into the teachers’ curricula appears to depend on a dialectic between the teacher and the students that determines student buy-in and attitude about the games. As discussed in the study, in classrooms where this buy-in was high, results for the games classrooms relative to their control classrooms were impressive.

Thus even with typical educational games, the role of the teacher and quality of implementation as well as cultural variables of the school and classroom are critical. Research and development of digital games for learning often focuses only on the interactions of players with the game (Martinez & Clark, 2013). The findings of the current study underscore the importance of attending to the role of the teacher as well as to overarching socio-cultural and community variables in thinking about digital games for the classroom.


The positive outcomes suggest substantial potential for educational games as a valuable addition to teachers’ repertoire of curricular tools and resources, particularly when one considers the high engagement levels reported by the students and teachers. Even if the games had led only to the same level of learning, the games could be considered successful given the high engagement overall and the supports for Special Education students. Yet they performed better than that. Augmenting regular instruction with the games for three weeks significantly improved student learning.


Anderson, J. & Rainie, L. (2012). Gamification: Experts expect ‘game layers’ to expand in the future, with positive and negative results. Available at:

Barab, S. A., Sadler, T., Heiselt, C., Hickey, D., Zuiker, S. (2007). Relating narrative, inquiry, and inscriptions: A framework for socio-scientific inquiry. Journal of Science Education and Technology, 16(1), 59-82.

Barab, S. A., Scott, B., Siyahhan, S., Goldstone, R., Ingram-Goble, A., Zuiker, S., & Warrant, S. (2009). Transformational play as a curricular scaffold: Using videogames to support science education. Journal of Science Education and Technology 18, 305-320.

Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284-290.

Clark, D. B., & Martinez-Garza, M. (2012). Prediction and explanation as design mechanics in conceptually-integrated digital games to help players articulate the tacit understandings they build through gameplay. In C. Steinkuhler, K. Squire, & S. Barab (Eds.), Games, Learning, And Society: Learning And Meaning In The Digital Age (pp. 279-305). Cambridge: Cambridge University Press.

Clark, D. B., Tanner-Smith, E., & Killingsworth, S. (First Online 2015). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research. DOI: 10.3102/0034654315582065, first published on April 16, 2015.

Dieterle, E. (2009). Neomillennial learning styles and River City. Children, Youth and Environments, 19(1), 245–278.

Dieterle, E. (2010). Games for science education. In A. Hirumi (Ed.), Playing Games in School: Video Games and Simulations for Primary and Secondary Education. Eugene, OR: International Society for Technology in Education, pp. 89-112.

Duggan, M. (2015). Gaming and Gamers.” Pew Research Center. Available at:

Federation of American Scientists. (2006). Report: Summit on Educational Games: Harnessing the Power of Video Games for Learning. Washington, D.C.

Gee, J. P. (2003/2007). What video games have to teach us about learning and literacy. New York: Palgrave Macmillan.

Gee, J. P. (2007). What video games have to teach us about learning and literacy (2nd ed.). New York, NY: Palgrave Macmillan.
Habgood, M. P. J., & Ainsworth, S. E. (2011). Motivating children to learn effectively: Exploring the value of intrinsic integration in educational games. Journal of the Learning Sciences, 20, 169 –206. doi:10.1080/

Hines, P. J., Jasny, B. R., & Merris, J. (2009). Adding a T to the three R’s. Science, 323, 53. doi: 10.1126/science.323.5910.53a

Honey, M. A., & Hilton, M. (Eds.). (2010). Learning science through computer games and simulations. National Research Council. Washington, DC: National Academy Press.

Kafai, Y. B., Quintero, M., & Feldon, D. (2010). Investigating the “‘Why’” in Whypox: Casual and systematic explorations of a virtual epidemic. Games and Culture, 5, 116–135. doi: 10.1177/1555412009351265

Ketelhut, D. J. (2007). The impact of student self-efficacy on scientific inquiry skills: An exploratory investigation in River City, a multi-user virtual environment. The Journal of Science Education and Technology, 16(1), 99–111.

Klopfer, E., Scheintaub, H., Huang, W., Wendal, D. & Roque, R. (2009). The Simulation Cycle: combining games, simulations, engineering and science using StarLogo TNG, E-Learning, 6(1), 71-96.

Lenhart, A., Kahne, J., Middaugh, E., Macgill, A., Evans, C., & Vitak, J. (2008). Teens, video games, and civics (p. 64). Pew Internet & American Life Project. Retrieved from

Martinez-Garza, M., & Clark, D. B. (2013). Teachers and teaching in game-based learning theory and practice. In M. Khine & I. Saleh (Eds.), Approaches and Strategies in Next Generation Science Learning (pp. 147-163). Hershey, PA: IGI Global. doi:10.4018/978-1-4666-2809-0.ch008.

Martinez-Garza, M., Clark, D. B. & Nelson, B. (2013). Digital games and the US National Research Council’s science proficiency goals. Studies in Science Education, 49, 170-208. doi:10.1080/03057267.2013.839372

Mayo, M. J. (2009). Video games: A route to large-scale STEM education? Science, 323, 79-82.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.

Sitzmann T. (2011). A meta-analytic examination of the instructional effectiveness of computer-based simulation games. Personnel Psychology, 64, 489–528. doi:10.1111/j.1744-6570.2011.01190.x

Squire, K. (2011). Video games and learning: Teaching and participatory culture in the digital age. New York, NY: Teachers College Press.

Squire, K. D. & Jan, M. (2007). Mad City Mystery: Developing scientific argumentation skills with a place-based augmented reality game on handheld computers. Journal of Science Education and Technology, 16(1) 5-29.

Squire, K.D., & Klopfer, E. (2007). Augmented reality simulations on handheld computers. Journal of the Learning Sciences, 16(3), 371-413.

Steinkuehler, D., & Duncan, S. (2008). Scientific habits of mind in virtual worlds. Journal of Science Education and Technology, 17(6), 530-543.

Steinkuehler, D., & Duncan, S. (2008). Scientific habits of mind in virtual worlds. Journal of Science Education and Technology, 17, 530-543. doi:10.1007/s10956-008-9120-8

Tennessee Department of Education (2016). Eighth Grade History Standards. Retrieved from

Vogel J.J., Vogel D.S., Cannon-Bowers J., Bowers C.A., Muse K. & Wright M. (2006) Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research 34, 229–243. doi:10.2190/FLHV-K4WA-WPVQ-H0YM

Walz, S. P. & Deterding, S. (2015): An Introduction to the Gameful World, in Walz, S. P. & Deterding, S. (eds.), The Gameful World: Approaches, Issues, Applications (pp. 1–14). Cambridge, MA, London: MIT Press.

Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105, 249-265. doi:10.1037/a0031311

Young, M. F., Slota, S., Cutter, A. B., Jalette, G., Mullin, G., Lai, B., Simeoni, Z., Tran, M., Yukhymenko, M. (2012). Our Princess Is in Another Castle: A Review of Trends in Serious Gaming for Education. Review of Educational Research, 82(1), 61–89. doi:10.3102/0034654312436980



Vadim Polikov

learning starts here

San Antonio

June 25-28, 2017

© 2017 International Society for Technology in Education (ISTE), All Rights Reserved