Educational App Review Websites: What Are They Talking About?

Framework

Meta-analytic studies have found evidence of overall effectiveness of educational app implementation (Kim et al., 2021; Fabian et al., 2016). Yet, various studies have revealed that many commercially available apps are unlikely to promote learning (e.g., Dubé et al., 2020; Meyer et al., 2021; Papadakis et al., 2018). For example, Meyer and colleagues (2021) found that less than half of the top educational apps in the Google Play and Apple App Store met their standards for a high-quality educational app.

Dubé and colleagues (2020) argue that high-quality educational apps should meet five evidence-based benchmarks: they should 1) cover a particular curriculum; 2) use teaching methods informed by learning theory; 3) scaffold students’ learning by adapting to their needs and providing supports; 4) provide feedback on students’ performance; and 5) include experts in education, such as teachers or researchers, on their development team. For effective app implementation, educators should choose apps that meet these five benchmarks. Indeed, educators do prefer apps that mention curriculum, learning theory, scaffolding, feedback, and development team in their app store descriptions (Liptrot et al., 2024; Montazami et al., 2022).

Unfortunately, the average app description in the Apple App Store mentions fewer than two educational benchmarks (Dubé et al., 2020). Consequently, educators may turn to other sources of information, such as reviews, to evaluate app quality. Liptrot and colleagues (2024) found that educators rely heavily on user ratings in the app store to make app download decisions, but caution that these ratings are not associated with researchers’ evaluations of educational quality (Papadakis et al., 2018), nor the number of educational benchmarks mentioned in the app’s description (Dubé et al., 2020).

With app store information being so unreliable for educators, there is a need for external app review systems to help educators identify high-quality apps. Many websites exist for this purpose, compiling lists of the “best” educational apps and evaluating their contents. However, the extent to which these websites are useful is unclear. Taylor and colleagues (2022) compared their own assessment of app educational quality to the evaluations of two app review websites. They determined that the apps that were positively evaluated by the websites have higher educational potential than those that were negatively evaluated. However, the “best” apps were still lacking in some metrics of educational quality. Thus, while app review websites can be a useful tool to help identify apps with some potential, their evaluations may not consider all criteria needed to identify the highest-quality educational apps. As such, it is necessary to investigate the content of app review websites to reveal which criteria are covered, and which criteria are not adequately addressed.

Methods

Data for this study involved the contents of 30 app reviews across five app review websites: Common Sense Media, Educational App Store, Children & Media Australia, Bridging Apps, and Best Apps for Kids. These websites were chosen following a Google search for app review websites based on their relevance as determined by six criteria:
1. The website includes reviews of educational apps for young children.
2. Each app reviewed on the website has a dedicated review page.
3. The reviews are up-to-date.
4. The reviews include original content and are not simply duplicated from other sources (e.g., the Apple App Store).
5. The review content originates from the website itself, rather than allowing submissions from external users.
6. The reviews incorporate some discussion of the educational value of the apps.

From each website, six app reviews were selected for analysis. Specifically, reviews of two mathematics apps, two literacy apps, and two apps covering a variety of educational topics were chosen from each website. For all five websites, Endless Alphabet and Khan Academy Kids were selected for analysis. However, all other app reviews were unique to each website due to an absence of apps that were available across all websites.

Data were analysed using thematic analysis, following methods as described by Braun and Clarke (2012). First, the primary coder read through the entire set of data to gain a comprehensive understanding of the topics covered by all reviews. Next, the primary coder created an initial list of codes to classify the text based on recurring concepts, patterns, and phrases in the content of the app reviews. Each review was then read individually and each line of data was closely examined to apply codes. Through the coding process, new codes were generated. Thus, following the initial coding of all reviews, all reviews were reread to apply a final, consistent set of codes. A definition and example from the text were written to define each code. Next, the primary coder reviewed the list of codes to identify an initial list of themes. These themes grouped codes together based on overlap in their contents. At this point, all researchers reviewed the list of themes to ensure that they captured the most important elements of the data. Finally, each theme was defined and named based on its unique features.

Results

Nine themes were identified: learning content, skills, variety, educational support, app design, community, cost/advertising, engaging, and gamification.

1. Learning Content. A theme emerged whereby the app reviews frequently provided information about the learning content, including discussion of the curriculum standards addressed by the app, the types of tasks used, and how learners apply their knowledge. This theme also encompasses instances where the app reviews made statements on the effectiveness of the app’s approach to teaching content.
2. Skills. In addition to learning content, app reviews often discussed the specific skills that students could gain from the educational apps, such as subject-specific skills like math or language skills, or logic skills.
3. Variety. The app reviews emphasized variety and comprehensiveness of content. Variety in activities, subjects, content, tools, targeted skills, and resources were discussed. This theme also encapsulates the discussion of app accessibility, in that the reviews discussed the app’s applicability to various learners. The presence of this theme demonstrates that the capacity for apps to go beyond a one-size-fits all model and provide options is valued.
4. Educational Support. Educational Support discusses the mechanisms within an educational app that guide learners towards meeting learning goals. The app’s capacity to progress students through levels of difficulty, adapt to meet individual needs and differentiate instruction were some topics covered by the app reviews. Also included within this theme is discussion of whether and how the app provides feedback to learners.
5. App Design. The review websites often discussed the design and functionality of educational apps. This theme covers usability-related concerns, such as age-appropriateness and ease of app use, device compatibility, privacy and safety, and convenient features such as user profile customization. This theme also covers discussion of the development team involved in the app’s design.
6. Community. The community theme refers to instances where the reviews discussed how the app allows for connection between learners, between learners and their parents, and between various learning contexts. The presence of this theme demonstrates that the websites emphasize the practical implementation of educational apps to children’s real-world learning community.
7. Cost/Advertising. Another theme that emerged within the app reviews was discussion of the cost and advertising within the app. Reviews often mentioned whether the app was free to use, had an associated cost or in-app purchases, or included advertising.
8. Engaging. Much of the reviews’ content discussed the degree to which the educational apps were engaging to students. Engagement was often discussed in terms of the app’s entertainment value and fun. Animations, interactivity, attractive artwork, and appealing music were some of the most common topics that were considered engaging.
9. Gamification. The gamification theme refers to instances where the reviews discuss gamification elements of the educational apps. Under this theme, the reviews touched on games, play, and reward systems.

Importance

Our findings indicate that app review websites consider a range of themes in their app reviews, from the specific learning content of the apps, to practical considerations like cost and usefulness in various learning contexts, to their entertainment value. However, the identified themes rarely move beyond a surface-level discussion of app contents and features to evaluate the pedagogical value of the app. Notably absent from the themes covered by app review websites is a focus on the learning theories guiding the app’s teaching approach. Educational app design should be theory-based to ensure that it teaches content in a way that can actually promote learning. For example, Dubé and colleagues (2019) point out that educational apps can seem to “engage” students with visual content that is extraneous to the app’s learning goals, making it essential to determine how an app is engaging to assess its educational value. Educators would therefore benefit from discussion of how an app applies learning theory to engage students in learning. Yet, our investigation reveals that the review websites’ discussion of “engaging” centres mainly on extraneous details like appealing graphics rather than the extent to which engagement is meaningful. Thus, “engaging” as discussed by the app review websites is no more than a buzzword, providing minimal information about the app’s educational value.

When educators are choosing educational apps, they prefer apps that meet evidence-based educational benchmarks over apps that are only described using buzzwords (Montazami et al., 2022). Our analysis reveals that the capacity for review websites to aid educators in determining whether apps meet Dubé and colleagues’ (2020) five educational benchmarks is mixed. For educators seeking to learn about the app’s connection to curriculum, the app reviews discuss the learning content and specific skills covered. The review websites’ discussion of educational supports provides educators with some information about how the app scaffolds students’ learning and provides feedback. Details of an app’s design, including its development team, are contained within the app design theme, although development team is not a major focus of the theme. Learning theory was not identified as a theme in the app reviews at all. Notably, learning theory is also the benchmark that is least covered by app store descriptions (Dubé et al., 2020), meaning app review websites are not helping educators to assess the most-difficult-to-identify benchmark.

For educators, the present study provides valuable information about what they can expect to learn from visiting app review websites, and where these websites fall short. Overall, the results of the present study suggest that app review websites may be useful for finding information about the contents and usability of an educational app, but are insufficient for evaluating the theoretical support behind the app’s teaching methods. For leaders in education, the present study reveals a need for tools that include a deeper discussion of apps’ educational value, moving beyond buzzwords like “engaging” to talk about the validity of the app’s teaching approach.

References

Braun, V., & Clarke, V. (2012). Thematic analysis. In APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 57–71). American Psychological Association. https://doi.org/10.1037/13620-004
Dubé, A. K., Alam, S. S., Xu, C., Wen, R., & Kacmaz, G. (2019). Tablets as elementary mathematics education tools: Are they effective and why. In K. M. Robinson, H. P. Osana, & D. Kotsopoulos (Eds.), Mathematical Learning and Cognition in Early Childhood: Integrating Interdisciplinary Research into Practice (pp. 223–248). Springer International Publishing. https://doi.org/10.1007/978-3-030-12895-1_13
Dubé, A. K., Kacmaz, G., Wen, R., Alam, S. S., & Xu, C. (2020). Identifying quality educational apps: Lessons from ‘top’ mathematics apps in the Apple App store. Education and Information Technologies, 25(6), 5389–5404. https://doi.org/10.1007/s10639-020-10234-z
Fabian, K., Topping, K. J., & Barron, I. G. (2016). Mobile technology and mathematics: Effects on students’ attitudes, engagement, and achievement. Journal of Computers in Education, 3(1), 77–104. https://doi.org/10.1007/s40692-015-0048-8
Kim, J., Gilbert, J., Yu, Q., & Gale, C. (2021). Measures matter: A meta-analysis of the effects of educational apps on preschool to grade 3 children’s literacy and math skills. AERA Open, 7. https://doi.org/10.1177/23328584211004183
Liptrot, E., Pearson, H. A., Montazami, A., & Dubé, A. K. (2024). Why this app? How user ratings and app store rankings impact educators’ selection of educational apps. Computers & Education, 218. https://doi.org/10.1016/j.compedu.2024.105080
Meyer, M., Zosh, J. M., McLaren, C., Robb, M., McCaffery, H., Golinkoff, R. M., Hirsh-Pasek, K., & Radesky, J. (2021). How educational are “educational” apps for young children? App store content analysis using the Four Pillars of Learning framework. Journal of Children and Media, 15(4), 526–548. https://doi.org/10.1080/17482798.2021.1882516
Montazami, A., Ann Pearson, H., Kenneth Dubé, A., Kacmaz, G., Wen, R., & Shajeen Alam, S. (2022). Why this app? How educators choose a good educational app. Computers & Education, 184, 104513. https://doi.org/10.1016/j.compedu.2022.104513
Papadakis, S., Kalogiannakis, M., & Zaranis, N. (2018). Educational apps from the Android Google Play for Greek preschoolers: A systematic review. Computers & Education, 116, 139–160. https://doi.org/10.1016/j.compedu.2017.09.007
Pocket Gamer. (2023). App Store Metrics. https://www.pocketgamer.biz/metrics/app-store/
Taylor, G., Kolak, J., Bent, E. M., & Monaghan, P. (2022). Selecting educational apps for preschool children: How useful are website app rating systems? British Journal of Educational Technology, 53(5), 1262–1282. https://doi.org/10.1111/bjet.13199