Hello everyone. This is Yan, a research student. After completing my graduate studies at The Education University of Hong Kong, I enrolled as a research student this April. I intend to conduct research on the design and development of virtual reality in science education. Regarding that design aspect, I believe gamification is one effective method. Therefore, I read and would like to introduce the following paper.

Paper Title: Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts

Authors: BAI, S. T., HEW, K. F., & HUANG, B.

Journal: Educational Research Review

Year of Publication: 2020

DOI: https://doi.org/10.1016/j.edurev.2020.100322

1. Introduction

Gamification is attracting attention in the field of education and is widely recommended as a method to increase learning motivation. However, it is also a subject of significant debate, and the results of existing empirical studies are inconsistent. Some studies suggest that learning outcomes improve, others find no significant difference, and some even show negative effects, such as a decline in test scores. Therefore, this study aims to systematically answer the question of whether gamification can actually improve students’ academic performance.

2. Previous Research

Previous studies have several common shortcomings. There has been no meta-analysis specifically focused on “academic performance,” many studies confuse gamification with game-based learning or serious games, there is a lack of systematic analysis of moderating factors such as the number of game elements or the duration of implementation, and there is a shortage of research that synthesizes students’ subjective feedback. In contrast, this study contributes by focusing solely on pure gamification, integrating only studies that included a control group, analyzing both quantitative effects and students’ raw feedback, and systematically examining moderating factors. The research objectives are to quantify the impact of gamification on academic performance through meta-analysis and to clarify the reasons why students like or dislike gamification. Based on this, the study sets quantitative analysis tasks to ask how gamification affects academic performance in K-12 and higher education, and qualitative analysis tasks to ask what aspects of gamification students like and dislike.

3. Research Methods

For literature collection, seven databases were used: ACM Digital Library, EBSCO, Emerald Insight, Science Direct, Scopus, and Web of Science, using combinations of “gamif*” and keywords related to education and learning. An initial search yielded 828 items, and 3 items were collected from other sources. Following screening based on PRISMA criteria, 24 quantitative studies and 32 qualitative studies were finally selected. The selection criteria for quantitative studies were that they used an experimental or quasi-experimental design to measure objective learning outcomes and clearly used at least one type of game element. The selection criteria for qualitative studies were that they collected students’ actual thoughts through interviews or open-ended questionnaires and presented specific statements. The selection process was conducted independently by two researchers, achieving a 90% agreement rate for quantitative studies and a 100% agreement rate for qualitative studies, with disagreements resolved through discussion.

In data analysis, for the quantitative analysis, effect sizes were calculated using Hedges’ g for 30 independent interventions (total participants: 3,202) obtained from the 24 quantitative studies, and a random-effects model was adopted. Heterogeneity between studies was examined using the I² statistic and Q-test, and publication bias was checked using funnel plots, rank correlation, and fail-safe N tests. Furthermore, as moderating factors, the types and number of game elements, the level of control in the study, course characteristics, and participant characteristics were analyzed. For the qualitative analysis, a thematic analysis method was used to extract common themes from the 32 qualitative studies.

4. Results

As a result of the quantitative analysis for RQ1, the overall effect size was Hedges’ g = 0.504, indicating a moderate effect, which was statistically significant (p < 0.001), confirming that gamification significantly outperforms non-gamified learning. No publication bias was observed, and the funnel plot was symmetrical; however, because heterogeneity between studies was significant, an analysis of moderating factors was necessary. In the analysis of moderating factors, while many factors did not significantly change the effect of gamification, three—sample size, intervention duration, and geographical region—had a significant impact, leading to the conclusion that a higher number of game elements does not necessarily lead to better effects.

In the qualitative analysis for RQ2, reasons students liked gamification included increased learning motivation (22 studies), tracking progress through rankings and badges (17 studies), satisfying the need for recognition through badges (11 studies), and motivation to set higher goals (11 studies). On the other hand, reasons for disliking it included the pressure or jealousy induced in students ranked lower (14 studies) and a decrease in motivation when there are no tangible rewards (7 studies) (7 studies).

5. Discussion and Unresolved Issues

Reasons why gamification is effective include the promotion of goal setting, the satisfaction of the need for recognition, and the provision of feedback and social comparison. However, unresolved issues remain. Since students hope that badges can be exchanged for actual grades or rewards, proposals have been made to link gamification points to grade evaluations, e-books, or the unlocking of exclusive content. Regarding the optimal use of rankings, it is considered important to introduce them in small groups of 10–20 people and to avoid displaying lower-ranked students.

6. Conclusion

As a major finding of this study, it was shown that gamification generally improves students’ academic performance to a moderate degree (g=0.504), but it was also clarified that the effect is influenced by factors such as learning time, sample size, and region, and that a higher number of game elements does not lead to better results. Limitations of the study include incomplete data in some studies and insufficient sample sizes in certain categories. Future research directions include long-term follow-up studies, research in non-traditional learning environments such as MOOCs, customization based on individual characteristics, clarification of educational design details, and a focus on teachers’ attitudes.

Reflections

Through this paper, I encountered the concept of gamified learning for the first time, and I felt that the analysis results demonstrated that gamified learning has a positive impact on student performance. However, it is important to consider influencing factors such as learning time and region to maximize its effectiveness. In future research, I would like to work on how different game elements affect students to varying degrees, as well as the construction of gamification systems tailored to individuals.

Report by: Yan Yuqi