Hello, everyone. I am Chu, a research student. In this article, I would like to introduce a paper I read in our most recent English Literature Seminar.

• Paper Title: Evaluating Computer Science Students’ Reading Comprehension of Educational Multimedia-Enhanced Text Using Scalable Eye-Tracking Methodology

• Journal: Smart Learning Environments

• Volume: 29

• Year of Publication: 2024

• Authors: Milan Turčáni, Zoltan Balogh & Michal Kohútek

The following is an overview of the content.

Overview

In recent years, the evolution of technology in the educational field has been remarkable, and among such advancements, “eye-tracking technology” has been attracting significant attention. In this study, the researchers analyzed learning effectiveness using eye-tracking technology with university students and verified how comprehension is improved.

The focus of the research is to analyze the gaze movements of learners when reading text and to clarify which parts act as barriers to understanding. In particular, they aimed to understand reading strategies when reading specialized texts and investigated the relationship between gaze movements and learning outcomes. Previous research has suggested that using eye-tracking technology allows for a more detailed understanding of how learners process information.

In this study, an experiment was conducted with 80 university students, dividing the subjects into a “group with key terms highlighted (40 students)” and a “group without highlighting (40 students)” to investigate differences in reading comprehension. During the experiment, the authors had the subjects read the following three texts and subsequently conducted comprehension tests.

• Text A (Basic Concepts): Average fixation duration (With highlighting: 3.5 sec, Without highlighting: 4.2 sec)

• Text B (Intermediate Concepts): Average fixation duration (With highlighting: 5.1 sec, Without highlighting: 6.0 sec)

• Text C (Applied Concepts): Average fixation duration (With highlighting: 6.8 sec, Without highlighting: 7.5 sec)

As a result of the comprehension tests, the group with highlighted keywords obtained an average score of 72.4 points, while the group without highlighting scored 65.3 points. However, no statistically significant difference (p = 0.08) was observed, and this paper also indicated that simply highlighting keywords is not enough to obtain a sufficient effect.

Furthermore, this study compared the gaze patterns of “skilled readers” and “unskilled readers” to explore differences in comprehension. While skilled readers (top 25%) moved their gaze efficiently and were able to acquire necessary information quickly, unskilled readers (bottom 25%) tended to keep their gaze on specific words for a long time. For example, the average fixation duration for unskilled readers was 8.2 seconds, whereas it was 5.6 seconds for skilled readers. Consequently, the authors believe that the development of educational tools that support different reading strategies for each learner is required.

The authors suggest the potential for their research to improve the quality of online education and e-learning, emphasizing the importance of utilizing eye-tracking technology in future educational design and asserting its significance. In particular, they state that this technology is expected to contribute greatly to the development of individually optimized learning support systems and the improvement of educational content design.

In future research, they state that they will also investigate the improvement of learning effects through real-time feedback by AI utilizing gaze data and the combination with other multimedia elements (audio and video).

My Thoughts

I believe that the high price of eye-tracking monitoring devices has hindered their widespread use in multimodal Learning Analytics (LA) until now. I feel that this research is highly contributive to this field as it proposes a method to collect gaze data using webcams and analyze student behavior, demonstrating a solution and analytical method for realizing large-scale eye-tracking collection and analysis at a low cost.

Additionally, the descriptions in the analysis and discussion sections of this study are very clear and easy to understand, allowing the main points of the argument and their importance to be easily grasped. Furthermore, it is commendable that the references cited in the experimental section are appropriate for the experimental content and that detailed explanations are provided.

However, on the other hand, I believe that a major challenge of this study lies in the issue of the reliability of the devices used. From the perspective of data analysis, favorable results are shown, but I feel that if the reliability of the devices used for data collection is low, the reliability of the research as a whole is compromised (the authors themselves point this out). While the starting point of this research is “low-cost data collection,” the use of data with low reliability becomes a fundamental problem for the research as a whole. Although the methodology is interesting, I believe that the reliability of the research will remain a challenge for the future, and it is something that researchers in this field as a whole should consider.

By: Chu