Hi, Everyone.
I will introduce the paper discussed in this session of the English literature seminar.

I am conducting development research on a VR system that supports the learning of onomatopoeic expressions, and I have just recently completed a formative evaluation. This system allows for the tracking of eye movement data through a VR headset as learners interact with the system. It is considered a key indicator for analyzing how learners acquire onomatopoeic expressions through the developed VR system. However, a challenge of how to utilize eye movement data in learning analysis still exists. Therefore, I chose this article for our English literature seminar to gain insights into how to utilize eye movement data.

Title: Identifying the Effects of Scaffolding on Learners’ Temporal Deployment of Self-regulated Learning Operations during Game-based Learning using Multimodal Data.

Journal: Frontiers in Psychology, 14, 1280566.

Authors: Daryn A. Dever, Megan D. Wiedbusch, Sarah M. Romero, Kevin Smith, Milouni Patel, Nathan Sonnenfeld, James Lester, Roger Azevedo

Published year: 2023

Below is the abstract of this paper. If you are interested, please feel free to read.

—————————————————————————————————————————————

According to the 2019 report of the National Assessment of Educational Progress (NAEP) in the United States, it was revealed that students’ science scores decline as they progress through school grades. To address this issue, the combination of traditional classroom instruction and Game-Based Learning Environments (GBLE) suggests potential as an advanced learning technology. However, another report on the use of educational technology indicates that only about 33% of teachers strongly support its effectiveness.

This study explores how scaffolding techniques within a GBLE impact learners’ Self-Regulated Learning (SRL) processes and learning outcomes. Through analysis based on multimodal data, it proposes effective methods of scaffolding. As an approach to SRL, the study adopts SMART operations based on Winne’s (2018) COPES model, considering the interaction between working memory and long-term memory. This includes searching, where learners focus attention on specific information while engaging with a task; monitoring, which involves comparing specific information against a standard; assembling, the process of associating information once relevant pieces are identified; rehearsing, actively maintaining information in working memory for reassessment; and translation, altering the processing of information to find solutions if it does not fit the current problem representation. This comprehensive cognitive flow forms the basis for understanding interactions between learners and materials, as well as the methods of information processing within a GBLE. These operations facilitate the effective use of SRL strategies by learners. Existing studies have shown that automatic detection of these SMART operations is effective in predicting learning outcomes.

The study involved 94 undergraduate students who were randomly assigned to conditions of “full agency” and “partial agency” to analyze the use of SMART operations. As a result, learners who received scaffolding showed more effective deployment of SMART operations and improved learning outcomes compared to those who did not receive scaffolding. Additionally, the study used multimodal data to analyze the correlation between the level of scaffolding provided in the GBLE and the overall learning outcomes of the learners. Learners who received scaffolding performed significantly fewer SMART operations due to restricted agency, with particularly notable differences observed in the searching operation. Furthermore, in terms of the transition probabilities among SMART operations, learners who received scaffolding used a more diverse set of operations, facilitating an efficient learning process. This revealed the impact of specific transitions between SMART operations on learning outcomes, indicating that instruction enabling more effective SRL based on these transitions is necessary.

However, in the methodology of the study, since eye tracking and log files could not isolate specific processes, the operations of assembling and rehearsing were considered as a single action, and were used to identify the transition probabilities between the two operations. Future research should explore how learners’ past behaviors influence transition probabilities over time. Additionally, the findings of this study provide new perspectives in the field of SRL and suggest future research directions concerning the implementation of scaffolding within GBLEs and the efficient use of SMART operations for learners’ SRL

Below are my thoughts.

learning process using eye-tracking and log files. Specifically, it details the methods required to gather data for each action based on the SMART operations of SRL, such as eye tracking and log files, for example. This approach allows for a clearer understanding of learners’ behaviors and outcomes at each step, enabling information-based analysis. I believe this analytical approach is well-founded and provides answers on how to apply rules and patterns in data analysis. Additionally, this paper provided insights on utilizing “eye tracking.” My eye movement data is represented by recording and visualizing the learners’ gaze positions in videos. Reading this paper has given me ideas on how to utilize that data effectively.

However, I have several questions regarding this paper. While it proposes the SMART approach of SRL (Self-Regulated Learning) to support learning in a GBLE (Game-Based Learning Environment), it lacks a clear explanation of the necessity for self-regulated learning and the advantages of choosing the SMART model, leaving me wondering why this model was selected. Additionally, the paper does not sufficiently detail how the presence or absence of scaffolding in SRL affects learners based on their current knowledge level of microbiology. The authors do not provide enough explanation on this matter, and it is unclear whether scaffolding is effective for learners at all levels.

Furthermore, the paper discusses the use of eye-tracking for data collection and analysis in the submission of “concept matrices” and the review of “posters, books, and research papers.” However, the terms “submission of concept matrices” and “review of posters, books, and research papers” may be somewhat understated expressions. This might be because similar information could also be obtained by monitoring the movements of the learner’s mouse. However, I believe that data collection and analysis of “submission of concept matrices” and “review of posters, books, and research papers” are in themselves of significant importance. Eye-tracking data not only determines what the learners are looking at but also, through blinking and fixations, can indicate the learners’ current state of learning and behavior. I am very much looking forward to future research on how learners’ visual behaviors will be incorporated into learning analytics.

Finally, after reading this paper, I have some reflections to share. Although I am not an expert in self-regulated learning (SRL), I perceive the difficulty in providing the right support without limiting the learners’ ability to advance their SRL. In the SRL process, learners acquire new knowledge according to their own learning habits and styles, yet unnecessary assistance could potentially hinder their learning. Therefore, it is crucial to consider the learners’ own knowledge and needs when providing scaffolding. I am looking forward to future research clarifying the appropriate methods of scaffolding.

By: Li Tang (M2 student)