Hello everyone. I am Tanaka, a second-year doctoral student. In this article, I would like to introduce a paper we read at our recent English seminar.

Paper Title: Leveraging Process-Action Epistemic Network Analysis to Illuminate Student Self-Regulated Learning with a Socratic Chatbot
Authors: Joel Weijia Lai, Wei Qiu, Maung Thway, Lei Zhang, Nurabidah Binti Jamil, Chit Lin Su,
Samuel S. H. Ng, Fun Siong Lim
Journal: Journal of Learning Analytics (2025)

1. Background and Previous Research
Learning Analytics (LA) is a widely used method for understanding learning processes and improving educational environments. This field focuses on clarifying how learners engage with the learning process. “Self-Regulated Learning (SRL),” which is the focus of this study, refers to the process by which learners actively manage their own learning environments, behaviors, and strategies to achieve their goals. Previous studies have shown that SRL has positive effects on academic achievement, metacognition, self-efficacy, motivation, and even emotional resilience. In recent years, there has also been a growing trend toward viewing SRL not as a fixed stage model, but as a “dynamic and adaptive process” that changes according to the situation.
In modern educational settings, opportunities for students to utilize generative AI for learning are rapidly increasing, but how this affects students’ Self-Regulated Learning (SRL) has not yet been sufficiently verified. As students fluidly engage in questioning, confirming, and revising their understanding during interactions with AI, this perspective of SRL becomes essential for achieving effective learning. Therefore, this study aims to capture dialogue data between students and generative AI chatbots as clues for understanding SRL and to examine its characteristics.

2. Research Questions
While generative AI holds great potential for personalized support, existing research has not sufficiently examined how interactions with AI can be analyzed from the perspective of Self-Regulated Learning. Given this gap, this study establishes the following two research questions (RQs).
RQ1: How can conversations with educational chatbots be processed and analyzed from the perspective of Self-Regulated Learning?
RQ2: What are the similarities and differences in interactions with the chatbot depending on students’ proficiency levels and the extent of their learning gains?

3. Research Methodology
Development of chatbots:
In this study, we used a chatbot (based on GPT-4) grounded in the “Socratic method,” which prompts students to think for themselves by asking questions in return. The design encourages students to think independently and regulate their own learning rather than relying on AI. By referencing the course knowledge base and past conversation history, the chatbot breaks down students’ questions step-by-step and provides follow-up inquiries.

Experimental Procedure:
The study was conducted with 34 undergraduate students enrolled in an introductory statistics course at the freshman level. It was carried out over a three-week blended learning period (the first week was face-to-face, and the second and third weeks were online, involving the viewing of video materials and the use of a chatbot), and the overall flow consisted of a “pre-test → three weeks of chatbot usage → post-test.”

SRL Framework and Coding: “Process-Action Framework”
The collected conversation logs were coded based on the “Process-Action Framework” from previous research (Lai, 2024), categorized into four Self-Regulated Learning processes (Defining / Seeking / Engaging / Reflecting) and nine associated specific actions (e.g., goal setting, comprehension checking, information organizing).

Network Analysis Methods:
For the analysis, we used “ENA (Epistemic Network Analysis),” which analyzes connections between codes, and “ONA (Ordered Network Analysis),” which can distinguish and analyze the sequence of behaviors. Using these, we conducted a comparative analysis for each “ability group (A1/A2)” based on pre-test scores and “growth group (G1/G2/G3)” based on test improvement.

4. Results and Discussion
Comparing the pre- and post-tests, students’ scores improved significantly, and a large effect size was confirmed. In the initial code distribution of the coded conversation logs (6,716 lines in total), reflecting the chatbot design, “Checking understanding or answers (E. RV)” and “Seeking information or explanations (S.S)” appeared frequently. Based on network analysis using this data, the following characteristics were revealed for each group.

Differences by proficiency level in the pre-test (Group A1: low score group vs. Group A2: high score group):
A2 showed more complex learning behaviors than A1, with actions such as goal setting (D.G), monitoring comprehension (E. RV), and organizing information (E.O) being strongly interconnected. There is a tendency not only to acquire knowledge but also to manage what should be learned and to systematize information.
On the other hand, A1 showed a tendency to lean toward information seeking (S.S). Epistemic Network Analysis (ENA) revealed that for A1, the transition frequency of “searching for information immediately after confirming (E. RV → S.S)” was approximately twice as high as “confirming after searching for information (S.S → E. RV),” indicating a reactive behavioral pattern. Based on these results, the authors state that for the group with lower initial ability, it is necessary to provide not only information but also “scaffolding” (guidance) that leads them toward goal setting and information organization.

Comparison between groups based on the gain from pre- to post-test (G1: small gain, G2: medium gain, G3: large gain):
As a result of the ONA analysis, significant differences were observed between G1 and G2, and between G2 and G3, but no significant difference was found between G1 and G3. This is interpreted as being because many in G3 were originally low-ability students with room for growth, while many in G1 were high-ability students near the ceiling effect.
G3 (and G1, which shares common characteristics), which showed significant growth, strongly exhibited cycles of information seeking and verification (S.S ⇄ E. RV) as well as behaviors of requesting examples and practicing (E. RH), engaging in interactions such as asking about basic concepts, confirming responses, and further requesting examples. On the other hand, G2, which showed moderate growth, was characterized by cycles of problem identification and verification (D.I ⇄ E. RV), which is interpreted as a passive tendency of simply inputting task questions as they are and confirming the answers.
These results indicate that Self-Regulated Learning is not a fixed sequence of “goal setting → performance → self-reflection,” but rather a process that fluidly shifts according to the situation, confirming that chat logs are a useful data source for capturing these dynamic aspects. Furthermore, it was suggested that the design of the chatbot itself can elicit Self-Regulated Learning behaviors in students, and that there is potential for providing personalized support by monitoring logs in real-time from the perspective of Learning Analytics.

5. Conclusion and Limitations
Conclusion: It was demonstrated that by appropriately coding conversation logs with generative AI using the “Process-Action Framework” and employing methods such as ONA, it is possible to visualize differences in the network structure of fluid Self-Regulated Learning behaviors according to students’ abilities and growth. The significance of this study lies in utilizing conversation logs not merely as usage records, but as learning data for analyzing Self-Regulated Learning.
Limitations: The limitations include the small sample size of 34 participants, the lack of a control group which prevents a causal conclusion regarding the chatbot’s effectiveness, and the fact that only student utterances were analyzed, excluding an analysis of the chatbot’s responses.

Impressions
In this paper, I found it very instructive that the interactions (dialogue logs) between students and chatbots were classified based on the SRL “process-action framework,” and that the connections and sequences of actions were further analyzed using ENA and ONA. In particular, the suggestion that the way students engage with chatbots may differ depending on their grades and the growth of their learning outcomes is something I feel is important when considering learning support using generative AI.
On the other hand, I feel that there are parts where it is difficult to make judgments when classifying data into Self-Regulated Learning codes. Depending on the context, there is a possibility that interpretations may differ as to whether a student’s utterance is merely a confirmation, or if it is monitoring of understanding or self-evaluation. I believe it is important to clarify and make transparent the criteria for coding. Furthermore, I thought it would be necessary to include the chatbot’s questions and responses in the analysis, not just the students’ utterances. Since students’ responses can be influenced by the chatbot’s prompts, analyzing only the students’ utterances might make it difficult to distinguish whether it is a spontaneous Self-Regulated Learning behavior or a reaction to the chatbot. Additionally, I felt that if the total number of chats and the frequency of use were also shown, it would be easier to interpret the results of comparisons between groups. I would like to apply the methodology of interpreting learning behaviors from the perspective of Self-Regulated Learning in this study to my own research.