Hello everyone. This is Tanaka, a first-year doctoral student. In this post, I will introduce a paper I read at the recent English seminar.

Paper Title: Decoding GenAI-assisted Academic Writing Processes: Insights from Lag Sequential and Epistemic Network
(Decoding GenAI-assisted Academic Writing Processes: Insights from Lag Sequential and Epistemic Network Analysis)
Authors: Belle Dang, Andy Nguyen, and Yvonne Hong
Year of Publication: 2024
Journal/Conference: Pacific-Asia Conference on Information Systems (PACIS 2024), Ho Chi Minh City 2024.

1. Abstract
This study analyzes GenAI-assisted academic writing in higher education (doctoral level) using learning process data (10 participants, 626 actions in total) to examine the behavioral sequences and epistemic connections that distinguish high-performing students from low-performing ones. While both groups used GenAI, high-performing students tended to integrate the evaluation of generated content with reading literature and writing, whereas low-performing students tended to rely on pasting generated text and minor paraphrasing. As an educational implication, the integration of scaffolding prompts to deepen interaction with GenAI is proposed.

2. Background and Theoretical Framework
• Unlike traditional proofreading tools, GenAI can interactively support generation and revision, but there are concerns that it may undermine originality, academic integrity, and the development of critical thinking. Furthermore, the study identifies a gap in existing research, which often focuses on perceptions of GenAI, noting that how GenAI is actually incorporated into the writing process has not been sufficiently elucidated.
• This study adopts a quantitative ethnography framework to view learner behavior as a process rather than an “outcome.” The research questions examine how GenAI is incorporated into the writing process through (1) Lag Sequential Analysis (LSA) to investigate the sequential chain of behaviors, and (2) Epistemic Network Analysis (ENA) to visualize the clusters of activities.
• As a theoretical background, the study first refers to the cognitive process model by Flower & Hayes (1981), which views writing as an iterative process of planning, translating, and reviewing. Next, from the perspective of Cognitive Load Theory, it is stated that while GenAI can reduce extraneous load, it may lead to dependency if it replaces thinking and reflection. Furthermore, based on Affordance Theory (Gibson, 1977), it is argued that since the effects of features like immediate feedback change depending on the learner’s perception and usage, it is necessary to analyze not just “what can be done” but “how it was used” as a process.

3. Research Methodology
Ten doctoral students from universities in Finland and New Zealand wrote a 30-minute, approximately 500-word short essay (topic: AI use in education) on Zoom. The use of ChatGPT, Google Scholar, etc., was permitted, and the sequence of actions during writing was captured via screen recording. Performance was graded by university faculty based on five criteria (content, analysis, structure, writing quality, and word count & APA referencing), and participants were classified into high-performing and low-performing groups based on the median.

4. Analysis (LSA and ENA)
• LSA (Lag Sequential Analysis) examines “which order behaviors are likely to occur in.” It distinguishes between lag1 (immediate transition) and lag2 (transition with one intervening action), with the significance of transitions based on z ≥ 1.96.
• ENA (Epistemic Network Analysis) visualizes how multiple activity codes are connected as a “cluster” in a network. The unit was “participant × performance group,” the moving window was 4 lines (the current line + the previous 3 lines) to handle co-occurrence, and group differences were confirmed using the Mann-Whitney U test.

5. Results
Differences in Descriptive Statistics
The results report that the high-performing group had longer sequences and relatively more instances of reviewing generated content, reading papers, and citing, while the low-performing group had more instances of pasting generated content and fewer instances of reviewing generated content. It is summarized that the high-performing group showed iterative and reflective usage, such as “prompt → review → integrated writing,” while the low-performing group showed usage leaning toward “direct application of generated output.”

Differences in LSA (Behavioral Chains)
• In the high-performing group, significant transitions were observed from repeating prompts or checking task requirements to prompting, indicating that they adjusted their dialogue according to the task. Also, as an example of lag2, an iterative pattern of integrating while moving back and forth between reviewing generated content and writing was shown, suggesting the possibility that they were integrating the output into their own writing while inspecting it.
• In the low-performing group, the focus was on moving back and forth between pasting and paraphrasing, and it is stated that significant transitions related to reviewing generated content were difficult to see. As a result, the performance gap is manifested as a difference in usage strategy: “evaluating and integrating generated output” versus “directly applying generated output.”

Differences in ENA (Clusters of Activities)
In ENA, it is explained that while the high-performing group showed clusters of activities centered on reading papers, prompting, reviewing generated content, and writing, the low-performing group had relatively weak connections between “reading/reviewing” and “application (writing),” resulting in greater fragmentation. As a difference in connectivity, it is shown that the high-performing group integrated output into writing after evaluation, whereas the low-performing group tended to use it as a “shortcut of generation → pasting.”

6. Implications and Limitations
The methodological contribution is positioned as combining LSA (sequence) and ENA (relationship) to show the interaction with GenAI from both sides. Furthermore, the need to integrate scaffolding prompts that encourage “more reading” and “critical evaluation of content” before and after draft generation is suggested to engage learners more deeply in the interaction with GenAI. The need for personalized support adjustment is also mentioned. As for limitations, the small sample size and the potential influence of prior writing skills and AI literacy differences are cited. It is summarized that future research should aim for larger-scale studies, the addition of qualitative research, comparisons across fields and educational stages, and longitudinal studies.

Reflections
I selected this paper because I intend to incorporate ENA into my own research in the future, and it examines the English writing process under GenAI assistance, which is also my research theme. It was very helpful to see the visualization of writing processes where GenAI usage methods differ depending on performance levels.
This study shows superficial GenAI usage in the low-performing group, and there is a concern that if writing without deep thinking becomes routine, it will trigger a negative spiral leading to stagnation in learning. To avoid inducing such superficial processes, I believe it is important to integrate scaffolding that optimizes the affordances of GenAI into course design and systems. Specifically, I think it is necessary to integrate “scaffolding prompts” that are individually optimized according to the learner’s proficiency, such as providing intensive support to encourage review of generated content and verification of evidence for low-performing learners, and offering higher-level suggestions such as refinement of arguments for high-performing learners.
Although this study analyzes a single 30-minute task, I would like to proceed with longitudinal research that tracks changes in long-term learning processes using methods like LSA × ENA with a larger sample in the future.