Hello everyone, I am Xuewang, Geng. In this article, I would like to introduce a paper I read in the English Literature Seminar and my thoughts on it.
-
Paper Title: Dissecting learning tactics in MOOC using ordered network analysis
-
Journal: Journal of Computer Assisted Learning
-
Volume/Pages: 39(1), 154-166
-
Year: 2023
-
Authors: Yizhou Fan, Yuanru Tan, Mladen Raković, Yeyu Wang, Zhiqiang Cai, David Williamson Shaffer, Dragan Gašević
Overview of the Paper: The number of people using MOOCs (Massive Open Online Courses) is increasing because they allow users to take courses over the internet for free or at low cost. However, unlike face-to-face classes, MOOCs offer limited immediate support from teachers or classmates, and it is reported that students frequently drop out mid-course, with dropout rates reaching 75% to 90%. On the other hand, previous research suggests that students who use self-regulated learning strategies have higher course completion rates (e.g., Kizilcec et al., 2017). Therefore, research is underway to investigate what kind of learning strategies learners use in MOOCs. The authors summarized four aspects of analyzing learning strategies from learning behaviors based on previous research:
-
Frequency: How many times a behavior occurs in a learning strategy.
-
Continuity: Whether a behavior is performed repeatedly.
-
Sequence: The order of behavior occurrence, considering the time series of performing a behavior before or after other behaviors.
-
Role: The role that a behavior itself plays in the overall learning strategy.
While many studies identify strategies from the aspects of frequency, continuity, and temporal sequence of learning behaviors, few analyses consider the “role” of the behavior. In this study, Ordered Network Analysis (ONA) was cited as a method for analyzing learning strategies, and by comparing it with process mining, the researchers investigated to what extent learning strategies could be identified across the four aspects mentioned above.
Process mining clarifies the transitions between learning behaviors using algorithms such as Heuristic Miner, Inductive Miner, and Fuzzy Miner, and generates a process map (transition diagram) of the behaviors. A process map represents learning behaviors as nodes and observed behavior transition probabilities as edges. Therefore, process mining focuses on the transition of learning behaviors and is particularly useful for analyzing sequence and continuity. On the other hand, ONA extends Epistemic Network Analysis (ENA) to quantify the connections between behaviors while considering the order of behaviors, visualizing these connections in a network model. In the generated network graph, not only the frequency of actions and the strength and direction of connections but also meaningful metrics based on position are displayed.
To compare process mining and ONA, this study conducted an experiment and collected learning data. In the experiment, data on learning behaviors were collected from 8,788 learners who participated in a 7-week MOOC “Flipped Classroom” course. Every week, students were required to spend 3 to 5 hours watching videos, participating in discussions, and completing quizzes and peer reviews. From the learning logs, behaviors were classified into nine categories, such as content access, content re-watching, discussion, and forum participation, and learning sessions were divided. In process mining, learning sessions were analyzed using a first-order Markov model and an Expectation-Maximization (EM) algorithm to identify behavior transition sequences, and learning sessions with similar behavior patterns were classified into the same strategy. Based on the process mining results, a behavior transition diagram was created. In the transition diagram, each node is a behavior, and arrows between nodes represent the transition probabilities between those behaviors. In ONA, the presence or absence of learning behaviors in the identified learning strategies was binary-coded, and the transitions between learning behaviors were modeled. The ONA results were visualized as a network graph.
From the analysis results, eight learning strategies were identified. In this study, one of these strategies was selected, and the results of process mining and ONA were explained in terms of the four aspects: frequency, continuity, sequence, and role of learning behaviors. In the process mining results, continuity and sequence of learning behaviors were confirmed. From the nodes and arrows of the transition diagram, behaviors such as continuous use of forums and searching, and transitions between behaviors like assessment and forum use, became clear. However, the transition diagram did not display the frequency of behaviors, and the roles of different learning behaviors were not elucidated.
On the other hand, from the ONA network graph, the frequency of behaviors and the frequency of self-transitions could be judged by the size of the nodes, showing high-frequency behaviors such as assessment, content re-watching, and overview, as well as the continuous performance of forum and search behaviors. Furthermore, from the color and direction of the arrows, it was confirmed that students most frequently viewed general course information (overview) before working on quizzes or assignments (assessment). Additionally, at the bottom of the network were the two main learning behaviors in MOOCs—access to content and assessment—while the top included supplementary or optional learning behaviors such as re-watching content, searching, and requesting help. Thus, it was found that the network vertically distinguishes between primary and supplementary learning behaviors. The nodes related to learning new content located on the far left suggested a tendency to participate in forums and evaluations rather than studying new content, due to the relatively weak overall connections. Therefore, ONA was able to interpret the relative roles and relevance of learning behaviors from the spatial arrangement of nodes. Based on the above, the authors report that ONA is more effective than process mining in the four aspects of analyzing learning strategies (frequency, continuity, sequence, and role).
My Thoughts: This study compares different analytical methods—process mining and ONA—and provides detailed explanations for each approach. This deepened my understanding of both methods and provided insights into the roles of learning behaviors, which was very helpful. This study analyzes “Learning Tactics” focusing on behavioral characteristics, but I am also very interested in how ONA can be utilized for analyzing “Learning Strategies” that integrate multiple Learning Tactics. Furthermore, I would like to know about the analytical methods necessary to grasp the overall picture of the self-regulated learning process at a higher level. The development of learning process analysis at different granularities is a very interesting point. I also look forward to future research on how these analytical results can help educators, such as instructors, improve the learning environment. Learning process analysis is often used in the research of the Yamada Lab, but since there are various methods, I feel we should actively incorporate them into our research approach.
