Hello everyone. I am Li, a research student. I would like to introduce the content of a paper I read in a recent English Literature Seminar along with my personal thoughts.

• Paper Title: Interpretable early warning recommendations in interactive learning environments: a deep-neural network approach based on learning behavior knowledge graph

• Year of Publication: 2023

• Author: Xiaona Xia & Wanxue Qi

• Journal: Humanities and Social Sciences Communications

• Volume/Issue: 10

Background

With the rapid development of information technology, interactive learning environments are widely used in the educational field. These environments break through traditional temporal and spatial constraints, enabling online collaboration, individual optimization, and data-driven decision-making. However, learners often exhibit numerous inefficient or ineffective learning behaviors in such environments, and there is an urgent need for methods to detect problems early and intervene. The interpretable early warning recommendation mechanism proposed in this study was designed to respond to this current situation. The main idea is to perform time-series modeling and semantic interpretation of vast amounts of learning behavior data using knowledge graphs and deep neural networks (DNNs) to identify critical periods and behavioral patterns. This presents a scientific basis for learning intervention.

Problems and Research Objectives

This paper points out the following three major problems in interactive learning environments:

1. Diversity of Behavioral Patterns and Difficulty in Evaluation Learning behaviors exhibit diversity, but due to the complex non-linear and time-series characteristics of the data, it is difficult to judge from static indicators alone which behaviors are truly effective and which might be masking potential risks.

2. Lack of Correlation Between Timing and Content In many cases, learning content lacks clear time-series divisions, and its internal logical relationships or conceptual associations are not sufficiently clear. Consequently, it becomes difficult to accurately identify a learner’s weaknesses when making content recommendations or providing behavioral guidance.

3. Limitations of Individual Cognition and Knowledge Structure Due to differences in learners’ own cognitive levels and knowledge structures, it is difficult to quickly construct effective learning behavior patterns suited to oneself, which directly impacts learning outcomes.

Based on this, the objective of this study is to design an interpretable early warning recommendation mechanism that combines knowledge graphs and deep neural networks. This mechanism aims not only to achieve high prediction accuracy in a large-scale data environment but also to clarify the internal logic of predictions through time-series analysis and vector decomposition, providing an intuitive basis for intervention measures.

Technology and Evaluation

In this study, feature extraction is first performed using a Convolutional Neural Network (CNN) on time-series data generated during the interactive learning process. This allows for obtaining representative weight vectors. Next, the obtained weight vectors are analyzed using vector decomposition techniques and decomposed into multiple interpretable feature components corresponding to learners, learning content, concepts, and more. This method improves prediction accuracy while providing detailed explanatory information for each prediction result.

At the same time, to fully clarify the intrinsic semantic relationships among elements in the learning process, a knowledge graph centered on learners, learning content, and concepts was constructed. This graph defines relationships such as “inclusion,” “hierarchy,” and “prerequisite,” clearly showing the interactions and dependencies between each entity. This provides semantic support for the learning of the deep neural network.

Furthermore, through the preprocessing and cleaning of large amounts of data, significant time periods in the learning process were identified. In the experiments, it was confirmed that specific periods—such as weeks 3–7 or weeks 15–19—hold prominent meaning in early warning, serving as the theoretical basis for intervention strategies. Finally, the overall effectiveness of the system was systematically evaluated using indicators such as AUC, F1 score, and Multi-Task Learning Gain (MTL-Gain). As a result, the proposed method showed high accuracy and robustness in identifying risky behaviors and recommending intervention strategies.

Results and Discussion

From the experimental results, the model constructed in this study not only achieved high AUC and F1 values in terms of prediction but also demonstrated a useful contribution in terms of interpreting learning behaviors. Specifically, through vector decomposition and the construction of the knowledge graph, it was possible to clearly show the contribution of features behind each prediction result. This allows teachers and learners to intuitively grasp which learning behaviors play an important role in the prediction.

Additionally, the experiments revealed that in some learning content, specific periods such as weeks 3–7 or weeks 15–19 have a clear warning function. These periods may function as a “golden time” for intervention for learners who have not yet mastered key concepts. However, this research focused on offline simulation verification based on large-scale crowd data of 1.3 PB, and intervention experiments in actual educational settings have not yet been conducted. In the future, it is necessary to conduct evaluations targeting actual users and verify the actual effects on improving learning pass rates and behavioral patterns.

Personal Thoughts

The innovation of this study lies in the integration of a Learning Behavior Knowledge Graph (LBKG) and a Deep Neural Network (DNN) to propose a practical early warning system. Unlike conventional methods, this system does not treat learning behavior as mere historical data; instead, it utilizes knowledge graphs to clarify relationships between concepts and combines this with time-series analysis to identify critical timings for learning. Because of these characteristics, it can provide feedback for concrete improvements rather than just performance prediction for the learner, making it highly practical with potential for application in the educational field.

From the experimental results, this model surpassed conventional methods in terms of AUC and F1 scores, and an improvement in its accuracy was confirmed. However, several challenges remain in this study, the largest of which is that the system has not undergone verification by actual learners. While this research conducted simulation verification using 1.3 PB of learning data, intervention experiments in actual educational settings remain unconducted, and trials introducing the system to students to compare learning outcomes or behavioral changes have not yet been performed. Consequently, it is unclear whether this system will function effectively in an actual educational environment.

Furthermore, an individually adaptive recommendation system that considers the characteristics of each learner and real-time learning support has not been implemented in this system, and due to this lack, the advice given to learners may not be sufficiently appropriate.

While this research holds great potential in the field of educational data analysis, it actually needs further optimization. Experiments in educational settings and the development of an adaptive recommendation system that considers the characteristics of each individual learner, as mentioned above, will be the focus of my future research.