Yamada Laboratory, Kyushu University

How to Create and Utilize Knowledge Maps to Support the Modeling of Learning Behaviors in Digital Learning Environments

2026年02月26日

Hello everyone. I am Li, a research student. I would like to introduce a paper I read in a recent English Literature Seminar, incorporating its content along with my own thoughts.

Paper Title: Knowledge Map Creation for Modeling Learning Behaviors in Digital Learning Environments Year of Publication: 2019 Authors: Brendan Flanagan, Rwitajit Majumdar, Gökhan Akçapınar, Jingyun Wang and Hiroaki Ogata Journal: Companion Proceedings of the 9th International Conference on Learning Analytics and Knowledge (LAK’19) Pages: 428-436

The following is a full translation of the content described in the paper.

Overview

Along with the rapid development of educational technology in digital learning environments, modeling and analyzing students’ knowledge states using learning behavior data has become an important direction in educational research. A “Knowledge Map,” which visually represents key concepts of a knowledge structure and their interrelationships, is a vital tool for promoting “meaningful learning” as proposed by Ausubel (1963). However, conventional knowledge map creation is typically performed manually by experts and requires a significant amount of time and labor, making it difficult to respond to dynamically changing educational needs. Furthermore, previous research has primarily focused on modeling learners by linking test results with knowledge maps.

To address these challenges, Flanagan et al. proposed a framework capable of automatically generating and managing knowledge maps from unstructured text. This framework provides a new solution for the educational field by utilizing learning analytics technology to link students’ learning behaviors with knowledge maps.

System Design and Functions

This study aims to design and implement a “Knowledge Extraction Processor” and integrate it with existing Learning Analytics (LA) infrastructure.

The Knowledge Extraction Processor analyzes digital learning materials (such as PDF documents) to extract important concepts and generates a knowledge map based on them. The generated knowledge maps are saved in a Knowledge Map Store, where teachers can manage and revise them. Students and teachers can access these maps through a user-facing Knowledge Portal to confirm the relationships between concepts related to the learning content and adjust learning plans based on learning behavior data (such as attendance, material viewing, and test results).

This system emphasizes integration with existing LA infrastructure, realizing coordination with Learning Management Systems (LMS), digital reader tools, and testing systems. Through a Learning Record Store (LRS) and an analysis processor, students’ learning behavior data is collected in real-time and mapped onto the nodes of the knowledge map. In this way, students’ knowledge states and learning behaviors are intuitively visualized, making it possible for students to better understand their own learning progress.

Knowledge Map Generation Method

To extract meaningful knowledge concepts from unstructured learning materials, this system adopts a knowledge map generation method based on text mining. First, a PDF analysis tool is used to extract the text content of the material, and a morphological analysis tool (e.g., MeCab) is used to segment the text and tag parts of speech. Next, significant noun phrases are extracted from the sentences, and a word co-occurrence matrix is constructed to identify relationships between concepts. Finally, a minimum spanning tree algorithm is used to optimize the complexity of the graph, leaving only the strongest concept associations.

The generated knowledge map is not used directly for teaching but is saved in the system as a first draft. Teachers can further review and adjust the map through an editing portal. This process includes deleting unnecessary nodes, adding missing important concepts, and linking relevant learning resources or test questions. These procedures not only improve the efficiency of knowledge map generation but also ensure the accuracy and practicality of the map.

Experiment and Evaluation

To verify the effectiveness of the system, the research team conducted an experiment in an information science class. Comparing the automatically generated knowledge map with a manually created “gold standard” map, the results showed that the automatically generated map demonstrated good performance in terms of precision and recall, particularly in retaining major important nodes at low complexity thresholds.

While automatically generated maps still require manual adjustment by teachers, it became clear that they can significantly reduce the time and labor involved compared to creating a map entirely by hand.

Application Scenarios and Potential Value

The application of the knowledge map system is not limited to visualizing knowledge structures; it also plays an important role in analyzing learning behaviors and determining educational policies.

For students, the knowledge map functions as a learning navigation tool, helping them grasp what knowledge they have already acquired and what they still need to learn. Additionally, they can fill knowledge gaps by utilizing linked learning materials and test questions. Furthermore, the knowledge map supports students in conducting long-term reviews and reflections on their knowledge, helping them track the accumulation of knowledge over time and identify weaknesses in their knowledge systems.

For teachers, on the other hand, the knowledge map serves as a powerful tool for grasping the overall picture. For example, by integrating the knowledge maps of all students to create a single comprehensive map, it becomes possible to identify knowledge blind spots common to students and adjust lesson content or pacing. Moreover, by utilizing this system, teachers can understand how important a specific knowledge concept is within the academic system or what content is most difficult for students.

My Thoughts

I feel that this study has two main points of originality. The first is that it proposed a method for automatically generating knowledge maps from unstructured text data, resolving the complexity and difficulty of the conventional knowledge map construction process. The second is that it integrated behavioral data other than students’ test performance into the knowledge map, reflecting students’ learning status more comprehensively. This provided reference information for students and teachers when adjusting learning or lesson plans. While these innovations were advanced for the research at that time, looking at them from a current perspective, I feel there is still significant room for improvement.

Regarding the knowledge map generation part, since five years have passed since this paper was published, the algorithms proposed in the study have already become dated. I truly feel the rapid pace of development in information technology. Currently, there is an increasing amount of research utilizing pre-trained Large Language Models (LLMs) to generate knowledge maps from unstructured text. This method can more effectively extract knowledge points and their logical relationships from large amounts of unstructured text, significantly reducing the cost of manual correction of knowledge maps. However, due to issues such as the ambiguity inherent in text itself, knowledge maps generated from text often have low precision. Overcoming the intrinsic limitations of text to generate more accurate knowledge maps is one of the important challenges in my future research.

Regarding the function of integrating students’ learning status into the knowledge map, this study incorporates not only students’ test score data but also behavioral data such as attendance, material viewing, and answering, objectively showing students’ learning states from multiple perspectives. However, I believe that while these behavioral data and test scores are essentially classified as manifest data, and the volume of data increases, the dimensionality of the data has not been substantially expanded. With technical progress, it is now possible to collect more diverse student behavioral data such as clicks, page transitions, dwell time, and annotations. For example, these data are showing a widening range of applications, such as effectively reflecting student learning status in Digital Game-Based Assessments (DGBAs).

Since I am also planning to conduct research related to knowledge maps, this paper has given me significant insights into my research direction. In the future, I aim to proceed with improving the algorithms shown in this and related studies, while also considering the utilization of more diverse and meaningful behavioral data. By combining this with a scientific analysis framework, I hope to reflect students’ learning status more comprehensively.

PAGE TOP