Hello everyone. I am Naohiro Higuchi, a second-year master’s student.
This time, I would like to share a paper I introduced in the English seminar.

The topic this time is research on “Learning Analytics,” which is increasingly being introduced into educational settings. In particular, it is an interesting paper that tracks, over a long period, the psychological changes and hurdles teachers face as they accept new digital technologies and analytical tools and incorporate them into their classes.

Paper Information
・Paper Title: Exploring Teachers’ Adoption of Learning Analytics Enhanced Pedagogical Practices: A Longitudinal Intervention Study
(Exploring Teachers’ Adoption of Learning Analytics Enhanced Pedagogical Practices: A Longitudinal Intervention Study)
・Authors: Khulbe, M., Tammets, K., Ley, T., Coelho, R., Kurvits, J., & Cukurova, M.
・Year of Publication: 2025
・Journal: Technology, Knowledge and Learning
・Pages: 1-23

1. Introduction
In recent years, in the context of “personalized learning” and “data-driven instruction” in educational settings, the goal has been for teachers to visualize each student’s learning history and level of understanding as data and utilize it for instruction. However, it is not easy for teachers in the field to actually practice this on a daily basis.

The paper I am introducing this time involved introducing a tool called a “Learning Analytics Dashboard (LA Dashboard),” which analyzes and provides advice on students’ learning status, to mathematics teachers, and investigated how their awareness and behavior changed over a long period of approximately seven months.

The unique aspect of this study is that it is not just a story of “creating a convenient tool and handing it over.” Based on the concept of the “Knowledge Appropriation Model,” it focuses on the process by which teachers master new technology and make it their own.

Specifically, it analyzes how teachers foster “trust” in the tool and how that trust influences their final “adoption intention” (whether they want to continue using it). For new technology to take root in the field, not only the superiority of its functions but also the psychology of the users and social factors are deeply involved.

2. Background
In the field of Educational Technology, it has long been pointed out that “tools considered effective” developed by researchers are often not used in actual classrooms, or are used in ways completely different from the developers’ intentions.

This paper cites the lack of sufficient consideration for “Human-Centered Design” in conventional development as one of the causes. In other words, it suggests that developers have been pursuing technical novelty while ignoring the context and needs that teachers face daily. It is said that for teachers to effectively integrate new technology, they need to possess an integrated knowledge of the three areas known as “TPACK”: Technology, Pedagogy, and Content Knowledge. However, it is pointed out that in reality, many teachers simply replace old teaching methods with digital ones.

The authors argue that the keywords that become important here are “Trust” and “Transparency.”
In particular, in “advisory LA dashboards” where AI or algorithms provide advice to learners, if the teacher cannot trust the tool, it will naturally not be adopted for instruction. Trust here is defined as “an attitude of relying on the tool to achieve one’s goals even in uncertain situations.” And it was found that what supports that trust is “transparency”—the understanding of why that advice was given.

In addition, this study uses the framework of the “Knowledge Appropriation Model (KAM).” This is a concept that emphasizes the process by which teachers, when adopting new technology, do not just learn how to operate it, but “appropriate” the technology to fit their own context through collaboration with colleagues and practice. This study was conducted under the hypothesis that the process of researchers and teachers forming a partnership and creating tools and lessons together is essential for the entrenchment of technology.

3. Method
In this study, a seven-month training program (TPD program) called “TIL4Math” was conducted for 26 mathematics teachers in Estonia. The period was from November 2022 to April 2023, and a very careful and intensive intervention was carried out, involving a cycle of design, execution, and reflection through a 6-hour session once a month.

The goal of the program is to enable teachers themselves to design and implement lessons that utilize digital teaching materials and LA dashboards so that students can acquire “mathematical problem-solving skills” based on the PISA (Programme for International Student Assessment) framework.

The flow of the research is broadly divided into six phases.

・Phase I–II (Understanding/Introduction): First, learn and discuss educational theories and technologies (such as H5P, a teaching material creation tool). The dashboard is not operated at this stage.
・Phase III (Dialogue/Transparency): Here, the prototype of the LA dashboard appears. Teachers receive an explanation of the logic (transparency) behind how data is collected and analyzed, deepening their understanding of the system.
・Phase IV–V (Co-design/Implementation): Teachers work in groups to create digital teaching materials that they will actually have their students solve. Then, they first implement the materials in the classroom without using the dashboard.
・Phase VI (Full Implementation/Reflection): Finally, they conduct lessons using both the created materials and the completed LA dashboard. They support students while referring to the analysis results and advice presented by the dashboard, and reflect on the experience after the lesson.

For data collection, questionnaire surveys were conducted multiple times to measure teachers’ “knowledge,” “digital skills,” “trust in the dashboard,” “perception of transparency,” and “adoption intention” (whether they want to continue using it). Furthermore, at the end of the program, qualitative data such as interviews and written responses were collected to explore the true feelings of the teachers behind the changes in numbers.

4. Results and Discussion
In this study, analysis was conducted on teacher changes due to the intervention (RQ1) and factors influencing adoption intention (RQ2).

RQ1: Teacher changes due to intervention (knowledge, trust, adoption intention)
First, teachers’ “pedagogical knowledge” improved significantly through the training, but there was no statistically significant difference in “digital skills.”
“Trust” was higher after actual use in the classroom (Phase VI) than at the stage of seeing the prototype (Phase III). It can be said that confirming how it actually functions in the field was most effective for building trust.
On the other hand, “adoption intention” did not simply increase steadily. It followed a “V-shaped” transition, where the initially high motivation dropped significantly at the material design stage (Phase III) and recovered after implementation (Phase VI). This suggests that motivation temporarily declined as they faced the complexity of creating ideal teaching materials.

RQ2: Changes in factors determining adoption intention
In the initial stage (Phase III), “trust in the tool” strongly influenced adoption intention. However, in the final stage (Phase VI), the influence of reliability decreased, and instead, the degree of engagement in “Knowledge Appropriation Practice”—the process of collaborating with colleagues and incorporating technology into one’s own practice—became the biggest factor determining adoption intention.
In other words, whether they ultimately want to continue using it depends not on the performance of the tool, but on whether they were able to “integrate it as part of their own practice.”

Challenges
From the qualitative survey, the “burden of material design” emerged as a major challenge. Since creating high-quality teaching materials based on educational theory takes time, it is thought that this sense of burden was one of the factors that caused adoption intention to remain stagnant even as trust in the tool increased.

5. Conclusion and Limitations
As a conclusion of this study, it was shown that whether teachers adopt new educational technology (especially advanced ones like LA dashboards) involves a complex interplay of social and psychological factors, not just simple technical factors.

A particularly important finding is the change in the role of “reliability.” In the early stages of introduction, it is extremely important for teachers to think, “This tool is not suspicious; it is reliable.” However, once they enter the implementation stage, reliability becomes a prerequisite that is “taken for granted,” and the process of “how it was integrated into our practice” and “how we collaborated with colleagues to create it” becomes more important.
Therefore, in Educational Technology interventions, it can be said that the key to dissemination is not just handing over and explaining tools, but carefully designing the “place” and “time” for teachers to collaborate and create lessons.

Of course, this study also has limitations. Since there were only 26 participants and they were teachers who participated voluntarily (and were highly motivated), it is unclear whether this can be applied directly to the entire general school setting. Also, other factors not separated in this analysis may have had an influence. Nevertheless, the data that followed the consciousness changes of teachers in the field over a long period of seven months provides very valuable suggestions for advancing future educational DX.

6. Reasons for Selection and Impressions
The reason I chose this paper is that I am currently conducting development and intervention research on an LAD (Learning Analytics Dashboard) for teachers in the field to use. In particular, I wanted to learn from concrete examples about the process of at what stage to present a prototype and how to involve teachers in the evaluation.

As an impression, the design of the “phases” was very helpful. From the system development side, the method of dividing even small functional additions into “phases” and having teachers touch them step-by-step to deepen their understanding and sense of conviction (appropriation) without strain is a point I would like to incorporate into my own future research design.

I also learned from the fact that a solid framework (in this case, KAM, TPACK, etc.) was used for evaluation. By measuring consciousness change based on theory rather than just asking, “Was it easy to use?”, it becomes possible to scientifically explain why adoption intention dropped and what the biggest factor was.

On the other hand, I would like to know a little more about the specific operational aspects, such as how the text of the “suggestion” presented by the dashboard is generated (whether it is rule-based or generative AI-like), and whether teachers look at it during class (real-time) or during reflection after class.

In any case, I was greatly stimulated by the stance of this research, which creates new teaching styles together with teachers in the field. I would like to devote myself to research so that I can also develop systems that are truly valuable to the field.

Text by: Naohiro Higuchi