Hello everyone. I am Ozaki, a first-year Master’s student. I would like to introduce an overview and my thoughts on a paper I read in a recent English literature seminar.

• Title: Artificial Intelligence (AI)-enhanced learning analytics (LA) for supporting career decisions: advantages and challenges

• Journal: Education and Information Technologies

• Volume & Pages: 29:297-322

• Year of Publication: 2024

• Authors: Egle Gedrimiene, Ismail Celik, Antti Kaasila, Kati Mäkitalo, Hanni Muukkonen

Background and Objectives

In modern society, supporting career decision-making is gaining importance. In a social environment filled with high uncertainty, the decision-making process for course selection and career formation has become a major challenge for students and the younger generation. Against this backdrop, AI-enhanced Learning Analytics (LA) tools are drawing attention. These tools have the potential to efficiently support the decision-making process by analyzing vast amounts of data and providing personalized information and alternatives. However, the current reality is that there has not been sufficient examination into how these tools function in actual career decision-making support and what their advantages and challenges are. This study aims to clarify the impact of AI-enhanced LA tools on career decision-making support and provide suggestions for future tool development. In particular, it focuses on clarifying the “usability and benefits of the tools from a user perspective” and the “possibility of utilization during career transitions rather than just for students.”

Theoretical Framework

Two frameworks were adopted as the theoretical basis for this study: the Technology Acceptance Model (TAM) and the Career Decision-Making (CDM) model. TAM is a model that explains technology acceptance through two factors: “usefulness” and “ease of use,” showing that these elements influence intention to use and actual usage behavior. In this study, we analyzed how AI-enhanced LA tools were evaluated by users in terms of these two factors. On the other hand, the CDM model explains career decision-making across three stages: “pre-screening,” “in-depth exploration,” and “final choice.” By using this model, we aimed to clarify at which stage and how AI-enhanced LA tools help users in their career decision-making process. By combining these two theoretical frameworks, we took an approach to comprehensively capture the advantages and challenges of tools in career guidance.

Research Method and Overview

In this study, a survey was conducted with 106 students belonging to vocational education and training institutions in Finland. The survey was conducted using an online open-ended questionnaire centered on five main questions: “In what situations is the tool used?”, “Does the tool help in applying for education or jobs?”, “What are the benefits of the information obtained?”, “Feedback on the tool,” and “Desired additional features.” Through these questions, we collected specific information regarding user experiences, evaluations, expectations, and points for improvement. Data analysis utilized the TAM and CDM frameworks to classify and organize the obtained data from a theoretical perspective. As a result, while the tool was highly rated for providing information and diversifying career paths, challenges were identified regarding visual design and clarity of instructions.

Research Results

In terms of detailed results, it was found that AI-enhanced LA tools function effectively particularly during the “in-depth exploration” and “final choice” stages of the career decision-making process. Specifically, the tool’s information-provision capability plays a vital role when users grasp available options and compare/evaluate them. Furthermore, it was found that the data provided by the tool has the potential to broaden the range of course and career choices. On the other hand, a lack of elements that prompt reflection or provide information about the user’s own values was cited as a challenge. Additionally, from the TAM perspective, while high ratings were obtained for “usefulness,” it was found that improvements in visual design and enhancements to the user interface are required from the viewpoint of “ease of use.” Furthermore, “diversification and individual optimization of information” was identified as a need for further development of LA career guidance tools. Specifically, there was a demand for creating data more optimized for the individual regarding the diversification of ideas for career paths and reflection. These findings provide important suggestions for deeply understanding the impact tools have on career decision-making.

Discussion and Limitations of This Study

Through this study, it became clear that AI-enhanced LA tools are useful in supporting career decision-making, but it was also shown that there is room for further improvement. In particular, there is a demand for functions that prompt user self-reflection through the information provided by the tool, enabling them to make decisions more proactively. Regarding this, it is necessary for the tool to compensate for a lack of self-awareness by increasing information regarding future goals, relevant skills, and personal interests. Furthermore, since the limited amount of information regarding occupations and educational institutions might affect the volume of information provided to users, it is also necessary to consider the collection of more comprehensive information.

Additionally, improvements to visual design and interfaces are expected to not only improve the “ease of use” of the tool but also to enhance the overall user experience. It is thought that career guidance tools are also required to consider how to design improvements to the interface or human support according to user needs.

Moreover, the results of this study indicated the possibility that changes may occur in the user’s decision-making process. Because the information analysis process and the data used by the tool are invisible to the user, with only the final list of recommendations displayed, there is a possibility that this may hinder “learning how to make career decisions,” which is an important goal of the CDM model. Therefore, in development, it is considered important to enable reflection by the user through various methods.

As a limitation of this study, it does not consider the larger socio-economic context thought to be related to individual career choices, such as the social environment, individual aptitude, economic situation, and health status. Various exclusions and biases relate to an individual’s career path and may also influence digital guidance.

The results of this study suggest the importance of incorporating a user perspective in the future development of AI-enhanced career support tools. Through this, it is expected that more people will be able to make career decisions with confidence in a highly uncertain social environment.

My Thoughts

While considering learning designs aimed at promoting career self-regulation, I chose this paper because I wanted to learn how LA can support career decision-making. As a result, I understood that there is potential for utilization in areas such as providing individual information for career decision-making. I also realized that reflection is indeed a critical element in career decision-making and is difficult to promote through information provision and recommendations alone. I concluded that this is an item that should be supported within the overall learning design, rather than relying on LA alone.

On the other hand, I would like to further review preceding research in the future regarding support for reflective activities using LA, such as promoting reflection through portfolio creation on the web and its text analysis, and the possibility of evaluating and improving learners’ career exploration abilities using LA.