Hello everyone. This is Ozaki, a third-year master’s student.

In this article, I will introduce a study that utilizes educational games as a tool to assess students’ vocational interests.

Authors: Gutierrez, R., Navarro, A. M., Villegas-Ch, W., & Mera-Navarrete, A.

Title: The Use of Interactive Narratives in Educational Games to Assess Vocational Interests: An Application of the RIASEC Test Integrated With OpenAI.

Journal: IEEE Transactions on Learning Technologies.

Year: 2025

1. Introduction: Why combine “games” and “AI”?

In modern educational settings, guiding students to identify their own vocational interests for future career choices has become an essential task. A representative example of traditional assessment methods is the “RIASEC test.” Based on Holland’s theory of vocational choice, this is a highly useful tool for assessing people’s interests across six categories. On the other hand, the monotony of traditional test formats can potentially undermine the authenticity of responses. Furthermore, these assessments are often perceived as disconnected from the context of students’ real lives and engagement mechanisms.

This is where the use of “educational games” is gaining attention. Previous research has demonstrated that using games in career education can simulate real-world contexts, help students identify their interests and aptitudes in a risk-free environment, enhance career decision-making, and make vocational exploration more accessible and meaningful. However, static and predefined decision paths that limit personalization and adaptability can reduce the authenticity of vocational responses. Interactive narratives deepen immersion and personalize the learning experience.

The purpose of this paper is to develop and evaluate a serious game that leverages a gaming approach to embed RIASEC test questions into interactive scenarios, thereby achieving natural decision-making and reducing the feeling of undergoing a formal assessment. The innovation of this paper lies in the integration of an LLM (GPT-4 Turbo) to improve both the student’s experience and the methodological reliability of vocational assessment through personalized game scenarios.

2. Research Methodology

Game Description and Scenarios

The scenarios integrate RIASEC test questions into a narrative, following the adventures of a character who makes realistic decisions that are immersive for the student. Each question reflects a test item, with the aim of preventing students from distorting their answers due to the awareness that they are being evaluated.

Technically, the Unity platform is used and integrated with the OpenAI API to realize real-time, dynamic content generation based on responses. The story consists of multiple chapters, introducing diverse scenarios. RIASEC test questions are not presented directly but are embedded as decisions within the narrative.

For example, a choice between helping a character in a predicament or prioritizing a mission reflects either social or enterprising interests. AI-based algorithms dynamically adapt questions to the student’s previous choices and the flow of the story, ensuring a personalized experience.

Algorithms and Technology

Regarding the use of AI, the GPT-4 Turbo model accessed via the OpenAI API is employed for scenario generation. An example of a prompt to OpenAI when generating scenarios with AI is: “The player is on an exploration mission in a remote colony and encounters a person asking for device repairs. Generate a story where the player chooses whether to continue the mission or stay to help with the repairs.” The scenario generated by OpenAI in response was: “Lyla looks at you with urgency in her eyes. ‘I can’t fix this alone, can you help me?’ If you help, there is a risk of being late for your meeting. If you continue, you will complete the mission sooner. What will you do?” The player’s progress and RIASEC test responses are saved in a database.

Participants and Procedure

The participants were high school seniors or college freshmen who are at a critical stage of vocational decision-making and have a clear interest in career guidance. The sample size was 200, divided equally between 100 high school students and 100 college students, with a 1:1 gender ratio maintained. The implementation period was 4 weeks, with 8 sessions in total, twice a week, each lasting 45–60 minutes.

Evaluation and Analysis

Evaluation was conducted using the RIASEC-based questions embedded in the narrative and a self-developed 12-item post-satisfaction survey. For data collection and analysis, RIASEC test responses before and after the game were analyzed using analysis of variance (ANOVA) and correlation analysis. For qualitative analysis, individual interviews were conducted with 30 students (15 high school and 15 college students) to gain deeper insights.

3. Results

・Stability of RIASEC test responses: The intraclass correlation coefficient was calculated, and the results showed that it exceeded 0.65 for all dimensions, indicating that the assessment through the game is stable over time.

・Accuracy of responses: Compared to the traditional RIASEC test, a strong positive correlation of 0.75 or higher was observed in all dimensions, indicating a stable relationship with traditional questionnaire-based tests.

・Comparison of pre- and post-RIASEC test values: The results of the analysis of variance showed significant differences (p<0.05) in responses before and after the game in 5 out of the 6 dimensions: Realistic, Investigative, Social, Enterprising, and Conventional. This suggests that the use of the game had a clear impact on the students’ self-perception.

・Evaluation of the game experience: Students reported high levels of satisfaction in all categories. The average for “narrative” was 4.5, and the average for “vocational impact” was 4.4. Regression analysis revealed that the quality of the narrative and the vocational impact were the main factors determining overall satisfaction.

・Interviews: Many positive opinions were expressed, such as “it was immersive” and “I was able to think about my interests from a new perspective.”

・Impact on career guidance and correlation with academic performance: Significant positive correlations were confirmed in subjects such as science (r=0.42), social studies (r=0.38), and mathematics (r=0.35). This confirms that the vocational interests identified through the game are consistent with the students’ actual academic strengths.

4. Discussion: The Potential of Serious Games

As a result of this study, quantitative research showed that the game’s AI-driven dynamic scenario generation provides real-time adaptability, indicating that students’ responses reflect their interests. Qualitative research highlighted high levels of satisfaction and underscored that engagement with the narrative and usability are key strengths. Furthermore, the correlation between in-game responses and academic performance showed that vocational interests and academic strengths are aligned.

Additionally, the innovative aspect of this paper is the use of AI for real-time scenario personalization, which enhances the authenticity of responses and provides an immersive experience. By embedding the RIASEC test within a narrative structure, it engaged students in a way impossible with traditional methods and encouraged a clearer understanding of their vocational interests.

On the other hand, as limitations of the study, there are constraints regarding the sample size and student population. The satisfaction survey is self-reported, which may introduce bias. Also, as a methodological consideration, the RIASEC test was adapted into an interactive narrative format, and this modification requires an examination of validity. Although high consistency and strong correlation with the traditional test were observed, changes in the presentation of items and context may affect students’ interpretation and responses. There is a need for validation studies to confirm whether the adapted items retain their original psychometric properties across various implementation forms.

Reason for Selection and Impressions

[Reason for Selection]

I chose this paper because I wanted to read about research in career education that focuses on personalization. I was particularly interested in the roles that games and AI can play.

[Impressions]

I found the mechanism of designing a game based on the RIASEC test and conducting an assessment while playing it to be very informative. On the other hand, since RIASEC is also considered to have elements of stages and gradients, I have doubts about whether an assessment can be made with only a few choices in a game. Also, considering that the generated scenarios and options differ for each learner, I find it interesting to consider how those outputs relate to learning outcomes through logs and text data. Regarding the target audience, I am also curious about whether there are differences in perception and behavior between high school and college students.

I understand that there are diverse possibilities in combining career learning with games and AI. I believe there is much room for application in the future, such as adaptive game development like in this study, or connecting to coaching using AI chat after reflection. I would like to learn about what the goals are, what can be realized, and how to implement them.

Report by: Kohei Ozaki