AI in Formative Assessment: a Mixed-Methods Study on High School Students’ Productive Language Skills

Author Information

Zeynep Erturk Icen, Bursa Uludag University, Turkey
Esi̇m Gursoy, Bursa Uludag University, Turkey

Abstract

Integrating AI-driven tools in education can transform formative assessment practices, providing innovative approaches that enhance learners’ engagement and productive language skills. As a Mixed-Methods Quasi-Experimental Approach within an Explanatory Sequential Design, this study aims to investigate the impact of AI-driven formative assessment tools (Magic School and Mizou AI) on 10th-grade EFL students’ speaking and writing performances. Conducted at a public high school in Turkiye, the study involved 54 participants selected using a non-random convenience sampling method, with diverse proficiency levels, divided into experimental (n=28) and control (n=26) groups. Quantitative data were collected through pre-tests and post-tests, using a standardized TOEFL rubric to evaluate differences in productive language skill performances of the groups. The experimental group used AI-driven formative assessment tasks, while the control group completed traditional tasks. ANCOVA was employed to analyze quantitative data, controlling for initial performance level. Qualitative data were gathered through students’ reflection papers, and thematically analyzed to explore the benefits and challenges of AI-driven assessment. The findings highlight AI's role in raising self-regulated learners through its instant feedback and detailed evaluations. Also, the majority of students found AI tools useful, emphasizing their benefits, particularly in providing instant feedback and enhancing engagement during AI-driven assessment. However, they demonstrated a lower ethical awareness regarding AI usage, highlighting the urgent need for AI ethics training. Despite their positive perceptions and increased engagement, no significant difference was observed between groups in improving productive language skills. Nevertheless, students expressed satisfaction with AI-integrated formative assessment methods as an alternative approach.

Save as a Favourite