Author Information
Kayo Tsuji, Osaka Metropolitan University, JapanAbstract
Given the growing challenges in providing individualized writing instruction, this study explores the potential of large language models—specifically ChatGPT—as autonomous tools for formative assessment in English writing education. The research investigates how ChatGPT can generate pedagogically valuable feedback on learner-produced opinion-based essays, focusing on the role of prompt design in eliciting meaningful evaluations. Drawing on prior research, a prototype prompt was developed to guide the model’s assessment process. A qualitative analysis of the feedback generated by ChatGPT revealed initial limitations, prompting a cycle of iterative prompt refinement aimed at enhancing the clarity, specificity, and educational relevance of the output. The final version of the prompt was then employed to generate both evaluative scores and qualitative comments on learner essays. These outputs were analyzed by two experts in English language education, who found a high degree of alignment between the scores assigned by ChatGPT and those of human raters. Moreover, the qualitative feedback offered by the model was found to parallel the kind typically provided by experienced educators, particularly in areas such as organization, argumentation, and language use. These findings indicate that, when guided by well-crafted prompts, ChatGPT is capable of producing valid and instructionally supportive feedback. The study underscores the potential of integrating large language models into language education as scalable tools for promoting autonomous writing development, particularly in contexts where individualized teacher feedback is limited. Further research is recommended to explore broader applications and long-term impacts on learner outcomes.
Comments
Powered by WP LinkPress