Abstract
In a current research project at the Ansbach University of Applied Science, an AI-based quiz function was created to serve as a voluntary student-oriented support offer to determine their learning progress in their respective courses by means of conducting self-assessment quizzes. The application takes lecture scripts as input and applies a question generation model to create questions that students can answer. In order to evaluate the given answers, another language model is involved to perform Natural Language Inference (NLI). Users can engage with the system via a graphical user interface currently provided via a web app. To assess preliminary feasibility and perception of the model prototype, a qualitative focus group discussion following a semi-structured interview guideline prepared by the research team according to similar studies in the education field (Sek et al. 2012) was conducted with five participants. A transcript of the discussion was prepared and analyzed using the qualitative content analysis method according to Kuckartz. Overall, the quiz function was well received by the participants of the focus group. However, the prototype still has potential when it comes to generating meaningful questions and transparently assigning categories to the given answers. Furthermore, the quiz parameters should be individually adjustable by users. In the following paper, the development of the service is illustrated by outlining the considerations for the application design and the training procedure of the language models. Afterwards, the design of the qualitative focus group is described including the presentation of the results.
Author Information
Betiel Woldai, University of Applied Science Ansbach, Germany
Sophie Henne, University of Applied Science Ansbach, Germany
Mascha-Lea Fersch, University of Applied Science Ansbach, Germany
Sudarshan Kamath Barkur, University of Applied Science Ansbach, Germany
Sigurd Schacht, University of Applied Science Ansbach, Germany
Paper Information
Conference: PCE2023
Stream: Design
This paper is part of the PCE2023 Conference Proceedings (View)
Full Paper
View / Download the full paper in a new tab/window
To cite this article:
Woldai B., Henne S., Fersch M., Barkur S., & Schacht S. (2023) A Qualitative Evaluation of an AI-Supported Quiz Application to Assess Learning Progress ISSN: 2758-0962 The Paris Conference on Education 2023: Official Conference Proceedings https://doi.org/10.22492/issn.2758-0962.2023.39
To link to this article: https://doi.org/10.22492/issn.2758-0962.2023.39
Comments
Powered by WP LinkPress