Abstract
The following paper presents the evaluation of an artificially intelligent assistant system (DIAS) with a service-oriented chatbot as a central communication element. The conversational AI (Artificial Intelligence) is supposed to increase information transparency in higher education environments and thus support students, teachers, and administrative staff. The exploratory study had two objectives: first, we intended to find out about the usability and utility of the DIAS chatbot using the CUQ (Chatbot Usability Questionnaire) score and benchmark the results against other conversational agents. Secondly, we were interested in possible effects among the different variables of interest, which could contribute to further theory development of chatbots in education. The results show that the DIAS chatbot scored above average, and can support students in finding relevant information, particularly if they use the assistant frequently. Positive aspects included the intuitive use, a welcoming persona (expressed in design & language) and easy navigation. The negative feedback showed potential for improvement particularly in content quality and handling dialogue mistakes, which is a general shortcoming of conversational AI at this development stage. The results can be used as a guidance for future research and theory building. However, they must be considered carefully due to several study limitations.
Author Information
Mascha-Lea Fersch, University of Applied Sciences Ansbach, Germany
Sigurd Schacht, University of Applied Sciences Ansbach, Germany
Betiel Woldai, University of Applied Sciences Ansbach, Germany
Paper Information
Conference: PCE2023
Stream: Higher education
This paper is part of the PCE2023 Conference Proceedings (View)
Full Paper
View / Download the full paper in a new tab/window
To cite this article:
Fersch M., Schacht S., & Woldai B. (2023) Exploring AI in Education: A Quantitative Study of a Service-Oriented University Chatbot ISSN: 2758-0962 The Paris Conference on Education 2023: Official Conference Proceedings https://doi.org/10.22492/issn.2758-0962.2023.37
To link to this article: https://doi.org/10.22492/issn.2758-0962.2023.37
Comments
Powered by WP LinkPress