Abstract
The launch of OpenAI’s ChatGPT in late 2022 sent ripples across higher education, particularly in attempts to detect its use by students using it to complete assignments. This paper involved a simple experiment that compared responses generated by artificial intelligence (AI) tools, namely ChatGPT and Google Gemini, in response to a case study-based Social Work assessment question. The outputs generated by these AI tools were analysed vis-à-vis a set of marking rubrics, and found that the responses were generally not of a high quality and were unable to integrate information from the case study in their responses. Although the AI tools were able to provide succinct summaries of the case study’s key points, they generally performed poorly in accurately applying the specific tool to the case study, in identifying relevant legislation to the local context of the case study, as well as in producing sufficient words that addressed the question. Strategies to improve the authenticity of assessments revolve around enhancing their complexity viz. real life scenarios, and could include incorporating recent events (e.g. in the news) in the case study, having a mix of essential and peripheral information requiring students to discern and synthesize information in case study, and for marking rubrics to rewards points that explicitly apply information from the case study. In sum, the quality of AI-generated responses were generally inadequate and well-crafted case studies appear to be AI tools’ Achilles heel which could ensure ‘authentic assessments.’
Author Information
Nicholas Netto, Singapore University of Social Sciences, Singapore
Paper Information
Conference: ACSS2024
Stream: Education and Social Welfare
This paper is part of the ACSS2024 Conference Proceedings (View)
Full Paper
View / Download the full paper in a new tab/window
Comments
Powered by WP LinkPress