Abstract
This paper reports on the design and evaluation of an innovative intertextual reading-into- writing task for use in academic admissions. Existing tests of English used for university admission avoid intertextual reading (Weir and Chan, 2019) and do not ask test takers to synthesize information from multiple texts into a single piece of writing. Therefore, there is a divergence between the design of language tests for university entrance and subsequent ‘academic writing’ required at university. We designed a mediation-focused Summary task which requires test takers to read two texts on the same topic (a total of 300 words) and to summarize the information using their own words (up to a maximum of 100 words). Seven trained judges provided CEFR ratings for 48 internally benchmarked test taker scripts across four Summary tasks (n=24) and four Essay tasks (n=24). Data were analyzed using many-facet Rasch analysis to investigate task, judge and rating scale performance. We also analyzed the level of agreement with internal benchmarking. We found a strong level of agreement between internal and external expert assessors and that the task is highly effective for distinguishing B2 from C1-level performances. Assessors were able to score responses using an analytic rating scale incorporating source use across four individual components at similar levels of reliability to a more traditional Essay task. Test takers’ lack of familiarity with the task design means the introduction of such tasks will have a significant washback effect.
Author Information
Nathaniel Owen, Oxford University Press, United Kingdom
Paper Information
Conference: ECLL2023
Stream: Assessment
This paper is part of the ECLL2023 Conference Proceedings (View)
Full Paper
View / Download the full paper in a new tab/window
To cite this article:
Owen N. (2023) Designing and Researching an Intertextual Reading-Into-Writing Summary Task ISSN: 2188-112X The European Conference on Language Learning 2023: Official Conference Proceedings https://doi.org/10.22492/issn.2188-112X.2023.23
To link to this article: https://doi.org/10.22492/issn.2188-112X.2023.23
Comments
Powered by WP LinkPress