Abstract
In typical classroom settings, students collaborate on tasks and submit group work for grading, often relying on peer evaluations to determine individual grades. We are concerned with the method of converting pairwise peer evaluations into individual final grades. The most common way to do it is as follows: every group member has a certain number of points to distribute among the rest of the group, and the final score of every student is the average number of points she receives from other group members. We call this Pie-to-others. Assessments should be psychometrically valid and reliable. We argue that the Pie-to-others method of evaluating individual contributions to group work is valid but not reliable. Moreover, by constructing a mathematical model of peer evaluation, we can measure exactly how much Pie-to-others (or, more generally, any assessment method of individual contribution to group work) deviates from being reliable. We will explain the worst-case scenario, i.e., derive the theoretical largest possible difference between the outcome of the Pie-to-others and the fair grade a student deserves. By analyzing a large dataset (1201 students, 220 project groups, 6619 evaluations) collected in large undergraduate classes in an Asian university, we estimate that, in practice, about 1% of all students are misgraded by the Pie-to-others. Finally, we will present an easy fix to the pie-to-others method that makes it reliable.
Author Information
Fedor Duzhin, Nanyang Technological University, Singapore
Megan Zheng Chi Lee, Nanyang Technological University, Singapore
Paper Information
Conference: PCE2024
Stream: Assessment Theories & Methodologies
This paper is part of the PCE2024 Conference Proceedings (View)
Full Paper
View / Download the full paper in a new tab/window
To cite this article:
Duzhin F., & Lee M. (2024) Toward Valid and Reliable Assessment of Individual Contributions to Teamwork ISSN: 2758-0962 The Paris Conference on Education 2024: Official Conference Proceedings (pp. 563-575) https://doi.org/10.22492/issn.2758-0962.2024.43
To link to this article: https://doi.org/10.22492/issn.2758-0962.2024.43
Comments
Powered by WP LinkPress