Abstract
This paper explores the application of canonical and gradient analysis to evaluate and visualize student performance and acceptance of a learning system platform. The subject of evaluation is a first year BSc module for computer programming. This uses ‘Ceebot’, an animated and immersive game-like development environment. Multivariate ordination approaches are widely used in ecology to explore species distribution along environmental gradients. Environmental factors are represented here by three ‘assessment’ gradients; one for the overall module mark and two independent tests of programming knowledge and skill. Response data included Lickert expressions for behavioural, acceptance and opinion traits. Behavioural characteristics (such as attendance, collaboration and independent study) were regarded to be indicative of learning activity. Acceptance and opinion factors (such as perceived enjoyment and effectiveness of Ceebot) were treated as expressions of motivation to engage with the learning environment. Ordination diagrams and summary statistics for canonical analyses suggested that the logbook grades (the basis for module assessment) and ‘code understanding’ were only weakly correlated. Thus strong module performance was not a reliable predictor of programming ability. The three assessment indices were correlated with behaviours of independent study and peer collaboration, but were only weakly associated with attendance. Results were useful for informing teaching practice and suggested: (1) realigning assessments to more fully capture code-level skills (important in the workplace); (2) re-evaluating attendance-based elements of module design; and (3) the overall merit of multivariate canonical and other gradient approaches for evaluating and visualizing the effectiveness of a learning system platform.
Author Information
Richard Alistair Mather
Paper Information
Conference: ECTC2014
Stream: Learning Systems Platforms
This paper is part of the ECTC2014 Conference Proceedings (View)
Full Paper
View / Download the full paper in a new tab/window
Comments
Powered by WP LinkPress