Does AI Ideation Feel Collaborative? Collaboration Potential as a Structurally Distinct Evaluation Dimension in an LLM-Based Ideation Tool

Author Information

Stefania Zourlidou, University of Koblenz, Germany
Kamyab Farokhi, University of Koblenz, Germany
Frank Hopfgartner, University of Koblenz, Germany

Abstract

Large language models are increasingly used to help students generate project ideas, but an idea that is easy to generate is not necessarily one that students see as useful for working with others. This paper re-analyses a pilot dataset from EduIDEAtor, an LLM-based ideation tool, to ask a narrow question that was not addressed in two earlier papers from the same deployment: does perceived collaboration potential move together with other quality ratings, or does it show a different response pattern? Sixteen students completed the post-use survey, with 14 valid responses for collaboration potential. Collaboration was rated positively: 78.6% selected Better or Much Better than their usual brainstorming method (median = 4 on the intended five-point scale). However, collaboration potential was weakly related to comparative engagement (rho = .27), creativity (rho = -.02), and ease of use (rho = .20). Its only clear association within the comparative block was with project relevance (rho = .65, p = .013). It was not significantly associated with any of the seven broader experience items. Reliability of the five-item comparative block increased from alpha = .814 to .860 when collaboration was removed. With this small sample and a single collaboration item, the result does not establish a separate construct. It does suggest that perceived collaboration potential may capture something different from general usability and engagement, and should be tested directly in future group-based studies.

Save as a Favourite

• Category: Design, Implementation & Assessment of Innovative Technologies in Education

Posted by James Alexander Gordon

All Posts