Interactive Sound Wave: Visualizing Audio and Emotion Through Real-Time Sound–Gesture Interaction in Technology-Enhanced Learning

Author Information

Naphatsanan Suwannawong, King Mongkut’s University of Technology Thonburi, Thailand
Yanee Posuk, King Mongkut’s University of Technology Thonburi, Thailand

Abstract

This study developed Interactive Sound Wave, a technology-enhanced learning artifact that visualizes audio and emotion through real-time sound–gesture interaction. The study had 3 objectives: (1) to develop an interactive learning artifact that transforms learners’ sound input and hand gestures into real-time particle visualizations, (2) to evaluate the quality of the artifact, and (3) to assess user satisfaction with the artifact among 30 participants. A research-and-development approach was implemented in 3 phases. Phase 1 focused on designing and building the system to capture audio features (e.g., rhythm and spectrum) and hand-gesture signals from a camera, then mapping these inputs to particle motion and visual effects to provide immediate audiovisual feedback. Phase 2 evaluated media quality through expert review using a 5-point Likert-scale form covering course suitability, interaction consistency, clarity of control, visual quality, and classroom feasibility. Phase 3 conducted a try-out session with users followed by a 5-point satisfaction questionnaire. Results indicated that overall media quality was good. Experts rated interaction consistency as very good, while clarity of user experience and control was rated good; other criteria were rated good. Overall user satisfaction was very high, with engagement and interaction rated the highest, followed by the clarity of linking sound–gesture–visuals to support concept understanding. The study demonstrated that real-time, responsive audiovisual interaction can promote learner engagement and help make abstract audio concepts more observable and discussable, providing a practical classroom-ready artifact and an evaluation framework for technology-enhanced learning in multimedia and creative media education.

Save as a Favourite