Abstract
The article describes the development and assessment of a computer-assisted pronunciation training (CAPT) environment to improve prosody practice focusing its particular setup for Japanese. Users interact with the system by replicating model utterances, while the system feedback is provided in the form of pitch curves contrasting the learner’s pronunciation against the model. Complementing the pitch graphs with the audible output and quantitative metrics provide objective evaluation of pitch quality. However, we observed that learners of Japanese, a mora-timed language, faced significant challenges in understanding and applying the feedback effectively, since Japanese rhythmic patterns are not encoded well in the pitch curves. To address this, we explore the use of additional instructive feedback models such as musical and even choreographic notations. Along with rhythmically enhanced phonetic transcription, these interfaces have sufficient flexibility to include syncopations and silence. Music notation has standard features, which can represent the important components of language rhythm including the time signatures capturing the assigned rhythm for a phrase, the notes symbolizing variations in pitch, and the timed rests that can be helpful in visualizing the mapping of the phrase to rhythmic units. We share practical CAPT scenarios supporting pronunciation exercises in Japanese. In particular, we use examples of Yodel music to model the phrases characterized by complex language rhythmic patterns. Vocal music can be helpful and inject fun into the learning process. We also noticed that rhythmic structure of utterances can be described with dance movement patterns, for example using known graphic dance notations.
Author Information
Evgeny Pyshkin, University of Aizu, Japan
Akemi Kusakari, National Institute of Technology, Hachinohe College, Japan
Natalia Bogach, Peter the Great St. Petersburg Polytechnic University, Russia
John Blake, University of Aizu, Japan
Paper Information
Conference: KAMC2023
Stream: Language and Cultural Studies
This paper is part of the KAMC2023 Conference Proceedings (View)
Full Paper
View / Download the full paper in a new tab/window
To cite this article:
Pyshkin E., Kusakari A., Bogach N., & Blake J. (2023) Music and Choreography Metaphors in Spoken Language Rhythm Modelling and Their Application to Computer-Assisted Pronunciation Training for Mora-Timed Japanese ISSN: 2436-0503 – The Kyoto Conference on Arts, Media & Culture 2023: Official Conference Proceedings https://doi.org/10.22492/issn.2436-0503.2023.43
To link to this article: https://doi.org/10.22492/issn.2436-0503.2023.43
Comments
Powered by WP LinkPress