Background Research and Related Work

Where the inspirations came

Past research has explored a motivation-driven approach for educational game design. Giving power to players is one important design principle in combining learning materials and video games. In music games, players are not limited by instrumental techniques or knowledge in music theory. Therefore they would be motivated to learn music or encouraged to start playing an instrument.

Previous research on music visualization indicated that synchronous visual and auditory stimulus could help beginning learners understand music. Several existing works for music visualization have used a circle of 12 notes to present the repetitive pattern of musical scale.

In terms of harmony learning, experiments results suggested that statistical information in patterning of chords is beneficial to new musical structures learning. Analogy was made between the learning processes in language and music. Acquisition of regular musical combinations or grammars happens through exposure to native music system.

Design and Development

1.Game play design

We adapted the appealing interaction design in most rhythm action games to increase players immersion. We further applied accurate music visualization to naturally integrate the learning materials into gameplay as stimulating visual elements.

In Chorlody, melody and chords to be played are displayed in a scrolling way. Players follow the triangle hints inside the circle and press buttons in the same

Before playing a song, an introduction scene is designed to lead players to preview and listen to what chords will appear in the song. The name of chord is shown in the center of the triangle. When playing a song, players are scored by their precision of button pressing time. Bonus track is played as rewards when players accomplish combos. After the song, statistics on players performance is displayed.

2.Music Visualization

According to our research on music visualization, we decided to use a ring of 12 buttons to visualize the chromatic scale. Each button shows a pitch name in a specific color. Players can touch the button to play the corresponding note.

We used a triangle to visualize which three notes are consisted in a triad (chords of three notes). The triangle is displayed in same color as the root note button.1 Melody is visualized as flowing light near the note button with the same pitch name. The combination of visualized melody and chord helps players experience the harmony of the song.

The progress bar on the left shows the musical structure of the song. The pattern of color changes reflects the progression of chords.

Testing and Evaluation

Get feedback from players.

We have conducted in-game observation and post playing survey with nine players. Reactions of players varied according to their familiarity with rhythm action games or how much musical experience they had.

Test players with rich gaming experience (6 of 9) became familiar with the interaction after playing a song. They recognized the similar appealing features in Chorlody with existing popular rhythm games. 4 of 9 players expressed excitement with the sensor device Leap Motion.

5 of 9 players have discovered the repetitive feature of triangle changes. Players with musical instrument playing experience (4 of 9) expressed they enjoyed the fun in exploring accompaniment in different chords.