ToneBender

About


ToneBender is an online, collaborative game that illustrates concept of musical timbre through interactive gaming interfaces. In addition, the game serves as a tool for engineering education and the collection of human evaluation data. ToneBender contains two interfaces, ToneBender and ToneListener, which allow for the creation and identification of musical sounds, respectively. The purpose of ToneBender is to have players modify the timbre of musical instrument sounds and to collect those sounds in a database. Players can then listen to the sounds others have created in the ToneListener interface in order to determine if they can identify the original instrument.
Read on to learn about the different game interfaces.
Try the game out at METgames!

ToneBender


The ToneBender interface allows users to modify the timbre of musical instrument sounds by allowing control of the time varying amplitude envelope and spectral envelope. Modification can be accomplished by simply clicking and dragging the on-screen control points or by using the drop down menus, which have pre-defined amplitude shapes. After modification, the user can playback their new sounds and they are shown a potential score which is based on the SNR between the original sound and their modified sound. The score is directly proportional to the change they impart on the sound (i.e. more change, more points). The player's incentive is to create a sound that differs in some way from the original, but is still recognizable. They are awarded the potential score if someone can guess the original identity of their modified instrument in the ToneListener interface. When the player is finished with a sound, they submit the parameters to a database and load another sound to modify.

ToneListener


The ToneListener interface allows players to test their ability to identify a musical instrument from a modified instrument created in the ToneBender interface. Players are given a randomly selected sound that is synthesized from parameters defined by a player in ToneBender. After listening to the sound, they attempt to guess the family and instrument type that the modified sound was generated from. If the correctly guess the instrument, they receive a score proportional to the SNR of the modified sound, so that identification of "harder" sounds is rewarded more than "easier" ones. On the other hand, if they guess the family correctly, they receive half of the potential score.

Background


Little research has been conducted regarding human response to modified audio stimulus, in particular speech and musical sounds. The ToneBender serves as a tool that can educate users on the underlying psychoacoustic concepts that comprise musical sounds. Additionally, it provides a method to collect data regarding how well humans identify musical sounds subject to timbre modification. Timbre is often described as the "quality" or "color" of a sound. To avoid ambiguities, this game treats timbre as the time varying amplitude envelope and spectral envelope that characterize a musical sound. ToneBender allows users to modify the sound of a musical instrument with the incentive to change it in some way, but retain the identity of the original instrument. This interface provides an opportunity to teach players how the time and spectral envelopes affect perceived sound. Players are shown a potential score that they will receive if another player can correctly identify the original instrument. This projected score is related to the deviation of the altered sound compared to the original, which is measured using SNR. Lower SNR correlates to a higher potential score so that players are encouraged to create intelligible sounds. In the listening component of the game, a player will listen to modified sounds and attempt to correctly identify the original instrument's identity and family. The listening component of the game allows us to analyze how humans perform when identifying musical sounds with modified timbre.

Evaluation


In previous work using the initial ToneBender platform, we have reported human performance as a function of SNR for a target group of middle-school students. The results of this analysis showed that human performance exhibited an upward trend as the SNR value of the created instrument sounds increased. While this is an expected result, the trend was not strictly linear, suggesting that there may be other salient acoustic properties that impact the perception of instrument identity.

Currently, we are exploring methods for analyzing collected performance data based on the acoustic properties of created instruments and their relationship to the source instrument. (Figure 3) provides a 3D visualization of the amplitude and spectral envelopes of a source instrument and the envelopes associated with the created instruments derived from it. The left column indicates the sounds that were correctly identified, while the right column indicates those that were incorrectly identified. Our analysis tools, based in MATLAB, communicate directly with the MySQL game database on the webserver to access live data. The data is easily plotted for visual comparison of the parameters, and the corresponding sounds can be auditioned for an audio comparison.


To obtain additional insight regarding the correlation between identification performance and the acoustic attributes of the created instruments, we have computed the amplitude and spectral deviation between the envelopes of the source and modified instruments. Table 1 provides statistics describing the spectral and amplitude deviation of created instruments derived from various source instruments in the game’s database. These values were calculated by averaging the amplitude and spectral distortion in- curred from all created instruments corresponding to a particular source that has been played at least four times. The distortion mea- sures are further separated by instruments that have been correctly identified and those that were never correctly identified. Table 1 shows that many of the instruments have a small amount of ampli- tude distortion, thus indicating that players in the creation interface felt it was important to preserve the amplitude envelope in order to maintain identifiability, even with applied spectral distortion. However, this analysis warrants further data collection in order to obtain more evaluations for each of the modified instruments in order to determine the perceptual interactions between the amplitude and spectral envelopes. Additional metrics may also be useful in identifying the relationships of the source and modified signals, such as describing the smoothness and coherence of the spectra.