Research projects
 |
Musically-Aware Humanoid Robots Human musicians make use of substantial auditory and visual information throughout a performance. Our humanoids research focuses on providing such capabilities (e.g., audio and visual beat detection, note onset and pitch detection, and basic control for musical keyboard performance), with the long-term goal of enabling a large humanoid to be an interactive participant in a live music ensemble. |
 |
In developing automated systems to recognize the emotional content of music, we are faced with a problem spanning two disparate domains: the space of human emotions and the acoustic signal of music. To address this problem, we must develop models for both data collected from humans describing their perceptions of musical mood and quantitative features derived from the audio signal. |
 |
The MRP is an augmentation of the acoustic grand piano. By using electromagnets to induce the strings to vibration, the MRP expands the piano's vocabulary to include infinite sustain, crescendos from silence, harmonics and new timbres. An optical keyboard sensor system lets the pianist continuously shape the sound of every note.. |
 |
Structured audio is a representation of sound content using symbolic or semantic information as a means of encoding (or representing) the data. It provides a framework to easily manipulate or transform content after it reaches the user. . |
 |
Analysis-Synthesis of Guitar Expression During performance, a guitarist employs specific techniques in order to convey expressive intentions for a desired effect. These intentions often correspond to simultaneous variation of the source (guitarist-string interaction) and the filter (resonant string) parameters. We propose modeling expression at the signal level via joint estimation of the source and filter parameters of plucked guitar sounds in order to achieve realistic re-synthesis while incorporating expressive intentions. |
 |
This system uses machine learning techniques to automatically combine individual instrument tracks into a single mixed song. Acoustic features are extracted from the audio and used to train the system to predict fader values for each instrument to produce a final mix. |
 |
ALF: Audio processing Library for Flash A research project dedicated to providing Flash developers with sophisticated signal processing routines for use in web-based MIR or game applications. We hope that this library encourages the growth of audio-centric Flash applications for research and entertainment. |
 |
iNotes: Orchestral Performance Companion We have developed a system that helps users by guiding them through the performance using a handheld application in real-time. Using audio features, we attempt to align the live performance audio with that of a previously annotated reference recording. The aligned position is transmitted to users’ handheld devices and pre-annotated inform-ation about the piece is displayed synchronously. |
 |
There has been a great deal of recent interest in the use of multi-touch displays (MTDs) as an interface for human-computer interaction. A multi-touch display (as opposed to a standard touch-screen) is able to detect and track multiple points of contact (e.g., fingers). This capability enables simultaneous control of multiple objects as well as multi-point gestures.. |