Music Emotion Recognition



Quick Links


Get the MoodSwings Turk Dataset Here
Emotion in Music Database (1000 Songs)

Overview


The development of systems for automatic content-based music emotion recognition spans a wide breadth of areas including psychology, signal processing, and machine learning.

Projects


MoodSwings: A collaborative game for music mood labeling


MoodSwings is a collaborative "game with a purpose", designed to enable the collection of continuous mood ratings of musical excerpts through a fun and engaging activity. You play with others across the internet with the goal of agreeing on current mood of a song within a grid defined by valence (happy vs. sad) and intensity (energetic vs. calm). The more you and your partner agree, the more points you receive!

Learning Emotion-Based Acoustic Features


Learning emotion-based acoustic features with deep belief networks.

Representing Emotion in Music as a Stochastic Distribution


Prediction of time-varying musical mood distributions from audio.

Predicting Time-Varying Emotion Space Heatmaps


Predicting Time-Varying Emotion Space Heatmaps with Conditional Random Fields.

Analyzing the Perceptual Salience of Audio Features


Analyzing the Perceptual Salience of Audio Features for Musical Emotion Recognition via Acoustic Feature Reconstructions.


Theses:


  • Schmidt, E. M. (2012). Modeling and Predicting Emotion in Music. Unpublished Ph.D. Thesis, Department of Electrical and Computer Engineering, Drexel University, Philadelphia, PA. [PDF]

  • Review Papers:


    • Kim, Y. E., Schmidt, E. M., Migneco, R., Morton, B. G., Richardson, P., Scott, J., Speck, J. A. and Turnbull, D. (2010). Music emotion recognition: a state of the art review. Proceedings of the 2010 International Society for Music Information Retrieval Conference, Utrecht, Netherlands: ISMIR. [PDF]

    • Published Work:


      • Schmidt, E. M and Kim, Y. E. (2013). Learning rhythm and melody features with deep belief networks. Proceedings of the 14th International Society for Music Information Retrieval Conference. Curitiba, Brazil. [PDF]

      • M. Soleymani, M. Caro, E. M. Schmidt, C. Sha, Y. Yang. 1000 Songs for Emotional Analysis of Music. Proceedings of the ACM multimedia 2013 workshop on Crowdsourcing for Multimedia. ACM, ACM , 2013.

      • Schmidt, E. M., Scott, J., and Kim, Y. E. (2012). Feature Learning in Dynamic Environments: Modeling the Acoustic Structure of Musical Emotion. Proceedings of the 2012 International Society for Music Information Retrieval Conference, Porto, Portugal: ISMIR. [PDF]

      • Schmidt, E. M., Prockup, M., Scott, J., Dolhansky, B., Morton, B. and Kim, Y. E. (2012). Relating perceptual and feature space invariances in music emotion recognition. Proceedings of the International Symposium on Computer Music Modeling and Retrieval, London, U.K.: CMMR. Best Student Paper. [PDF] [Oral Presentation]

      • Scott, J., Schmidt, E. M., Prockup, M., Morton, B. and Kim, Y. E. (2012). Predicting time-varying musical emotion distributions from multi-track audio. Proceedings of the International Symposium on Computer Music Modeling and Retrieval, London, U.K.: CMMR. [PDF]

      • Schmidt, E. M. and Kim, Y. E. (2012). Modeling and Predicting Emotion in Music. Music, Mind, and Invention Workshop, Ewing, NJ: MMI. [PDF]

      • Schmidt, E. M. and Kim, Y. E. (2011). Modeling the acoustic structure of musical emotion with deep belief networks. NIPS Workshop on Music and Machine Learning, Sierra Nevada, Spain: NIPS-MML. [Oral Presentation]

      • Schmidt, E. M. and Kim, Y. E. (2011). Modeling musical emotion dynamics with conditional random fields. Proceedings of the 2011 International Society for Music Information Retrieval Conference, Miami, Florida: ISMIR. [PDF]

      • Speck, J. A., Schmidt, E. M., Morton, B. G., and Kim, Y. E. (2011). A comparative study of collaborative vs. traditional annotation methods. Proceedings of the 2011 International Society for Music Information Retrieval Conference, Miami, Florida: ISMIR. [PDF]

      • Schmidt, E. M. and Kim, Y. E. (2011). Learning emotion-based acoustic features with deep belief networks. Proceedings of the 2011 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY: WASPAA. [PDF]

      • Kim, Y. E., Batula, A. M., Migneco, R., Richardson, P., Dolhansky, B., Grunberg, D., Morton, B., Prockup, M., Schmidt, E. M., and Scott, J. (2011). Teaching STEM concepts through music technology and DSP. Proceedings of the 14th IEEE Digital Signal Processing Workshop and 6th IEEE Signal Processing Education Workshop, Sedona, AZ: DSP/SPE. [PDF]

      • Schmidt, E. M. and Kim, Y. E. (2010). Prediction of time-varying musical mood distributions using Kalman filtering. Proceedings of the 2010 IEEE International Conference on Machine Learning and Applications, Washington, D.C.: ICMLA. [PDF]

      • Schmidt, E. M. and Kim, Y. E. (2010). Prediction of time-varying musical mood distributions from audio. Proceedings of the 2010 International Society for Music Information Retrieval Conference, Utrecht, Netherlands: ISMIR. [PDF]

      • Morton, B. G., Speck, J. A., Schmidt, E. M., and Kim, Y. E. (2010). Improving music emotion labeling using human computation. Proceedings of the ACM SIGKDD Workshop on Human Computation, Washington, D.C.: HCOMP [PDF]

      • Schmidt, E. M., Turnbull, D., and Kim, Y. E. (2010). Feature selection for content-based, time-varying musical emotion regression. Proc. ACM SIGMM International Conference on Multimedia Information Retrieval, Philadelphia, PA. [PDF]

      • Schmidt, E. M. and Kim, Y. E. (2009). Projection of acoustic features to continuous valence-arousal mood labels via regression. Accepted to the 2009 International Society for Music Information Retrieval Conference, Kobe, Japan: ISMIR. [PDF]

      • Kim, Y. E., Schimdt, E., and Emelle, L. (2008). Moodswings: a collaborative game for music mood label collection. Proceedings of the 2008 International Conference on Music Information Retrieval, Philadelphia, PA: ISMIR. [PDF]