Skip to main content
SpringerLink
Log in
Book cover

International Symposium on Computer Music Modeling and Retrieval

CMMR 2012: From Sounds to Music and Emotions pp 1–18Cite as

The Six Emotion-Face Clock as a Tool for Continuously Rating Discrete Emotional Responses to Music

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 7900))

Abstract

Recent instruments measuring continuous self-reported emotion responses to music have tended to use dimensional rating scale models of emotion such as valence (happy to sad). However, numerous retrospective studies of emotion in music use checklist style responses, usually in the form of emotion words, (such as happy, angry, sad…) or facial expressions. A response interface based on six simple sketch style emotion faces aligned into a clock-like distribution was developed with the aim of allowing participants to quickly and easily rate emotions in music continuously as the music unfolded. We tested the interface using six extracts of music, one targeting each of the six faces: ‘Excited’ (at 1 o’clock), ‘Happy’ (3), ‘Calm’ (5), ‘Sad’ (7), ‘Scared’ (9) and ‘Angry’ (11). 30 participants rated the emotion expressed by these excerpts on our ‘emotion-face-clock’. By demonstrating how continuous category selections (votes) changed over time, we were able to show that (1) more than one emotion-face could be expressed by music at the same time and (2) the emotion face that best portrayed the emotion the music conveyed could change over time, and (3) the change could be attributed to changes in musical structure. Implications for research on orientation time and mixed emotions are discussed.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Yang, Y.H., Lin, Y.C., Su, Y.F., Chen, H.H.: A regression approach to music emotion recognition. IEEE Transactions on Audio, Speech, and Language Processing 16(2), 448–457 (2008)

    Article  Google Scholar 

  2. Schmidt, E.M., Turnbull, D., Kim, Y.E.: Feature selection for content-based, time-varying musical emotion regression. In: MIR 2010 Proceedings of the International Conference on Multimedia Information Retrieval. ACM, New York (2010)

    Google Scholar 

  3. Korhonen, M.D., Clausi, D.A., Jernigan, M.E.: Modeling emotional content of music using system identification. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics 36(3), 588–599 (2006)

    Article  Google Scholar 

  4. Schubert, E.: Continuous self-report methods. In: Juslin, P.N., Sloboda, J.A. (eds.) Handbook of Music and Emotion: Theory, Research, Applications, pp. 223–253. OUP, Oxford (2010)

    Google Scholar 

  5. Madsen, C.K., Frederickson, W.E.: The experience of musical tension: A replication of Nielsen’s research using the continuous response digital interface. Journal of Music Therapy 30(1), 46–63 (1993)

    Article  Google Scholar 

  6. Nielsen, F.V.: Musical tension and related concepts. In: Sebeok, T.A., Umiker-Sebeok, J. (eds.) The Semiotic Web 1986. An International Year-Book. Mouton de Gruyter, Berlin (1987)

    Google Scholar 

  7. Russell, J.A.: Affective space is bipolar. Journal of Personality and Social Psychology 37(3), 345–356 (1979)

    Article  Google Scholar 

  8. Russell, J.A.: A circumplex model of affect. Journal of Social Psychology 39, 1161–1178 (1980)

    Google Scholar 

  9. Krumhansl, C.L.: An exploratory study of musical emotions and psychophysiology. Canadian Journal of Experimental Psychology 51(4), 336–352 (1997)

    Article  Google Scholar 

  10. Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., Schröder, M.: FEELTRACE: An instrument for recording perceived emotion in real time. In: Cowie, R., Douglas-Cowie, E., Schroede, M. (eds.) Speech and Emotion: Proceedings of the ISCA workshop, pp. 19–24. Newcastle, Co. Down (2000)

    Google Scholar 

  11. Nagel, F., Kopiez, R., Grewe, O., Altenmüller, E.: EMuJoy: Software for continuous measurement of perceived emotions in music. Behavior Research Methods 39(2), 283–290 (2007)

    Article  Google Scholar 

  12. Schubert, E.: Measuring emotion continuously: Validity and reliability of the two-dimensional emotion-space. Australian Journal of Psychology 51(3), 154–165 (1999)

    Article  Google Scholar 

  13. Schimmack, U., Rainer, R.: Experiencing activation: Energetic arousal and tense arousal are not mixtures of valence and activation. Emotion 2(4), 412–417 (2002)

    Article  Google Scholar 

  14. Schimmack, U., Grob, A.: Dimensional models of core affect: A quantitative comparison by means of structural equation modeling. European Journal of Personality 14(4), 325–345 (2000)

    Article  Google Scholar 

  15. Wundt, W.: Grundzüge der physiologischen Psychologie. Engelmann, Leipzig (1905)

    Google Scholar 

  16. Plutchik, R.: The emotions: Facts, theories and a new model. Random House, New York (1962)

    Google Scholar 

  17. Russell, J.A., Mehrabian, A.: Evidence for a 3-factor theory of emotions. Journal of Research in Personality 11(3), 273–294 (1977)

    Article  Google Scholar 

  18. Barrett, L.F., Wager, T.D.: The Structure of Emotion: Evidence From Neuroimaging Studies. Current Directions in Psychological Science 15(2), 79–83 (2006)

    Article  Google Scholar 

  19. Barrett, L.F.: Discrete emotions or dimensions? The role of valence focus and arousal focus. Cognition & Emotion 12(4), 579–599 (1998)

    Article  Google Scholar 

  20. Lewis, M., Haviland, J.M. (eds.): Handbook of emotions (1993)

    Google Scholar 

  21. Izard, C.E.: The psychology of emotions. Plenum Press, New York (1991)

    Book  Google Scholar 

  22. Izard, C.E.: Organizational and motivational functions of discrete emotions. In: Lewis, M., Haviland, J.M. (eds.) Handbook of Emotions, pp. 631–641. The Guilford Press, New York (1993)

    Google Scholar 

  23. Namba, S., Kuwano, S., Hatoh, T., Kato, M.: Assessment of musical performance by using the method of continuous judgment by selected description. Music Perception 8(3), 251–275 (1991)

    Article  Google Scholar 

  24. Juslin, P.N., Laukka, P.: Communication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin 129(5), 770–814 (2003)

    Article  Google Scholar 

  25. Laukka, P., Gabrielsson, A., Juslin, P.N.: Impact of intended emotion intensity on cue utilization and decoding accuracy in vocal expression of emotion. International Journal of Psychology 35(3-4), 288 (2000)

    Google Scholar 

  26. Juslin, P.N.: Communicating emotion in music performance: A review and a theoretical framework. In: Juslin, P.N., Sloboda, J.A. (eds.) Music and Emotion: Theory and Research, pp. 309–337. Editors. Oxford University Press, London (2001)

    Google Scholar 

  27. Schubert, E., Ferguson, S., Farrar, N., McPherson, G.E.: Sonification of Emotion I: Film Music. In: The 17th International Conference on Auditory Display (ICAD-2011), International Community for Auditory Display (ICAD), Budapest (2011)

    Google Scholar 

  28. Hevner, K.: Expression in music: a discussion of experimental studies and theories. Psychological Review 42, 187–204 (1935)

    Article  Google Scholar 

  29. Hevner, K.: The affective character of the major and minor modes in music. American Journal of Psychology 47, 103–118 (1935)

    Article  Google Scholar 

  30. Hevner, K.: Experimental studies of the elements of expression in music. American Journal of Psychology 48, 246–268 (1936)

    Article  Google Scholar 

  31. Hevner, K.: The affective value of pitch and tempo in music. American Journal of Psychology 49, 621–630 (1937)

    Article  Google Scholar 

  32. Rigg, M.G.: The mood effects of music: A comparison of data from four investigators. The Journal of Psychology 58(2), 427–438 (1964)

    Article  Google Scholar 

  33. Han, B., Rho, S., Dannenberg, R.B., Hwang, E.: SMERS: Music emotion recognition using support vector regression. In: Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR 2009), Kobe International Conference Center, Kobe, Japan, October 26-30 (2009)

    Google Scholar 

  34. Schlosberg, H.: The description of facial expressions in terms of two dimensions. Journal of Experimental Psychology 44, 229–237 (1952)

    Article  Google Scholar 

  35. Russell, J.A.: Reading emotion from and into faces: Resurrecting a dimensional-contextual perspective. In: Russell, J.A., Fernández-Dols, J.M. (eds.) The Psychology of Facial Expression, pp. 295–320. Cambridge University Press, Cambridge (1997)

    Chapter  Google Scholar 

  36. Dimberg, U., Thunberg, M.: Rapid facial reactions to emotional facial expressions. Scandinavian Journal of Psychology 39(1), 39–45 (1998)

    Article  Google Scholar 

  37. Britton, J.C., Taylor, S.F., Sudheimer, K.D., Liberzon, I.: Facial expressions and complex IAPS pictures: common and differential networks. Neuroimage 31(2), 906–919 (2006)

    Article  Google Scholar 

  38. Waller, B.M., Cray Jr, J.J., Burrows, A.M.: Selection for universal facial emotion. Emotion 8(3), 435–439 (2008)

    Article  Google Scholar 

  39. Ekman, P.: Facial expression and emotion. American Psychologist 48(4), 384–392 (1993)

    Article  Google Scholar 

  40. Lang, P.J.: Behavioral treatment and bio-behavioral assessment: Computer applications. In: Sidowski, J.B., Johnson, J.H., Williams, T.A. (eds.) Technology in Mental Health Care Delivery Systems, pp. 119–137. Ablex, Norwood (1980)

    Google Scholar 

  41. Bradley, M.M., Lang, P.J.: Measuring emotion - the self-assessment mannequin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49–59 (1994)

    Article  Google Scholar 

  42. Ekman, P., Rosenberg, E.L. (eds.): What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Series in affective science. Oxford University Press, London (1997)

    Google Scholar 

  43. Eerola, T., Vuoskoski, J.K.: A comparison of the discrete and dimensional models of emotion in music. Psychology of Music 39(1), 18–49 (2011)

    Article  Google Scholar 

  44. Kratus, J.: A developmental study of children’s interpretation of emotion in music. Psychology of Music 21, 3–19 (1993)

    Article  Google Scholar 

  45. Schubert, E., McPherson, G.E.: The perception of emotion in music. In: McPherson, G.E. (ed.) The Child as Musician: A Handbook of Musical Development, pp. 193–212. Oxford University Press, Oxford (2006)

    Chapter  Google Scholar 

  46. Ekman, R., Friesen, W.V., Ellsworth, R.: Emotion in the human face: Guidelines jbr research and an integration of findings. Pergamon Press, New York (1972)

    Google Scholar 

  47. Zentner, M., Grandjean, D., Scherer, K.R.: Emotions evoked by the sound of music: characterization, classification, and measurement. Emotion 8(4), 494–521 (2008)

    Article  Google Scholar 

  48. Schubert, E.: Update of the Hevner adjective checklist. Perceptual and Motor Skills 96(3), 1117–1122 (2003)

    Article  Google Scholar 

  49. Kostov, V., Yanagisawa, H., Johansson, M., Fukuda, S.: Method for Face-Emotion Retrieval Using A Cartoon Emotional Expression Approach. JSME International Journal Series C 44(2), 515–526 (2001)

    Article  Google Scholar 

  50. Schubert, E.: Continuous measurement of self-report emotional response to music. In: Juslin, P.N., Sloboda, J.A. (eds.) Music and Emotion: Theory and Research, pp. 393–414. Oxford University Press, Oxford (2001)

    Google Scholar 

  51. Schubert, E.: Reliability issues regarding the beginning, middle and end of continuous emotion ratings to music. Psychology of Music 41(3), 350–371 (2013)

    Article  MathSciNet  Google Scholar 

  52. Bachorik, J.P., Bangert, M., Loui, P., Larke, K., Berger, J., Rowe, R., Schlaug, G.: Emotion in motion: Investigating the time-course of emotional judgments of musical stimuli. Music Perception 26(4), 355–364 (2009)

    Article  Google Scholar 

  53. Schubert, E., Dunsmuir, W.: Regression modelling continuous data in music psychology. In: Yi, S.W. (ed.) Music, Mind, and Science, pp. 298–352. Seoul National University, Seoul (1999)

    Google Scholar 

  54. Juslin, P.N., Friberg, A., Bresin, R.: Toward a computational model of expression in music performance: The GERM model. Musicae Scientiae. Special Issue: pp. 63-122 (2001)

    Google Scholar 

  55. Lucas, B.J., Schubert, E., Halpern, A.R.: Perception of emotion in sounded and imagined music. Music Perception 27(5), 399–412 (2010)

    Article  Google Scholar 

  56. Upham, F.: Quantifying the temporal dynamics of music listening: A critical investigation of analysis techniques for collections of continuous responses to music. McGill University (2011)

    Google Scholar 

  57. Schubert, E., Vincs, K., Stevens, C.J.: Identifying regions of good agreement among responders in engagement with a piece of live dance. Empirical Studies of the Arts 13(1), 1–20 (2013)

    Article  Google Scholar 

  58. Hunter, P.G., Schellenberg, E.G., Schimmack, U.: Mixed affective responses to music with conflicting cues. Cognition & Emotion 22(2), 327–352 (2008)

    Article  Google Scholar 

  59. Hunter, P.G., Schellenberg, E.G., Schimmack, U.: Feelings and perceptions of happiness and sadness induced by music: Similarities, differences, and mixed emotions. Psychology of Aesthetics, Creativity, and the Arts 4(1), 47–56 (2010)

    Article  Google Scholar 

  60. Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.: Multilabel classification of music into emotions. In: Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR 2008), Philadelphia, PA (2008)

    Google Scholar 

  61. Levy, M., Sandler, M.: A semantic space for music derived from social tags. In: Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007), Vienna, Austria (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Empirical Musicology Group, University of New South Wales, Sydney, Australia

    Emery Schubert, Natasha Farrar & David Taylor

  2. University of Technology, Sydney, Australia

    Sam Ferguson

  3. Melbourne Conservatorium of Music, University of Melbourne, Melbourne, Australia

    Gary E. McPherson

Authors
  1. Emery Schubert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sam Ferguson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Natasha Farrar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gary E. McPherson
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. CNRS - LMA, 31 Chemin Joseph Aiguier, 13402, Marseille Cedex 20, France

    Mitsuko Aramaki , Richard Kronland-Martinet  & Sølvi Ystad ,  & 

  2. Centre for Digital Music, Queen Mary University of London, Mile End Road, E1 4NS, London, UK

    Mathieu Barthet

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Schubert, E., Ferguson, S., Farrar, N., Taylor, D., McPherson, G.E. (2013). The Six Emotion-Face Clock as a Tool for Continuously Rating Discrete Emotional Responses to Music. In: Aramaki, M., Barthet, M., Kronland-Martinet, R., Ystad, S. (eds) From Sounds to Music and Emotions. CMMR 2012. Lecture Notes in Computer Science, vol 7900. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41248-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-41248-6_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-41247-9

  • Online ISBN: 978-3-642-41248-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation