Utilizing context-relevant keywords extracted from a large collection of user-generated documents for music discovery

https://doi.org/10.1016/j.ipm.2017.04.006Get rights and content

Highlights

  • Our system develops generalized context-relevant music descriptors by extracting keywords from a large collection of user-generated documents.

  • The music descriptors contain various context-relevant terms which could enhance semantic music search/discovery.

  • We identified a correlation between the proposed music descriptors with conventional features such as acoustic features or lyrics.

  • User studies confirm that the proposed method can be applied to semantic music search/discovery and context-aware music recommendation.

Abstract

The contextual background of a user is one of the important criteria when deciding what music to listen to. In this paper, we propose a novel method to embed the user context for music search and retrieval. The proposed system extracts keywords from a large collection of documents written by users. Each of these documents contains a personal story about the writer’s situation and/or mood, followed by a song request. We consider that there is a strong correlation between the story and the song. Therefore, by extracting keywords from these documents, it is possible to develop a list of terms that can generally be used to describe the user context when requesting a song, which may then be employed to represent a music item in a richer manner. Once each song is represented using the proposed context-relevant music descriptors, we perform Latent Dirichlet Allocation to retrieve similar music based on context similarity. By conducting a series of experiments, we identified a correlation between the proposed music descriptors and conventional approaches, such as acoustic features or lyrics. The identified correlation can be used to auto-tag songs with no document association. We also qualitatively evaluated our system by comparing the performance of our proposed music descriptors with other conventional features for music retrieval. The results showed that the performance of the proposed music descriptors was competitive with conventional features, thereby suggesting their potential use for describing music in semantic music search/retrieval.

Introduction

Proliferation of music data available for users increased the demand of searching and retrieving music that perfectly suits the individual listener’s situation and entertainment needs. In order to meet such demands, Schedl and Knees (2013) emphasized the importance of personalized and user context-aware systems. As a consequence, music exploration systems implemented various functionalities in an attempt to provide more satisfying results. Some of the methods to search and retrieve music from such systems are searching music using metadata, retrieving recommended music, and browsing predefined playlists (Nanopoulos, Rafailidis, Ruxanda, & Manolopoulos, 2009).

Metadata includes music-related information such as the artist, title, and genre information. Users can query the music exploration system using text to retrieve the exact match. However, metadata lack contextual information, so using a metadata query will not satisfy the user if he or she seeks music in a certain context such as mood or situation. Recently, the utilization of social tags to enhance music description has been attempted in the research field of Music Information Retrieval. Symeonidis, Ruxanda, Nanopoulos, and Manolopoulos (2008) gathered social tags obtained from Last.fm to recommend music. However, as Lamere (2008) pointed out, most social tags are related to the artist, title, genre, and instrument, whereas only a small proportion is related to the user context. These social tags enhance text-based music search to some extent but they have difficulties with context-related terms.

Music exploration systems also provide music recommendations to users using various similarity measures. Collaborative filtering algorithms retrieve similar music by discovering similar user preference (Xing, Wang, & Wang, 2014). User preference is inferred by analyzing user ratings and/or user playlist. On the other hand, content-based algorithms utilize music-centric features such as timbre, pitch, and lyrics to discover similar music (Bogdanov, Haro, Fuhrmann, Xambó, Gómez, Herrera, 2013, Li, Myaeng, Kim, 2007, Mayer, Neumayer, Rauber, 2008). These music-centric features include information obtained from the audio signal itself, but they lack information about the listener. Therefore, these music-centric features have a limited capacity to reflect the needs of the user.

Additionally, playlists tagged with predefined terms are also available in most of the music exploration systems. For instance, Allmusic1 provides playlists that are tagged with predefined moods and themes. However, the recommendations provided are unbalanced in terms of the song distribution per mood/theme. For instance, the system suggests various songs for the in love theme but it only provides one song under the theme of work. Another problem is that it is difficult to build a consensus among the users because there is no specific standard for selecting moods and themes.

In this paper, we extract context-relevant keywords from a large collection of user-generated documents to capture the user contextual background when searching for music. Each document comes from a radio program’s Internet bulletin board, where it comprises a personal story and a song request. Fig. 1 illustrates the concept of the proposed system. Hyung, Lee, and Lee (2014) showed that there is a strong correlation between personal stories and song requests. There are large number of such documents, so we consider that some general terms will be used to describe the user contextual background when requesting songs. Therefore, by performing keyword extraction based on these documents, it will be possible to create generalized context-relevant music descriptors, which can be applied to music search and retrieval. We collected 186,656 documents sent from the listeners of a radio program aired between 6:00 p.m. and 8:00 p.m.. We chose this program as they only air Western pop songs.

In this paper, we introduce a novel approach that utilizes user-generated documents to extract user context for music search and retrieval. We described music using context-relevant keywords extracted from a large collection of user-generated documents. By utilizing context-relevant music descriptors, our system facilitates natural language text querying when searching for music and thus, it can retrieve music that satisfies the entertainment needs of users. Additionally, the keywords describing the user context are extracted from a large collection of documents, so the coverage of the explainable contextual background when listening to music will be broad. Therefore, users will be able to discover music in various contexts. Finally, we determined a correlation between our proposed context-relevant music descriptors and conventional features, such as acoustic features or lyrics, which could provide some insights into how to overcome the cold start problem where pieces of music with no document associations are never discovered.

The remainder of this paper is organized as follows. The next section reviews recent papers that embed the user context for music discovery. Studies that exploited text to describe music are also reviewed. Section 3 explains the proposed system in detail. In 4 Social tags vs. context-relevant music descriptors, 5 Examining the correlation with conventional features, and 6, various experiments that were conducted in an attempt to validate the proposed approach are described. Finally, Section 7 presents the conclusions by providing a summary and suggesting directions for future research.

Section snippets

Background

Recently, novel approaches to the inclusion of contextual information related to users have been attempted actively in order to provide a richer context-relevant representation of music. In this section, we review recent studies that embedded contextual information related to users during music retrieval. We also describe some studies that employed text to enhance music descriptions.

Context-relevant music descriptors

In this section, we explain our system, which uses context-relevant keywords to describe music. Fig. 3 shows a block diagram that illustrates our system and the evaluation process.

Our system comprises three steps: preprocessing, music descriptor generation, and music discovery. In the preprocessing step, we extract the song information from each document to create song-document associations. We also remove stop-words and stemmed words during this step. In the subsequent music descriptor

Social tags vs. context-relevant music descriptors

In order to show that our music descriptors contained more context-relevant terms than social tags, we compared the terms in gKD with the social tags provided by Last.fm. For this comparison, we first categorized the terms and the tags with several labels. Lamere (2008) used the labels Genre, Locale, Mood, Opinion, Instrumentation, Style, Misc, Personal, and Organization to categorize Last.fm tags. Our aim was to examine the distribution of music-related words and context-related words, so we

Dataset

In order to examine the correlation with conventional features, we built a noise-free dataset. The dataset comprises songs with more than 20 document associations. We manually removed documents associated with incorrect songs. Thus, we had 350 songs with a minimum of 20 and a maximum of 365 documents associated with a song. The lyrics data were manually downloaded from various Internet sites, such as MetroLyrics3 and LyricsFreak.4 We selected these two

Qualitative evaluation

By comparing the label distribution of social tags obtained from Last.fm with the label distribution of terms within our proposed context-relevant music descriptors, we showed that our approach contains richer context-relevant terms. Additionally, we examined correlation between the proposed music descriptors and conventional features. However, this does not verify if the proposed music descriptors can be used for music discovery. Therefore, in an attempt to show that our music descriptors are

Conclusion

In this paper, we proposed a novel method for representing music by conducting keyword extraction on user-generated documents. Utilizing a large collection of documents containing the contextual information from users and a song request, we first built song-document associations by discovering the request song through calculating the Levenstein distance. We then extracted keywords from each document to obtain context-relevant music descriptors, which we used to explain the user’s general

Acknowledgement

This research was supported by the MSIP (Ministry of Science, ICT, and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2016-H8501-16-1016) supervised by the IITP (Institute for Information & communications Technology Promotion).

References (43)

  • M.-K. Shan et al.

    Emotion-based music recommendation by affinity discovery from film music

    Expert Systems with Applications

    (2009)
  • L. Baltrunas et al.

    Incarmusic: Context-aware music recommendations in a car.

    Ec-web

    (2011)
  • T. Bertin-Mahieux et al.

    Automatic tagging of audio: The state-of-the-art

    Machine Audition: Principles, Algorithms and Systems

    (2010)
  • T. Bertin-Mahieux et al.

    The million song dataset.

    Ismir

    (2011)
  • D.M. Blei et al.

    Latent dirichlet allocation

    Journal of machine Learning research

    (2003)
  • M. Braunhofer et al.

    Location-aware music recommendation

    International Journal of Multimedia Information Retrieval

    (2013)
  • Z. Cheng et al.

    Just-for-me: An adaptive personalization system for location-aware social music recommendation

    Proceedings of international conference on multimedia retrieval

    (2014)
  • S. Cunningham et al.

    Saturday night or fever? context-aware music playlists

    Proceeding audio mostly

    (2008)
  • T.L. Griffiths et al.

    Finding scientific topics

    Proceedings of the National academy of Sciences

    (2004)
  • B.-j. Han et al.

    Music emotion classification and context-based music recommendation

    Multimedia Tools and Applications

    (2010)
  • N. Hariri et al.

    Using social tags to infer context in hybrid music recommendation

    Proceedings of the twelfth international workshop on web information and data management

    (2012)
  • Cited by (19)

    • University students’ use of music for learning and well-being: A qualitative study and design implications

      2021, Information Processing and Management
      Citation Excerpt :

      Biographies of singers could be explored to identify their similarity in life experience, to complement existing artist recommendation mechanism. Recent research has also explored the possibility of exploiting user-generated documents or song interpretations to facilitate music search (Choi, Lee, Hu & Downie, 2016; Hyung et al., 2017). Taking into account the role of music in shaping/developing one's meaning for life, online music services could also consider users’ interpretations and/or reviews of songs as an input of the recommendation system.

    • Cochleogram-based approach for detecting perceived emotions in music

      2020, Information Processing and Management
      Citation Excerpt :

      Their context-based model used social media data to collect information about public perception of songs, while the recommendation was based on common opinions. Hyung, Park, and Lee (2017) proposed extracting context-relevant keywords from user documents. Their solution utilized these context-relevant music descriptors to aid users in performing text querying while searching for music.

    • A multi-centrality index for graph-based keyword extraction

      2019, Information Processing and Management
      Citation Excerpt :

      It is very useful in organizing the documents without the need of human annotators since assigning keywords manually can be costly and time-consuming. An automatic analysis that extracts information over a huge amount of data has been proposed in different domains (Hyung, Park, & Lee, 2017; Morillo & Álvarez-Bornstein, 2018; Nasar, Jaffry, & Malik, 2018; Raamkumar, Foo, & Pang, 2017; Xu et al., 2018). Several approaches have been reported addressing the automatic keyword extraction problem (Bharti & Babu, 2017; Hasan & Ng, 2014).

    • Beyond the Big Five personality traits for music recommendation systems

      2023, Eurasip Journal on Audio, Speech, and Music Processing
    • An innovative method for topic modeling employing speed reading

      2023, Proceedings - SITA 2023: 2023 14th International Conference on Intelligent Systems: Theories and Applications
    View all citing articles on Scopus
    View full text