Utilizing context-relevant keywords extracted from a large collection of user-generated documents for music discovery
Introduction
Proliferation of music data available for users increased the demand of searching and retrieving music that perfectly suits the individual listener’s situation and entertainment needs. In order to meet such demands, Schedl and Knees (2013) emphasized the importance of personalized and user context-aware systems. As a consequence, music exploration systems implemented various functionalities in an attempt to provide more satisfying results. Some of the methods to search and retrieve music from such systems are searching music using metadata, retrieving recommended music, and browsing predefined playlists (Nanopoulos, Rafailidis, Ruxanda, & Manolopoulos, 2009).
Metadata includes music-related information such as the artist, title, and genre information. Users can query the music exploration system using text to retrieve the exact match. However, metadata lack contextual information, so using a metadata query will not satisfy the user if he or she seeks music in a certain context such as mood or situation. Recently, the utilization of social tags to enhance music description has been attempted in the research field of Music Information Retrieval. Symeonidis, Ruxanda, Nanopoulos, and Manolopoulos (2008) gathered social tags obtained from Last.fm to recommend music. However, as Lamere (2008) pointed out, most social tags are related to the artist, title, genre, and instrument, whereas only a small proportion is related to the user context. These social tags enhance text-based music search to some extent but they have difficulties with context-related terms.
Music exploration systems also provide music recommendations to users using various similarity measures. Collaborative filtering algorithms retrieve similar music by discovering similar user preference (Xing, Wang, & Wang, 2014). User preference is inferred by analyzing user ratings and/or user playlist. On the other hand, content-based algorithms utilize music-centric features such as timbre, pitch, and lyrics to discover similar music (Bogdanov, Haro, Fuhrmann, Xambó, Gómez, Herrera, 2013, Li, Myaeng, Kim, 2007, Mayer, Neumayer, Rauber, 2008). These music-centric features include information obtained from the audio signal itself, but they lack information about the listener. Therefore, these music-centric features have a limited capacity to reflect the needs of the user.
Additionally, playlists tagged with predefined terms are also available in most of the music exploration systems. For instance, Allmusic1 provides playlists that are tagged with predefined moods and themes. However, the recommendations provided are unbalanced in terms of the song distribution per mood/theme. For instance, the system suggests various songs for the in love theme but it only provides one song under the theme of work. Another problem is that it is difficult to build a consensus among the users because there is no specific standard for selecting moods and themes.
In this paper, we extract context-relevant keywords from a large collection of user-generated documents to capture the user contextual background when searching for music. Each document comes from a radio program’s Internet bulletin board, where it comprises a personal story and a song request. Fig. 1 illustrates the concept of the proposed system. Hyung, Lee, and Lee (2014) showed that there is a strong correlation between personal stories and song requests. There are large number of such documents, so we consider that some general terms will be used to describe the user contextual background when requesting songs. Therefore, by performing keyword extraction based on these documents, it will be possible to create generalized context-relevant music descriptors, which can be applied to music search and retrieval. We collected 186,656 documents sent from the listeners of a radio program aired between 6:00 p.m. and 8:00 p.m.. We chose this program as they only air Western pop songs.
In this paper, we introduce a novel approach that utilizes user-generated documents to extract user context for music search and retrieval. We described music using context-relevant keywords extracted from a large collection of user-generated documents. By utilizing context-relevant music descriptors, our system facilitates natural language text querying when searching for music and thus, it can retrieve music that satisfies the entertainment needs of users. Additionally, the keywords describing the user context are extracted from a large collection of documents, so the coverage of the explainable contextual background when listening to music will be broad. Therefore, users will be able to discover music in various contexts. Finally, we determined a correlation between our proposed context-relevant music descriptors and conventional features, such as acoustic features or lyrics, which could provide some insights into how to overcome the cold start problem where pieces of music with no document associations are never discovered.
The remainder of this paper is organized as follows. The next section reviews recent papers that embed the user context for music discovery. Studies that exploited text to describe music are also reviewed. Section 3 explains the proposed system in detail. In 4 Social tags vs. context-relevant music descriptors, 5 Examining the correlation with conventional features, and 6, various experiments that were conducted in an attempt to validate the proposed approach are described. Finally, Section 7 presents the conclusions by providing a summary and suggesting directions for future research.
Section snippets
Background
Recently, novel approaches to the inclusion of contextual information related to users have been attempted actively in order to provide a richer context-relevant representation of music. In this section, we review recent studies that embedded contextual information related to users during music retrieval. We also describe some studies that employed text to enhance music descriptions.
Context-relevant music descriptors
In this section, we explain our system, which uses context-relevant keywords to describe music. Fig. 3 shows a block diagram that illustrates our system and the evaluation process.
Our system comprises three steps: preprocessing, music descriptor generation, and music discovery. In the preprocessing step, we extract the song information from each document to create song-document associations. We also remove stop-words and stemmed words during this step. In the subsequent music descriptor
Social tags vs. context-relevant music descriptors
In order to show that our music descriptors contained more context-relevant terms than social tags, we compared the terms in gKD with the social tags provided by Last.fm. For this comparison, we first categorized the terms and the tags with several labels. Lamere (2008) used the labels Genre, Locale, Mood, Opinion, Instrumentation, Style, Misc, Personal, and Organization to categorize Last.fm tags. Our aim was to examine the distribution of music-related words and context-related words, so we
Dataset
In order to examine the correlation with conventional features, we built a noise-free dataset. The dataset comprises songs with more than 20 document associations. We manually removed documents associated with incorrect songs. Thus, we had 350 songs with a minimum of 20 and a maximum of 365 documents associated with a song. The lyrics data were manually downloaded from various Internet sites, such as MetroLyrics3 and LyricsFreak.4 We selected these two
Qualitative evaluation
By comparing the label distribution of social tags obtained from Last.fm with the label distribution of terms within our proposed context-relevant music descriptors, we showed that our approach contains richer context-relevant terms. Additionally, we examined correlation between the proposed music descriptors and conventional features. However, this does not verify if the proposed music descriptors can be used for music discovery. Therefore, in an attempt to show that our music descriptors are
Conclusion
In this paper, we proposed a novel method for representing music by conducting keyword extraction on user-generated documents. Utilizing a large collection of documents containing the contextual information from users and a song request, we first built song-document associations by discovering the request song through calculating the Levenstein distance. We then extracted keywords from each document to obtain context-relevant music descriptors, which we used to explain the user’s general
Acknowledgement
This research was supported by the MSIP (Ministry of Science, ICT, and Future Planning), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2016-H8501-16-1016) supervised by the IITP (Institute for Information & communications Technology Promotion).
References (43)
- et al.
Studying emotion induced by music through a crowdsourcing game
Information Processing & Management
(2016) - et al.
Semantic audio content-based music recommendation and visualization based on user preference examples
Information Processing & Management
(2013) - et al.
On effective location-aware music recommendation
ACM Transactions on Information Systems (TOIS)
(2016) - et al.
Exploring user emotion in microblogs for music recommendation
Expert Systems with Applications
(2015) - et al.
Music recommendation using text analysis on song requests to radio stations
Expert Systems with Applications
(2014) - et al.
A probabilistic music recommender considering user opinions and audio features
Information processing & management
(2007) - et al.
A novel method for personalized music recommendation
Expert Systems with Applications
(2009) - et al.
Music search engines: Specifications and challenges
Information Processing & Management
(2009) - et al.
Term-weighting approaches in automatic text retrieval
Information processing & management
(1988) - et al.
A music information system automatically generated via web content mining techniques
Information Processing & Management
(2011)
Emotion-based music recommendation by affinity discovery from film music
Expert Systems with Applications
Incarmusic: Context-aware music recommendations in a car.
Ec-web
Automatic tagging of audio: The state-of-the-art
Machine Audition: Principles, Algorithms and Systems
The million song dataset.
Ismir
Latent dirichlet allocation
Journal of machine Learning research
Location-aware music recommendation
International Journal of Multimedia Information Retrieval
Just-for-me: An adaptive personalization system for location-aware social music recommendation
Proceedings of international conference on multimedia retrieval
Saturday night or fever? context-aware music playlists
Proceeding audio mostly
Finding scientific topics
Proceedings of the National academy of Sciences
Music emotion classification and context-based music recommendation
Multimedia Tools and Applications
Using social tags to infer context in hybrid music recommendation
Proceedings of the twelfth international workshop on web information and data management
Cited by (19)
University students’ use of music for learning and well-being: A qualitative study and design implications
2021, Information Processing and ManagementCitation Excerpt :Biographies of singers could be explored to identify their similarity in life experience, to complement existing artist recommendation mechanism. Recent research has also explored the possibility of exploiting user-generated documents or song interpretations to facilitate music search (Choi, Lee, Hu & Downie, 2016; Hyung et al., 2017). Taking into account the role of music in shaping/developing one's meaning for life, online music services could also consider users’ interpretations and/or reviews of songs as an input of the recommendation system.
Cochleogram-based approach for detecting perceived emotions in music
2020, Information Processing and ManagementCitation Excerpt :Their context-based model used social media data to collect information about public perception of songs, while the recommendation was based on common opinions. Hyung, Park, and Lee (2017) proposed extracting context-relevant keywords from user documents. Their solution utilized these context-relevant music descriptors to aid users in performing text querying while searching for music.
A multi-centrality index for graph-based keyword extraction
2019, Information Processing and ManagementCitation Excerpt :It is very useful in organizing the documents without the need of human annotators since assigning keywords manually can be costly and time-consuming. An automatic analysis that extracts information over a huge amount of data has been proposed in different domains (Hyung, Park, & Lee, 2017; Morillo & Álvarez-Bornstein, 2018; Nasar, Jaffry, & Malik, 2018; Raamkumar, Foo, & Pang, 2017; Xu et al., 2018). Several approaches have been reported addressing the automatic keyword extraction problem (Bharti & Babu, 2017; Hasan & Ng, 2014).
Beyond the Big Five personality traits for music recommendation systems
2023, Eurasip Journal on Audio, Speech, and Music ProcessingMusic emotion recognition based on a modified brain emotional learning model
2023, Multimedia Tools and ApplicationsAn innovative method for topic modeling employing speed reading
2023, Proceedings - SITA 2023: 2023 14th International Conference on Intelligent Systems: Theories and Applications