Gawesh Jawaheer
Skip Abstract Section
Abstract
Recommender systems are firmly established as a standard technology for assisting users with their choices; however, little attention has been paid to the application of the user model in recommender systems, particularly the variability and noise that are an intrinsic part of human behavior and activity. To enable recommender systems to suggest items that are useful to a particular user, it can be essential to understand the user and his or her interactions with the system. These interactions typically manifest themselves as explicit and implicit user feedback that provides the key indicators for modeling users’ preferences for items and essential information for personalizing recommendations. In this article, we propose a classification framework for the use of explicit and implicit user feedback in recommender systems based on a set of distinct properties that include Cognitive Effort, User Model, Scale of Measurement, and Domain Relevance. We develop a set of comparison criteria for explicit and implicit user feedback to emphasize the key properties. Using our framework, we provide a classification of recommender systems that have addressed questions about user feedback, and we review state-of-the-art techniques to improve such user feedback and thereby improve the performance of the recommender system. Finally, we formulate challenges for future research on improvement of user feedback.
- Silvana Aciar, Debbie Zhang, Simeon Simoff, and John Debenham. 2006. Recommender system based on consumer product reviews. In Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence (WI’06). IEEE Computer Society, Washington, DC, 719--723. Google Scholar
Digital Library
- Gediminas Adomavicius and YoungOk Kwon. 2008. Overcoming accuracy-diversity tradeoff in recommender systems: A variance-based approach. In Proceedings of the 18th Workshop on Information and Technology and Systems (WITS’08).Google Scholar
- Gediminas Adomavicius, Bamshad Mobasher, Francesco Ricci, and Alex Tuzhilin. 2011. Context-aware recommender systems. AI Magazine 32, 3, 67--80.Google ScholarDigital Library
- Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering 17, 6, 734--749. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1423975. Google ScholarDigital Library
- Xavier Amatriain. 2011. Recommender Systems: We’re Doing It (All) Wrong. Retrieved May 26, 2014, from http://technocalifornia.blogspot.com/2011/04/recommender-systems-were-doing-it-all.html.Google Scholar
- Xavier Amatriain, Josep Pujol, and Nuria Oliver. 2009a. I like it. I like it not: Evaluating user ratings noise in recommender systems. In G.-J. Houben, G. McCalla, F. Pianesi, and M. Zancanaro (Eds.), User Modeling, Adaptation, and Personalization. Springer, Berlin, Heidelberg, 247--258. Google ScholarDigital Library
- Xavier Amatriain, Josep M. Pujol, Nava Tintarev, and Nuria Oliver. 2009b. Rate it again: Increasing recommendation accuracy by user re-rating. In Proceedings of the 3rd Conference on Recommender Systems. ACM Press, New York, NY, 173--180. Google ScholarDigital Library
- Justin Basilico and Thomas Hofmann. 2004. Unifying collaborative and content-based filtering. In Proceedings of the 21st International Conference on Machine Learning (ICML’04). ACM Press, New York, NY, 9. Google ScholarDigital Library
- Robert M. Bell and Yehuda Koren. 2007. Lessons from the Netflix prize challenge. ACM SIGKDD Explorations Newsletter 9, 2, 75. http://portal.acm.org/citation.cfm?doid=1345448.1345465. Google ScholarDigital Library
- James Bennett and Stan Lanning. 2007. The Netflix prize. In Proceedings of the KDD-Cup and Workshop at the 13th SIGKDD International Conference on Knowledge Discovery and Data Mining.Google Scholar
- Mark Claypool, Phong Le, Makoto Wased, and David Brown. 2001. Implicit interest indicators. In Proceedings of the 6th International Conference on Intelligent User Interfaces (IUI’01). ACM Press, New York, NY, 33--40. Google ScholarDigital Library
- Dan Cosley, Shyong K. Lam, Istvan Albert, Joseph A. Konstan, and John Riedl. 2003. Is seeing believing? How recommender interfaces affect users. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’03). ACM Press, New York, NY, 585--592. http://portal.acm.org/ citation.cfm?doid=642611.642713. Google ScholarDigital Library
- James Davidson, Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi, Sujoy Gupta, Yu He, Mike Lambert, Blake Livingston, and Dasarathi Sampath. 2010. The YouTube video recommendation system. In Proceedings of the 4th Conference on Recommender Systems. ACM Press, New York, NY, 293--296. Google ScholarDigital Library
- Mukund Deshpande and George Karypis. 2004. Item-based top-N recommendation algorithms. ACM Transactions on Information Systems 22, 1, 143--177. http://portal.acm.org/citation.cfm?doid=963770.963776. Google ScholarDigital Library
- Christian Desrosiers and George Karypis. 2010. A novel approach to compute similarities and its application to item recommendation. In Proceedings of the 11th Pacific Rim International Conference on Trends in Artificial Intelligence. Springer-Verlag, Berlin, Heidelberg, 39--51. Google ScholarDigital Library
- Andy Field and Graham Hole. 2003. How to Design and Report Experiments. Sage Publications.Google Scholar
- Sandra C. Gadanho and Nicolas Lhuillier. 2007. Addressing uncertainty in implicit preferences. In Proceedings of the 2007 Conference on Recommender Systems. ACM Press, New York, NY, 97--104. Google ScholarDigital Library
- Sandra Garcia Esparza, Michael P. O’Mahony, and Barry Smyth. 2012. Mining the real-time Web: A novel approach to product recommendation. Journal of Knowledge-Based Systems: Special Issue on Innovative Techniques and Applications of Artificial Intelligence 29, 3--11. Google ScholarDigital Library
- Ken Goldberg, Theresa Roeder, Dhruv Gupta, and Chris Perkins. 2001. Eigentaste: A constant time collaborative filtering algorithm. Information Retrieval 4, 2, 133--151. http://www.springerlink.com/index/ M5458KV8LJ602646.pdf. Google ScholarDigital Library
- GroupLens Research. 2011a. EachMovie Dataset. Retrieved May 26, 2014, from http://www.grouplens.org/ node/76.Google Scholar
- GroupLens Research. 2011b. MovieLens Dataset. Retrieved May 26, 2014, from http://www.grouplens.org/ node/73.Google Scholar
- Maxwell F. Harper, Xin Li, Yan Chen, and Joseph A. Konstan. 2005. An economic model of user rating in an online recommender system. In L. Ardissono, P. Brna, and A. Mitrovic (Eds.), User Modeling 2005. Springer, 307--316. Google ScholarDigital Library
- Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems 22, 1, 5--53. http://portal.acm.org/citation.cfm?doid=963770.963772. Google ScholarDigital Library
- Oscar Celma Herrada, 2009a. Last.fm Dataset. Retrieved May 26, 2014, from http://www.dtic.upf.edu/∼ocelma/MusicRecommendationDataset/index.html.Google Scholar
- Oscar Celma Herrada. 2009b. Music Recommendation and Discovery in the Long Tail. Ph.D. Dissertation.Google Scholar
- Will Hill, Larry Stead, Mark Rosenstein, and George Furnas. 1995. Recommending and evaluating choices in a virtual community of use. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’95). 194--201. http://portal.acm.org/citation.cfm?doid=223904.223929. Google ScholarDigital Library
- Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In Proceedings of the 2008 8th IEEE International Conference on Data Mining. 263--272. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4781121. Google ScholarDigital Library
- Zan Huang. 2007. Selectively acquiring ratings for product recommendation. In Proceedings of the 9th International Conference on Electronic Commerce. ACM Press, New York, NY, 379--388. Google ScholarDigital Library
- IMDB.com Inc. 1990. The Internet Movie Database. Retrieved May 26, 2014, from http://www.imdb.com.Google Scholar
- Dietmar Jannach, Markus Zanker, Alexander Felfernig, and Friedrich Gerhard. 2011. Recommender Systems: An Introduction. Cambridge University Press. Google Scholar
- Gawesh Jawaheer, Martin Szomszor, and Patty Kostkova. 2010a. Characterisation of explicit feedback in an online music recommendation service. In Proceedings of the 4th Conference on Recommender Systems. ACM Press, New York, NY, 317--320. Google ScholarDigital Library
- Gawesh Jawaheer, Martin Szomszor, and Patty Kostkova. 2010b. Comparison of implicit and explicit feedback from an online music recommendation service. In Proceedings of the 1st International Workshop on Information Heterogeneity and Fusion in Recommender Systems. ACM Press, New York, NY, 47--51. Google ScholarDigital Library
- Jester Dataset. 2012. Anonymous Ratings from the Jester Online Joke Recommender System. Retrieved May 26, 2014, from http://eigentaste.berkeley.edu/jester-data/.Google Scholar
- Jinmook Kim and Douglas W. Oard. 2001. User modeling for information access based on implicit feedback. In Proceedings of the Symposium of ISKO-France. 1--11.Google Scholar
- Daniel Kluver, Tien T. Nguyen, Michael Ekstrand, Shilad Sen, and John Riedl. 2012. How many bits per rating? In Proceedings of the 6th Conference on Recommender Systems. ACM Press, New York, NY, 99--206. Google ScholarDigital Library
- Joseph A. Konstan, Bradley N. Miller, David Maltz, Jonathan L. Herlocker, Lee R. Gordon, and John Riedl. 1997. GroupLens: Applying collaborative filtering to Usenet news. Communications of the ACM 40, 3, 87. http://portal.acm.org/citation.cfm?id=245108.245126. Google ScholarDigital Library
- Yehuda Koren. 2010. Factor in the neighbors: Scalable and accurate collaborative filtering. ACM Transactions on Knowledge Discovery from Data 4, 1, 1--24. http://portal.acm.org/citation.cfm?doid=1644873.1644874. Google ScholarDigital Library
- Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer 42, 8, 30--37. Google ScholarDigital Library
- Yehuda Koren and Joe Sill. 2011. OrdRec: An ordinal model for predicting personalized item rating distributions. In Proceedings of the 5th Conference on Recommender Systems (RecSys’11). ACM Press, New York, NY, 117--124. Google ScholarDigital Library
- Patty Kostkova, Gayo Diallo, and Gawesh Jawaheer. 2008. User profiling for semantic browsing in medical digital libraries. In The Semantic Web: Research and Applications. Springer, 827--831. Google ScholarDigital Library
- Patty Kostkova and Gemma Madle. 2009. User-centered evaluation model for medical digital libraries. In Knowledge Management for Health Care Procedures. Springer, 92--103. Google ScholarDigital Library
- Patty Kostkova and Gemma Madle. 2013. What impact do healthcare digital libraries have? An evaluation of national resource of infection control at the point of care using the Impact-ED framework. International Journal on Digital Libraries 13, 2, 77--90. Google ScholarDigital Library
- YoungOk Kwon. 2008. Improving top-N recommendation techniques using rating variance. In Proceedings of the 2008 Conference on Recommender Systems (RecSys’08). ACM Press, New York, NY, 307--310. Google ScholarDigital Library
- Juha Leino and Kari-Jouko Raiha. 2007. Case amazon: Ratings and reviews as part of recommendations. In Proceedings of the 2007 Conference on Recommender Systems (RecSys’07). ACM Press, New York, NY, 137--140. Google ScholarDigital Library
- Sarah Lichtenstein and Paul Slovic (Eds.). 2006. The Construction of Preference. Cambridge University Press.Google Scholar
- Nathan N. Liu, Evan W. Xiang, Min Zhao, and Qiang Yang. 2010. Unifying explicit and implicit feedback for collaborative filtering. In Proceedings of the 19th Conference on Information and Knowledge Management (CIKM’10). ACM Press, New York, NY, 1445--1448. Google ScholarDigital Library
- Yue Lu, ChengXiang Zhai, and Neel Sundaresan. 2009. Rated aspect summarization of short comments. In Proceedings of the 18th International Conference on World Wide Web (WWW’09). ACM Press, New York, NY, 131--140. Google ScholarDigital Library
- Gemma Madle, Anouk Berger, Sebastien Cognat, Sylvio Menna, and Patty Kostkova. 2009. User information seeking behaviour: Perceptions and reality. An evaluation of the WHO Labresources Internet portal. Informatics for Health and Social Care 34, 1, 30--38.Google ScholarCross Ref
- Benjamin M. Marlin and Richard S. Zemel. 2009. Collaborative prediction and ranking with non-random missing data. In Proceedings of the 3rd ACM Conference on Recommender Systems (RecSys’09). ACM Press, New York, NY, 5--12. http://portal.acm.org/citation.cfm?doid=1639714.1639717. Google ScholarDigital Library
- Lorraine McGinty and Barry Smyth. 2002. Evaluating preference-based feedback in recommender systems. In Artificial Intelligence and Cognitive Science. Springer, 209--214. Google ScholarDigital Library
- Lorraine McGinty and Barry Smyth. 2005. Improving the Performance of Recommender Systems That Use Critiquing. Springer.Google Scholar
- Omar Moling, Linas Baltrunas, and Francesco Ricci. 2012. Optimal radio channel recommendations with explicit and implicit feedback. In Proceedings of the 6th ACM Conference on Recommender Systems (RecSys’12). ACM Press, New York, NY, 75--82. Google ScholarDigital Library
- David M. Nichols. 1997. Implicit rating and filtering. In Proceedings of the 5th DELOS Workshop on Filtering and Collaborative Filtering.Google Scholar
- Douglas W. Oard. 1998. Implicit feedback for recommender systems. In Proceedings of the AAAI Workshop on Recommender Systems.Google Scholar
- Douglas W. Oard and Jinmook Kim. 2001. Modeling information content using observable behavior. In Proceedings of the 64th Annual Conference of the American Society for Information Science and Technology.Google Scholar
- Helen Oliver, Gayo Diallo, Ed De Quincey, Dimitra Alexopoulou, Bianca, Habermann, Patty Kostkova, Michael Schroeder, Simon Jupp, Khaled Khelif, Robert Stevens, Gawesh Jawaheer, and Madle Gawesh. 2009. A user-centered evaluation framework for the Sealife semantic Web browsers. BMC Bioinformatics 10, Suppl. 10, 1.Google Scholar
- Michael P. O’Mahony, Neil J. Hurley, and Guénolé C. M. Silvestre. 2006. Detecting noise in recommender system databases. In Proceedings of the 11th International Conference on Intelligent User Interfaces (IUI’06). ACM Press, New York, NY, 109. http://portal.acm.org/citation.cfm?doid=1111449.1111477. Google ScholarDigital Library
- Denis Parra and Xavier Amatriain. 2011. Walk the talk: Analyzing the relation between implicit and explicit feedback for preference elicitation. In Proceedings of the 19th International Conference on User Modeling, Adaptation, and Personalization (UMAP’11). Springer-Verlag, Berlin, Heidelberg, 255--268. Google ScholarDigital Library
- Denis Parra, Alexandros Karatzoglou, Xavier Amatriain, and Idil Yavuz. 2011. Implicit feedback recommendation via implicit-to-explicit ordinal logistic regression mapping. In Proceedings of CARS 2011.Google Scholar
- Al Manunur Rashid, George Karypis, and John Riedl. 2008. Learning preferences of new users in recommender systems: An information theoretic approach. ACM SIGKDD Explorations Newsletter 10, 2, 90--100. http://dl.acm.org/citation.cfm?id=1540302. Google ScholarDigital Library
- Steffen Rendle, Christoph Freudenthaler, Gantner Zeno, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. AUAI Press, Arlington, VA, 452--461. Google ScholarDigital Library
- Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor (Eds.). 2010. Recommender Systems Handbook. Springer. http://www.springerlink.com/index/10.1007/978-0-387-85820-3. Google ScholarDigital Library
- Judy Robertson. 2011. Stats: We’re Doing It Wrong. Retrieved May 26, 2014, from http://cacm.acm.org/blogs/ blog-cacm/107125-stats-were-doing-it-wrong/fulltext.Google Scholar
- Anjana Roy, Patty Kostkova, Mike Catchpole, and Ewart Carson. 2010. “Do users do what they think they do?” A comparative study of user perceived and actual information searching behaviour in the National Electronic Library of Infection. In Electronic Healthcare. Springer, 96--103.Google Scholar
- Ruslan Salakhutdinov and Andriy Mnih. 2008. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems. Vol. 20, 1257--1264.Google ScholarDigital Library
- J. Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative filtering recommender systems. In P. Brusilovsky, A. Kobsa, and W. Nejdl (Eds.), The Adaptive Web. Springer, Berlin, Heidelberg, 291--324. Google ScholarDigital Library
- Stefan Siersdorfer, Sergiu Chelaru, Wolfgang Nejdl, and Jose San Pedro. 2010. How useful are your comments?: Analyzing and predicting YouTube comments and comment ratings. In Proceedings of the 19th International Conference on World Wide Web (WWW’10). ACM Press, New York, NY, 891--900. Google ScholarDigital Library
- E. Isaac Sparling and Shilad Sen. 2011. Rating: How difficult is it? In Proceedings of the 5th Conference on Recommender Systems. ACM Press, New York, NY, 149--156. Google ScholarDigital Library
- Martin Szomszor, Ciro Cattuto, Harith Alani, Kieron O’Hara, Andrea Baldassarri, Vittorio Loreto, and Vito D. P. Servedio. 2007. Folksonomies, the semantic Web, and movie recommendation. In Proceedings of the 4th European Semantic Web Conference, Bridging the Gap Between Semantic Web and Web 2.0.Google Scholar
- Akhmed Umyarov and Alexander Tuzhilin. 2007. Leveraging aggregate ratings for better recommendations. In Proceedings of the 2007 Conference on Recommender Systems (RecSys’07). ACM Press, New York, NY, 161. http://portal.acm.org/citation.cfm?doid=1297231.1297261. Google ScholarDigital Library
- Akhmed Umyarov and Alexander Tuzhilin. 2008. Improving collaborative filtering recommendations using external data. In Proceedings of the 2008 8th International Conference on Data Mining. IEEE Computer Society, Washington, DC, 618--627. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber= 4781157. Google ScholarDigital Library
- Akhmed Umyarov and Alexander Tuzhilin. 2009. Improving rating estimation in recommender systems using aggregation- and variance-based hierarchical models. In Proceedings of the 3rd Conference on Recommender Systems (RecSys’09). ACM Press, New York, NY, 37. http://portal.acm.org/citation.cfm? doid=1639714.1639722. Google ScholarDigital Library
- Markus Weimer, Alexandros Karatzoglou, and Alex Smola. 2008. Improving maximum margin matrix factorization. Machine Learning 72, 3, 263--276. Google ScholarDigital Library
- Diyi Yang, Tianqi Chen, Weinan Zhang, Qiuxia Lu, and Yong Yu. 2012. Local implicit mining for music recommendation. In Proceedings of the 5th Conference on Recommender Systems. ACM Press, New York, NY, 91--98. Google ScholarDigital Library
- Markus Zanker and Markus Jessenitschnig. 2009. Case-studies on exploiting explicit customer requirements in recommender systems. User Modeling and User-Adapted Interaction 19, 1, 133--166. Google ScholarDigital Library
- Cai-Nicolas Ziegler. 2004. Book-Crossing Dataset. Retrieved May 26, 2014, from http://www.informatik.uni-freiburg.de/∼cziegler/BX/.Google Scholar
Index Terms
- Modeling User Preferences in Recommender Systems: A Classification Framework for Explicit and Implicit User Feedback
Recommendations
Comparison of implicit and explicit feedback from an online music recommendation service
HetRec '10: Proceedings of the 1st International Workshop on Information Heterogeneity and Fusion in Recommender SystemsExplicit and implicit feedback exhibits different characteristics of users' preferences with both pros and cons. However, a combination of these two types of feedback provides another paradigm for recommender systems (RS). Their combination in a user ...
Unifying explicit and implicit feedback for collaborative filtering
CIKM '10: Proceedings of the 19th ACM international conference on Information and knowledge managementMost collaborative filtering algorithms are based on certain statistical models of user interests built from either explicit feedback (eg: ratings, votes) or implicit feedback (eg: clicks, purchases). Explicit feedbacks are more precise but more ...
Building recommender systems for scholarly information
SWM '17: Proceedings of the 1st Workshop on Scholarly Web MiningThe depth and breadth of research now being published is overwhelming for an individual researcher to keep track of let alone consume. Recommender systems have been developed to make it easier for researchers to discover relevant content. However, these ...
Comments