Skip to main content
SpringerLink
Log in

Deep autoencoders for feature learning with embeddings for recommendations: a novel recommender system solution

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

We propose “Deep Autoencoders for Feature Learning in Recommender Systems,” a novel discriminative model based on the incorporation of features from autoencoders in combination with embeddings into a deep neural network to predict ratings in recommender systems. The work has two major motivations. The first is to engineer features for recommender systems in a domain-agnostic way using autoencoders. The second is to develop a method that sets a benchmark for predictive accuracy. In our proposed solution, we build a user autoencoder and item autoencoder that extract latent features for the users and items, respectively. The additional features engineered are the latent features for the users and items, and these come from the bottleneck activations of the autoencoder. Our method of feature engineering is domain agnostic, as the inner-most activations would differ for domains without any additional effort required on part of the modeler. Next, we then use the activations of the inner-most layers of the autoencoders as features in a subsequent deep neural network to predict the ratings along-with user and item embeddings. Our method incorporates additional linear and nonlinear latent features from the autoencoders to improve predictive accuracy. This is different from the existing approaches that use autoencoders as full-fledged recommender systems or use autoencoders to generate features for a subsequent supervised learning algorithm or without using embeddings. We demonstrate the out performance of our solution on four different datasets of varying sizes and sparsity, namely MovieLens 100 K, MovieLens 1 M, FilmTrust and BookCrossing datasets, with strong experimental results. We have compared our DAFERec method against mDA-CF, TrustSVD, SVD variants, BiasedMF, ItemKNN and I-AutoRec methods. The results demonstrate that our proposed solution beats the benchmarks and is a highly flexible model that works on different datasets solving different business problems like book recommendations, movie recommendations and trust.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

Publicly datasets used with links below.

Book-Crossing: http://www2.informatik.uni-freiburg.de/~cziegler/BX/BX-CSV-Dump.zip, MovieLens100K: http://files.grouplens.org/datasets/movielens/ml-latest-small.zip, MovieLens1M: http://files.grouplens.org/datasets/movielens/ml-1m.zip, FilmTrust: https://www.librec.net/datasets/filmtrust.zip.

Code availability

Code made available for reproducible research at https://github.com/efpm04013/experiment3.

Notes

  1. https://github.com/efpm04013/experiment3.

References

  1. Aggarwal CC (2016) An introduction to recommender systems. RecomSyst. https://doi.org/10.1007/978-3-319-29659-3_1

    Article  Google Scholar 

  2. Ketkar N (2017) Introduction to pytorch. Deep learning with python. Apress, Berkeley, pp 195–208. https://doi.org/10.1007/978-1-4842-2766-4_12

    Chapter  Google Scholar 

  3. Chen T, Li M, Li Y, Lin M, Wang N, Wang M, Xiao T, Xu B, Zhang C, Zhang, Z. (2015) MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems. https://doi.org/10.1145/2532637

  4. Google (2018) Tensorflow. 2018

  5. Kiran R, Kumar P, Bhasker B (2020) DNNRec: a novel deep learning based hybrid recommender system. Expert SystAppl. https://doi.org/10.1016/j.eswa.2019.113054

    Article  Google Scholar 

  6. Lee W, Song K, Moon IC (2017) Augmented variational autoencoders for collaborative filtering with auxiliary information. In: Proceedings of the 2017 ACM on conference on information and knowledge management - CIKM ’17. pp 1139–1148. https://doi.org/10.1145/3132847.3132972

  7. Langville A (2005) Nonnegative matrix factorization. Matrix 25(6):1336–1353. https://doi.org/10.1109/TPAMI.2011.18

    Article  Google Scholar 

  8. Salakhutdinov R, Mnih A (2008) Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In: Proceedings of the 25th international conference on machine learning - ICML ’08. https://doi.org/10.1145/1390156.1390267

  9. Sarwar BM., Karypis G, Konstan JA, Riedl JT (2000) Application of dimensionality reduction in recommender system - a case study. Architecture. https://doi.org/10.1138/744

  10. Guo G, Zhang J, Yorke-Smith N (2015) TrustSVD: collaborative filtering with both the explicit and implicit influence of user trust and of item ratings. In: Proceedings of the national conference on artificial intelligence.

  11. Salakhutdinov, R., Mnih, A., & Hinton, G. (2007). Restricted Boltzmann machines for collaborative filtering. In: Proceedings of the 24th international conference on Machine learning - ICML ’07. pp 791–798. https://doi.org/10.1145/1273496.1273596

  12. He X, Liao L, Zhang H, Nie L, Hu X, Chua T-S (2017) (NCF) Neural collaborative filtering. https://doi.org/10.1145/3038912.3052569

    Article  Google Scholar 

  13. Catherine R, Cohen W (2017) Transnets. In: Proceedings of the eleventh ACM conference on recommender systems - RecSys ’17. pp 288–296. https://doi.org/10.1145/3109859.3109878

  14. Cheng HT, Ispir M, Anil R, Haque Z, Hong L, Jain V, et al. (2016) Wide & deep learning for recommender systems. In: Proceedings of the 1st workshop on deep learning for recommender systems - DLRS 2016. pp 7–10. https://doi.org/10.1145/2988450.2988454

  15. van den Oord A, Dieleman S, Schrauwen B (2013) Deep content-based music recommendation. Electron InfSyst Depart (ELIS). https://doi.org/10.1109/MMUL.2011.34.van

    Article  Google Scholar 

  16. Liang D (2014) Content-aware collaborative music recommendation using pre-trained neural networks. In: ISMIR 2015: Proceedings of the 16th international society for music information retrieval conference. pp 295–301

  17. Tan J, Wan X, Xiao J (2016) A neural network approach to quote recommendation in writings. In: Proceedings of the 25th ACM international on conference on information and knowledge management - CIKM ’16. pp 65–74. https://doi.org/10.1145/2983323.2983788

  18. Bansal T, Belanger D, McCallum A (2016) Ask the GRU. In Proceedings of the 10th ACM conference on recommender systems - RecSys ’16. pp 107–114. https://doi.org/10.1145/2959100.2959180

  19. Ko Y-J, Maystre L, Grossglauser M, Durrant RJ, Kim K-E (2016) Collaborative recurrent neural networks for dynamic recommender systems. JMLR 63:366–381. https://doi.org/10.1109/ICDE.2016.7498326

    Article  Google Scholar 

  20. Balakrishnan A (2014) Deepplaylist: using recurrent neural networks to predict song similarity. In: Stanfort University, pp 1–7. Retrieved from https://cs224d.stanford.edu/reports/BalakrishnanDixit.pdf

  21. Wu Y, DuBois C, Zheng AX, Ester M (2016) Collaborative denoising auto-encoders for top-N recommender systems. In: Proceedings of the ninth ACM international conference on web search and data mining - WSDM ’16. pp 153–162. https://doi.org/10.1145/2835776.2835837

  22. Li X, She J (2017) Collaborative variational autoencoder for recommender systems. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining - KDD ’17. pp 305–314. https://doi.org/10.1145/3097983.3098077

  23. Sedhain S, Menon AK, Sanner S, Xie L (2015) Autorec: autoencoders meet collaborative filtering. In: WWW 2015 companion: proceedings of the 24th international conference on world wide web. pp 111–112. https://doi.org/10.1145/2740908.2742726

  24. Sedhain S, Menon AK, Sanner S, Xie L (2015) Autorec: autoencoders meet collaborative filtering. In: 24th international conference on world wide web. https://doi.org/10.1145/2740908.2742726

  25. Vincent P, Larochelle H, Bengio Y, Manzagol PA (2008) Extracting and composing robust features with denoising autoencoders. In: Proceedings of the 25th international conference on machine learning - ICML ’08. https://doi.org/10.1145/1390156.1390294

  26. Li S, Kawale J, Fu Y (2015) Deep collaborative filtering via marginalized denoising auto-encoder. In: Proceedings of the 24th ACM international on conference on information and knowledge management - CIKM ’15. pp 811–820. https://doi.org/10.1145/2806416.2806527

  27. Strub F, Gaudel R, Mary J (2016) Hybrid recommender system based on autoencoders. In: proceedings of the 1st workshop on deep learning for recommender systems - DLRS 2016. pp 11–16. https://doi.org/10.1145/2988450.2988456

  28. Braida F, Mello CE, Pasinato MB, Zimbrão G (2015) Transforming collaborative filtering into supervised learning. Expert SystAppl. https://doi.org/10.1016/j.eswa.2015.01.023

    Article  Google Scholar 

  29. Barbieri J, Alvim LGM, Braida F, Zimbrão G (2017) Autoencoders and recommender systems: COFILS approach. Expert SystAppl 89:81–90. https://doi.org/10.1016/j.eswa.2017.07.030

    Article  Google Scholar 

  30. Strub F, Mary J, Strub F, Mary J, Filtering C, Autoencoders D, Strub F (2015) Collaborative filtering with stacked denoising autoencoders and sparse inputs. In: NIPS workshop on machine learning for ecommerce, December 2015

  31. Ziegler CN, McNee SM, Konstan JA, Lausen G (2005) Improving recommendation lists through topic diversification. https://doi.org/10.1145/1060745.1060754

  32. Guo G, Zhang J, Yorke-Smith N (2013) A novel bayesian similarity measure for recommender systems. In: IJCAI international joint conference on artificial intelligence

  33. Li S, Kawale J, Fu Y (2015) Deep collaborative filtering via marginalized denoising auto-encoder. https://doi.org/10.1145/2806416.2806527

  34. LibRec - Examples & Comparison with Other Recommendation Libraries (n.d.). Retrieved 24 Nov 2018, from https://www.librec.net/release/v1.3/example.html

  35. LibRec - Examples & comparison with other recommendation libraries. (n.d.). Retrieved 27 July 2019, from https://www.librec.net/release/v1.3/example.html

  36. MyMediaLite: Example Experiments. (n.d.). Retrieved 24 Nov 2018, from http://www.mymedialite.net/examples/datasets.html

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Information Technology and Systems, Indian Institute of Management Lucknow, IIM Road, Prabandh Nagar, Mubarakpur, Lucknow, Uttar Pradesh, 226013, India

    Kiran Rama, Pradeep Kumar & Bharat Bhasker

  2. Indian Institute of Management Raipur, Naya Raipur, India

    Bharat Bhasker

Authors
  1. Kiran Rama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pradeep Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bharat Bhasker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kiran Rama.

Ethics declarations

Conflict of interests

Not Applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rama, K., Kumar, P. & Bhasker, B. Deep autoencoders for feature learning with embeddings for recommendations: a novel recommender system solution. Neural Comput & Applic 33, 14167–14177 (2021). https://doi.org/10.1007/s00521-021-06065-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06065-9

Keywords

Search

Navigation