Abstract
In this concept paper, we explore some of the aspects of quality of continuous learning artificial intelligence systems as they interact with and influence their environment. We study an important problem of implicit feedback loops that occurs in recommendation systems, web bulletins and price estimation systems. We demonstrate how feedback loops intervene with user behavior on an exemplary housing prices prediction system. Based on a preliminary model, we highlight sufficient existence conditions when such feedback loops arise and discuss possible solution approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Such as openrent.co.uk or zillow.com or any other similar website.
- 2.
The source code for the experiment is available at https://github.com/prog-autom/hidden-demo.
References
Amershi, S., et al.: Software engineering for machine learning: a case study. In: Proceedings of the IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2019), pp. 291–300. IEEE (2019)
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., Mané, D.: Concrete problems in AI safety. arXiv preprint arXiv:1606.06565 (2016)
Arpteg, A., Brinne, B., Crnkovic-Friis, L., Bosch, J.: Software engineering challenges of deep learning. In: Proceedings of the 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2018), pp. 50–59. IEEE (2018)
Bottou, L., et al.: Counterfactual reasoning and learning systems: the example of computational advertising. J. Mach. Learn. Res. 14(1), 3207–3260 (2013)
Brun, Y., et al.: Engineering self-adaptive systems through feedback loops. In: Cheng, B.H.C., de Lemos, R., Giese, H., Inverardi, P., Magee, J. (eds.) Software Engineering for Self-Adaptive Systems. LNCS, vol. 5525, pp. 48–70. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02161-9_3
Chouldechova, A., Roth, A.: The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810 (2018)
Colleoni, E., Rozza, A., Arvidsson, A.: Echo chamber or public sphere? predicting political orientation and measuring political homophily in twitter using big data. J. Commun. 64(2), 317–332 (2014)
DiFranzo, D., Gloria-Garcia, K.: Filter bubbles and fake news. XRDS Crossroads ACM Mag. Students 23(3), 32–35 (2017)
Ensign, D., Friedler, S.A., Neville, S., Scheidegger, C., Venkatasubramanian, S.: Runaway feedback loops in predictive policing (2017)
Hadzic, O., Pap, E.: Fixed Point Theory in Probabilistic Metric Spaces, vol. 536. Springer Science & Business Media, Dordrecht (2013)
Harrison Jr., D., Rubinfeld, D.L.: Hedonic housing prices and the demand for clean air. J. Environ. Econ. Manage. 5, 81–102 (1978)
Joshi, M.C., Bose, R.K.: Some Topics in Nonlinear Functional Analysis. John Wiley & Sons, New York (1985)
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., Zhang, G.: Learning under concept drift: a review. IEEE Trans. Knowl. Data Eng. 31(12), 2346–2363 (2018)
Lwakatare, L.E., Raj, A., Bosch, J., Olsson, H.H., Crnkovic, I.: A taxonomy of software engineering challenges for machine learning systems: an empirical investigation. In: Kruchten, P., Fraser, S., Coallier, F. (eds.) XP 2019. LNBIP, vol. 355, pp. 227–243. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19034-7_14
Martin Jr, D., Prabhakaran, V., Kuhlberg, J., Smart, A., Isaac, W.S.: Participatory problem formulation for fairer machine learning through community based system dynamics. arXiv preprint arXiv:2005.07572 (2020)
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge (2012)
Pariser, E.: The Filter Bubble: What the Internet is Hiding from You. Penguin, New York (2011)
Pedregosa, F., et al.: Scikit-learn: Machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Sculley, D., et al.: Hidden technical debt in machine learning systems. In: Advances in Neural Information Processing Systems, pp. 2503–2511 (2015)
Sinha, A., Gleich, D.F., Ramani, K.: Deconvolving feedback loops in recommender systems. In: Advances in Neural Information Processing Systems, pp. 3243–3251 (2016)
Suryn, W., Abran, A., April, A.: ISO/IEC SQuaRE: the second generation of standards for software product quality (2003)
Wan, Z., Xia, X., Lo, D., Murphy, G.C.: How does machine learning change software development practices? IEEE Trans. Softw. Eng. 9, 4492–4500 (2019)
Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)
Zhang, J.M., Harman, M., Ma, L., Liu, Y.: Machine learning testing: survey, landscapes and horizons. IEEE Trans. Softw. Eng. (2020). https://doi.org/10.1109/TSE.2019.2962027
Acknowledgements
Authors are thankful to the anonymous reviewers whose useful feedback helped to improve the paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Khritankov, A. (2021). Hidden Feedback Loops in Machine Learning Systems: A Simulation Model and Preliminary Results. In: Winkler, D., Biffl, S., Mendez, D., Wimmer, M., Bergsmann, J. (eds) Software Quality: Future Perspectives on Software Engineering Quality. SWQD 2021. Lecture Notes in Business Information Processing, vol 404. Springer, Cham. https://doi.org/10.1007/978-3-030-65854-0_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-65854-0_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-65853-3
Online ISBN: 978-3-030-65854-0
eBook Packages: Computer ScienceComputer Science (R0)