Xinru Wang
ABSTRACT
Internet users make numerous decisions online on a daily basis. With the rapid advances in AI recently, AI-assisted decision making—in which an AI model provides decision recommendations and confidence, while the humans make the final decisions—has emerged as a new paradigm of human-AI collaboration. In this paper, we aim at obtaining a quantitative understanding of whether and when would human decision makers adopt the AI model’s recommendations. We define a space of human behavior models by decomposing the human decision maker’s cognitive process in each decision-making task into two components: the utility component (i.e., evaluate the utility of different actions) and the selection component (i.e., select an action to take), and we perform a systematic search in the model space to identify the model that fits real-world human behavior data the best. Our results highlight that in AI-assisted decision making, human decision makers’ utility evaluation and action selection are influenced by their own judgement and confidence on the decision-making task. Further, human decision makers exhibit a tendency to distort the decision confidence in utility evaluations. Finally, we also analyze the differences in humans’ adoption behavior of AI recommendations as the stakes of the decisions vary.
- OA Adeogun, AM Ajana, OA Ayinla, MT Yarhere, and MO Adeogun. 2008. Application of logit model in adoption decision: A study of hybrid clarias in Lagos State, Nigeria. American-Eurasian Journal of Agriculture and Environmental Sciences 4, 4(2008), 468–472.Google Scholar
- Amos Azaria, Ya’akov Gal, Sarit Kraus, and Claudia V Goldman. 2016. Strategic advice provision in repeated human-agent interactions. Autonomous Agents and Multi-Agent Systems 30, 1 (2016), 4–29.Google ScholarDigital Library
- Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, and Daniel S Weld. 2021. Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork. (2021).Google Scholar
- Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. 2019. Beyond accuracy: The role of mental models in human-AI team performance. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 7. 2–11.Google ScholarCross Ref
- Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S Weld, Walter S Lasecki, and Eric Horvitz. 2019. Updates in human-ai teams: Understanding and addressing the performance/compatibility tradeoff. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 2429–2437.Google ScholarDigital Library
- Jonathan Baron, Barbara A Mellers, Philip E Tetlock, Eric Stone, and Lyle H Ungar. 2014. Two reasons to make aggregated probability forecasts more extreme. Decision Analysis 11, 2 (2014), 133–145.Google ScholarDigital Library
- David V Budescu and Hsiu-Ting Yu. 2006. To Bayes or not to Bayes? A comparison of two classes of models of information aggregation. Decision analysis 3, 3 (2006), 145–162.Google Scholar
- Chun-Wei Chiang and Ming Yin. 2021. You’d Better Stop! Understanding Human Reliance on Machine Learning Models under Covariate Shift. In 13th ACM Web Science Conference 2021. 120–129.Google ScholarDigital Library
- Abir De, Paramita Koley, Niloy Ganguly, and Manuel Gomez-Rodriguez. 2020. Regression under human assistance. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 2611–2620.Google ScholarCross Ref
- Diansheng Dong, Chanjin Chung, and Harry M Kaiser. 2004. Modelling milk purchasing behaviour with a panel data double-hurdle model. Applied Economics 36, 8 (2004), 769–779.Google ScholarCross Ref
- Sebastian Ebert and Philipp Strack. 2015. Until the bitter end: on prospect theory in a dynamic context. American Economic Review 105, 4 (2015), 1618–33.Google ScholarCross Ref
- Drew Fudenberg, Jon Kleinberg, Annie Liang, and Sendhil Mullainathan. 2019. Measuring the completeness of theories. (2019).Google Scholar
- Ruijiang Gao, Maytal Saar-Tsechansky, Maria De-Arteaga, Ligong Han, Min Kyung Lee, and Matthew Lease. 2021. Human-AI Collaboration with Bandit Feedback. arXiv preprint arXiv:2105.10614(2021).Google Scholar
- Yaohui Guo and X Jessie Yang. 2020. Modeling and Predicting Trust Dynamics in Human–Robot Teaming: A Bayesian Inference Approach. International Journal of Social Robotics(2020), 1–11.Google Scholar
- Yaohui Guo, Chongjie Zhang, and X Jessie Yang. 2020. Modeling Trust Dynamics in Human-robot Teaming: A Bayesian Inference Approach. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1–7.Google Scholar
- Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531(2015).Google Scholar
- Wan-Lin Hu, Kumar Akash, Neera Jain, and Tahira Reid. 2016. Real-time sensing of trust in human-machine interactions. IFAC-PapersOnLine 49, 32 (2016), 48–53.Google ScholarCross Ref
- Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. 2016. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems 29 (2016).Google Scholar
- Isaac Lage, Andrew Slavin Ross, Been Kim, Samuel J Gershman, and Finale Doshi-Velez. 2018. Human-in-the-loop interpretability prior. Advances in neural information processing systems 31 (2018).Google Scholar
- Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency. 29–38.Google ScholarDigital Library
- Pamela K Lattimore, Joanna R Baker, and Ann D Witte. 1992. The influence of probability on risky choice: A parametric examination. Journal of economic behavior & organization 17, 3(1992), 377–400.Google ScholarCross Ref
- John Lee and Neville Moray. 1992. Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35, 10 (1992), 1243–1270.Google ScholarCross Ref
- Han Liu, Vivian Lai, and Chenhao Tan. 2021. Understanding the effect of out-of-distribution examples and interactive explanations on human-ai decision making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2(2021), 1–45.Google ScholarDigital Library
- Siqi Liu, J Benjamin Miller, and Alexandros Psomas. 2019. Risk Robust Mechanism Design for a Prospect Theoretic Buyer. In International Symposium on Algorithmic Game Theory. Springer, 95–108.Google ScholarDigital Library
- Yidu Lu. 2020. Detecting and overcoming trust miscalibration in real time using an eye-tracking based technique. Ph. D. Dissertation.Google Scholar
- Zhuoran Lu and Ming Yin. 2021. Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–16.Google ScholarDigital Library
- Daniel McFadden. 2001. Economic choices. American economic review 91, 3 (2001), 351–378.Google Scholar
- Robert Mislavsky and Celia Gaertig. 2020. Combining Probability Forecasts: 60% and 60% Is 60%, but Likely and Likely is Very Likely. Forthcoming at Management Science, Johns Hopkins Carey Business School Research Paper20-14(2020).Google Scholar
- Gali Noti, Effi Levi, Yoav Kolumbus, and Amit Daniely. 2016. Behavior-based machine-learning: A hybrid approach for predicting human decision making. arXiv preprint arXiv:1611.10228(2016).Google Scholar
- Joshua C Peterson, David D Bourgin, Mayank Agrawal, Daniel Reichman, and Thomas L Griffiths. 2021. Using large-scale experiments and machine learning to discover theories of human decision-making. Science 372, 6547 (2021), 1209–1214.Google Scholar
- Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–52.Google ScholarDigital Library
- Drazen Prelec. 1998. The probability weighting function. Econometrica (1998), 497–527.Google Scholar
- John Quiggin. 2012. Generalized expected utility theory: The rank-dependent model. Springer Science & Business Media.Google Scholar
- Maithra Raghu, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer, and Sendhil Mullainathan. 2019. The algorithmic automation problem: Prediction, triage, and human effort. arXiv preprint arXiv:1903.12220(2019).Google Scholar
- Amy Rechkemmer and Ming Yin. 2022. When Confidence Meets Accuracy: Exploring the Effects of Multiple Performance Indicators on Trust in Machine Learning Models. In Proceedings of the 2022 chi conference on human factors in computing systems.Google ScholarDigital Library
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ” Why should i trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135–1144.Google ScholarDigital Library
- Kenneth E Train. 2009. Discrete choice methods with simulation. Cambridge university press.Google Scholar
- Amos Tversky and Craig R Fox. 1995. Weighing risk and uncertainty.Psychological review 102, 2 (1995), 269.Google Scholar
- Amos Tversky and Daniel Kahneman. 1992. Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and uncertainty 5, 4 (1992), 297–323.Google ScholarCross Ref
- John Von Neumann and Oskar Morgenstern. 1947. Theory of games and economic behavior, 2nd rev. (1947).Google Scholar
- Xinru Wang and Ming Yin. 2021. Are Explanations Helpful? A Comparative Study of the Effects of Explanations in AI-Assisted Decision-Making. In 26th International Conference on Intelligent User Interfaces. 318–328.Google ScholarDigital Library
- Anqi Xu and Gregory Dudek. 2015. Optimo: Online probabilistic trust inference model for asymmetric human-robot collaborations. IEEE, 221–228.Google ScholarDigital Library
- Yash. 2020. Lending club 2007-2020q3 | Kaggle. https://www.kaggle.com/ethon0426/lending-club-20072020q1?select=Loan_status_2007-2020Q3.gzipGoogle Scholar
- Ming Yin and Yu-An Sun. 2015. Human behavior models for virtual agents in repeated decision making under uncertainty. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. 581–589.Google ScholarDigital Library
- Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–12.Google ScholarDigital Library
- Bianca Zadrozny and Charles Elkan. 2001. Obtaining calibrated probability estimates from decision trees and naive bayesian classifiers. In Icml, Vol. 1. Citeseer, 609–616.Google Scholar
- Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. 295–305.Google ScholarDigital Library
Index Terms
- Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making
Index terms have been assigned to the content through auto-classification.
Recommendations
Human-Centered Evaluation of Explanations in AI-Assisted Decision-Making
IUI '24 Companion: Companion Proceedings of the 29th International Conference on Intelligent User InterfacesAI explanations have been increasingly used to help people better utilize AI recommendations in AI-assisted decision making. While numerous technical transparency approaches have been established, a human-centered perspective is needed for understanding ...
Modeling human trust and reliance in AI-assisted decision making: a markovian approach
AAAI'23/IAAI'23/EAAI'23: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial IntelligenceThe increased integration of artificial intelligence (AI) technologies in human workflows has resulted in a new paradigm of AI-assisted decision making, in which an AI model provides decision recommendations while humans make the final decisions. To best ...
Human Behavior Models for Virtual Agents in Repeated Decision Making under Uncertainty
AAMAS '15: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent SystemsTo design virtual agents that simulate humans in repeated decision making under uncertainty, we seek to quantitatively characterize the actual human behavior in these settings. We collect our data from 800 real human subjects through a large-scale ...
Comments