Ziyi Ye
Yiqun Liu
Min Zhang
ABSTRACT
With the growth of information on the Web, most users heavily rely on information access systems (e.g., search engines, recommender systems, etc.) in their daily lives. During this procedure, modeling users' satisfaction status plays an essential part in improving their experiences with the systems. In this paper, we aim to explore the benefits of using Electroencephalography (EEG) signals for satisfaction modeling in interactive information access system design. Different from existing EEG classification tasks, the arisen of satisfaction involves multiple brain functions, such as arousal, prototypicality, and appraisals, which are related to different brain topographical areas. Thus modeling user satisfaction raises great challenges to existing solutions. To address this challenge, we propose BTA, a Brain Topography Adaptive network with a multi-centrality encoding module and a spatial attention mechanism module to capture cognitive connectives in different spatial distances. We explore the effectiveness of BTA for satisfaction modeling in two popular information access scenarios, i.e., search and recommendation. Extensive experiments on two real-world datasets verify the effectiveness of introducing brain topography adaptive strategy in satisfaction modeling. Furthermore, we also conduct search result re-ranking task and video rating prediction task based on the satisfaction inferred from brain signals on search and recommendation scenarios, respectively. Experimental results show that brain signals extracted with BTA help improve the performance of interactive information access systems significantly.
Supplemental Material
- Mashael Aldayel, Mourad Ykhlef, and Abeer Al-Nafjan. 2020. Deep learning for EEG-based preference classification in neuromarketing. Applied Sciences, Vol. 10, 4 (2020), 1525.Google Scholar
Cross Ref
- Rashid Ali and MM Sufyan Beg. 2011. An overview of Web search evaluation methods. Computers & Electrical Engineering, Vol. 37, 6 (2011), 835--848.Google ScholarDigital Library
- Xavier Amatriain, Josep M Pujol, and Nuria Oliver. 2009. I like it... i like it not: Evaluating user ratings noise in recommender systems. In International Conference on User Modeling, Adaptation, and Personalization. Springer, 247--258.Google ScholarDigital Library
- Thomas Baumgartner, Lilian Valko, Michaela Esslen, and Lutz J"ancke. 2006. Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an EEG and psychophysiology study. CyberPsychology & Behavior, Vol. 9, 1 (2006), 30--45.Google ScholarCross Ref
- Daniel E Berlyne. 1970. Novelty, complexity, and hedonic value. Perception & psychophysics, Vol. 8, 5 (1970), 279--286.Google Scholar
- Anjan Chatterjee. 2003. Prospects for a cognitive neuroscience of visual aesthetics. (2003).Google Scholar
- Xuesong Chen, Ziyi Ye, Xiaohui Xie, Yiqun Liu, Xiaorong Gao, Weihang Su, Shuqi Zhu, Yike Sun, Min Zhang, and Shaoping Ma. 2022. Web Search via an Efficient and Effective Brain-Machine Interface. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 1569--1572.Google ScholarDigital Library
- Ye Chen, Yiqun Liu, Min Zhang, and Shaoping Ma. 2017. User satisfaction prediction with mouse movement information in heterogeneous search environment. IEEE Transactions on Knowledge and Data Engineering, Vol. 29, 11 (2017), 2470--2483.Google ScholarDigital Library
- Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7--10.Google ScholarDigital Library
- Keith M Davis III, Michiel Spapé, and Tuukka Ruotsalo. 2021. Collaborative filtering with preferences inferred from brain signals. In Proceedings of the Web Conference 2021. 602--611.Google Scholar
- Ruo-Nan Duan, Jia-Yi Zhu, and Bao-Liang Lu. 2013. Differential entropy feature for EEG-based emotion classification. In 6th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, 81--84.Google ScholarCross Ref
- Steve Fox, Kuldeep Karnawat, Mark Mydland, Susan Dumais, and Thomas White. 2005. Evaluating implicit measures to improve web search. ACM Transactions on Information Systems (TOIS), Vol. 23, 2 (2005), 147--168.Google ScholarDigital Library
- Martin Gjoreski, Blagoj Mitrevski, Mitja Luvs trek, and Matjavz Gams. 2018. An inter-domain study for arousal recognition from physiological signals. Informatica, Vol. 42, 1 (2018).Google Scholar
- Shan Guan, Kai Zhao, and Shuning Yang. 2019. Motor imagery EEG classification based on decision tree framework and Riemannian geometry. Computational intelligence and neuroscience , Vol. 2019 (2019).Google Scholar
- Jacek Gwizdka, Rahilsadat Hosseini, Michael Cole, and Shouyi Wang. 2017. Temporal dynamics of eye-tracking and EEG during reading and relevance decisions. Journal of the Association for Information Science and Technology, Vol. 68, 10 (2017), 2299--2312.Google ScholarDigital Library
- Ahmed Hassan, Ryen W White, Susan T Dumais, and Yi-Min Wang. 2014. Struggling or exploring? Disambiguating long search sessions. In Proceedings of the 7th ACM international conference on Web search and data mining. 53--62.Google ScholarDigital Library
- Tiffany C Ho, Colm G Connolly, Eva Henje Blom, Kaja Z LeWinn, Irina A Strigo, Martin P Paulus, Guido Frank, Jeffrey E Max, Jing Wu, Melanie Chan, et al. 2015. Emotion-dependent functional connectivity of the default mode network in adolescent depression. Biological psychiatry, Vol. 78, 9 (2015), 635--646.Google Scholar
- Jingzhao Hu, Chen Wang, Qiaomei Jia, Qirong Bu, Richard Sutcliffe, and Jun Feng. 2021. ScalingNet: extracting features from raw EEG data for emotion recognition. Neurocomputing , Vol. 463 (2021), 177--184.Google ScholarDigital Library
- Kalervo J"arvelin and Jaana Kek"al"ainen. 2017. IR evaluation methods for retrieving highly relevant documents. In ACM SIGIR Forum, Vol. 51. ACM New York, NY, USA, 243--250.Google Scholar
- Ziyu Jia, Youfang Lin, Jing Wang, Zhiyang Feng, Xiangheng Xie, and Caijie Chen. 2021. HetEmotionNet: two-stream heterogeneous graph recurrent neural network for multi-modal emotion recognition. In Proceedings of the 29th ACM International Conference on Multimedia. 1047--1056.Google ScholarDigital Library
- Ziyu Jia, Youfang Lin, Jing Wang, Ronghao Zhou, Xiaojun Ning, Yuanlai He, and Yaoshuai Zhao. 2020. GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification.. In IJCAI. 1324--1330.Google Scholar
- Diane Kelly. 2009. Methods for evaluating interactive information retrieval systems with users. Now Publishers Inc.Google ScholarDigital Library
- Hyun Hee Kim and Yong Ho Kim. 2019. ERP/MMR algorithm for classifying topic-relevant and topic-irrelevant visual shots of documentary videos. Journal of the Association for Information Science and Technology, Vol. 70, 9 (2019), 931--941.Google ScholarDigital Library
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Demetres Kostas, Stephane Aroca-Ouellette, and Frank Rudzicz. 2021. BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data. Frontiers in Human Neuroscience , Vol. 15 (2021).Google ScholarCross Ref
- Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. 2004. Estimating mutual information. Physical review E, Vol. 69, 6 (2004), 066138.Google Scholar
- Victor Lavrenko and W Bruce Croft. 2017. Relevance-based language models. In ACM SIGIR Forum, Vol. 51. ACM New York, NY, USA, 260--267.Google ScholarDigital Library
- Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance. 2018. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. Journal of neural engineering , Vol. 15, 5 (2018), 056013.Google ScholarCross Ref
- Rui Li, Yiting Wang, and Bao-Liang Lu. 2021. A Multi-Domain Adaptive Graph Convolutional Network for EEG-based Emotion Recognition. In Proceedings of the 29th ACM International Conference on Multimedia. 5565--5573.Google ScholarDigital Library
- Xiang Li, Dawei Song, Peng Zhang, Guangliang Yu, Yuexian Hou, and Bin Hu. 2016. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. In 2016 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE, 352--359.Google ScholarCross Ref
- Mengyang Liu, Yiqun Liu, Jiaxin Mao, Cheng Luo, Min Zhang, and Shaoping Ma. 2018. " Satisfaction with Failure" or" Unsatisfied Success" Investigating the Relationship between Search Success and User Satisfaction. In Proceedings of the 2018 world wide web conference. 1533--1542.Google Scholar
- Yiqun Liu, Chao Wang, Ke Zhou, Jianyun Nie, Min Zhang, and Shaoping Ma. 2014. From skimming to reading: A two-stage examination model for web search. In Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management. 849--858.Google ScholarDigital Library
- Colin Martindale and Kathleen Moore. 1988. Priming, prototypicality, and preference. Journal of Experimental Psychology: Human Perception and Performance, Vol. 14, 4 (1988), 661.Google ScholarCross Ref
- Juan Abdon Miranda-Correa, Mojtaba Khomami Abadi, Nicu Sebe, and Ioannis Patras. 2018. Amigos: A dataset for affect, personality and mood research on individuals and groups. IEEE Transactions on Affective Computing , Vol. 12, 2 (2018), 479--493.Google ScholarDigital Library
- Yashar Moshfeghi, Peter Triantafillou, and Frank Pollick. 2019. Towards predicting a realisation of an information need based on brain signals. In The World Wide Web Conference. 1300--1309.Google ScholarDigital Library
- Yashar Moshfeghi, Peter Triantafillou, and Frank E Pollick. 2016. Understanding information need: An fMRI study. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 335--344.Google ScholarDigital Library
- Marcos Nadal, Enric Munar, Miquel Angel Capo, Jaume Rossello, and Camilo Jose Cela-Conde. 2008. Towards a framework for the study of the neural correlates of aesthetic preference. Spatial vision, Vol. 21, 3 (2008), 379.Google Scholar
- Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research , Vol. 12 (2011), 2825--2830.Google Scholar
- David MW Powers. 2020. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv preprint arXiv:2010.16061 (2020).Google Scholar
- B Pyakillya, N Kazachenko, and N Mikhailovsky. 2017. Deep learning for ECG classification. In Journal of physics: conference series, Vol. 913. IOP Publishing, 012004.Google Scholar
- Steffen Rendle. 2010. Factorization machines. In 2010 IEEE International conference on data mining. IEEE, 995--1000.Google ScholarDigital Library
- Matthew Richardson, Ewa Dominowska, and Robert Ragno. 2007. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web. 521--530.Google ScholarDigital Library
- Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Now Publishers Inc.Google Scholar
- S Rasoul Safavian and David Landgrebe. 1991. A survey of decision tree classifier methodology. IEEE transactions on systems, man, and cybernetics, Vol. 21, 3 (1991), 660--674.Google Scholar
- Louis A Schmidt and Laurel J Trainor. 2001. Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cognition & Emotion, Vol. 15, 4 (2001), 487--500.Google ScholarCross Ref
- Paul J Silvia. 2005. Cognitive appraisals and interest in visual art: Exploring an appraisal theory of aesthetic emotions. Empirical studies of the arts , Vol. 23, 2 (2005), 119--133.Google Scholar
- Tengfei Song, Wenming Zheng, Peng Song, and Zhen Cui. 2018. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Transactions on Affective Computing , Vol. 11, 3 (2018), 532--541.Google ScholarCross Ref
- Akara Supratak and Yike Guo. 2020. TinySleepNet: An efficient deep learning model for sleep stage scoring based on raw single-channel EEG. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 641--644.Google ScholarCross Ref
- Johan AK Suykens and Joos Vandewalle. 1999. Least squares support vector machine classifiers. Neural processing letters , Vol. 9, 3 (1999), 293--300.Google ScholarDigital Library
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems , Vol. 30 (2017).Google Scholar
- Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021a. Denoising implicit feedback for recommendation. In Proceedings of the 14th ACM international conference on web search and data mining. 373--381.Google ScholarDigital Library
- Wenjie Wang, Fuli Feng, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2021b. Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1288--1297.Google ScholarDigital Library
- Xiao-Wei Wang, Dan Nie, and Bao-Liang Lu. 2014. Emotional state classification from EEG data using machine learning approach. Neurocomputing , Vol. 129 (2014), 94--106.Google ScholarDigital Library
- Yingying Wu, Yiqun Liu, Yen-Hsi Richard Tsai, and Shing-Tung Yau. 2019. Investigating the role of eye movements and physiological signals in search satisfaction prediction using geometric analysis. Journal of the Association for Information Science and Technology, Vol. 70, 9 (2019), 981--999.Google ScholarDigital Library
- Dezhong Yao, Yun Qin, Shiang Hu, Li Dong, Maria L Bringas Vega, and Pedro A Valdés Sosa. 2019. Which reference should we use for EEG and ERP practice? Brain topography, Vol. 32, 4 (2019), 530--549.Google Scholar
- Ziyi Ye, Xiaohui Xie, Yiqun Liu, Zhihong Wang, Xuancheng Li, Jiaji Li, Xuesong Chen, Min Zhang, and Shaoping Ma. 2022. Why Don't You Click: Understanding Non-Click Results in Web Search with Brain Signals. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval.Google ScholarDigital Library
- Zhongliang Yin, Jun Li, Yun Zhang, Aifeng Ren, Karen M Von Meneen, and Liyu Huang. 2017. Functional brain network analysis of schizophrenic patients with positive and negative syndrome based on mutual information of EEG time series. Biomedical Signal Processing and Control , Vol. 31 (2017), 331--338.Google ScholarCross Ref
- George Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. 2021. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 2114--2124.Google ScholarDigital Library
- Guanhua Zhang, Minjing Yu, Yong-Jin Liu, Guozhen Zhao, Dan Zhang, and Wenming Zheng. 2021. SparseDGCNN: recognizing emotion from multichannel EEG signals. IEEE Transactions on Affective Computing (2021).Google Scholar
- Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, et al. 2021. Recbole: Towards a unified, comprehensive and efficient framework for recommendation algorithms. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 4653--4664.Google ScholarDigital Library
- Peixiang Zhong, Di Wang, and Chunyan Miao. 2020. EEG-based emotion recognition using regularized graph neural networks. IEEE Transactions on Affective Computing (2020).Google ScholarCross Ref
- Mu Zhu. 2004. Recall, precision and average precision. Department of Statistics and Actuarial Science, University of Waterloo, Waterloo, Vol. 2, 30 (2004), 6.ioGoogle Scholar
Index Terms
- Brain Topography Adaptive Network for Satisfaction Modeling in Interactive Information Access System
Recommendations
Using a trust network to improve top-N recommendation
RecSys '09: Proceedings of the third ACM conference on Recommender systemsTop-N item recommendation is one of the important tasks of recommenders. Collaborative filtering is the most popular approach to building recommender systems which can predict ratings for a given user and item. Collaborative filtering can be extended ...
Information Recommendation Method Research Based on Trust Network and Collaborative Filtering
ICEBE '11: Proceedings of the 2011 IEEE 8th International Conference on e-Business EngineeringInformation recommender system is considered to be one of the most effective tools to solve the problem of information overload. Collaborative Filtering (CF), which utilizes similar neighbors to generate recommendations, is believed to be the most ...
Understanding post-adoption behaviors of e-service users in the context of online travel services
This study examines the motivators of continuance intention and WOM of e-service users.This study also examines the relationship between continuance intention and WOM.Satisfaction and perceived usefulness determine continuance intention.Continuance ...
Comments