Cross-subject transfer learning in human activity recognition systems using generative adversarial networks
Introduction
Remarkable enrichment of sensor technology and consequently smart environments alongside with huge progress in machine learning techniques have pervasively brought intelligent solutions into every aspect of human life. Recognition of what the human subject is doing, widely considered to be one of the most important tasks of an intelligent system known as an active field of research named Human Activity Recognition (HAR). The Activity Recognition problem could include various sub-categories of tasks such as sensing, detection, and classification. In this research, the HAR approaches are mainly considered to be addressing the classification task.
Previous studies on HAR can be generally categorized based on sensor modalities and data utilized for recognition of activity details, including vision and sensors based approaches. Vision-based sensors are exploited to capture images, videos or surveillance camera features to recognize activity [1]. Despite the successful performance of vision-based solutions, non-visual sensors are still required to address their existing limitations such as laborious processing and privacy problems. Non-visual sensors can be installed on the human’s body (wearable sensors) or in the environment (ambient sensors). Utilizing a network of heterogeneous sensors has become widespread interest as well.
Let us consider collecting data from sensors measuring k attributes, while the human user is performing activities. Given:while W is a set of m time windows such that each contains a set of time series values from each of the k measured attributes, and A is a set of activity labels, the task of Human Activity Recognition can be defined as finding a mapping function such that is as similar as possible to the actual activity performed during [2].
Diverse supervised and semi-supervised machine learning models have been proposed for activity recognition. These models deliver promising accuracy conditioning on training with enough labeled data. However, the pitfall is that their performance will dramatically fall against data, from new unseen distributions. The difference may root in feature space or label space distribution. Therefore, recognizing the activities of a new user remains challenging for the model which was trained by samples of other users’ behavior. Nevertheless, collecting and labeling sufficient training data is not feasible for every new user since it requires a relatively long time observation of human subjects’ behavior which is time-consuming and sometimes impractical [3], [4].
Transfer Learning techniques aim to prevent the aforementioned performance leak by adapting obtained knowledge from the source domain (training users) to the target domain (new users). These approaches should address the primary questions of What, How, and From where to transfer. Thought, it should be noted that when the prior experiences (from other sources of knowledge) are not relevant enough to the new domain, a brutal-force transfer may degrade the learning performance, resulting in the negative knowledge transfer phenomena [5]. Transfer learning is researched under a variety of different names such as life-long learning, knowledge transfer, learning to learn, inductive transfer, context-sensitive learning, and meta-learning in the field of machine learning [6].
The remainder of this paper is organized as follows in five sections. Section 2 examines the previous research works have been devoted to the study of HAR and Transfer Learning. Section 3 describes SA-GAN and its related training details. Evaluation, experimental results and their analysis are presented in Section 4. Finally, Section 5 summarizes the results of this work, draws conclusions, and highlights issues for future researches.
Section snippets
Human activity recognition
Traditional machine learning approaches including K-Nearest Neighbor (KNN), Hidden Markov Model (HMM), Support Vector Machine (SVM), Random Forest (RF), and Naïve Bayes (NB) have shown satisfactory results on recognizing human activities [7], [8]. A major criticism of these models is that they mainly rely on handcrafted feature extraction or heuristic information. Besides the demand for the domain specialist, extracted features are not abstracted enough. Therefore, models are not suitable for
Proposed Model: SA-GAN
Following our semi-supervised knowledge transfer setting, we have labeled data of the source domain and unlabeled data of target domain . In our Cross-Subject Transfer Learning problem the difference between the domains roots in the distribution of feature space and the conditional distribution of label space. It can be interpreted as the scenario when there is a model trained with limited samples from distribution , and it is required to test the model against samples drawn
Experiments
Our experiments are broken down into two groups on the basis of their objectives. The first set of analysis was carried out in order to justify the necessity of applying knowledge transfer technique by investigating the performance drop in case of domain shifts. Another group of experiments was conducted with the aim of measuring the improvement achieved by SA-GAN model. Fig. 4, depicts a simple overview of the required steps to have the prediction of target domain , using SA-GAN.
Conclusions and Future Work
One of the most important limitations of HAR models lies in lacking a sufficient amount of labeled data. Furthermore, the discovered patterns through available labeled data might not be well generalizable to the samples from unseen subjects. However, data acquisition and labeling are not feasible for newcomers due to limitations of interaction with human users. This paper has highlighted an innovative cutting-edge solution for cross-subject knowledge transfer in the domain of Human Activity
CRediT authorship contribution statement
Elnaz Soleimani: Writing - original draft, Investigation. Ehsan Nazerfard: Supervision, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Ehsan Nazerfard received the BS and MS degrees in Computer Engineering and Artificial Intelligence from Amirkabir University of Technology and Sharif University of Technology, Tehran, Iran, in 2003 and 2006, respectively. He also received the PhD degree in Computer Science from Washington State University, WA USA, in 2014. He is currently an Assistant Professor in the department of Computer Engineering at Amirkabir University of Technology. His research interests include Machine Learning, Deep
References (45)
- et al.
Kernel fusion based extreme learning machine for cross-location activity recognition
Inform. Fusion
(2017) - et al.
Human activity recognition with smartphone sensors using deep learning neural networks
Expert Syst. Appl.
(2016) - et al.
Cross-domain activity recognition via transfer learning
Pervasive Mobile Comput.
(2011) - et al.
Stratified transfer learning for cross-domain activity recognition
- et al.
The opportunity challenge: A benchmark database for on-body sensor-based activity recognition
Pattern Recogn. Lett.
(2013) - H.F. Nweke, Y.W. Teh, M.A. Al-Garadi, U.R. Alo, Deep learning algorithms for human activity recognition using mobile...
- et al.
A survey on human activity recognition using wearable sensors
IEEE Commun. Surveys Tutorials
(2012) - et al.
A survey on transfer learning
IEEE Trans. Knowl. Data Engineering
(2010) - et al.
Active prior tactile knowledge transfer for learning tactual properties of new objects
Sensors
(2018) - et al.
Transfer learning for activity recognition: A survey
Knowl. Inform. Syst.
(2013)
Machine learning methods for classifying human physical activity from on-body accelerometers
Sensors
By train or by car? detecting the user’s motion type through smartphone sensors data
A-wristocracy, Deep learning on wrist-worn sensing for recognition of user complex activities
Human activity recognition from sensor-based large-scale continuous monitoring of parkinson’s disease patients
Human activity recognition using wearable sensors by deep convolutional neural networks, in
Unsupervised feature learning for human activity recognition using smartphone sensors
Cited by (0)
Ehsan Nazerfard received the BS and MS degrees in Computer Engineering and Artificial Intelligence from Amirkabir University of Technology and Sharif University of Technology, Tehran, Iran, in 2003 and 2006, respectively. He also received the PhD degree in Computer Science from Washington State University, WA USA, in 2014. He is currently an Assistant Professor in the department of Computer Engineering at Amirkabir University of Technology. His research interests include Machine Learning, Deep Learning and Internet of Things.
Elnaz Soleimani received the BS and MS degrees in Computer Engineering and Artificial Intelligence from Amirkabir University of Technology, Tehran, Iran, in 2016 and 2018, respectively. She also received master degree in AI from University of Paris Est Creteil, Paris, France, in 2019. Her main research interests include Machine Learning, Transfer Learning and Computer Vision.