Transferring activity recognition models in FOG computing architecture

https://doi.org/10.1016/j.jpdc.2018.07.020Get rights and content

Highlights

  • Transfer learning for in-home activity recognition on Cloud-Fog Architecture.

  • Activity Recognition for imperfect datasets.

  • Building a meta-data description for abstracting smart home’s activities at the fog level.

  • Fog-to-Fog Data Models Transfer for Predicting Human Behavior.

Abstract

A major focus of research in the field of in-home activity recognition (AR) and home automation (HA) is the ability to transfer data models to other homes for the purpose of applying new services, annotating classified data, and generating datasets due to lack of training ones. The wide spread of fog computing as an architecture for organizing edge devices in Internet-of-Things (IoT) systems lends support to the sharing of different environmental characteristics between different fogs (smart homes). In this paper, we propose a framework that serves the transfer of data models between different smart homes in a bid to overcome the lack of training data, which prevents the development of high-performance models that utilize fog computing characteristics. Our technique incorporates the sharing of environmental characteristics (by Fogs) in order to analyze the data features at the source and target smart homes. The features, then, are mapped onto each other using a fusion method that guarantees to keep the variations between different homes by reducing the divergence between them. The hidden Markov model has also been applied in order to model activities at target homes. Three experiments have been conducted to measure the performance of the proposed framework: first, against the accuracy of feature-mapping techniques; second, measuring the performance of classifying data at target homes; and, third, the ability of the proposed framework to function well due to noise data. The results show promising indicators and highlight the limitations of the proposed methodology.

Introduction

Over the last decade, cloud computing has encountered scalability challenges in terms of meeting the new and complicated requirements of the emerging Internet-of-Things (IoT) paradigms. The increasing number of smart devices, powerful end-users, services, and the centralized nature of cloud computing architecture introduce several challenges such as timely services, bandwidth constraints, connectivity, and security issues. In fact, the demand is for several IoT applications to be functioning and plugged in at different and new places such as smart homes, hospitals, and schools.

Smart home technology provides a wide range of services, especially for the elderly and people with disabilities. It provides quality-of-life services through what is known as activity recognition (AR). Recognizing an inhabitant’s activities allows for developing smart services by predicting residents’ behaviors. This is done by modeling activities using a training set of real data from inside homes. A challenge met in this domain is the ability of such systems to transfer models to other homes. For instance, traditional deployment of a smart home requires data acquisition, processing, and the development of a recognition model for each activity. In practical terms, deploying new homes in a smart environment is an expensive and frustrating process.

Recent advances in cloud computing proposed the Fog architecture, in which data sharing is performed on two levels: local (Fog) and global (Cloud). Cloud-Fog architecture has proven its efficiency in terms of energy consumption, reliability, and service delivery [18], [29]. Such distribution of computing power facilitates transfer learning: a methodology for mapping source knowledge and target environment to a common space. Fig. 1 shows the general architecture of cloud-fog computing.

Transfer learning is the process of learning a mapping (model) between source domains and a target environment, where only a small or NO annotated training datasets currently exists. Annotating home activities is an essential task for modeling their behaviors [1]. In the literature, transfer learning has been applied to induce model parameters of target environments. This research aims to transfer knowledge from well-trained models into an environment that has a lack of trained ones.

In this paper, we are investigating the problem of transferring an activity recognition model from a source smart home into a target one for the purpose of developing data service to the target environment [26]. Our hypothesis states that if we have metadata describing the target domain, we can adjust source models in order to come up with a customized model for the target domain. Therefore, the contribution of this work is to propose a methodology that supports our hypothesis.

Our proposed methodology relies on analyzing the features of the source domain for coordinating them into a standard form. The next step involves proposing a solution to resolve variations in data distribution since source homes have different distributions of sensors according to their design. Such distribution results in a divergence of data that should be resolved before correlating selected features in the feature mapping phase. Thus, the selected features are used to develop a classification model according to the metadata provided from the target environment. The novelty of the proposed research methodology is its ability to benefit from the environment metadata (fog architecture) to transfer a source model into a target environment.

This paper is organized as follows: Section 2 provides a literature review to compare the presented work with related ones. Section 3 explains the system architecture and the relationship between its components. Section 4 details the research methodology with formal descriptions. Section 5 explains the experiments that have been conducted to measure the performance of the proposed technique. The paper is concluded in Section 6.

Section snippets

Related work

Activity recognition, the process of predicting the activity of the home’s residents through the sequence of actions they perform, has attracted many researchers interested in developing methods and techniques to enhance the accuracy of the process and to develop generalized models that are applicable for different environments and settings. Building the models have been based on many approaches, including naïve Bayes classifier, support vector machine [10], hidden Markov model and conditional

System architecture

In this research, our assumption is centered on organizing smart homes under a city — Fog groups in which each Fog is responsible for maintaining the characteristics of each smart home at a given city. Such characteristics include a meta description of the deployed IoT devices and the setting of the wireless networks.

Furthermore, the Fog controller maintains the distribution of actions (generated by sensors) of the original activities. Such information is classified as a meta description of the

Model transfer framework

This section illustrates the research methodology for inducing models at target smart homes using models generated at source homes. Our purpose is to maximize the accuracy of generated datasets. In addition, the generated datasets should represent the target environment in a way in which NO contradiction occurs with its original metadata. The overall methodology components are depicted in Fig. 3.

The feature analysis component is responsible for spanning the source datasets in order to handle

Experiments and results

We conducted three experiments to test our proposed methodology. The first one concentrated on measuring the accuracy of the feature-mapping technique. The second experiment tested the application of the features on classifying in-home data as a goal for transforming them. Finally, a sensitivity analysis test was applied to measure how our methodology performed due to the existence of noise data.

Conclusion

This paper introduced a transfer learning methodology based on Fog computing architecture, in which the methodology has been developed according to the Fog-Cloud communicationparadigm. The contribution of this work is to provide a methodology that is capable of transferring data models between different smart homes (fogs), while keeping the performance acceptable. The work addresses the activity recognition services with target environments lacking in training datasets. The proposed methodology

Acknowledgments

The authors are grateful to the Deanship of Scientific Research at King Saud University, Riyadh, Saudi Arabia for funding this work through the Vice Deanship of Scientific Research Chairs.

Samer Samarah is an associate professor in the Computer Information Systems Department at Yarmouk University, Jordan. He obtained his Ph.D. in Computer Science from University of Ottawa, Canada in 2008. He has many published journals and conferences in the area of data mining and wireless networks. His research focuses on discovering behavioral patterns from data collected by Wireless Sensor Networks and Vehicular ah-hoc Networks. He is a referee for many international journals and conferences

References (34)

  • HassanM.M. et al.

    Human activity recognition from body sensor data using deep learning

    J. Med. Syst.

    (2018)
  • HossainM.S. et al.

    Audio-visual emotion recognition using multi-directional regression and ridgelet transform

    J. Multimodal User Interfaces

    (2016)
  • HossainM.S. et al.

    Healthcare big data voice pathology assessment framework

    IEEE Access

    (2016)
  • HossainM.S. et al.

    Smart healthcare monitoring: a voice pathology detection paradigm for smart cities

    Multimedia Systems

    (2017)
  • HossainM.S.

    Big data-driven service composition using parallel clustered particle swarm optimization in mobile environment

    IEEE Trans. Trans. Serv. Comput.

    (2016)
  • HossainM.S.

    Toward end-to-end biometrics-based security for IoT infrastructure

    IEEE Wirel. Commun.

    (2016)
  • D. Hu, Q. Yang, Transfer learning for activity recognition via sensor mapping, in: Proceedings of the Twenty-Second...
  • Cited by (17)

    View all citing articles on Scopus

    Samer Samarah is an associate professor in the Computer Information Systems Department at Yarmouk University, Jordan. He obtained his Ph.D. in Computer Science from University of Ottawa, Canada in 2008. He has many published journals and conferences in the area of data mining and wireless networks. His research focuses on discovering behavioral patterns from data collected by Wireless Sensor Networks and Vehicular ah-hoc Networks. He is a referee for many international journals and conferences and served as external examiner for many theses.

    Mohammed AL Zamil is an Associate Professor in the department of computer information systems at Yarmouk University (YU) in Jordan. He obtained his Ph.D degree in Information Systems from Middle East Technical University, Ankara, Turkey (2010). His master degree is in Computer Science from YU. He had B.Sc degree in computer science from YU. His research interests include Data Mining, Wireless Sensor Networks, Model Checking, Software verification, and Software Engineering.

    Majdi Rawashdeh received his Ph.D. degree in Computer Science from the University of Ottawa, Canada. He is currently an Assistant Professor at Princess Sumaya University for Technology (PSUT), Jordan. His research interests include social media, recommender systems, data mining, and big data. He has served as a member of the organizing and technical committees of several international conferences and workshops.

    M. Shamim Hossain is a Professor at the Department of Software Engineering, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia. He is also an Adjunct Professor, School of Electrical Engineering and Computer Science (EECS), University of Ottawa, Canada. He received his Ph.D. in Electrical and Computer Engineering from the University of Ottawa, Canada. His research interests include Cloud networking, smart environment (smart city, smart health), social media, IoT, edge computing and multimedia for healthcare, deep learning approach for multimedia processing, and multimedia big data. He has authored and coauthored approximately 200 publications including refereed IEEE/ACM/Springer/Elsevier journals, conference papers, books, and book chapters. He has served as a member of the organizing and technical committees of several international conferences and workshops. He has served as co-chair, general chair, workshop chair, publication chair, and TPC for over 12 IEEE and ACM conferences and workshops. Currently, he is the co-chair of the 1st IEEE ICME workshop on Multimedia Services and Tools for smart-health (MUST-SH 2018). He is a recipient of a number of awards, including the Best Conference Paper Award, the 2016 ACM Transactions on Multimedia Computing, Communications and Applications (TOMM) Nicolas D. Georganas Best Paper Award, and the Research in Excellence Award from College of Computer and Information Sciences (CCIS) King Saud University (3 times). He is on the editorial board of IEEE Network, IEEE Multimedia, IEEE Access, Journal of Network and Computer Applications (Elsevier), Computers and Electrical Engineering (Elsevier), Human-centric Computing and Information Sciences (Springer), Games for Health Journal, and International Journal of Multimedia Tools and Applications (Springer). Currently, he serves as a lead guest editor of IEEE Communication Magazine, Future Generation Computer Systems (Elsevier), and IEEE Access. Previously, he served as a guest editor of IEEE Transactions on Information Technology in Biomedicine (currently JBHI), IEEE Transactions on Cloud Computing, International Journal of Multimedia Tools and Applications (Springer), Cluster Computing (Springer), Future Generation Computer Systems (Elsevier), Computers and Electrical Engineering (Elsevier), Sensors (MDPI), and International Journal of Distributed Sensor Networks. He is a Senior Member of IEEE, a Senior member of ACM and ACM SIGMM.

    Ghulam Muhammad is a Professor in the Department of Computer Engineering, College of Computer and Information Sciences at King Saud University, Riyadh, Saudi Arabia. He received his Ph.D. in Electrical and Computer Engineering from Toyohashi University and Technology, Japan in 2006. His research interests include serious games, cloud and multimedia for healthcare, resource provisioning for big data processing on media clouds and biologically inspired approach for multimedia and software system, image and speech processing. He has authored and co-authored many publications including refereed IEEE/ACM/Springer/Elsevier journals, conference papers, books, and book chapters. He has supervised a number of Ph.D. and Masters Theses. He owns a US patent.

    Atif Alamri is an Associate Professor in the Information Systems Department at the College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia. His research interests include multimedia-assisted health systems, ambient intelligence, and service-oriented architecture. He was Guest Editor of the IEEE Transactions on Instrumentation and Measurement, a co-chair of the first IEEE International Workshop on Multimedia Services and Technologies for E-health, a technical program co-chair for the 10th IEEE International Symposium on Haptic Audio Visual Environments and Games, and serves as a program committee member for many conferences in multimedia, virtual environments, and medical applications.

    View full text