Zhaolong Ning
Joel J. P. C. Rodrigues
Skip Abstract Section
Abstract
The development of smart vehicles brings drivers and passengers a comfortable and safe environment. Various emerging applications are promising to enrich users’ traveling experiences and daily life. However, how to execute computing-intensive applications on resource-constrained vehicles still faces huge challenges. In this article, we construct an intelligent offloading system for vehicular edge computing by leveraging deep reinforcement learning. First, both the communication and computation states are modelled by finite Markov chains. Moreover, the task scheduling and resource allocation strategy is formulated as a joint optimization problem to maximize users’ Quality of Experience (QoE). Due to its complexity, the original problem is further divided into two sub-optimization problems. A two-sided matching scheme and a deep reinforcement learning approach are developed to schedule offloading requests and allocate network resources, respectively. Performance evaluations illustrate the effectiveness and superiority of our constructed system.
- Zhiguang Cao, Hongliang Guo, and Jie Zhang. 2018. A multiagent-based approach for vehicle routing by considering both arriving on time and total travel time. ACM Trans. Intell. Syst. Technol. 9, 3 (2018), 25.Google Scholar
Digital Library
- Xu Chen, Lei Jiao, Wenzhong Li, and Xiaoming Fu. 2016. Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans. Netw. 5 (2016), 2795--2808.Google ScholarDigital Library
- Ruilong Deng, Rongxing Lu, Chengzhe Lai, and Tom H. Luan. 2015. Towards power consumption-delay tradeoff by workload allocation in cloud-fog computing. In Proceedings of the IEEE International Conference on Communications (ICC’15). IEEE, 3909--3914.Google Scholar
- Ruilong Deng, Rongxing Lu, Chengzhe Lai, Tom H. Luan, and Hao Liang. 2016. Optimal workload allocation in fog-cloud computing toward balanced delay and power consumption. IEEE Int. Things J. 3, 6 (2016), 1171--1181.Google Scholar
- Ruilong Deng, Zaiyue Yang, Jiming Chen, Navid Rahbari Asr, and Mo-Yuen Chow. 2014. Residential energy consumption scheduling: A coupled-constraint game approach. IEEE Trans. Smart Grid 5, 3 (2014), 1340--1350.Google ScholarCross Ref
- Boya Di, Lingyang Song, Yonghui Li, and Geoffrey Ye Li. 2017. Non-orthogonal multiple access for high-reliable and low-latency V2X communications in 5G systems. IEEE J. Select. Areas Commun. 35, 10 (2017), 2383--2397.Google ScholarDigital Library
- Lin Gu, Deze Zeng, Song Guo, Ahmed Barnawi, and Yong Xiang. 2017. Cost efficient resource management in fog computing supported medical cyber-physical system. IEEE Trans. Emerg. Topics Comput. 5, 1 (2017), 108--119.Google ScholarCross Ref
- Gabriel Guerrero-Contreras, Jose Luis Garrido, Sara Balderas-Diaz, and Carlos Rodriguez-Dominguez. 2017. A context-aware architecture supporting service availability in mobile cloud computing. IEEE Trans. Serv. Comput. 10, 6 (2017), 956--968.Google ScholarCross Ref
- Ying He, Chengchao Liang, Richard Yu, and Zhu Han. 2018. Trust-based social networks with computing, caching, and communications: A deep reinforcement learning approach. IEEE Trans. Netw. Sci. Eng. DOI:10.1109/TNSE.2018.2865183Google Scholar
- Ying He, F. Richard Yu, Nan Zhao, Victor C. M. Leung, and Hongxi Yin. 2017. Software-defined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach. IEEE Commun. Mag. 55, 12 (2017), 31--37.Google ScholarDigital Library
- Ying He, Nan Zhao, and Hongxi Yin. 2018. Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach. IEEE Trans. Vehic. Technol. 67, 1 (2018), 44--55.Google ScholarCross Ref
- Bhavesh Khemka, Ryan Friese, Luis D. Briceno, Howard Jay Siegel, Anthony A. Maciejewski, Gregory A. Koenig, Chris Groer, Gene Okonski, Marcia M. Hilton, Rajendra Rambharos et al. 2015. Utility functions and resource management in an oversubscribed heterogeneous computing environment. IEEE Trans. Comput. 64, 8 (2015), 2394--2407.Google ScholarDigital Library
- Jun Li, Xue Mei, Danil Prokhorov, and Dacheng Tao. 2017. Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans. Neural Netw. Learn. Syst. 28, 3 (2017), 690--703.Google ScholarCross Ref
- Yu-Jui Liu, Shin-Ming Cheng, and Yu-Lin Hsueh. 2017. eNB selection for machine type communications using reinforcement learning based Markov decision process. IEEE Trans. Vehic. Technol. 66, 12 (2017), 11330--11338.Google ScholarCross Ref
- Nguyen Cong Luong, Zehui Xiong, Ping Wang, and Dusit Niyato. 2018. Optimal auction for edge computing resource management in mobile blockchain networks: A deep learning approach. In Proceedings of the IEEE International Conference on Communications (ICC’18). IEEE, 1--6.Google ScholarCross Ref
- Yisheng Lv, Yanjie Duan, Wenwen Kang, Zhengxi Li, Fei-Yue Wang et al. 2015. Traffic flow prediction with big data: A deep learning approach. IEEE Trans. Intell. Transport. Syst. 16, 2 (2015), 865--873.Google Scholar
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski et al. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529.Google Scholar
- K. Mohamed, Etienne Côme, Latifa Oukhellou, and Michel Verleysen. 2017. Clustering smart card data for urban mobility analysis. IEEE Trans. Intell. Transport. Syst. 18, 3 (2017), 712--728.Google ScholarDigital Library
- Zhaolong Ning, Xiping Hu, Zhikui Chen, MengChu Zhou, Bin Hu, Jun Cheng, and Mohammad S. Obaidat. 2018. A cooperative quality-aware service access system for social Internet of vehicles. IEEE Int. Things J. 5, 4 (2018), 2506--2517.Google ScholarCross Ref
- Zhaolong Ning, Xiaojie Wang, and Jun Huang. 2019. Mobile edge computing-enabled 5G vehicular networks: toward the integration of communication and computing. IEEE Vehic. Technol. Mag. 14, 1 (2019), 54--61.Google ScholarCross Ref
- Zhaolong Ning, Xiaojie Wang, Feng Xia, and Joel Jose Rodrigues. 2019. Joint computation offloading, power allocation, and channel assignment for 5G-enabled traffic management systems. IEEE Trans. Industr. Inform. 15, 5 (2019), 3058--3067.Google ScholarCross Ref
- Zhaolong Ning, Feng Xia, Noor Ullah, Xiangjie Kong, and Xiping Hu. 2017. Vehicular social networks: Enabling smart mobility. IEEE Commun. Mag. 55, 5 (2017), 49--55.Google ScholarDigital Library
- Le Thanh Tan and Rose Qingyang Hu. 2018. Mobility-aware edge caching and computing in vehicle networks: A deep reinforcement learning. IEEE Trans. Vehic. Technol. 67, 11 (2018), 10190--10203.Google ScholarCross Ref
- David Tse and Pramod Viswanath. 2005. Fundamentals of Wireless Communication. Cambridge University Press.Google Scholar
- Hado Van Hasselt, Arthur Guez, and David Silver. 2016. Deep reinforcement learning with double Q-learning. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI’05), Vol. 2. 5.Google Scholar
- Chenmeng Wang, Chengchao Liang, F. Richard Yu, Qianbin Chen, and Lun Tang. 2017. Computation offloading and resource allocation in wireless cellular networks with mobile edge computing. IEEE Trans. Wirel. Commun. 16, 8 (2017), 4924--4938.Google ScholarDigital Library
- Xiaojie Wang, Zhaolong Ning, and Lei Wang. 2018. Offloading in Internet of vehicles: A fog-enabled real-time traffic management system. IEEE Trans. Industr. Inform. 14, 10 (2018), 4568--4578.Google ScholarCross Ref
- Xiaokang Wang, Laurence T. Yang, Xia Xie, Jirong Jin, and M. Jamal Deen. 2017. A cloud-edge computing framework for cyber-physical-social services. IEEE Commun. Mag. 55, 11 (2017), 80--85.Google ScholarCross Ref
- Zehua Wang, Derrick Wing Kwan Ng, Vincent W. S. Wong, and Robert Schober. 2017. Robust beamforming design in C-RAN with sigmoidal utility and capacity-limited backhaul. IEEE Trans. Wirel. Commun. 16, 9 (2017), 5583--5598.Google ScholarDigital Library
- Yifei Wei, F. Richard Yu, and Mei Song. 2010. Distributed optimal relay selection in wireless cooperative networks with finite-state Markov channels. IEEE Trans. Vehic. Technol. 59, 5 (2010), 2149--2158.Google ScholarCross Ref
- Jinming Wen, Chao Ren, and Arun Kumar Sangaiah. 2018. Energy-efficient device-to-device edge computing network: An approach offloading both traffic and computation. IEEE Commun. Mag. 56, 9 (2018), 96--102.Google ScholarCross Ref
- Guangquan Xu, Jia Liu, Yanrong Lu, Xianjiao Zeng, Yao Zhang, and Xiaoming Li. 2018. A novel efficient MAKA protocol with desynchronization for anonymous roaming service in global mobility networks. J. Netw. Comput. Appl. 107 (2018), 83--92.Google ScholarCross Ref
- Guangquan Xu, Yao Zhang, Arun Kumar Sangaiah, Xiaohong Li, Aniello Castiglione, and Xi Zheng. 2019. CSP-E2: An abuse-free contract signing protocol with low-storage TTP for energy-efficient electronic transaction ecosystems. Inform. Sci. 476 (2019), 505--515.Google ScholarCross Ref
- Jie Xu, Lixing Chen, and Shaolei Ren. 2017. Online learning for offloading and autoscaling in energy harvesting mobile edge computing. IEEE Trans. Cog. Commun. Netw. 3, 3 (2017), 361--373.Google ScholarCross Ref
- Chenggang Yan, Hongtao Xie, Dongbao Yang, Jian Yin, Yongdong Zhang, and Qionghai Dai. 2018. Supervised hash coding with deep neural network for environment perception of intelligent vehicles. IEEE Trans. Intell. Transport. Syst. 19, 1 (2018), 284--295.Google ScholarCross Ref
- Xianjiao Zeng, Guangquan Xu, Xi Zheng, Yang Xiang, and Wanlei Zhou. 2018. E-AUA: An efficient anonymous user authentication protocol for mobile IoT. IEEE Int. Things J. 6, 2 (2018), 1506--1519. DOI:https://doi.org/10.1109/JIOT.2018.2847447.Google ScholarCross Ref
- Desheng Zhang, Tian He, and Fan Zhang. 2018. Real-time human mobility modeling with multi-view learning. ACM Trans. Intell. Syst. Technol 9, 3 (2018), 22.Google ScholarDigital Library
- Shuhang Zhang, Boya Di, Lingyang Song, and Yonghui Li. 2017. Sub-channel and power allocation for non-orthogonal multiple access relay networks with amplify-and-forward protocol. IEEE Trans. Wirel. Commun. 16, 4 (2017), 2249--2261.Google ScholarDigital Library
- Zixing Zhang, Jürgen Geiger, Jouni Pohjalainen, Amr El-Desoky Mousa, Wenyu Jin, and Björn Schuller. 2018. Deep learning for environmentally robust speech recognition: An overview of recent developments. ACM Trans. Intell. Syst. Technol 9, 5 (2018), 49.Google ScholarDigital Library
- Bowen Zhou and Rajkumar Buyya. 2018. Augmentation techniques for mobile cloud computing: A taxonomy, survey, and future directions. ACM Comput. Surv. 51, 1 (2018), 13.Google ScholarDigital Library
- Chao Zhu, Giancarlo Pastor, Yu Xiao, Yong Li, and Antti Ylae-Jaeaeski. 2018. Fog following me: Latency and quality balanced task allocation in vehicular fog computing. In Proceedings of the 15th IEEE International Conference on Sensing, Communication, and Networking (SECON’18). IEEE, 1--9.Google ScholarDigital Library
Index Terms
- Deep Reinforcement Learning for Vehicular Edge Computing: An Intelligent Offloading System
Recommendations
Deep learning architectures in emerging cloud computing architectures: Recent development, challenges and next research trend
AbstractThe challenges of the conventional cloud computing paradigms motivated the emergence of the next generation cloud computing architectures. The emerging cloud computing architectures generate voluminous amount of data that are beyond ...
Graphical abstractDisplay Omitted
Highlights- The article presents literature review on the application of deep learning in emerging cloud computing architectures.
An RSU-crossed dependent task offloading scheme for vehicular edge computing based on deep reinforcement learning
Various interdependent and computationally intensive on-vehicle tasks have posed great pressure on the computing power of vehicles. Vehicular edge computing (VEC) is considered to be a promising paradigm to solve this problem. However, due to the high ...
Task offloading for vehicular edge computing with edge-cloud cooperation
AbstractVehicular edge computing (VEC) is emerging as a novel computing paradigm to meet low latency demands for computation-intensive vehicular applications. However, most existing offloading schemes do not take the dynamic edge-cloud computing ...
Comments