ABSTRACT
Learning-based Adaptive Bit Rate~(ABR) method, aiming to learn outstanding strategies without any presumptions, has become one of the research hotspots for adaptive streaming. However, it is still suffering from several issues, i.e., low sample efficiency and lack of awareness of the video quality information. In this paper, we propose Comyco, a video quality-aware ABR approach that enormously improves the learning-based methods by tackling the above issues. Comyco trains the policy via imitating expert trajectories given by the instant solver, which can not only avoid redundant exploration but also make better use of the collected samples. Meanwhile, Comyco attempts to pick the chunk with higher perceptual video qualities rather than video bitrates. To achieve this, we construct Comyco's neural network architecture, video datasets and QoE metrics with video quality features. Using trace-driven and real world experiments, we demonstrate significant improvements of Comyco's sample efficiency in comparison to prior work, with 1700x improvements in terms of the number of samples required and 16x improvements on training time required. Moreover, results illustrate that Comyco outperforms previously proposed methods, with the improvements on average QoE of 7.5% - 16.79%. Especially, Comyco also surpasses state-of-the-art approach Pensieve by 7.37% on average video quality under the same rebuffering time.
- 2019. DASH Industry Forum | Catalyzing the adoption of MPEG-DASH. (2019). https://dashif.org/Google Scholar
- 2019. HTTP Live Streaming. https://developer.apple.com/streaming/. (2019).Google Scholar
- Anne Aaron, Zhi Li, Megha Manohara, Joe Yuchieh Lin, Eddy Chi-Hao Wu, and C-C Jay Kuo. 2015. Challenges in cloud based ingest and encoding for high quality streaming media. In 2015 IEEE International Conference on Image Processing (ICIP). IEEE, 1732--1736.Google ScholarDigital Library
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. TensorFlow: A System for Large-Scale Machine Learning.. In OSDI, Vol. 16. 265--283.Google ScholarDigital Library
- Tasnim Abar, Asma Ben Letaifa, and Sadok El Asmi. 2017. Machine learning based QoE prediction in SDN networks. In 2017 13th International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 1395--1400.Google ScholarCross Ref
- Zahaib Akhtar and et al. 2018. Oboe: auto-tuning video ABR algorithms to network conditions. In SIGCOMM 2018. ACM, 44--58.Google Scholar
- David M Beazley et al. 1996. SWIG: An Easy to Use Tool for Integrating Scripting Languages with C and C++.. In Tcl/Tk Workshop. 43.Google Scholar
- Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009. Pearson correlation coefficient. In Noise reduction in speech processing. Springer, 1--4.Google ScholarDigital Library
- Abdelhak Bentaleb, Ali C Begen, and Roger Zimmermann. 2016. SDNDASH: Improving QoE of HTTP adaptive streaming using software defined networking. In Proceedings of ACM MultiMedia 2016. ACM, 1296--1305.Google ScholarDigital Library
- Abdelhak Bentaleb, Bayan Taani, Ali C Begen, Christian Timmerer, and Roger Zimmermann. 2018. A Survey on Bitrate Adaptation Schemes for Streaming Media over HTTP. IEEE Communications Surveys & Tutorials (2018).Google Scholar
- Cisco. 2017. Cisco Visual Networking Index: Forecast and Methodology, 2016--2021. (2017). https://www.cisco.com/c/dam/en/us/ solutions/collateral/service-provider/visual-networking-index-vni/ complete-white-paper-c11--481360.pdfGoogle Scholar
- Zhengfang Duanmu, Abdul Rehman, and Zhou Wang. 2018. A Quality-of- Experience Database for Adaptive Video Streaming. IEEE Transactions on Broadcasting 64, 2 (June 2018), 474--487.Google ScholarCross Ref
- Zhengfang Duanmu, Kai Zeng, Kede Ma, Abdul Rehman, and Zhou Wang. 2017. A quality-of-experience index for streaming video. IEEE Journal of Selected Topics in Signal Processing 11, 1 (2017), 154--166.Google ScholarCross Ref
- FFmpeg. [n. d.]. FFmpeg. ([n. d.]). http://ffmpeg.org/Google Scholar
- M. Gadaleta, F. Chiariotti, M. Rossi, and A. Zanella. 2017. D-DASH: A Deep QLearning Framework for DASH Video Streaming. IEEE Transactions on Cognitive Communications and Networking 3, 4 (Dec 2017), 703--718. https://doi.org/10. 1109/TCCN.2017.2755007Google ScholarCross Ref
- GPAC. [n. d.]. MP4BOX. ([n. d.]). https://gpac.wp.imt.fr/mp4box/Google Scholar
- Alain Hore and Djemel Ziou. 2010. Image Quality Metrics: PSNR vs. SSIM. (2010), 2366--2369.Google Scholar
- Tianchi Huang, Xin Yao, Chenglei Wu, Rui-Xiao Zhang, and Lifeng Sun. 2018. Tiyuntsong: A Self-Play Reinforcement Learning Approach for ABR Video Streaming. arXiv preprint arXiv:1811.06166 (2018).Google Scholar
- Tianchi Huang, Rui-Xiao Zhang, Chao Zhou, and Lifeng Sun. 2018. QARC: Video Quality Aware Rate Control for Real-Time Video Streaming based on Deep Reinforcement Learning. In 2018 ACM Multimedia Conference on Multimedia Conference. ACM, 1208--1216.Google ScholarDigital Library
- Te-Yuan Huang, Chaitanya Ekanadham, Andrew J. Berglund, and Zhi Li. 2019. Hindsight: Evaluate Video Bitrate Adaptation at Scale. In Proceedings of the 10th ACM Multimedia Systems Conference (MMSys '19). ACM, New York, NY, USA, 86--97. https://doi.org/10.1145/3304109.3306219Google ScholarDigital Library
- Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. 2015. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. ACM SIGCOMM Computer Communication Review 44, 4 (2015), 187--198.Google ScholarDigital Library
- Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne. 2017. Imitation learning: A survey of learning methods. ACM Computing Surveys (CSUR) 50, 2 (2017), 21.Google ScholarDigital Library
- Junchen Jiang, Vyas Sekar, and Hui Zhang. 2014. Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. TON 22, 1 (2014), 326--340.Google Scholar
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- Michael Laskey, Jonathan Lee, Roy Fox, Anca Dragan, and Ken Goldberg. 2017. Dart: Noise injection for robust imitation learning. arXiv preprint arXiv:1703.09327 (2017).Google Scholar
- Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun. 2018. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European Conference on Computer Vision (ECCV). 116--131.Google ScholarDigital Library
- Mao. 2017. hongzimao/pensieve. (Jul 2017). https://github.com/hongzimao/ pensieveGoogle Scholar
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the 2017 ACM SIGCOMM Conference. ACM, 197--210.Google ScholarDigital Library
- Hongzi Mao, Shaileshh Bojja Venkatakrishnan, Malte Schwarzkopf, and Mohammad Alizadeh. 2019. Variance Reduction for Reinforcement Learning in Input-Driven Environments. international conference on learning representations (2019).Google Scholar
- Russell Mendonca, Abhishek Gupta, Rosen Kralev, Pieter Abbeel, Sergey Levine, and Chelsea Finn. 2019. Guided Meta-Policy Search. arXiv preprint arXiv:1904.00956 (2019).Google Scholar
- Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning. 1928--1937.Google ScholarDigital Library
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).Google Scholar
- Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. 2016. Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440 (2016).Google Scholar
- Ravi Netravali, Anirudh Sivaraman, Somak Das, Ameesh Goyal, Keith Winstein, James Mickens, and Hari Balakrishnan. 2015. Mahimahi: accurate record-andreplay for HTTP. (2015), 417--429.Google Scholar
- Takayuki Osa, Joni Pajarinen, Gerhard Neumann, J AndrewBagnell, Pieter Abbeel, Jan Peters, et al. 2018. An algorithmic perspective on imitation learning. Foundations and Trends® in Robotics 7, 1--2 (2018), 1--179.Google Scholar
- Pablo Gil Pereira, Andreas Schmidt, and Thorsten Herfet. 2018. Cross-Layer Effects on Training Neural Algorithms for Video Streaming. In Proceedings of the 28th ACM SIGMM Workshop on Network and Operating Systems Support for Digital Audio and Video. ACM, 43--48.Google ScholarDigital Library
- Yanyuan Qin, Shuai Hao, Krishna R Pattipati, Feng Qian, Subhabrata Sen, Bing Wang, and Chaoqun Yue. 2018. ABR streaming of VBR-encoded videos: characterization, challenges, and solutions. In Proceedings of CoNeXT 2018. ACM, 366--378.Google ScholarDigital Library
- Reza Rassool. 2017. VMAF reproducibility: Validating a perceptual practical video quality metric. In Broadband Multimedia Systems and Broadcasting (BMSB), 2017 IEEE International Symposium on. IEEE, 1--2.Google ScholarCross Ref
- Abdul Rehman, Kai Zeng, and Zhou Wang. 2015. Display device-adapted video quality-of-experience assessment. In Human Vision and Electronic Imaging XX, Vol. 9394. International Society for Optics and Photonics, 939406.Google Scholar
- Measuring Fixed Broadband Report. 2016. Raw Data Measuring Broadband America 2016. https://www.fcc.gov/reports-research/reports/measuring-broadbandamerica/ raw-data-measuring-broadband-america-2016. (2016). [Online; accessed 19-July-2016].Google Scholar
- Haakon Riiser, Paul Vigmostad, Carsten Griwodz, and Pål Halvorsen. 2013. Commute path bandwidth traces from 3G networks: analysis and applications. In Proceedings of the 4th ACM Multimedia Systems Conference. ACM, 114--118.Google ScholarDigital Library
- Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics. 627--635.Google Scholar
- Kevin Spiteri, Ramesh Sitaraman, and Daniel Sparacio. 2018. From theory to practice: improving bitrate adaptation in the DASH reference player. In Proceedings of the 9th MMSys. ACM, 123--137.Google ScholarDigital Library
- Kevin Spiteri, Rahul Urgaonkar, and Ramesh K Sitaraman. 2016. BOLA: Nearoptimal bitrate adaptation for online videos. In INFOCOM 2016, IEEE. IEEE, 1--9.Google Scholar
- Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction. MIT press.Google ScholarDigital Library
- F. Tang, B. Mao, Z. M. Fadlullah, N. Kato, O. Akashi, T. Inoue, and K. Mizutani. 2018. On Removing Routing Protocol from Future Wireless Networks:AReal-time Deep Learning Approach for Intelligent Traffic Control. IEEE Wireless Communications 25, 1 (February 2018), 154--160. https://doi.org/10.1109/MWC.2017.1700244Google ScholarCross Ref
- Yuan Tang. 2016. TF. Learn: TensorFlow's high-level module for distributed machine learning. arXiv preprint arXiv:1612.04251 (2016).Google Scholar
- Usc-Nsl. 2018. USC-NSL/Oboe. (Oct 2018). https://github.com/USC-NSL/OboeGoogle Scholar
- Zhou Wang. 2017. Video QoE: Presentation Quality vs. Playback Smoothness. (Jul 2017). https://www.ssimwave.com/science-of-seeing/ video-quality-of-experience-presentation-quality-vs-playback-smoothness/Google Scholar
- Francis Y Yan, Jestin Ma, Greg D Hill, Deepti Raghavan, Riad S Wahby, Philip Levis, and Keith Winstein. 2018. Pantheon: the training ground for Internet congestion-control research. In 2018 {USENIX} Annual Technical Conference ({USENIX} {ATC} 18). 731--743.Google Scholar
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A controltheoretic approach for dynamic adaptive video streaming over HTTP. In ACM SIGCOMM Computer Communication Review. ACM, 325--338.Google Scholar
Index Terms
- Comyco: Quality-Aware Adaptive Video Streaming via Imitation Learning
Recommendations
Quality-of-Experience of Adaptive Video Streaming: Exploring the Space of Adaptations
MM '17: Proceedings of the 25th ACM international conference on MultimediaWith the remarkable growth of adaptive streaming media applications, especially the wide usage of dynamic adaptive streaming schemes over HTTP (DASH), it becomes ever more important to understand the perceptual quality-of-experience (QoE) of end users, ...
Feedback control for adaptive live video streaming
MMSys '11: Proceedings of the second annual ACM conference on Multimedia systemsMultimedia content feeds an ever increasing fraction of the Internet traffic. Video streaming is one of the most important applications driving this trend. Adaptive video streaming is a relevant advancement with respect to classic progressive download ...
Cumulative Quality Modeling for HTTP Adaptive Streaming
HTTP Adaptive Streaming has become the de facto choice for multimedia delivery. However, the quality of adaptive video streaming may fluctuate strongly during a session due to throughput fluctuations. So, it is important to evaluate the quality of a ...
Comments