Xiao Zhu
ABSTRACT
Over-the-top (OTT) live video traffic has grown significantly, fueled by fundamental shifts in how users consume video content (e.g., increased cord-cutting) and by improvements in camera technologies, computing power, and wireless resources. A key determining factor for the end-to-end live streaming QoE is the design of the first-mile upstream ingest path that captures and transmits the live content in real-time, from the broadcaster to the remote video server. This path often involves either a Wi-Fi or cellular component, and is likely to be bandwidth-constrained with time-varying capacity, making the task of high-quality video delivery challenging. Today, there is little understanding of the state of the art in the design of this critical path, with existing research focused mainly on the downstream distribution path, from the video server to end viewers.
To shed more light on the first-mile ingest aspect of live streaming, we propose Livelyzer, a generalized active measurement and black-box testing framework for analyzing the performance of this component in popular live streaming software and services under controlled settings. We use Livelyzer to characterize the ingest behavior and performance of several live streaming platforms, identify design deficiencies that lead to poor performance, and propose best practice design recommendations to improve the same.
- ITU-TP. 910. 2008. Subjective Video Quality Assessment Methods for Multimedia Applications.Google Scholar
- Adeel Abbas. 2020. Introducing VMAF percentiles for video quality measurements. https://blog.twitter.com/engineering/en_us/topics/infrastructure/2020/introducing-vmaf-percentiles-for-video-quality-measurements.htmlGoogle Scholar
- Adobe. 2012. Adobe's Real Time Messaging Protocol. https://www.adobe.com/content/dam/acom/en/devnet/rtmp/pdf/rtmp_specification_1.0.pdf.Google Scholar
- Roman Arutyunyan. 2020. NGINX-based Media Streaming Server. https://github.com/arut/nginx-rtmp-module.Google Scholar
- Abdelhak Bentaleb, Christian Timmerer, Ali C Begen, and Roger Zimmermann. 2019. Bandwidth prediction in low-latency chunked streaming. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video. 7--13.Google ScholarDigital Library
- Bitmovin. 2019. Bitmovin Video Developer Report 2019. https://cdn2.hubspot.net/hubfs/3411032/Bitmovin%20Magazine/Video%20Developer%20Report%202019/bitmovin-video-developer-report-2019.pdf.Google Scholar
- Francesco Bronzino, Paul Schmitt, Sara Ayoubi, Guilherme Martins, Renata Teixeira, and Nick Feamster. 2020. Inferring streaming video quality from encrypted traffic: Practical models and deployment experience. ACM SIGMETRICS Performance Evaluation Review 48, 1 (2020), 27--28.Google ScholarDigital Library
- Jan De Cock, Aditya Mavlankar, Anush Moorthy, and Anne Aaron. 2016. A large-scale video codec comparison of x264, x265 and libvpx for practical VOD applications. In Applications of Digital Image Processing XXXIX, Vol. 9971. International Society for Optics and Photonics, 997116.Google Scholar
- Jie Deng, Gareth Tyson, Felix Cuadrado, and Steve Uhlig. 2017. Internet scale user-generated live video streaming: The Twitch case. In International Conference on Passive and Active Network Measurement. Springer, 60--71.Google ScholarCross Ref
- Google Developers. 2020. Real-time communication for the web. https://webrtc.org.Google Scholar
- Facebook. 2020. FB: How to Go Live on Mobile? https://www.facebook.com/business/help/1884140525218868.Google Scholar
- Facebook. 2020. How do I go live from my Facebook Page? https://www.facebook.com/help/1916203341847533.Google Scholar
- Facebook. 2020. How do I set up streaming software to work with Facebook? https://www.facebook.com/help/755943624557739.Google Scholar
- Facebook. 2020. What are the video format guidelines for live streaming on Facebook? https://www.facebook.com/help/1534561009906955.Google Scholar
- FFmpeg. 2021. A complete, cross-platform solution to record, convert and stream audio and video. https://ffmpeg.orgGoogle Scholar
- International Organization for Standardization. 2014. ISO/IEC 14496-15:2014 Information technology --- Coding of audio-visual objects --- Part 15: Carriage of network abstraction layer (NAL) unit structured video in ISO base media file format. https://www.iso.org/standard/65216.htmlGoogle Scholar
- International Organization for Standardization. 2020. ISO/IEC 23000-19:2020 Information technology --- Multimedia application format (MPEG-A) --- Part 19: Common media application format (CMAF) for segmented media. https://www.iso.org/standard/79106.html.Google Scholar
- DASH Industry Forum. 2020. Low-latency Modes for DASH. https://dashif.org/docs/CR-Low-Latency-Live-r8.pdf.Google Scholar
- DASH Industry Forum. 2021. DASH-IF Live Media Ingest Protocol Technical Specification, 26 February 2021. https://dashif-documents.azurewebsites.net/Ingest/master/DASH-IF-Ingest.html.Google Scholar
- Sadjad Fouladi, John Emmons, Emre Orbay, Catherine Wu, Riad S Wahby, and Keith Winstein. 2018. Salsify: Low-latency network video through tighter integration between a video codec and a transport protocol. In 15th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 18).Google Scholar
- Maximilian Grüner, Melissa Licciardello, and Ankit Singla. 2020. Reconstructing proprietary video streaming algorithms. In 2020 {USENIX} Annual Technical Conference ({USENIX} {ATC} 20).Google Scholar
- Oliver L Haimson and John C Tang. 2017. What makes live events engaging on Facebook Live, Periscope, and Snapchat. In Proceedings of the 2017 CHI conference on human factors in computing systems. 48--60.Google ScholarDigital Library
- Dexter Tan Guan Hao. 2019. Mixer's Faster Than Light streaming protocol explained. https://dotesports.com/streaming/news/mixers-faster-than-light-streaming-protocol-explained.Google Scholar
- Cisco Visual Networking Index. 2019. Forecast and Methodology 2017--2022. Cisco: San Jose, CA, USA (2019).Google Scholar
- Jaehong Kim, Youngmok Jung, Hyunho Yeo, Juncheol Ye, and Dongsu Han. 2020. Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 107--125.Google ScholarDigital Library
- Emily Krings. 2021. What is RTMPS and Why is it Important to Secure Streaming? https://www.dacast.com/blog/rtmps-streaming/#:~:text=that%20it%20provides.-,RTMP%20vs.,stream%20with%20the%20secure%20alternativeGoogle Scholar
- Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward A Practical Perceptual Video Quality Metric. https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652Google Scholar
- Zhi Li, Christos Bampis, Julie Novak, Anne Aaron, Kyle Swanson, Anush Moorthy, and Jan De Cock. 2018. VMAF: The Journey Continues. https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12Google Scholar
- Joe Yuchieh Lin, Tsung-Jung Liu, Eddy Chi-Hao Wu, and C-C Jay Kuo. 2014. A fusion-based video quality assessment (FVQA) index. In Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific. IEEE, 1--5.Google ScholarCross Ref
- Tsung-Jung Liu, Yu-Chieh Lin, Weisi Lin, and C-C Jay Kuo. 2013. Visual quality assessment: recent developments, coding applications and future trends. APSIPA Transactions on Signal and Information Processing 2 (2013).Google Scholar
- Xing Liu, Bo Han, Feng Qian, and Matteo Varvello. 2019. LIME: understanding commercial 360° live video streaming services. In Proceedings of the 10th ACM Multimedia Systems Conference. 154--164.Google ScholarDigital Library
- Tarun Mangla, Emir Halepovic, Ellen Zegura, and Mostafa Ammar. 2020. Drop the packets: using coarse-grained data to detect video performance issues. In Proceedings of the 16th International Conference on emerging Networking EXperiments and Technologies. 71--77.Google ScholarDigital Library
- Tarun Mangla, Ellen Zegura, Mostafa Ammar, Emir Halepovic, Kyung-Wook Hwang, Rittwik Jana, and Marco Platania. 2018. VideoNOC: Assessing video QoE for network operators using passive measurements. In Proceedings of the 9th ACM Multimedia Systems Conference. 101--112.Google ScholarDigital Library
- Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 197--210.Google ScholarDigital Library
- Matthew K Mukerjee, David Naylor, Junchen Jiang, Dongsu Han, Srinivasan Seshan, and Hui Zhang. 2015. Practical, real-time centralized control for cdn-based live video delivery. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. 311--324.Google ScholarDigital Library
- Ashkan Nikravesh, Qi Alfred Chen, Scott Haseley, Xiao Zhu, Geoffrey Challen, and Z Morley Mao. 2018. QoE inference and improvement without end-host control. In 2018 IEEE/ACM Symposium on Edge Computing (SEC). IEEE, 43--57.Google ScholarCross Ref
- Yong Niu, Yong Li, Depeng Jin, Li Su, and Athanasios V Vasilakos. 2015. A survey of millimeter wave communications (mmWave) for 5G: opportunities and challenges. Wireless networks 21, 8 (2015), 2657--2676.Google Scholar
- Roger Pantos. 2020. HTTP Live Streaming 2nd Edition. https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-07.Google Scholar
- Karine Pires and Gwendal Simon. 2015. YouTube live and Twitch: a tour of user-generated live streaming systems. In Proceedings of the 6th ACM multimedia systems conference. 225--230.Google ScholarDigital Library
- Yanyuan Qin, Shuai Hao, Krishna R Pattipati, Feng Qian, Subhabrata Sen, Bing Wang, and Chaoqun Yue. 2018. ABR streaming of VBR-encoded videos: characterization, challenges, and solutions. In Proceedings of the 14th International Conference on emerging Networking EXperiments and Technologies. 366--378.Google ScholarDigital Library
- Devdeep Ray, Jack Kosaian, KV Rashmi, and Srinivasan Seshan. 2019. Vantage: optimizing video upload for time-shifted viewing of social live streams. In Proceedings of the ACM Special Interest Group on Data Communication. 380--393.Google ScholarDigital Library
- Eric Rescorla, Hannes Tschofenig, and Nagendra Modadugu. 2018. The Datagram Transport Layer Security (DTLS) Protocol Version 1.3. https://tools.ietf.org/id/draft-ietf-tls-dtls13-01.html.Google Scholar
- Selenium. 2021. Selenium automates browsers. https://www.selenium.devGoogle Scholar
- Matti Siekkinen, Teemu Kämäräinen, Leonardo Favario, and Enrico Masala. 2018. Can you see what I see? Quality-of-experience measurements of mobile live video broadcasting. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 14, 2s (2018), 1--23.Google ScholarDigital Library
- Matti Siekkinen, Enrico Masala, and Teemu Kämäräinen. 2016. A first look at quality of mobile live streaming experience: the case of periscope. In Proceedings of the 2016 Internet Measurement Conference. 477--483.Google ScholarDigital Library
- Iraj Sodagar. 2011. The mpeg-dash standard for multimedia streaming over the internet. IEEE multimedia 18, 4 (2011), 62--67.Google ScholarDigital Library
- Denny Stohr, Tao Li, Stefan Wilk, Silvia Santini, and Wolfgang Effelsberg. 2015. An analysis of the YouNow live streaming platform. In 2015 IEEE 40th local computer networks conference workshops (LCN Workshops). IEEE, 673--679.Google Scholar
- OBS Studio. 2020. Open Broadcast Software. https://obsproject.com/.Google Scholar
- Gary J Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on circuits and systems for video technology 22, 12 (2012), 1649--1668.Google ScholarDigital Library
- John C Tang, Gina Venolia, and Kori M Inkpen. 2016. Meerkat and periscope: I stream, you stream, apps stream for live streams. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 4770--4780.Google ScholarDigital Library
- Twitch. 2020. Twitch Recommended Software for Broadcasting. https://help.twitch.tv/s/article/recommended-software-for-broadcasting?language=en_US.Google Scholar
- umlaeute. 2020. v4l2loopback - a kernel module to create V4L2 loopback devices. https://github.com/umlaeute/v4l2loopback.Google Scholar
- VideoLAN. 2021. x265. https://www.videolan.org/developers/x265.htmlGoogle Scholar
- Bolun Wang, Xinyi Zhang, Gang Wang, Haitao Zheng, and Ben Y Zhao. 2016. Anatomy of a personalized livestreaming system. In Proceedings of the 2016 Internet Measurement Conference. 485--498.Google ScholarDigital Library
- Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612.Google ScholarDigital Library
- Wikipedia. [n.d.]. Reed-Solomon error correction. https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correctionGoogle Scholar
- Wikipedia. 2020. QR code. https://en.wikipedia.org/wiki/QR_code.Google Scholar
- Shichang Xu, Eric Petajan, Subhabrata Sen, and Z Morley Mao. 2020. What you see is what you get: measure ABR video streaming QoE via on-device screen recording. In Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video. 60--66.Google ScholarDigital Library
- Shichang Xu, Subhabrata Sen, and Z Morley Mao. 2020. CSI: inferring mobile ABR video adaptation behavior under HTTPS and QUIC. In Proceedings of the Fifteenth European Conference on Computer Systems. 1--16.Google ScholarDigital Library
- Shichang Xu, Subhabrata Sen, Z Morley Mao, and Yunhan Jia. 2017. Dissecting VOD services for cellular: performance, root causes and best practices. In Proceedings of the 2017 Internet Measurement Conference. 220--234.Google ScholarDigital Library
- Yang Xu, Chenguang Yu, Jingjiang Li, and Yong Liu. 2012. Video telephony for end-consumers: measurement study of Google+, iChat, and Skype. In Proceedings of the 2012 Internet Measurement Conference. 371--384.Google ScholarDigital Library
- Francis Y Yan, Hudson Ayers, Chenzhi Zhu, Sadjad Fouladi, James Hong, Keyi Zhang, Philip Levis, and Keith Winstein. 2020. Learning in situ: a randomized experiment in video streaming. In 17th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 20). 495--511.Google Scholar
- Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication.Google ScholarDigital Library
- Youtube. 2020. Create a live stream on mobile - Android - Youtube Help. https://support.google.com/youtube/answer/9228390?hl=en.Google Scholar
- Youtube. 2020. Create a live stream via webcam - Youtube Help. https://support.google.com/youtube/answer/9228389?hl=en&ref_topic=9257984.Google Scholar
- Youtube. 2020. How to Live Stream On Youtube - How Youtube Works. https://www.youtube.com/howyoutubeworks/product-features/live/#youtube-live.Google Scholar
- Youtube. 2020. YouTube Live verified encoders. https://support.google.com/youtube/answer/2907883?hl=en&ref_topic=9257984#zippy=%2Csoftware-encoders.Google Scholar
- Cong Zhang and Jiangchuan Liu. 2015. On crowdsourced interactive live streaming: a twitch. tv-based measurement study. In Proceedings of the 25th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video.Google ScholarDigital Library
- Huanhuan Zhang, Anfu Zhou, Jiamin Lu, Ruoxuan Ma, Yuhan Hu, Cong Li, Xinyu Zhang, Huadong Ma, and Xiaojiang Chen. 2020. OnRL: improving mobile video telephony via online reinforcement learning. In The 26th Annual International Conference on Mobile Computing and Networking. 1--14.Google ScholarDigital Library
- Anfu Zhou, Huanhuan Zhang, Guangyuan Su, Leilei Wu, Ruoxuan Ma, Zhen Meng, Xinyu Zhang, Xiufeng Xie, Huadong Ma, and Xiaojiang Chen. 2019. Learning to Coordinate Video Codec with Transport Protocol for Mobile Video Telephony. In The 25th Annual International Conference on Mobile Computing and Networking. 1--16.Google ScholarDigital Library
Index Terms
- Livelyzer: analyzing the first-mile ingest performance of live video streaming
Recommendations
An SDN-aided low-latency live video streaming over HTTP
AbstractDynamic adaptive streaming over HTTP (DASH) is the crucial factor in the rapid penetration of over-the-top (OTT) service providers for on-demand video streaming. It can also be used for live video streaming by the OTT providers. The recent ...
MUSLIN: Achieving High, Fairly Shared QoE Through Multi-Source Live Streaming
PV '18: Proceedings of the 23rd Packet Video WorkshopDelivering video content with a high and fairly shared quality of experience is a challenging task in view of the drastic video traffic increase forecasts. Currently, content delivery networks provide numerous servers hosting replicas of the video ...
MUSLIN demo: high QoE fair multi-source live streaming
MMSys '18: Proceedings of the 9th ACM Multimedia Systems ConferenceDelivering video content with a high and fairly shared quality of experience is a challenging task in view of the drastic video traffic increase forecasts. Currently, content delivery networks provide numerous servers hosting replicas of the video ...
Comments