Athula Balachandran
Ion Stoica
ABSTRACT
Improving users' quality of experience (QoE) is crucial for sustaining the advertisement and subscription based revenue models that enable the growth of Internet video. Despite the rich literature on video and QoE measurement, our understanding of Internet video QoE is limited because of the shift from traditional methods of measuring video quality (e.g., Peak Signal-to-Noise Ratio) and user experience (e.g., opinion scores). These have been replaced by new quality metrics (e.g., rate of buffering, bitrate) and new engagement centric measures of user experience (e.g., viewing time and number of visits). The goal of this paper is to develop a predictive model of Internet video QoE. To this end, we identify two key requirements for the QoE model: (1) it has to be tied in to observable user engagement and (2) it should be actionable to guide practical system design decisions. Achieving this goal is challenging because the quality metrics are interdependent, they have complex and counter-intuitive relationships to engagement measures, and there are many external factors that confound the relationship between quality and engagement (e.g., type of video, user connectivity). To address these challenges, we present a data-driven approach to model the metric interdependencies and their complex relationships to engagement, and propose a systematic framework to identify and account for the confounding factors. We show that a delivery infrastructure that uses our proposed model to choose CDN and bitrates can achieve more than 20\% improvement in overall user engagement compared to strawman approaches.
- Buyer's Guide: Content Delivery Networks. http://goo.gl/B6gMK.Google Scholar
- Cisco study. http://goo.gl/tMRwM.Google Scholar
- Driving Engagement for Online Video. http://goo.gl/pO5Cj.Google Scholar
- Microsoft Smooth Streaming. http://goo.gl/6JOXh.Google Scholar
- P.800 : Methods for subjective determination of transmission quality. http://www.itu.int/rec/T-REC-P.800-199608-I/en.Google Scholar
- P.910: Subjective video quality assessment methods for multimedia applications. http://goo.gl/QjFhZ.Google Scholar
- Peak signal to noise ratio. http://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio.Google Scholar
- SPEC philosophy. http://www.spec.org/spec/#philosophy.Google Scholar
- Turbobytes: How it works. http://www.turbobytes.com/products/optimizer/.Google Scholar
- Video quality metrics. http://goo.gl/Ga9Xz.Google Scholar
- Vqeg. http://www.its.bldrdoc.gov/vqeg/vqeg-home.aspx.Google Scholar
- S. Akhshabi, L. Anantakrishnan, C. Dovrolis, and A. C. Begen. What Happens when HTTP Adaptive Streaming Players Compete for Bandwidth? In Proc. NOSSDAV, 2012. Google ScholarDigital Library
- S. Akhshabi, A. Begen, and C. Dovrolis. An Experimental Evaluation of Rate Adaptation Algorithms in Adaptive Streaming over HTTP. In MMSys, 2011. Google ScholarDigital Library
- R. H. Allen and R. D. Sriram. The Role of Standards in Innovation. Elsevier: Technology Forecasting and Social Change, 2000.Google ScholarCross Ref
- A. Balachandran, V. Sekar, A. Akella, S. Seshan, I. Stoica, and H. Zhang. A quest for an internet video quality-of-experience metric. In Hotnets, 2012. Google ScholarDigital Library
- K. Chen, C. Huang, P. Huang, and C. Lei. Quantifying skype user satisfaction. In SIGCOMM, 2006. Google ScholarDigital Library
- S. Chikkerur, V. Sundaram, M. Reisslein, and L. J. Karam. Objective video quality assessment methods: A classification, review, and performance comparison. In IEEE Transactions on Broadcasting, 2011.Google Scholar
- N. Cranley, P. Perry, and L. Murphy. User perception of adapting video quality. International Journal of Human-Computer Studies, 2006. Google ScholarDigital Library
- H. Deng, G. Runger, and E. Tuv. Bias of importance measures for multi-valued attributes and solutions. In ICANN, 2011. Google ScholarDigital Library
- F. Dobrian, V. Sekar, A. Awan, I. Stoica, D. A. Joseph, A. Ganjam, J. Zhan, and H. Zhang. Understanding the impact of video quality on user engagement. In Proc. SIGCOMM, 2011. Google ScholarDigital Library
- J. Esteban, S. Benno, A. Beck, Y. Guo, V. Hilt, and I. Rimac. Interactions Between HTTP Adaptive Streaming and TCP. In Proc. NOSSDAV, 2012. Google ScholarDigital Library
- A. Finamore, M. Mellia, M. Munafo, R. Torres, and S. G. Rao. Youtube everywhere: Impact of device and infrastructure synergies on user experience. In Proc. IMC, 2011. Google ScholarDigital Library
- A. Halevy, P. Norvig, and F. Pereira. The unreasonable effectiveness of data. In IEEE Intelligent Systems, 2009. Google ScholarDigital Library
- L. Huang, J. Jia, B. Yu, B. G. Chun, P. Maniatis, and M. Naik. Predicting Execution Time of Computer Programs Using Sparse Polynomial Regression. In Proc. NIPS, 2010.Google Scholar
- A. Khan, L. Sun, and E. Ifeachor. Qoe prediction model and its applications in video quality adaptation over umts networks. In IEEE Transactions on Multimedia, 2012.Google Scholar
- S. S. Krishnan and R. K. Sitaraman. Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs. In IMC, 2012. Google ScholarDigital Library
- B. Liu, M. Hu, and W. Hsu. Intuitive Representation of Decision Trees Using General Rules and Exceptions. In Proc. AAAI, 2000. Google ScholarDigital Library
- X. Liu, F. Dobrian, H. Milner, J. Jiang, V. Sekar, I. Stoica, and H. Zhang. A case for a coordinated internet video control plane. In SIGCOMM, 2012. Google ScholarDigital Library
- A. Mahimkar, Z. Ge, A. Shaikh, J. Wang, J. Yates, Y. Zhang, and Q. Zhao. Towards Automated Performance Diagnosis in a Large IPTV Network. In Proc. SIGCOMM, 2009. Google ScholarDigital Library
- V. Menkvoski, A. Oredope, A. Liotta, and A. C. Sanchez. Optimized online learning for qoe prediction. In BNAIC, 2009.Google Scholar
- T. Mitchell. Machine Learning. McGraw-Hill.Google Scholar
- R. K. P. Mok, E. W. W. Chan, X. Luo, and R. K. C. Chang. Inferring the QoE of HTTP Video Streaming from User-Viewing Activities . In SIGCOMM W-MUST, 2011. Google ScholarDigital Library
- L. Plissonneau and E. Biersack. A Longitudinal View of HTTP Video Streaming Performance. In MMSys, 2012. Google ScholarDigital Library
- I. Sodagar. The MPEG-DASH Standard for Multimedia Streaming Over the Internet. IEEE Multimedia, 2011. Google ScholarDigital Library
- H. H. Song, Z. Ge, A. Mahimkar, J. Wang, J. Yates, Y. Zhang, A. Basso, and M. Chen. Q-score: Proactive Service Quality Assessment in a Large IPTV System. In Proc. IMC, 2011. Google ScholarDigital Library
- M. Watson. Http adaptive streaming in practice. In MMSys - Keynote, 2011.Google Scholar
- I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, 2000. Google ScholarDigital Library
- C. Wu, B. Li, and S. Zhao. Diagnosing Network-wide P2P Live Streaming Inefficiencies. In Proc. INFOCOM, 2009.Google ScholarCross Ref
- W. Xu, L. Huang, A. Fox, D. Patterson, and M. Jordan. Detecting large-scale system problems by mining console logs. In Proc. SOSP, 2009. Google ScholarDigital Library
- H. Yin et al. Inside the Bird's Nest: Measurements of Large-Scale Live VoD from the 2008 Olympics.Google Scholar
- H. Yu, D. Z. B. Y. Zhao, and W. Zheng. Understanding User Behavior in Large-Scale Video-on-Demand Systems. In Proc. Eurosys, 2006. Google ScholarDigital Library
Index Terms
- Developing a predictive model of quality of experience for internet video
Recommendations
Understanding the impact of video quality on user engagement
SIGCOMM '11: Proceedings of the ACM SIGCOMM 2011 conferenceAs the distribution of the video over the Internet becomes main- stream and its consumption moves from the computer to the TV screen, user expectation for high quality is constantly increasing. In this context, it is crucial for content providers to ...
Understanding the impact of video quality on user engagement
SIGCOMM '11As the distribution of the video over the Internet becomes main- stream and its consumption moves from the computer to the TV screen, user expectation for high quality is constantly increasing. In this context, it is crucial for content providers to ...
Developing a predictive model of quality of experience for internet video
Improving users' quality of experience (QoE) is crucial for sustaining the advertisement and subscription based revenue models that enable the growth of Internet video. Despite the rich literature on video and QoE measurement, our understanding of ...
Comments