Srinivas Gudumasu
ABSTRACT
Versatile Video Coding (VVC) standard is currently being prepared as the latest video coding standard of the ITU-T and ISO/IEC. The primary goal of the VVC, expected to be finalized in 2020, is to further improve compression performance compared to its predecessor HEVC. The frame level, slice level or Wavefront parallel processing (WPP) existing in VTM (VVC Test Model) doesn't fully utilize the CPU capabilities available in today's multicore systems. Moreover, VTM decoder sequentially processes the decoding tasks. This design is not parallelization friendly. This paper proposes re-designed decoding tasks that parallelize the decoder using: 1. Load balanced task parallelization and 2. CTU (Coding Tree Unit) based data parallelization. The design overcomes the limitations of the existing parallelization techniques by fully utilizing the available CPU computation resource without compromising on the coding efficiency and the memory bandwidth. The parallelization of CABAC and the slice decoding tasks is based on a load sharing scheme, while parallelization of each sub-module of the slice decoding task uses CTU level data parallelization. The parallelization scheme may either remain restricted within an individual decoding task or utilize between task parallelization. Such parallelization techniques achieve real-time VVC decoding on multi-core CPUs, for bitstreams generated using VTM5.0 using Random-Access configuration. An overall average decoding time reduction of 88.97% (w.r.t. VTM5.0 decoder) is achieved for 4K sequences on a 10-core processor.
- High Efficiency Video Coding (HEVC), Rec. ITU-T H.265 and ISO/IEC 23008-2, January 2013 (and later editions).Google Scholar
- G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, "Overview of the High Efficiency Video Coding (HEVC) Standard", IEEE Trans. Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649--1668, December 2012.Google ScholarDigital Library
- J. Chen, Y. Chen, M. Karczewicz, X. Li, H. Liu, L. Zhang, X. Zhao, "Coding tools investigation for next generation video coding", ITU-T SG16 Doc. COM16-C806, February 2015.Google Scholar
- E. Alshina, A. Alshin, J. - H. Min, K. Choi, A. Saxena, M. Budagavi, "Known tools performance investigation for next generation video coding", VCEGAZ05, June 2015.Google Scholar
- A. Segall, V. Baroncini, J. Boyce, J. Chen, T. Suzuki, "Joint call for proposals on video compression with capability beyond HEVC", JVET-H1002, October 2017, Macau, China.Google Scholar
- B. Bross, K. Andersson, M. Blaser, V. Drugeon, S-H Kim, J. Lainema, J. Li, S. Liu, J. R. Ohm, G. J. Sullivan, "General Video Coding Technology in Responses to the Joint Call for Proposals on Video Compression with Capability beyond HEVC", IEEE Trans. Circuits and Systems for Video Technology, special issue October 2019Google Scholar
- J. Le. Tanou, M. Blestel, "Analysis of Emerging Video Codecs: Coding Tools, Compression Efficiency and Complexity", SMPTE Motion Imaging Journal, 128 (10), pp. 14--24, November 2019Google ScholarCross Ref
- J. Chen, Y. Ye, S. H. Kim, "Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6)", JVET-O2002, July 2019.Google Scholar
- I. Ahmad, Y. He and M. L. Liou, "Video compression with parallel processing," Parallel Computing, 28 (7-8): 1039-1078, August 2002.Google ScholarDigital Library
- DF De Souza, N. Roma, L. Sousa, "Co-operative CPU+GPU deblocking filter parallelization for high performance HEVC video codecs," IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4993--4997, May 2014.Google Scholar
- DF De Souza, A. llic, N. Roma, L. Sousa, "GPU assisted HEVC intra decoder," Journal of Real-Time Image Processing, vol. 12, Issue 2, pp. 531--547, August 2016.Google ScholarDigital Library
- DF De Souza, A. llic, L. Sousa, "HEVC in-loop GPU parallelization in embedded systems," International Conference on Embedded Computer System: Architectures, Modeling and Simulation (SAMOS XV), pp. 123--130, July 2015.Google Scholar
- W. Zhang, C. Guo, "Design and Implementation of Parallel Algorithms for Sample Adaptive Offset in HEVC Based on GPU", Sixth International Conference on Information Science and Technology, May 2016.Google Scholar
- Ittiam system private limited, "AHG16: Early Implementation of VVC software player and Demonstration on Mobile device", JVET-P0307, Geneva, CH, October 2019.Google Scholar
- A. M. Kotra, M. Raulet and O. Deforges, "Efficient Parallelization of Different HEVC Decoding Stages," 2013 Data Compression Conference, Snowbird, UT, pp. 502--502, March 2013.Google Scholar
- M. Alvarez-Mesa, C. C. Chi, B. Juurlink, V. George and T. Schierl, "Parallel video decoding in the emerging HEVC standard," 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, pp. 1545--1548, March 2012.Google Scholar
- S. Gudumasu, Y. He, Y. Ye, Y. He, E-S. Ryu, J. Dong, X. Xiu, "Real-time SHVC software decoding with multi-threaded parallel processing", Proceedings of SPIE, Applications of Digital Image Processing, vol. 9217, September 2014.Google Scholar
- M. Koziri et al., "Adaptive Tile Parallelization for Fast Video Encoding in HEVC," 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Chengdu, pp. 738--743, December 2016Google Scholar
- V. Sze, A. P. Chandrakasan, M. Budagavi and Minhua Zhou, "Parallel CABAC for low power video coding," 2008 15th IEEE International Conference on Image Processing, San Diego, CA, pp. 2096--2099, October 2008.Google Scholar
- Y. Huang, et.al., "A VVC Proposal with Quaternary Tree plus Binary-Ternary Tree Coding Block Structure and Advanced Coding Techniques," IEEE Transactions on Circuits and Systems for Video Technology, pp. 1--1, October 2019.Google Scholar
- M. Wang et al., "Extended Quad-Tree Partitioning for Future Video Coding," 2019 Data Compression Conference (DCC), Snowbird, UT, USA, pp. 300--309, May 2019.Google ScholarCross Ref
- J. Chen, M. Karczewicz, Y-W. Huang, K. Choi, J-R. Ohm, G. J. Sullivan, "The Joint Exploration Model (JEM) for Video Compression with Capability beyond HEVC", IEEE Transactions on Circuits and Systems for Video Technology, pp. 1--1, October 2019.Google Scholar
- H. Gao, S. Esenlik, Z. Zhao, E. Steinbach, J. Chen, "Decoder Side Motion Vector Refinement for Versatile Video Coding", IEEE 21st International Workshop on Multimedia Signal Processing (MMSP), pp. 27--29, September 2019.Google Scholar
- L. Zhang, K. Zhang, H. Liu, H. C. Chuang, Y. Wang, J. Xu, P. Zhao, D. Hong, "History-Based Motion Vector Prediction in Versatile Video Coding", IEEE Data Compression Conference (DCC), pp. 43--52, March 2019.Google Scholar
- C.-C. Chen, X. Xiu, Y. He, Y. Ye, "Generalized bi-prediction method for future video coding", Proc. IEEE Picture Coding Symp. (PCS), pp. 1--5, December 2016.Google ScholarCross Ref
- L. Zhao et al., "Wide Angular Intra Prediction for Versatile Video Coding," 2019 Data Compression Conference (DCC), Snowbird, UT, USA, pp. 53--62, May 2019.Google ScholarCross Ref
- Y. Chang et al., "Multiple Reference Line Coding for Most Probable Modes in Intra Prediction," 2019 Data Compression Conference (DCC), Snowbird, UT, USA, pp. 559--559, March 2019.Google ScholarCross Ref
- A. Segall, J. Zhao and I. Yamamoto, "Parallel intra prediction for video coding," 28th Picture Coding Symposium, Nagoya, pp. 310--313, December 2010.Google Scholar
- A. Kenneth, "Combined Intra Inter Prediction Coding Mode", Picture Coding Symposium ITU-T SG16/Q6 (VCEG) Doc. VCEGAD11, October 2006.Google Scholar
- Y. Wang, X. Guo, X. Fan, Y. Lu, D. Zhao and W. Gao, "Parallel In-Loop Filtering in HEVC Encoder on GPU," in IEEE Transactions on Consumer Electronics, vol. 64, no. 3, pp. 276--284, August 2018.Google ScholarDigital Library
- A. Norkin et al., "HEVC Deblocking Filter," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1746--1754, December 2012.Google ScholarDigital Library
- C. Fu et al., "Sample Adaptive Offset in the HEVC Standard," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1755--1764, December 2012.Google ScholarDigital Library
- VTM: the reference software for VVC development: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTMGoogle Scholar
- F. Bossen, J. Boyce, K. Suehring, X. Li and V. Seregin, "JVET common test conditions and software reference configurations", JVET-L1010, Macao, CN, October 2018.Google Scholar
Index Terms
- Software-based versatile video coding decoder parallelization
Recommendations
GPU Parallelization of HEVC In-Loop Filters
In the High Efficiency Video Coding (HEVC) standard, multiple decoding modules have been designed to take advantage of parallel processing. In particular, the HEVC in-loop filters (i.e., the deblocking filter and sample adaptive offset) were conceived ...
Accelerating QTMT-based CU partition and intra mode decision for versatile video coding
AbstractThe H.266/VVC achieves about 50% bitrate saving compared to its predecessor H.265/HEVC at the expense of exponentially increased computational complexity. The most efficient but complex technique for H.266/VVC intra frame coding is the QuadTree ...
Rate distortion optimization with adaptive content modeling for random-access versatile video coding
AbstractIn this paper, we study the capability of improving the rate-distortion (RD) performance based on the adaptive content modeling in the Versatile Video Coding (VVC) standard. In particular, the frame-level dependent relationships and ...
Comments