Skip to main content
SpringerLink
Log in

Convolutional Neural Network-based deblocking filter for SHVC in H.265

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The deblocking filter in the standard H.265 SHVC reduces blocking artifacts at the edges and block boundaries of coded frame with exhaustive computation. But the maximum removal of these blocking artifacts is not achieved; also the computational complexity is still a burden. This made many research works discussed related to maximum removal of blocking artifacts without considering computational complexity. In this paper, a convolutional neural network (CNN)-based deblocking filter is proposed for SHVC in H.265 that removes blocking artifacts with less computation. The proposed CNN framework learns the reconstructed samples of the input frame in a video sequence. The blocking artifacts are efficiently detected by the preprocessing unit which is considered as the first convolution layer. Next, the features are extracted for the frames in a video based on patch by employing kernel and stride for scanning the complete frames in a video. The normalization is applied from base layer to enhancement layer in the CNN framework to preserve sharpness in the video by removing artifacts generated due to inter-layer prediction and quantization. The network model is trained by rectified linear unit activation function to perform nonlinear mappings. In addition, the CNN-based deblocking filter achieves less computation due to max-pooling and soft-max layers in CNN. The simulation results are conducted that produce an average of 0.76 dB increase in PSNR and 57% time saving compared with the standard SHM reference encoder.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Park, C.-S., Kim, B.-G.: Performance analysis of inter-layer prediction in scalable extension of HEVC (SHVC) for adaptive media service. Displays 44, 27–36 (2016)

    Article  Google Scholar 

  2. Pourazad, M.T., Doutre, C., Azimi, M., Nasiopoulos, P.: HEVC: the new gold standard for video compression. IEEE Consum. Electron. Mag. 1(3), 36–46 (2012)

    Article  Google Scholar 

  3. Test model for scalable extensions of high efficiency video coding (HEVC), ISO/IEC and JTC1/SC29/WG11 (2013)

  4. Larochelle, H., Bengio, Y., Louradour, J., Lamblin, P.: Exploring strategies for training deep neural networks. J. Mach. Learn. Res. 10, 1–40 (2009)

    MATH  Google Scholar 

  5. Lee, H., Ekanadham, C., Ng, A.Y.: Sparse deep belief net model for visual area v2. In: Proceedings of the 20th International Conference on Neural Information Processing Systems, pp. 873–880 (2007)

  6. Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  7. Swietojanski, P., Ghoshal, A., Renals, S.: Convolutional neural networks for distant speech recognition. IEEE Signal Process. Lett. 21(9), 1120–1124 (2014)

    Article  Google Scholar 

  8. Wang, Z., Yan, D., Wang, R., Xiang, L., Wu, T.: Speech resampling detection based on inconsistency of band energy. Comput. Mater. Contin. 56(2), 247–259 (2018)

    Google Scholar 

  9. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems, ser. NIPS’12, 2012, vol. 1, pp. 1097–1105

  10. Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256–1272 (2017)

    Article  Google Scholar 

  11. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  Google Scholar 

  12. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In: IEEE International Conference on Computer Vision (ICCV), pp. 4549–4557 (2018)

  13. Lu, X., Martin, G.R.: Fast mode decision algorithm for the H.264/AVC scalable video coding extension. IEEE Trans. Circuits Syst. Video Technol. 23(5), 846–855 (2013)

    Article  Google Scholar 

  14. Balaji, L., Thyagharajan, K.K.: H.264/SVC mode decision based on mode correlation and desired mode list. Int. J. Autom. Comput. 11(5), 510–516 (2014)

    Article  Google Scholar 

  15. Khairy, M., Hamdy, A., Elsayed, A., Farouk, H.: Efficient enhancement for spatial scalable video coding transmission. Math. Probl. Eng. 2017, 1–13 (2017)

    Article  Google Scholar 

  16. Balaji, L., Thyagharajan, K.K.: H.264 SVC complexity reduction based on likelihood mode decision. Sci. World J. 2015, 1–11 (2015)

    Article  Google Scholar 

  17. Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: Content adaptive complexity reduction scheme for quality/fidelity scalable HEVC. In: Proceedings of ICASSP, pp. 1744–1748 (2013)

  18. Content adaptive complexity reduction scheme for quality/fidelity scalable HEVC, ISO/IEC JTC1/SC29/WG11, L0042 (2013)

  19. Tohidypour, H.R., Pourazad, M.T., Bashashati, H., Nasiopoulos, P.: Fast mode assignment for quality scalable extension of the high efficiency video coding (HEVC) standard: a Bayesian approach. In: Proceedings of BCI, pp. 61–65 (2013)

  20. Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: Adaptive search range method for spatial scalable HEVC. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 191–192 (2014)

  21. Bailleul, R., De Cock, J., Van de Walle, R.: Fast mode decision for SNR scalability in SHVC digest of technical papers. In: Proceedings of IEEE International Conference on Consumer Electronics, pp. 193–194 (2014)

  22. Cote, G., Erol, B., Gallant, M., Kossentini, F.: H.263 + : video coding at low bit rates. IEEE Trans. Circuits Syst. Video Technol. 8(7), 849–866 (1998)

    Article  Google Scholar 

  23. Sze, V., Budagavi, M., Madhukar, S., Sullivan, G.: High efficiency video coding (HEVC): algorithms and architectures. In: Chandrakasan, A.P. (ed.) Integrated Circuits and Systems. Springer, New York (2014)

    Google Scholar 

  24. Wien, M.: High efficiency video coding: coding tools and specification. In: Celebi, E., Chen, J., Gopi, E.S., Neustein, A., Poor, H.V. (eds.) Signals and Communication Technology. Springer, NewYork (2014)

    Google Scholar 

  25. Sullivan, G.J., Ohm, J., Woo-Jin, J.H., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  26. Aminlou, A., Lainema, J., Ugur, K., Hannuksela, M.M., Gabbouj, M.: Differential coding using enhanced inter-layer reference picture for the scalable extension of H.265/HEVC video codec. IEEE Trans. Circuits Syst. Video Technol. 24(11), 1945–1956 (2014)

    Article  Google Scholar 

  27. Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: Probabilistic approach for predicting the size of coding units in the quad-tree structure of the quality and spatial scalable HEVC. IEEE Trans. Multimed. 18(2), 182–195 (2016)

    Article  Google Scholar 

  28. Shen, L., An, P., Feng, G.: Low-complexity scalable extension of the high-efficiency video coding (SHVC) encoding system. ACM Trans. Multimed. Comput. Commun. Appl. 15(2), 44 (2019)

    Google Scholar 

  29. Tohidypour, H.R., Pourazad, M.T., Nasiopoulos, P.: An encoder complexity reduction scheme for quality/fidelity scalable HEVC. IEEE Trans. Broadcast. 62(3), 664–674 (2016)

    Article  Google Scholar 

  30. Raja, G., Mirza, M.J.: In-loop deblocking filter for H.264/AVC video. In: 2nd International Symposium on Communications, Control, and Signal Processing, ISCCSP (2006)

  31. Norkin, A., Bjøntegaard, G., Fuldseth, A., Narroschke, M., Ikeda, M., Andersson, K., Zhou, M., Van der Auwera, G.: HEVC deblocking filter. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1746–1754 (2012)

    Article  Google Scholar 

  32. Wang, Y., Guo, X., Fan, X.: Parallel in-loop filtering in HEVC encoder on GPU. IEEE Trans. Consum. Electron. 64(3), 276–284 (2018)

    Article  Google Scholar 

  33. Lee, Y., Kim, J., Choi, Y., Kim, B.: CNN-based approach for visual quality improvement on heVC. In: IEEE International Conference on Consumer Electronics (ICCE 2018), pp. 498–500 (2018)

  34. Balaji, L., Thyagharajan, K.K.: An enhanced performance for H.265/SHVC based on combined AEGBM3D filter and back-propagation neural network. Signal Image Video Process. 12(5), 809–817 (2018)

    Article  Google Scholar 

  35. Sheen, L., Feng, G.F., An, P.A.: SHVC CU processing aided by a feed forward neural network. IEEE Trans. Ind. Inform. 99, 1 (2019)

    Google Scholar 

  36. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)

    Article  Google Scholar 

  37. List, P., Joch, A., Lainema, J., Bjontegaard, G., Karczewicz, M.: Adaptive deblocking filter. IEEE Trans. Circuits Syst. Video Technol. 13(7), 614–619 (2003)

    Article  Google Scholar 

  38. Reeve III, H.C., Lim, J.S.: Reduction of blocking effects in image coding. Opt. Eng. 23(1), 230134 (1984)

    Google Scholar 

  39. Wang, C., Zhou, J., Liu, S.: Adaptive non-local means filter for image deblocking. Signal Process. Image Commun. 28(5), 522–530 (2013)

    Article  Google Scholar 

  40. Liew, A.-C., Yan, H.: Blocking artifacts suppression in block-coded images using overcomplete wavelet representation. IEEE Trans. Circuits Syst. Video Technol. 14(4), 450–461 (2004)

    Article  Google Scholar 

  41. Foi, A., Katkovnik, V., Egiazarian, K.: Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images. IEEE Trans. Image Process. 16(5), 1395–1411 (2007)

    Article  MathSciNet  Google Scholar 

  42. Jancsary, J., Nowozin, S., Rother, C.: Loss-specific training of nonparametric image restoration models: a new state of the art. In: European Conference on Computer Vision, pp. 112–125 (2012)

  43. Li, Y., Guo, F., Tan, R.T., Brown, M.S.: A contrast enhancement framework with jpeg artifacts suppression. In: European Conference on Computer Vision, pp. 174–188 (2014)

  44. SHM-7.0. [Online]. Available: https://hevc.hhi.fraunhofer.de/svn/svnSHVCSoftware/tags/SHM-7.0, Accessed 30 Oct 2015

  45. Bjontegaard, G.: Calculation of average PSNR differences between RDcurves. In: ITU-T Q. 6/SG16 VCEG, 15th Meeting, Austin, Texas, USA (2001)

Download references

Author information

Authors and Affiliations

  1. Department of ECE, School of Electrical and Electronics Engineering, Sathyabama Institute of Science and Technology, Chennai, India

    A. Dhanalakshmi

  2. Department of EIE, Panimalar Engineering College, Chennai, India

    A. Dhanalakshmi

  3. Department of Computer Science and Engineering, School of Computing, Sathyabama Institute of Science and Technology, Chennai, India

    G. Nagarajan

Authors
  1. A. Dhanalakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Nagarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Dhanalakshmi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dhanalakshmi, A., Nagarajan, G. Convolutional Neural Network-based deblocking filter for SHVC in H.265. SIViP 14, 1635–1645 (2020). https://doi.org/10.1007/s11760-020-01713-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-020-01713-4

Keywords

Search

Navigation