Skip to main content
SpringerLink
Log in

Object-based forgery detection in surveillance video using capsule network

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Due to the rapid growth of digital multimedia technology, the importance of multimedia data such as digital images/videos is increasing rapidly in diverse domains. This includes surveillance CCTV footages, which act as primary evidences in many court cases, including highly sensitive contexts. Today, with the wide availability of low-cost image/video manipulating software, digital images/videos have become highly vulnerable to manipulation/modification attacks; one such attack is the class of object-based forgery in surveillance videos. In this paper, we propose a Capsule Network based digital forensic technique for detection of object-based forgery in surveillance videos. In the proposed technique, we use motion residual, computed from every video frame, to extract intra- and inter-frame inherent statistical characteristics of the video sequence, as the input of capsule network. Our experimental results indicate that the proposed technique achieves significant performance in terms of authentic, double compressed and forged frame detection, irrespective of the group of pictures length and degree of compression in videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Aghamaleki JA, Behrad A (2016) Inter-frame video forgery detection and localization using intrinsic effects of double compression on quantization errors of video coding. Signal Process Image Commun 47:289–302

    Article  Google Scholar 

  • Amerini I, Becarelli R, Caldelli R, Del Mastio A (2014) Splicing forgeries localization through the use of first digit features. In: IEEE international workshop on information forensics and security (WIFS), pp 143–148

  • Amerini I, Galteri L, Caldelli R, Del Bimbo A (2019) Deepfake video detection through optical flow based CNN. In: IEEE/CVF international conference on computer vision workshop (ICCVW), pp 1205–1207

  • Bakas J, Naskar R (2018) A digital forensic technique for inter-frame video forgery detection based on 3D CNN. In: International conference on information systems security, (ICISS 2018). Springer, pp 304–317

  • Bhartiya G, Jalal AS (2017) Forgery detection using feature-clustering in recompressed JPEG images. Multim Tools Appl 76(20):20799–20814

    Article  Google Scholar 

  • Castiglione A, Cattaneo G, De Santis A (2011) A forensic analysis of images on online social networks. In: Third international conference on intelligent networking and collaborative systems, IEEE, pp 679–684

  • Chen S, Tan S, Li B, Huang J (2016) Automatic detection of object-based forgery in advanced video. IEEE Trans Circuits Syst Video Technol 26(11):2138–2151

    Article  Google Scholar 

  • D’Amiano L, Cozzolino D, Poggi G, Verdoliva L (2018) A patchmatch-based dense-field algorithm for video copy-move detection and localization. IEEE Trans Circuits Syst Video Technol 29(3):669–682

    Article  Google Scholar 

  • Fadl S, Han Q, Li Q (2019) Surveillance video authentication using universal image quality index of temporal average. In: International workshop on digital watermarking (IWDW 2018). Springer, pp 337–350

  • Fadl SM, Han Q, Li Q (2018) Inter-frame forgery detection based on differential energy of residue. IET Image Process 13(3):522–528

    Article  Google Scholar 

  • Gan Y, Yang J, Lai W (2019) Video object forgery detection algorithm based on VGG-11 convolutional neural network. In: International conference on intelligent computing. Automation and systems (ICICAS), IEEE, pp 575–580

  • Gonzalez RC, Woods RE, Masters BR (2008) Digital image processing third edition. Pearson Prentice Hall, New Jersey, pp 743–747

  • Gull S, Loan NA, Parah SA, Sheikh JA, Bhat GM (2020) An efficient watermarking technique for tamper detection and localization of medical images. J Ambient Intell Humaniz Comput 11(5):1799–1808

    Article  Google Scholar 

  • Hinton GE, Krizhevsky A, Wang SD (2011) Transforming auto-encoders. In: International conference on artificial neural networks. Springer, pp 44–51

  • Hinton GE, Sabour S, Frosst N (2018) Matrix capsules with EM routing. In: International conference on learning representations

  • Katebi R, Zhou Y, Chornock R, Bunescu R (2019) Galaxy morphology prediction using capsule networks. Mon Not R Astron Soc 486(2):1539–1547

    Article  Google Scholar 

  • Kim M, Chi S (2019) Detection of centerline crossing in abnormal driving using CapsNet. J Supercomput 75(1):189–196

    Article  Google Scholar 

  • Kodovskỳ J, Fridrich J (2009) Calibration revisited. In: Proceedings of the 11th ACM workshop on multimedia and security, pp 63–74

  • Kodovsky J, Fridrich J (2012) Steganalysis of JPEG images using rich models. In: Media watermarking, security, and forensics, International Society for Optics and Photonics, vol 8303, p 83030A

  • Kodovsky J, Fridrich J, Holub V (2011) Ensemble classifiers for steganalysis of digital media. IEEE Trans Inf Forensics Secur 7(2):432–444

    Article  Google Scholar 

  • Labartino D, Bianchi T, De Rosa A, Fontani M, Vázquez-Padín D, Piva A, Barni M (2013) Localization of forgeries in MPEG-2 video through GOP size and DQ analysis. In: IEEE 15th international workshop on multimedia signal processing (MMSP), IEEE, pp 494–499

  • Li Y, Zhou J (2019) Fast and effective image copy-move forgery detection via hierarchical feature point matching. IEEE Trans Inf Forensics Secur 14(5):1307–1322

    Article  Google Scholar 

  • Lin CS, Tsay JJ (2014) A passive approach for effective detection and localization of region-level video forgery with spatio-temporal coherence analysis. Dig Investig 11(2):120–140

    Article  Google Scholar 

  • Lin PY (2009) Basic image compression algorithm and introduction to JPEG standard. National Taiwan University, Taipei

    Google Scholar 

  • Liu Y, Huang T (2017) Exposing video inter-frame forgery by Zernike opponent chromaticity moments and coarseness analysis. Multim Syst 23(2):223–238

    Article  MathSciNet  Google Scholar 

  • Long C, Smith E, Basharat A, Hoogs A (2017) A C3D-based convolutional neural network for frame dropping detection in a single video shot. In: IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 1898–1906

  • Mohanarathinam A, Kamalraj S, Venkatesan GP, Ravi RV, Manikandababu C (2020) Digital watermarking techniques for image security: a review. J Ambient Intell Humaniz Comput 11(8):3221–3229

    Article  Google Scholar 

  • Nguyen HH, Tieu TND, Nguyen-Son HQ, Nozick V, Yamagishi J, Echizen I (2018) Modular convolutional neural network for discriminating between computer-generated images and photographic images. In: Proceedings of the 13th international conference on availability, reliability and security, pp 1–10

  • Nguyen HH, Yamagishi J, Echizen I (2019) Capsule-forensics: using capsule networks to detect forged images and videos. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2307–2311

  • Nie Y, Ma KK (2002) Adaptive rood pattern search for fast block-matching motion estimation. IEEE Trans Image Process 11(12):1442–1449

    Article  Google Scholar 

  • Pandey RC, Singh SK, Shukla KK (2017) A passive forensic method for video: exposing dynamic object removal and frame duplication in the digital video using sensor noise features. J Intell Fuzzy Syst 32(5):3339–3353

    Article  Google Scholar 

  • Pizzolante R, Castiglione A, Carpentieri B, De Santis A, Castiglione A (2014) Protection of microscopy images through digital watermarking techniques. In: International conference on intelligent networking and collaborative systems, IEEE, pp 65–72

  • Poncelet J, Renkens V, Van hamme H, (2021) Low resource end-to-end spoken language understanding with capsule networks. Comput Speech Lang 66:101142

    Article  Google Scholar 

  • Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Advances in neural information processing systems (NIPS 2017), pp 3856–3866

  • Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 3rd international conference on learning representations (ICLR2015)

  • Sitara K, Mehtre B (2016) Digital video tampering detection: an overview of passive techniques. Dig Investig 18(Supplement C):8–22

  • Su K, Kundur D, Hatzinakos D (2005) Statistical invisibility for collusion-resistant digital video watermarking. IEEE Trans Multim 7(1):43–51

    Article  Google Scholar 

  • Su L, Huang T, Yang J (2015) A video forgery detection algorithm based on compressive sensing. Multim Tools Appl 74(17):6641–6656

    Article  Google Scholar 

  • Su L, Luo H, Wang S (2019) A novel forgery detection algorithm for video foreground removal. IEEE Access 7:109719–109728

    Article  Google Scholar 

  • Vazquez-Padin D, Fontani M, Bianchi T, Comesana P, Piva A, Barni M (2012) Detection of video double encoding with GOP size estimation. In: IEEE international workshop on information forensics and security (WIFS), pp 151–156

  • Yu L, Wang H, Han Q, Niu X, Yiu SM, Fang J, Wang Z (2016) Exposing frame deletion by detecting abrupt changes in video streams. Neurocomputing 205:84–91

    Article  Google Scholar 

Download references

Acknowledgements

This research is partially funded by the following projects:(1) Department of Science and Technology (DST), Govt. of India, Grant No. DST/ICPS/Cluster/CS Research/2018 (General) dated: 13/03/2019. (2) Project titled “Deep learning applications for computer vision task” funded by National Institute of Technology Rourkela Overseas Alumni Association (NITROAA) with support of Lenovo P920 workstation and NVIDIA Corporation with support of NVIDIA Titan V and Quadro RTX 8000 GPUs.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Rourkela, Rourkela, 769008, India

    Jamimamul Bakas & Sambit Bakshi

  2. Department of Information Technology, Indian Institute of Engineering Science and Technology Shibpur, Shibpur, 711103, India

    Ruchira Naskar

  3. Department of Computer Science, University of Salerno, Salerno, Italy

    Michele Nappi

Authors
  1. Jamimamul Bakas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruchira Naskar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michele Nappi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sambit Bakshi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jamimamul Bakas or Sambit Bakshi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bakas, J., Naskar, R., Nappi, M. et al. Object-based forgery detection in surveillance video using capsule network. J Ambient Intell Human Comput 14, 3781–3791 (2023). https://doi.org/10.1007/s12652-021-03511-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-03511-3

Keywords

Search

Navigation