Skip to main content
SpringerLink
Log in

Camera operation estimation from video shot using 2D motion vector histogram

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a novel technique for classifying several camera operations in videos. First of all, we obtain a series of 2D motion vector (MV) fields by applying an existing MV estimation method. Then, a 2D MV histogram is generated in polar coordinates. The histogram shows that how many MVs in each frame share the similar magnitude and orientation. These two MV features are utilized simultaneously to classify the camera operations by representing on the 2D histogram. The proposed method can detect not only single camera operations but also a combination of two camera operations. The 2D histogram can describe the speed of the camera operations. Moreover, the 2D MV field itself can separate zoom-in and zoom-out camera operations that may produce exactly the same pattern in the 2D MV histogram. Especially, separating zoom-in and zoom-out camera operations because these two operations produce a similar 2D histogram. The proposed method can achieve a processing time of 5-10 millisecond per frame for a low-resolution video, while it takes 40-80 millisecond for a high-resolution video.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abdollahian G, Taskiran CM, Pizlo Z, Delp EJ (2010) Camera motion-based analysis of user generated video. IEEE Transactions on Multimedia 12(1):28–41

    Article  Google Scholar 

  2. Almeida J, Minetto R, Almeida TA, Torres RS, Leite NJ (2010) Estimation of camera parameters in video sequences with a large amount of scene motion. In: 2010 17th International conference on systems, signals and image processing (IWSSIP 2010), pp 348–358

  3. Bendraou Y, Essannouni F, Aboutajdine D, Salam A (2014) Video shot boundary detection method using histogram differences and local image descriptor. In: 2014 Second world conference on complex systems (WCCS), pp 665–670

  4. Block matching algorithms for motion estimation. https://www.mathworks.com/matlabcentral/fileexchange/8761-block-matching-algorithms-for-motion-estimation. Accessed 04 Apr 2018

  5. Chen Y, Zhang L, Lin B, Xu Y, Ren X (2011) Fighting detection based on optical flow context histogram. In: 2011 Second international conference on innovations in bio-inspired computing and applications, pp 95–98

  6. de Souza TT, Goularte R (2013) Video shot representation based on histograms. In: Proceedings of the 28th annual ACM symposium on applied computing SAC ’13. ACM, New York, pp 961–966

  7. Derue FX, Dahmane M, Lalonde M, Foucher S (2017) Exploiting semantic segmentation for robust camera motion classification. In: Image analysis and recognition, pp 173–181

  8. Duan LY, Jin JS, Tian Q, Xu CS (2006) Nonparametric motion characterization for robust classification of camera motion patterns. IEEE Trans Multimed 8(2):323–340

    Article  Google Scholar 

  9. Erturk S (2003) Digital image stabilization with sub-image phase correlation based global motion estimation. IEEE Trans Consum Electron 49(4):1320–1325

    Article  Google Scholar 

  10. Experimental data of duan’s method and okade’s method. http://www.facweb.iitkgp.ernet.in/~pkb/camera_classify.html. Accessed 04 Apr 2018

  11. Ewerth R, Schwalb M, Tessmann P, Freisleben B (2004) Estimation of arbitrary camera motion in mpeg videos. In: Proceedings of the 17th international conference on pattern recognition, 2004. ICPR 2004, vol 1, pp 512–515

  12. Fakhar B, Kanan HR, Behrad A (2019) Event detection in soccer videos using unsupervised learning of spatio-temporal features based on pooled spatial pyramid model. Multimed Tools Appl 78(12):16995–17025

    Article  Google Scholar 

  13. Gonzalez RC, Woods RE (2006) Digital image processing, 3rd edn. Prentice-Hall, Inc., Upper Saddle River

    Google Scholar 

  14. Hasan MA, Xu M, He X, Xu C (2014) Camhid: camera motion histogram descriptor and its application to cinematographic shot classification. IEEE Trans Circ Syst Video Technol 24(10):1682–1695

    Article  Google Scholar 

  15. Hasan MA, Xu M, He X, Wang Y (2015) A camera motion histogram descriptor for video shot classification. Multimed Tools Appl 74(24):11073–11098

    Article  Google Scholar 

  16. Hu WC, Chen CH, Chen TY, Peng MY, Su YJ (2018) Real-time video stabilization for fast-moving vehicle cameras. Multimed Tools Appl 77(1):1237–1260

    Article  Google Scholar 

  17. Kim JG, Chang HS, Kim J, Kim HM (2000) Efficient camera motion characterization for mpeg video indexing. In: 2000 IEEE International conference on multimedia and expo. ICME2000. Proceedings. Latest advances in the fast changing world of multimedia (Cat. No.00TH8532), vol 2, pp 1171– 1174

  18. Lee S, Hayes MH (2002) Real-time camera motion classification for content-based indexing and retrieval using templates. In: 2002 IEEE International conference on acoustics, speech, and signal processing, vol 4, pp IV–3664–IV–3667

  19. Luca C, Sergio B, Riccardo L (2013) Classifying cinematographic shot types. Multimed Tools Appl 62(1):51–73

    Article  Google Scholar 

  20. Mahabalagiri A, Ozcan K, Velipasalar S (2014) Camera motion detection for mobile smart cameras using segmented edge-based optical flow. In: 2014 11th IEEE International conference on advanced video and signal based surveillance (AVSS), pp 271–276

  21. Matlab the language of technical computing. https://www.mathworks.com. Accessed 04 Apr 2018

  22. Mpeg the moving picture experts group website. https://mpeg.chiariglione.org/. Accessed 13 Aug 2019

  23. Narayanan S, Makur A (2013) Camera motion estimation using circulant compressive sensing matrices. In: 2013 9th International conference on information, communications signal processing, pp 1–5

  24. Nguyen NT, Laurendeau D, Branzan-Albu A (2010) A robust method for camera motion estimation in movies based on optical flow. Int J Intell Syst Technol Appl 9(3/4):228–238

    Google Scholar 

  25. Nie Y, Ma KK (2002) Adaptive rood pattern search for fast block-matching motion estimation. IEEE Trans Image Process 11(12):1442–1449

    Article  Google Scholar 

  26. Nikitidis S, Zafeiriou S, Pitas I (2008) Camera motion estimation using a novel online vector field model in particle filters. IEEE Trans Circ Syst Vid Technol 18(8):1028–1039

    Article  Google Scholar 

  27. Okade M, Biswas PK (2012) Fast camera motion estimation using discrete wavelet transform on block motion vectors. In: 2012 Picture coding symposium, pp 333–336

  28. Okade M, Patel G, Biswas PK (2016) Robust learning-based camera motion characterization scheme with applications to video stabilization. IEEE Trans Circ Syst Vid Technol 26(3):453–466

    Article  Google Scholar 

  29. Patel NV, Sethi IK (1997) Video shot detection and characterization for video databases. Pattern Recogn 30(4):583–592

    Article  Google Scholar 

  30. Prasertsakul P (2018) A video analysis for camera motion estimation and its application to automatic retrieval of attractive moments in sport videos. Dissertation, Japan Advanced Institute of Science and Technology

  31. Prasertsakul P, Kondo T, Iida H (2017) Video shot classification using 2d motion histogram. In: 2017 14th International conference on electrical engineering/electronics, computer, telecommunications and information technology (ECTI-CON), pp 202–205

  32. Puthenpurayil SP, Chakrabarti I, Virdi R, Kaushik H (2016) Very large scale integration architecture for block-matching motion estimation using adaptive rood pattern search algorithm. IET Circ Dev Syst 10(4):309–316

    Article  Google Scholar 

  33. Shih H (2013) A novel attention-based key-frame determination method. IEEE Trans Broadcast 59(3):556–562

    Article  Google Scholar 

  34. Spampinato G, Bruna A, Naccari F, Tomaselli V (2019) Adaptive low cost algorithm for video stabilization. Multimed Tools Appl 78(10):13787–13804

    Article  Google Scholar 

  35. Srinivasan M, Venkatesh S, Hosie R (1997) Qualitative estimation of camera motion parameters from video sequences. Pattern Recogn 30(4):593–606

    Article  Google Scholar 

  36. Tan YP, Saur DD, Kulkami SR, Ramadge PJ (2000) Rapid estimation of camera motion from compressed video with application to video annotation. IEEE Trans Circ Syst Video Technol 10(1):133–146

    Article  Google Scholar 

  37. Tavassolipour M, Karimian M, Kasaei S (2014) Event detection and summarization in soccer videos using Bayesian network and copula. IEEE Trans Circ Syst Video Technol 24(2):291–304

    Article  Google Scholar 

  38. Tu Y, Zhang X, Liu B, Yan C (2017) Video description with spatial-temporal attention. In: Proceedings of the 25th ACM international conference on multimedia, pp 1014–1022

  39. Ultra video group. http://ultravideo.cs.tut.fi/#testsequences. Accessed 15 Aug 2018

  40. Video samples. http://www.divx.com/en/devices/profiles/video. Accessed 04 Apr 2018

  41. Wang R, Huang T (1999) Fast camera motion analysis in mpeg domain. In: Proceedings 1999 international conference on image processing (Cat. 99CH36348), vol 3, pp 691–694

  42. Weng Y, Jiang J (2011) Fast camera motion estimation in mpeg compressed domain. IEEE Trans Consum Electron 57(3):1329–1335

    Article  Google Scholar 

  43. Xiph.org video test media [derf’s collection]. https://media.xiph.org/video/derf/. Accessed 04 Apr 2018

  44. Xiao Q, Wang H, Li F, Gao Y (2011) 3d object retrieval based on a graph model descriptor. Neurocomputing 74(17):3486–3493

    Article  Google Scholar 

  45. Xiao Q, Luo Y, Wang H (2014) Motion retrieval based on switching Kalman filters model. Multimed Tools Appl 72(1):951–966

    Article  Google Scholar 

  46. Xiao Q, Wang Y, Wang H (2015) Motion retrieval using weighted graph matching. Soft Comput 19(1):133–144

    Article  Google Scholar 

  47. Yan C, Tu Y, Wang X, Zhang Y, Hao X, Zhang Y, Dai Q (2019) Stat: spatial-temporal attention mechanism for video captioning. IEEE Trans Multimed (Early Access), 1–1

  48. Yu J, Tao D, Wang M, Rui Y (2015) Learning to rank using user clicks and visual features for image retrieval. IEEE Trans Cybern 45(4):767–779

    Article  Google Scholar 

  49. Zhang L, Xu QK, Nie LZ, Huang H (2014) Videograph: a non-linear video representation for efficient exploration. Vis Comput 30(10):1123–1132

    Article  Google Scholar 

  50. Zhu X, Elmagarmid AK, Xue X, Wu L, Catlin AC (2005) Insightvideo: toward hierarchical video content organization for efficient browsing, summarization and retrieval. IEEE Trans Multimed 7(4):648–666

    Article  Google Scholar 

Download references

Acknowledgements

This research is financially supported by Sirindhorn International Institute of Technology (SIIT), Thammasat University (TU), Japan Advanced Institute of Science and Technology (JAIST), National Science and Technology Development Agency (NSTDA), and National Research University Project (NRU), Thailand Office of Higher Education Commission.

Author information

Authors and Affiliations

  1. School of Information, Computer, and Communication Technology, Sirindhorn International Institute of Technology, Thammasat University, 12000, Pathum Thani, Thailand

    Pawin Prasertsakul & Toshiaki Kondo

  2. School of Information Science, Japan Advanced Institute of Science and Technology, 923-1211, Ishikawa, Japan

    Pawin Prasertsakul & Hiroyuki Iida

  3. National Electronics and Computer Technology Center (NECTEC), National Science and Technology Development Agency (NSTDA), Khlong Nueng, Thailand

    Teera Phatrapornnant

Authors
  1. Pawin Prasertsakul
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toshiaki Kondo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hiroyuki Iida
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Teera Phatrapornnant
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pawin Prasertsakul.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Prasertsakul, P., Kondo, T., Iida, H. et al. Camera operation estimation from video shot using 2D motion vector histogram. Multimed Tools Appl 79, 17403–17426 (2020). https://doi.org/10.1007/s11042-019-08378-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08378-3

Keywords

Search

Navigation