Skip to main content
SpringerLink
Log in

Fuzzy reasoning framework to improve semantic video interpretation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A video retrieval system user hopes to find relevant information when the proposed queries are ambiguous. The retrieval process based on detecting concepts remains ineffective in such a situation. Potential relationships between concepts have been shown as a valuable knowledge resource that can enhance the retrieval effectiveness, even for ambiguous queries. Recent researches in multimedia retrieval have focused on ontology modeling as a common framework to manage knowledge. Handling these ontologies has to cope with issues related to generic knowledge management and processing scalability. Considering these issues, we suggest a context-based fuzzy ontology framework for video content analysis and indexing. In this paper, we focused on the way in which we modeled our fuzzy ontology: First, we populate automatically the generated ontology by gathering various available video annotation datasets. Then, the ontology content was used to infer enhanced video semantic interpretation. Finally, considering user feedback, the content of the ontology was improved. Experimental results showed that our approach achieves the goal of scalability while at the same time allowing better video content semantic interpretation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. http://www-lpir.nist.gov/projects/tv2010/tv10.semantic.indexing.relations2.txt

References

  1. (2006). LSCOM Lexicon Definitions Version, Annotations, Version 1.0., Tech. rep. Columbia University

  2. Adami N, Bugatti A, Leonardi R, Migliorati P (2001) Low level processing of audio and video information for extracting the semantics of content. In: 2001 IEEE Fourth Workshop on Multimedia Signal Processing, pp 607–612

  3. Ayache S (2007) Indexation de documents vidéos par concepts par fusion de caractristiques audio, image et texte, Ph.D. thesis, Institut National Polytechnique de Grenoble

  4. Ayache S (2008) Video Corpus Annotation using Active Learning. In: European Conference on Information Retrieval (ECIR). Glasgow, Scotland, pp 187–198

  5. Baader F, Calvanese D, McGuinness D L, Nardi D, Patel-Schneider PF (eds) (2003) The description logic handbook: theory, implementation, and applications. Cambridge University Press, New York

  6. Baghdadi S, Gravier G, Demarty C, Gros P (2008) Structure learning in a bayesian network-based video indexing framework. In: 2008 IEEE International Conference on Multimedia and Expo, pp. 677–680

  7. Bannour H, Hudelot C (2013) Building and using fuzzy multimedia ontologies for semantic image annotation. Multimed Tools Appl:1–35

  8. Benitez A, Chang SF (2003) Image classification using multimedia knowledge networks. In: 2003. ICIP 2003. Proceedings. 2003 International Conference on Image Processing, vol. 3, pp. III–613–16 vol.2

  9. Bobillo F, Delgado M, Gmez-Romero J, Straccia U (2012) Joining gödel and zadeh fuzzy logics in fuzzy description logics, vol 20

  10. Bosko B (1990) Fuzziness vs. probability. Int J Gen Syst 17(2-3):211–240

    Article  MATH  Google Scholar 

  11. Brilhault A (2009) Indexation et recherche par le contenu de documents vidéos, Joseph Fourier University

  12. Calegari S, Ciucci D (2007) Fuzzy ontology, fuzzy description logics and fuzzy-owl. In: Proceedings of the 7th international workshop on Fuzzy Logic and Applications: Applications of Fuzzy Sets Theory, WILF ’07. Springer-Verlag, Berlin, Heidelberg, pp 118–126

  13. Chattopadhyay C, Maurya A (2013) Genre-specific modeling of visual features for efficient content based video shot classification and retrieval. Int J Multimed Inf Retr 2(4):289–297. doi:10.1007/s13735-013-0034-8

    Article  Google Scholar 

  14. Cheng Y, Xiong Y (2012) Research on model of ontology-based semantic information retrieval. In: Jin D, Lin S (eds) Advances in Multimedia, Software Engineering and Computing Vol.1, vol 128. Springer , Berlin Heidelberg, pp 271–276

  15. Dasiopoulou S, Giannakidou E, Litos G, Malasioti P, Kompatsiaris Y (2011) A survey of semantic image and video annotation tools. In: Paliouras G, Spyropoulos C , Tsatsaronis G (eds) Knowledge-Driven Multimedia Information Extraction and Ontology Evolution, vol 6050. Springer, Berlin Heidelberg, pp 196–239

    Chapter  Google Scholar 

  16. Dean J (2009) Challenges in building large-scale information retrieval systems: invited talk. In: Proceedings of the Second ACM International Conference on Web Search and Data Mining, WSDM ’09, New York, pp 1–1

  17. DeMenthon D, Megret R (2002) Spatio-Temporal Segmentation of Video by Hierarchical Mean Shift Analysis. Tech. Rep. LAMP-TR-090, CAR-TR-978, CS-TR-4388, UMIACS-TR-2002-68, University of Maryland, College Park

  18. Dentler K, Cornet R, Ten Teije A, De Keizer N (2011) Comparison of reasoners for large ontologies in the owl 2 el profile. Semant. web 2(2):71–87

    Google Scholar 

  19. Du Y, Chen F, Xu W, Zhang W (2006) Interacting activity recognition using hierarchical durational-state dynamic bayesian network. In: Zhuang Y, Yang S Q, Rui Y, He Q (eds) Advances in Multimedia Information Processing - PCM 2006, vol 4261. Springer , Berlin Heidelberg, pp 185–192

    Chapter  Google Scholar 

  20. Egozi O, Markovitch S, Gabrilovich E (2011) Concept-based information retrieval using explicit semantic analysis. ACM Trans Inf Syst 29(2):8:1–8:34

    Article  Google Scholar 

  21. Elleuch N, Ben Ammar A, Alimi A M (2010) A generic system for semantic video indexing by visual concept. In: 2010 5th International Symposium on I/V Communications and Mobile Network (ISVC)

  22. Elleuch N, Zarka M, Ben Ammar A, Alimi MA (2011) A fuzzy ontology: based framework for reasoning in visual video content analysis and indexing. In: Proceedings of the Eleventh International Workshop on Multimedia Data Mining, MDMKDD ’11. New York, pp 1–1

  23. Elleuch N, Zarka M, Feki I, Ben Ammar A, Alimi MA (2010) Regimvid at trecvid 2010: Semantic indexing, TRECVID. 2010

  24. Faria C, Girardi R (2011) An information extraction process for semi-automatic ontology population. In: Corchado E, Snel V, Sedano J , Hassanien A , Calvo J , lzak D (eds) Soft Computing Models in Industrial and Environmental Applications, 6th International Conference SOCO 2011, Advances in Intelligent and Soft Computing, vol 87, pp 319–328. Springer , Berlin Heidelberg

    Google Scholar 

  25. Fellbaum C (2010) Wordnet. In: Poli R, Healy M, Kameas A (eds) Theory and Applications of Ontology: Computer Applications. Springer , Netherlands, pp 231–243

    Chapter  Google Scholar 

  26. Fernndez-López M (1999) Overview of methodologies for building ontologies. In: Proceedings of the IJCAI-99 Workshop on Ontologies and Problem Solving Methods (KRR5) Stockholm, Sweden, August 2, 1999

  27. Fu G, Jones C, Abdelmoty A (2005) Ontology-based spatial query expansion in information retrieval. In: Meersman R, Tari Z (eds) On the Move to Meaningful Internet Systems 2005: CoopIS, DOA, and ODBASE, vol 3761. Springer, Berlin Heidelberg, pp 1466–1482

    Chapter  Google Scholar 

  28. Gargouri F, Jaziri W (2010) Ontology Theory, Management and Design: Advanced Tools and Models. Premier Reference Source. Information Science Reference

  29. Horrocks I (2012) Semantics ; scalability: Journal of Zhejiang University - Science C 13(4) 241–244

  30. Huang YF, Wang SH (2012) Movie genre classification using svm with audio and video features. In: Huang R, Ghorbani A, Pasi G, Yamaguchi T, Yen N, Jin B (eds) Active Media Technology, vol 7669. Springer , Berlin Heidelberg, pp 1–10

    Google Scholar 

  31. Jiang Y G, Wang J, Chang S F, Ngo C W (2009) Domain adaptive semantic diffusion for large scale context-based video annotation. In: IEEE 12th International Conference on Computer Vision, pp 1420 –1427

  32. Kara S (2010) An ontology-absed retrieval system using semantic indexing, Ph.D. thesis, Middle East Technical University

  33. Ksentini N, Zarka M, Ben Ammar A, Alimi MA (2012) Toward an assisted context based collaborative annotation. In: 10th International Workshop on Content-Based Multimedia Indexing (CBMI), 2012, pp 1 –6

  34. Ksibi A, Ben Ammar A, Ben Amar C (2014) Adaptive diversification for tag-based social image retrieval. IJMIR 3(1):29–39

    Google Scholar 

  35. Ksibi A, Dammak M, Ben Ammar A, Mejdoub M, Ben Amar C (2012) Flickr-based semantic context to refine automatic photo annotation. In: Image Processing Theory, Tools and Applications (IPTA), 2012 3rd International Conference on, pp 377–382

  36. Kumar S, Rana RK, Singh P (2012) Ontology based semantic indexing approach for information retrieval system, vol 49, pp 14–18. Published by Foundation of Computer Science, New York, USA

  37. Leite M, Ricarte I (2008) A framework for information retrieval based on fuzzy relations and multiple ontologies. In: Geffner H, Prada R, Machado Alexandre I, David N (eds) Advances in Artificial Intelligence IBERAMIA 2008, vol 5290. Springer , Berlin Heidelberg, pp 292–301

    Chapter  Google Scholar 

  38. Li Z, Ramani K (2007) Ontology-based design information extraction and retrieval. AI EDAM 21:137–154

    Google Scholar 

  39. Mukesh R, Penchala S, Ingale A (2013) Ontology based zone indexing using information retrieval systems. In: Unnikrishnan S, Surve S , Bhoir D (eds) Advances in Computing, Communication, and Control, vol 361. Springer , Berlin Heidelberg, pp 181–186

    Chapter  Google Scholar 

  40. Muneesawang P, Zhang N, Guan L (2014) Scalable video genre classification and event detection. In: Multimedia Database Retrieval, Multimedia Systems and Applications, pp 247–278. Springer International Publishing

  41. Mustafa J, Khan S, Latif K (2008) Ontology based semantic information retrieval. In: 2008 IS ’08. 4th International IEEE Conference Intelligent Systems, vol 3, pp 22–14–22–19

  42. Mylonas P, Athanasiadis T, Wallace M, Avrithis Y, Kollias S (2008) Semantic representation of multimedia content: Knowledge representation and semantic indexing. Multimed Tools Appl 39(3):293–327

    Article  Google Scholar 

  43. Mylonas P, Spyrou E, Avrithis Y, Kollias S (2009) Using visual context and region semantics for high-level concept detection. Multimed, IEEE Trans on 11(2):229–243

    Article  Google Scholar 

  44. Nguyen C T (2010) Bridging semantic gaps in information retrieval: Context-based approaches. In: VLDB doctoral workshop, Singapore 2010

  45. Nikolopoulos S, Papadopoulos G, Kompatsiaris I, Patras I (2009) An evidence-driven probabilistic inference framework for semantic image understanding. In: Perner P (ed) Machine Learning and Data Mining in Pattern Recognition, vol 5632. Springer , Berlin Heidelberg, pp 525–539

    Chapter  Google Scholar 

  46. Nikolopoulos S, Papadopoulos G T, Kompatsiaris I, Patras I (2011) Evidence-driven image interpretation by combining implicit and explicit knowledge in a bayesian network, IEEE Transactions on Systems, Man, and Cybernetics, Part B 41(5),1366–1381

  47. Noy NF, Mcguinness D L (2001) Ontology development 101: A guide to creating your first ontology. Tech. Rep. KSL-01-05, Stanford Knowledge Systems Laboratory

  48. Over P, Awad G, Michel M, Fiscus J, Sanders G, Kraaij W, Smeaton AF, Quenot G (2013) Trecvid 2013 – an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID 2013. NIST USA

  49. Paliouras G, Spyropoulos C, Tsatsaronis G (2011) Bootstrapping ontology evolution with multimedia information extraction. In: Paliouras G , Spyropoulos C , Tsatsaronis G (eds) Knowledge-Driven Multimedia Information Extraction and Ontology Evolution, vol 6050 . Springer, Berlin Heidelberg, pp 1–17

    Chapter  Google Scholar 

  50. Paliouras G, Spyropoulos CD, Tsatsaronis G (2011) Bootstrapping ontology evolution with multimedia information extraction. Lect Notes in Comput Sci 6050

  51. Paliouras G, Spyropoulos C D, Tsatsaronis G (2011) Knowledge-Driven Multimedia Information Extraction and Ontology Evolution - Bridging the Semantic Gap, vol 6050. Springer

  52. Park S, Aggarwal J (2004) A hierarchical bayesian network for event recognition of human actions and interactions. Multimedia Systems 10(2):164–179

    Article  Google Scholar 

  53. Perpetual Coutinho F, Asnani K, Amos Caeiro D (2012) Context based information retrieval. International Journal of Advanced Research in Computer Science and Electronics Engineering (IJARCSEE) 1(7)

  54. Petasis G, Karkaletsis V, Paliouras G, Krithara A, Zavitsanos E (2011) Ontology population and enrichment: State of the art

  55. Petersohn C (2004) Fraunhofer hhi at trecvid 2004: Shot boundary detection system. In: TREC Video Retrieval Evaluation Online Proceedings, TRECVID

  56. Petridis K, Bloehdorn S, Saathoff C, Simou N, Dasiopoulou S, Tzouvaras V, Handschuh S, Avrithis Y, Kompatsiaris Y, Staab S (2006) Knowledge representation and semantic annotation of multimedia content. Vision, Image and Signal Processing. IEE Proceedings - 153(3):255–262

  57. Rodrguez-Garca M, Valencia-Garca R , Garca-Snchez F (2012) An ontology evolution-based framework for semantic information retrieval. In: Herrero P, Panetto H, Meersman R, Dillon T (eds) On the Move to Meaningful Internet Systems: OTM 2012 Workshops, vol 7567 . Springer , Berlin Heidelberg, pp 163–172

    Chapter  Google Scholar 

  58. Romero AA, Grau BC, Horrocks I, Jiménez-Ruiz E (2013) More: a modular owl reasoner for ontology classification. In: Bail S , Glimm B, Gonçalves R S, Jiménez-Ruiz E, Kazakov Y , Matentzoglu N, Parsia B (eds) ORE, CEUR Workshop Proceedings, vol 1015, pp 61–67. CEUR-WS.org

  59. Rozilawati binti D, Masaki A (2011) Ontology based approach for classifying biomedical text abstracts. International Journal of Data Engineering 2(1)

  60. Sanjaa B, Tsoozol P (2007) Fuzzy and probability. In: Strategic Technology, 2007. IFOST 2007. International Forum on, pp. 141–143

  61. Sari RF, Ayuningtyas N (2010) Implementation of web ontology and semantic application for electronic journal citation system. Journal Of Emerging Technologies in Web Intelligence 2:34–41

    Article  Google Scholar 

  62. Simou N, Kollias S (2007) Fire: A fuzzy reasoning engine for impecise knowledge. K-Space PhD Students Workshop, Berlin, Germany, 14 September 2007

  63. Smeaton AF, Over P, Kraaij W (2006) Evaluation campaigns and trecvid. In: Proceedings of the 8th ACM international workshop on Multimedia information retrieval, MIR ’06, pp 321–330, ACM, New York, NY, USA

  64. Snoek CGM, Worring M (2009) Concept-based video retrieval. Foundations and Trends in Information Retrieval 2(4):215–322

    Article  Google Scholar 

  65. Staab S, Studer R (2009) Handbook on Ontologies, 2nd edn. Springer Publishing Company, Incorporated

  66. Stoilos G, Stamou GB, Tzouvaras V, Pan JZ, Horrocks I (2005) The fuzzy description logic f-shin. International Workshop on Uncertainty Reasoning For the Semantic Web (2005)

  67. Thomee B, Popescu A (2012) Overview of the imageclef 2012 flickr photo annotation and retrieval task. In: Forner P , Karlgren J, Womser-Hacker C (eds) CLEF (Online Working Notes/Labs/Workshop)

  68. Vallet D, Castells P, Fernandez M, Mylonas P, Avrithis Y (2007) Personalized content retrieval in context using ontological knowledge. Circuits and Systems for Video Technology. IEEE Transactions on 17(3):336–346

    Google Scholar 

  69. Wu F, Wu G, Fu X (2008) Design and implementation of ontology-based query expansion for information retrieval. In: Xu L, Tjoa A, Chaudhry S (eds) Research and Practical Issues of Enterprise Information Systems II, vol 254, pp 293–298. Springer US,

  70. Wu J, Worring M (2012) Efficient genre-specific semantic video indexing. Multimedia, IEEE Transactions on 14(2 ):291–302 . doi:10.1109/TMM.2011.2174969

    Article  Google Scholar 

  71. Zadeh L (2014) Fuzzy set theory and probability theory: What is the relationship? In: Lovric M (ed) International Encyclopedia of Statistical Science. Springer , Berlin Heidelberg, pp 563–566

    Google Scholar 

  72. Zarka M, Ben Ammar A, Alimi M A (2011) Multimodale fuzzy fusion for semantic video indexing. In: IEEE Symposium Series in Computational Intelligence 2011 - CIMSIVP

  73. Zhai J, Li M, Li J (2012) Semantic information retrieval based on rdf and fuzzy ontology for university scientific research management. In: Luo J (ed) Affective Computing and Intelligent Interaction, vol 137. Springer , Berlin Heidelberg, pp 661–668

    Chapter  Google Scholar 

Download references

Acknowledgment

The authors would like to acknowledge the financial support of this work by grants from General Direction of Scientific Research (DGRST), Tunisia, under the ARUB program.

Author information

Authors and Affiliations

  1. REGIM: Reaserch Groups on Intelligent Machines, University of Sfax, ENIS, BP 1173, Sfax, 3038, Tunisia

    Mohamed Zarka, Anis Ben Ammar & Adel M. Alimi

Authors
  1. Mohamed Zarka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anis Ben Ammar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adel M. Alimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohamed Zarka.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zarka, M., Ben Ammar, A. & Alimi, A.M. Fuzzy reasoning framework to improve semantic video interpretation. Multimed Tools Appl 75, 5719–5750 (2016). https://doi.org/10.1007/s11042-015-2537-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2537-1

Keywords

Search

Navigation