Marco Bertini
Alberto Del Bimbo
Carlo Torniai
ABSTRACT
In this paper, we present the dynamic pictorial ontology paradigm for video annotation. Ontologies are often used to describe a given domain for different goals, including description of multimedia data. In the case of video annotation, the visual knowledge cannot be described using only abstract concepts but is more effectively represented in a visual form. To this aim, we introduce visual concepts, elicited from the data set as the most representative prototypes that specialize abstract concepts. The ontology created is intrinsically dynamic since it must embrace the perceptual and visual experience during annotation. Thus visual concepts can change, adapting to the multimedia content analyzed. Motivation for this new ontology paradigm are discussed together with a proposal of a framework for ontology creation, maintenance, and automatic annotation of video. The creation and usage of dynamic pictorial ontologies have been tested for soccer domain exploiting low level perceptual features and higher level domain features.
- S. Dasiopoulou, V. K. Papastathis, V. Mezaris, I. Kompatsiaris, M. G. Strintzis, "An Ontology Framework For Knowledge-Assisted Semantic Video Analysis and Annotation", Proc.of 4th International Workshop on SemAnnot at the 3rd International Semantic Web Conference, Hiroshima, Japan, 2004.Google Scholar
- S. Dasiopoulou, V. Mezaris, I. Kompatsiaris, V. K. Papastathis M. G. Strintzis, "Knowledge-Assisted Semantic Video Object Detection", IEEE Transactions on Circuits and Systems for Video Technology, Special Issue on Analysis and Understanding for Video Adaptation, vol. 15, no. 10, pp. 1210--1224, October 2005 Google ScholarDigital Library
- C. Tsinaraki, P. Polydoros, S. Christodoulakis, "Interoperability support for Ontology-based Video Retrieval Applications", Proc of Conference on Image and Video Retrieval (CIVR), 2004Google Scholar
- A. Jaimes, J.R. Smith, "Semi-automatic, data-driven construction of multimedia ontologies", Proc. of International Conference on Multimedia and Expo (ICME), 2003. Google ScholarDigital Library
- T. h. Athanasiadis, V. Tzouvaras, K. Petridis, F. Precioso, Y. Avrithis Y. Kompatsiaris, "Using a Multimedia Ontology Infrastructure for Semantic Annotation of Multimedia Content", Proc. of 5th International Workshop on Knowledge Markup and Semantic Annotation (SemAnnot '05), Galway, Ireland, November 2005Google Scholar
- "Towards a common multimedia ontology framework" analysis of the contribution 27/4/06 report on ACEMedia projectGoogle Scholar
- "Ontological and Epistemological Foundations", report on DELOS project http://www.idi.ntnu.no/grupper/su/publ/html/totland/ch032.htmGoogle Scholar
- R. Hirschheim, H. Klein, K. Lyytinen, "Information Systems Development and Data Modeling - Conceptual and Philosophical Foundations", Cambridge University Press, Cambridge, UK, 289 pages, 1995 Google ScholarDigital Library
- E. G. Guba, Y. S. Lincoln, "Competing Paradigms in Qualitative Research", in (Denzin and Lincoln, 1994), pp. 105--117, 1994Google Scholar
- L. Wittgenstein, "Tractatus Logico-Philosophicus", Hypertext Translated from the German by C.K. Ogden http://kfs.org/~jonathan/witt/tlph.html 200 Google ScholarDigital Library
- Y. Zhai, J. Liu, X. Cao, A. Basharat, A. Hakeem, S. Ali, M. Shah, C. Grana, R.Cucchiara, "Video Understanding and Content-based Retrieval", TREC Video Retrieval Evaluation Online Proceedings, http://wwwnlpir.nist.gov/projects/tvpubs/tv.pubs.org.html, 2005Google Scholar
- C. Grana, R. Cucchiara, "Linear Transition Detection as a Unified Shot Detection Approach" in IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, n. 4, pp. 483--489, 2007 Google ScholarDigital Library
- M. Bertini, R. Cucchiara, A. Del Bimbo C. Torniai, "Video Annotation with Pictorially Enriched Ontologies". In Proc. of International Conference on Multimedia and Expo (ICME), 2005.Google Scholar
- C. Grana, R.Vezzani, R. Cucchiara, "Enhancing HSV Histograms with achromatic points detection in video retrieval", Proc. of Conference on Image and Video Retrieval (CIVR), 2007 Google ScholarDigital Library
- D. Zhou, J. Li, H. Zha, "A new Mallows distance based metric for comparing clusterings", Proc. of the 22nd international conference on Machine Learning, Bonn, Germany, 2005 Google ScholarDigital Library
- M. Bertini, A. Del Bimbo C.Torniai, "Automatic Video Annotation using Ontologies Extended with Visual Information", Proc. of ACM Multimedia, November 2005 Google ScholarDigital Library
- J. C. Bezdek, "Pattern Recognition with Fuzzy Objective Function Algorithms", Plenum Press, New York, 1981 Google ScholarDigital Library
- A. Jaimes, B. Tseng, J. Smith, "Modal keywords, Ontologies, and Reasoning for Video Understanding" in Proc. of Conference on Image and Video Retrieval (CIVR), July 2003 Google ScholarDigital Library
- A. Benitez, S.-F. Chang, "Automatic Multimedia Knowledge Discovery, Summarization and Evaluation" IEEE Transactions on Multimedia, SubmittedGoogle Scholar
- J. Kender, M. Naphade, "Visual Concepts for News Story Tracking: Analyzing and Exploiting the NIST TRECVID Video Annotation Experiment" in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 1174--1181, 2005 Google ScholarDigital Library
- M. Naphade, J. Smith, J. Tesic, S. Chang, L. Kennedy, A. Hauptmann, J. Curtis, "Large-scale Concepts Ontology for Multimedia", IEEE Multimedia, vol. 13, no.3, pp. 86--91, July-Sept 2006 Google ScholarDigital Library
- D. Lenat, R. Guha, "Building Large Knowledge-based Systems: Representation and Inference in the Cyc Project". Reading, MA (USA): Addison-Wesley, 1990. Google ScholarDigital Library
- J. Strintzis, S. Bloehdorn, S. Handschuh, S. Staab, N. Simou, V. Tzouvaras, K. Petridis, I. Kompatsiaris, Y. Avrithis, "Knowledge representation for semantic multimedia content analysis and reasoning," in Proc. of European Workshop on the Integration of Knowledge, Semantics and Digital Media Technology, Nov. 2004.Google Scholar
- S. Vembu, M. Kiesel, M. Sintek, S. Bauman, "Towards bridging the semantic gap in multimedia annotation and retrieval," in Proc. of First International Workshop on Semantic Web Annotations for Multimedia (SWAMM), Edinburgh (Scotland), May 2006.Google Scholar
- V. Mezaris, I. Kompatsiaris, N. Boulgouris, M. Strintzis, "Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval," IEEE Transactions on Circuits and Systems for Video Technology, vol. 14,no. 5, pp. 606--621, 2004. Google ScholarDigital Library
- N. Simou, C. Saathoff, S. Dasiopoulou, E. Spyrou, N. Voisine, V. Tzouvaras, I. Kompatsiaris, Y. Avrithis, S. Staab, "An ontology infrastructure for multimedia reasoning," in Proc. International Workshop VLBV, Sardinia (Italy), September, 2005. Google ScholarDigital Library
- D. Kosmopoulos, S. Petridis, I. Pratikakis, V. Gatos, S. Perantonis, V. Karkaletsis, G. Paliouras, "Knowledge Acquisition from Multimedia Content using an Evolution Framework," in Proc. of 3rd IFIP Conference on Artificial Intelligence Applications & Innovations (AIAI), June, 2006.Google Scholar
- K. Petridis, S. Bloehdorn, C. Saathoff, N. Simou, S. Dasiopoulou, V. Tzouvaras, S. Handschuh, Y. Avrithis, I. Kompatsiaris, S. Staab, "Knowledge Representation and Semantic Annotation of Multimedia Content," IEE Proceedings on Vision Image and Signal Processing, Special issue on Knowledge-based Digital Media Processing, vol. 153, no. 3, pp. 255--262, June 2006.Google ScholarCross Ref
- C. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, M. Worring, "Adding semantics to detectors for video retrieval," IEEE Transactions on Multimedia, vol. 9, no. 5, August, 2007. Google ScholarDigital Library
- C. Grana, D. Bulgarelli, R. Cucchiara, "Video clip clustering for assisted creation of mpeg-7 pictorially enriched ontologies," in Proc. Second International Symposium on Communications, Control and Signal Processing (ISCCSP), Marrakech, Morocco, March 2006.Google Scholar
Index Terms
- Dynamic pictorial ontologies for video digital libraries annotation
Recommendations
Multimedia ontology learning for automatic annotation and video browsing
MIR '08: Proceedings of the 1st ACM international conference on Multimedia information retrievalIn this work, we offer an approach to combine standard multimedia analysis techniques with knowledge drawn from conceptual metadata provided by domain experts of a specialized scholarly domain, to learn a domain-specific multimedia ontology from a set ...
RDF-powered semantic video annotation tools with concept mapping to Linked Data for next-generation video indexing: a comprehensive review
Video annotation tools are often compared in the literature, however, most reviews mix unstructured, semi-structured, and the very few structured annotation software. This paper is a comprehensive review of video annotations tools generating structured ...
Video Annotation and Retrieval Using Ontologies and Rule Learning
An approach for automatic annotation and retrieval of video content uses semantic concept classifiers and ontologies to permit expanded queries to synonyms and concept specializations.
Comments