Skip to main content
SpringerLink
Log in

An MPEG-7 scheme for semantic content modelling and filtering of digital video

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Part 5 of the MPEG-7 standard specifies Multimedia Description Schemes (MDS); that is, the format multimedia content models should conform to in order to ensure interoperability across multiple platforms and applications. However, the standard does not specify how the content or the associated model may be filtered. This paper proposes an MPEG-7 scheme which can be deployed for digital video content modelling and filtering. The proposed scheme, COSMOS-7, produces rich and multi-faceted semantic content models and supports a content-based filtering approach that only analyses content relating directly to the preferred content requirements of the user. We present details of the scheme, front-end systems used for content modelling and filtering and experiences with a number of users.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Day, Y.F., Khokhar, A., Dagtas, S., Ghafoor, A.: A multi-level abstraction and modeling in video databases. Multimedia Syst. 7(5), 409–423 (1999)

    Article  Google Scholar 

  2. Al-Safadi, L., Getta, J.: Semantic content-based retrieval for video documents. In: Rahman, S.M. (ed.) Design and Management of Multimedia Information Systems: Opportunities & Challenges, pp. 165–200. IDEA Group Publishing, Hershey, PA (2001)

    Google Scholar 

  3. Löffler, J., Biatov, K., Eckes, C., Köhler, J.: iFinder: an MPEG-7-based retrieval system for distributed multimedia content. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 431–435. Juan-les-Pins, France (2002)

  4. Jaimes, A., Omura, K., Nagamine, T., Hirata, K.: Memory cues for meeting video retrieval. In: Proceedings of ACM CARPE'04, pp. 74–85, New York, NY, USA (2004)

  5. Bolle, R.M., Yeo, B.-L., Yeung, M.M.: Video query: research directions. IBM J. Res. Dev. 42(2) (1998)

  6. Naphande, M.R., Huang, T.S.: A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans. Multimedia 3(1), 141–151 (2001)

    Article  Google Scholar 

  7. Davis, M.: Media Streams: An iconic visual language for video annotation. Telektronikk 4.93, 59–71 (1993)

    Google Scholar 

  8. Tusch, R., Kosch, H., Böszörmenyi, L.: VIDEX: an integrated generic video indexing approach. In: Proceedings of the ACM Multimedia'00, pp. 448–451. Los Angeles, CA (2000)

  9. Vendrig, J.W., M.: Interactive adaptive movie annotation. In: Proceedings of the 2002 IEEE International Conference on Multimedia and Expo, vol. 1, pp. 93–96 (2002)

  10. Zhao, R., Grosky, W.I.: Bridging the semantic gap in image retrieval. In: Shih, T.K. (ed.). Distributed Multimedia Databases: Techniques and Applications, pp. 14–36. IDEA Group Publishing, Hershey, PA (2002)

    Google Scholar 

  11. Rowe, L.A., Boreczky, J.S., Eads, C.A.: Indices for user access to large video databases. In: Proceedings of the Storage and Retrieval for Image and Video Database II, Proceedings of the SPIE, vol. 2185, pp. 150–161 (1994)

  12. Hartley, E., Parkes, A.P., Hutchison, A.D.: A conceptual framework to support content-based multimedia applications. Lecture Notes in Computer Science 1629 (2000)

  13. Adami, N., Bugatti, A., Leonardi, R., Migliorati, P., Rossi, L.A.: The ToCAI description scheme for indexing and retrieval of multimedia documents. Multimedia Tools Appl. 14(2), 153–173 (2001)

    Article  MATH  Google Scholar 

  14. Golshani, F., Dimitrova, N.: A language for content-based video retrieval. Multimedia Tools Appl. 6(3), 289–312 (1998)

    Article  Google Scholar 

  15. ISO/IEC: Information Technology −Multimedia Content Description Interface – Part 5: Multimedia Description Schemes. Geneva, Switzerland, International Organisation for Standardisation (2002)

  16. Tseng, B.L., Lin, C.-Y., Smith, J.R.: Using MPEG-7 and MPEG-21 for personalizing video. IEEE Multimedia 11(1), 42–53 (2004)

    Article  Google Scholar 

  17. Correia, P., Pereira, F.: The role of analysis in content-based video coding and indexing. Signal Process. 66, 125–142, (1998)

    Article  MATH  Google Scholar 

  18. Kosch, H.: MPEG-7 and multimedia database systems. ACM SIGMOD Record 31(2), 34–39 (2002)

    Article  Google Scholar 

  19. Westermann, U., Klas, W.: An analysis of XML database solutions for the management of MPEG-7 media descriptions. ACM Comput. Surveys 35(4), 331–373 (2003)

    Article  Google Scholar 

  20. Kang, J.-H., Kim, C.-S., Ko, E.-J.: An XQuery engine for digital library systems. In: Proceedings of the 3rd ACM/IEEE-CS Joint Conference on Digital Libraries, p. 400. Houston, TX (2003)

  21. Chu, Y., Chia, L.-T., Bhowmick, S.S.: Looking at mapping, indexing & querying of MPEG-7 descriptors in RDBMS with SM3. In: Proceedings of ACM MMDB '04, pp. 55–64. Washington, DC, USA (2004)

  22. Döller, M., Kosch, H., Dörflinger, B., Bachlechner, A., Blaschke, G.: Demonstration of an MPEG-7 multimedia data cartridge. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 85–86. Juan-les-Pins, France (2002)

  23. Lin, C.-Y., Tseng, B.L., Naphade, M., Natsev, A., Smith, J.R.: MPEG-7 video automatic labeling system. In: Proceedings of the ACM MM'03, pp. 98–99. Berkeley, CA, USA (2003)

  24. Lin, C.-Y., Tseng, B.L., Naphade, M., Natsev, A., Smith, J.R.: VideoAL: a novel end-to-end MPEG-7 video automatic labeling system. In: Proceedings of the IEEE International Conference on Image Processing 2003, vol. 3, pp. III-53–56. Barcelona, Spain, IEEE Press, Piscataway, NJ (2003)

  25. Tseng, B.L., Ching-Yung, L., Smith, J.R.: Video personalization and summarization system. In: Proceedings of the IEEE Workshop on Multimedia Signal Processing, pp. 424–427 (2002)

  26. IBM: IBM VideoAnnEx website. http://www.research.ibm.com/VideoAnnEx/ (2005)

  27. Ryu, J., Sohn, Y., Kim, M.: MPEG-7 metadata authoring tool. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 267–270. Juan-les-Pins, France (2002)

  28. Lee, J.-H., Kim, H.-J., Kim, W.-Y.: Video image retrieval system based on MPEG-7 (VIRS). In: Proceedings of the International Conference on Information Technology: Research and Education (ITRE '03), pp. 79–83 (2003)

  29. Benitez, A.B., Chang, S.-F.: Extraction, description and application of multimedia using MPEG-7. In: Proceedings of the 37th Asilomar Conference on Signals, Systems & Computers, pp. 92–96. IEEE Press (2003)

  30. Tran-Thuong, T., Roisin, C.: Multimedia modeling using MPEG-7 for authoring multimedia integration. In: Proceedings of the 5th ACM SIGMM International Workshop on Multimedia Information Retrieval, pp. 171–178. Berkeley, CA, ACM Press, New York, NY (2003)

  31. Goularte, R., Cattelan, R.G., Camacho-Guerrero, J.A., Inácio Jr., V.R., Pimentel, M.D.G.C.: Interactive multimedia annotations: enriching and extending content. In: Proceedings of DocEng'04, pp. 84–86. Milwaukee, Wisconsin, USA (2004)

  32. Costa, M., Correia, N., Guimarães, N.: Annotations as multiple perspectives of video content. In: Proceedings of ACM Multimedia'02, pp. 283–286. Juan-les-Pins, France (2002)

  33. Eidenberger, H.: Distance Measures for MPEG-7-based Retrieval. In: Proceedings of ACM MIR'03, pp. 130–137. Berkeley, CA, USA (2003)

  34. Goularte, R., Moreira, E.D.S., Pimentel, M.D.G.C.: Structuring interactive TV documents. In: Proceedings of DocEng'03, pp. 42–51. Grenoble, France (2003)

  35. Pfeiffer, S., Srinivasan, U.: TV Anytime as an application scenario for MPEG-7. In: Proceedings of the ACM Multimedia Workshop, pp. 89–92. Marina Del Rey, CA (2000)

  36. Salembier, P., Qian, R., O'Connor, N., Correia, P., Sezan, I., van Beek, P.: Description schemes for video programs, users and devices. Signal Process.: Image Commun. 16, 211–234 (2000)

    Article  Google Scholar 

  37. Rehm, E.: Representing internet streaming media metadata using MPEG-7 multimedia description schemes. In: Proceedings of the ACM Multimedia Workshop, pp. 93–98. Marina Del Rey, CA, USA (2000)

  38. van Setten, M., Oltmans, E.: Demonstration of a distributed MPEG-7 video search and retrieval application in the educational domain. In: Proceedings of ACM Multimedia'01, pp. 595–596. Ottawa, Canada (2001)

  39. Vakali, A., Hacid, M.-S., Elmagarmid, A.: MPEG-7 based description schemes for multi-level video content classification. Image Vis. Comput. 22, 367–378 (2004)

    Article  Google Scholar 

  40. Kuo, P.-J., Aoki, T., Yasuda, H.: Building personal digital photograph libraries: An approach with ontology-based MPEG-7 dozen dimensional digital content architecture. In: Proceedings of IEEE Computer Graphics International (CGI'04) (2004)

  41. Hunter, J.: Enhancing the semantic interoperability of multimedia through a core ontology. IEEE Trans. Circuits Syst. Video Technol. 13(1), 49–58 (2003)

    Article  Google Scholar 

  42. Graves, A., Lalmas, M.: Video retrieval using an MPEG-7 based inference network. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, pp. 339–346. Finland, ACM Press, New York, NY (2002)

  43. Magalhães, J., Pereira, F.: Using MPEG standards for multimedia customization. Signal Process.: Image Commun. 19, 437–456 (2004)

    Article  Google Scholar 

  44. Martínez, J.M., González, C., Fernández, O., Garcia, C., de Ramón, J.: Towards universal access to content using MPEG-7. In: Proceedings of the 10th ACM International Conference on Multimedia (MM02), pp. 199–202. Juan-les-Pins, France (2002)

  45. Ferman, A.M., Beek, J.H.E.P.V., Sezan, M.I.: Content-based filtering and personalization using structured metadata. In: Proceedings of the 2nd ACM/IEEE-CS Joint Conference on Digital Libraries, p. 393. Portland, Oregon (2002)

  46. Ferman, A.M., van Beek, P., Errico, J.H., Sezan, M.I.: Multimedia content recommendation engine with automatic inference of user preferences. In: Proceedings of the IEEE International Conference on Image Processing, vol. 3, pp. 49–52 (2003)

  47. Rovira, M., González, J., López, A., Mas, J., Puig, A., Fabregat, J., Fernàndez, G.: IndexTV: A MPEG-7 Based Personalized Recommendation System for Digital TV. In: Proceedings of the 2004 IEEE International Conference on Multimedia and Expo (ICME), pp. 823–826 (2004)

  48. Tseng, B.L., Lin, C.-Y., Smith, J.R.: Video personalization and summarization system for usage environment. J. Vis. Commun. Image Recognit. 15, 370–392 (2004)

    Article  Google Scholar 

  49. Echigo, T., Masumitsu, K., Teraguchi, M., Etoh, M., Sekihuchi, S.: Personalized delivery of digest video managed on MPEG-7. In: Proceedings of the 2001 International Conference on Information Technology: Coding and Computing, pp. 216–220. Las Vegas, NV, USA, IEEE Press, Piscataway, NJ (2001)

  50. Jaimes, A., Echigo, T., Teraguchi, M., Satoh, F.: Learning personalized video highlights from detailed MPEG-7 metadata. In: Proceedings of the 2002 IEEE International Conference on Image Processing (ICIP-02), vol. 1, pp. 133–136. Rochester, NY, IEEE Press, Piscataway, NJ (2002)

  51. Fonseca, P.M., Pereira, F.: Automatic video summarization based on MPEG-7 descriptions. Signal Process.: Image Commun. 19, 685–699 (2004)

    Article  Google Scholar 

  52. Agius, H.W., Angelides, M.C.: COSMOS – Content oriented semantic modelling overlay scheme. Comput. J. 42(3), 153–176 (1999)

    Article  Google Scholar 

  53. Agius, H.W., Angelides, M.C.: A method for developing interactive multimedia from their semantic content. Data Knowledge Eng. 34(2), 165–187 (2000)

    Article  MATH  Google Scholar 

  54. Agius, H.W., Angelides, M.C.: Modelling content for semantic-level querying of multimedia. Multimedia Tools Appl. 15(1), 5–37 (2001)

    Article  MATH  Google Scholar 

  55. Appan, P., Sundaram, H.: Networked multimedia event exploration. In: Proceedings of the ACM Multimedia'04, pp. 40–47. New York, NY (2004)

  56. Dasiopoulou, S., Papastathis, V.K., Mezaris, V., Kompatsiaris, I., Strintzis, M.G.: An ontology framework for knowledge-assisted semantic video analysis and annotation. In: Proceedings of the 4th International Workshop on Knowledge Markup and Semantic Annotation (SemAnnot 2004) at the 3rd International Semantic Web Conference (ISWC 2004) (2004)

  57. Adali, S., Candan, K.S., Chen, S.-S., Erol, K., Subrahmanian, V.S.: The advanced video information system: Data structures and query processing. Multimedia Syst. 4(4), 172–186 (1996)

    Article  Google Scholar 

  58. Davis, M., King, S., Good, N., Sarvas, R.: From context to content: leveraging context to infer media metadata. In: Proceedings of the ACM Multimedia'04, pp. 188–195. New York, NY (2004)

  59. Goldfarb, C.S.: HyTime: A standard for structured hypermedia interchange. Computer 24(8), 81–84 (1991)

    Article  Google Scholar 

  60. Newcomb, S.R., Kipp, N.A., Newcomb, V.T.: The “HyTime” hypermedia/time-based document structuring language. Commun. ACM 34(11), 67–83 (1991)

    Article  Google Scholar 

  61. Newcomb, S.R.: Multimedia interchange using SGML/HyTime: Part II: Principles and Applications. IEEE Multimedia 2(3), 60–64 (1995)

    Article  MathSciNet  Google Scholar 

  62. Kretz, F., Colaïtis, F.: Standardizing hypermedia information objects. IEEE Commun. Mag. 30(5), 60–70 (1992)

    Article  Google Scholar 

  63. Meyer-Boudnik, T., Effelsberg, W.: MHEG explained. IEEE Multimedia 2(1), 26–38 (1995)

    Article  Google Scholar 

  64. Vieira, M.T.P., Santos, M.T.P.: Content-based search on an MHEG-5 standard-based multimedia database. In: Proceedings of the Eighth International Workshop on Database and Expert Systems Applications, pp. 154–159. Toulouse, France (1997)

  65. Echiffre, M., Marchisio, C., Marchisio, P., Panicciari, P., Del Rossi, S.: MHEG-5-aims, concepts, and implementation issues. IEEE Multimedia 5(1), 84–91 (1998)

    Article  Google Scholar 

  66. Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983)

    Article  MATH  Google Scholar 

  67. NIST: NIST MPEG-7 Validation Service. http://m7itb.nist.gov/M7Validation.html (2005)

  68. Wenyin, L., Chen, Z., Lin, F., Zhang, H., Ma, W.-Y.: Ubiquitous media agents: a framework for managing personally accumulated multimedia files. Multimedia Syst. 9(2), 144–156 (2003)

    Article  Google Scholar 

  69. Wallace, M.S.G.: Towards a context aware mining of user interests for consumption of multimedia documents. In: Proceedings of the 2002 IEEE International Conference on Multimedia and Expo, vol. 1, pp. 733–736 (2002)

  70. Angelides, M.C.: Multimedia content modelling and personalization. IEEE Multimedia 10(4), 12–15 (2003)

    Article  Google Scholar 

  71. Eirinaki, M., Vazirgiannis, M.: Web mining for web personalization. ACM Trans. Internet Technol. 3(1), 1–27 (2003)

    Article  Google Scholar 

  72. Kuflik, T., Shoval, P.: Generation of user profiles for information filtering – research agenda. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 313–315 (2000)

  73. van Meteren, R., van Someren, M.: Using content-based filtering for recommendation. In: Proceedings of the Machine Learning in the New Information Age: MLnet/ECML2000 Workshop, Barcelona, Spain (2000)

  74. Good, N., Schafer, J., Konstan, J., Borchers, A., Sarwar, B., Herlocker, J., Riedl, J.: Combining collaborative filtering with personal agents for better recommendations. In: Proceedings of the 16th National Conference on Artificial Intelligence, pp. 439–446 (1999)

  75. Specht, T.K.G., Kahabka, T.: Information filtering and personalisation in databases using Gaussian curves. In: Proceedings of the IEEE Databases Engineering and Applications Symposium. Yokohama, Japan, (2000)

  76. Lienhart, R., Pfeiffer, S., Effelsberg, W.: The MoCA workbench: Support for creativity in movie content analysis. In: Proceedings of the 3rd IEEE International Conference on Multimedia Computing and Systems, pp. 314–321. Hiroshima, Japan (1996)

  77. Okoli, C., Pawlowski, S.D.: The Delphi method as a research tool: an example, design considerations and applications. Inf. Manage. 42(1), 15–29 (2004)

    Google Scholar 

  78. Rowe, G., Wright, G., McColl, A.: Judgment change during Delphi-like procedures: the role of majority influence, expertise, and confidence. Technol. Forecasting Soc. Change 72(4), 377–399 (2005)

    Article  Google Scholar 

  79. Ambler, S.W.: Agile Modelling: Effective Practices for eXtreme Programming and the Unified Process John Wiley & Sons, New York, NY (2002)

    Google Scholar 

  80. Germain, É., Robillard, P.N.: Engineering-based processes and agile methodologies for software development: a comparative case study. J. Syst. Softw. 75(1/2), 17–27 (2005)

    Article  Google Scholar 

  81. Vetro, A.: MPEG-21 digital item adaptation: Enabling universal multimedia access. IEEE Multimedia 11(1), 84–87 (2004)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Systems, Computing and Mathematics, Brunel University, St John's, Uxbridge, Middlesex, UB8 3PH, UK

    Marios C. Angelides & Harry Agius

Authors
  1. Marios C. Angelides
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Harry Agius
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marios C. Angelides.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Angelides, M.C., Agius, H. An MPEG-7 scheme for semantic content modelling and filtering of digital video. Multimedia Systems 11, 320–339 (2006). https://doi.org/10.1007/s00530-006-0012-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-006-0012-6

Keywords

Search

Navigation