Klaus Schoeffmann
Werner Bailer
ABSTRACT
In this tutorial we discuss interactive video search tools and methods, review their need in the age of deep learning, and explore video and multimedia search challenges and their role as evaluation benchmarks in the field of multimedia information retrieval. We cover three different campaigns (TRECVID, Video Browser Showdown, and the Lifelog Search Challenge), discuss their goals and rules, and present their achieved findings over the last half-decade. Moreover, we talk about datasets, tasks, evaluation procedures, and examples of interactive video search tools, as well as how they evolved over the years. Participants of this tutorial will be able to gain collective insights from all three challenges and use them for focusing their research efforts on outstanding problems that still remain unsolved in this area.
- George Awad, Asad Butt, Jonathan Fiscus, Martial Michel, David Joy, Wessel Kraaij, Alan F. Smeaton, Georges Quénot, Maria Eskevich, Roeland Ordelman, Gareth J. F. Jones, and Benoit Huet. 2017. TRECVID 2017: Evaluating Ad-hoc and Instance Video Search, Events Detection, Video Captioning and Hyperlinking. In Proceedings of TRECVID 2017. NIST, USA.Google Scholar
- Werner Bailer and Herwig Rehatschek. 2009. Comparing Fact Finding Tasks and User Survey for Evaluating a Video Browsing Tool. In Proceedings of ACM Multimedia. Beijing, CN. Google ScholarDigital Library
- Nancy A Chinchor, James J Thomas, Pak Chung Wong, Michael G Christel, and William Ribarsky. 2010. Multimedia analysis+ visual analytics= multimedia analytics. IEEE computer graphics and applications , Vol. 30, 5 (2010), 52--60. Google ScholarDigital Library
- Claudiu Cobârzan, Klaus Schoeffmann, Werner Bailer, Wolfgang Hürst, Adam Blavz ek, Jakub Lokovc, Stefanos Vrochidis, Kai Uwe Barthel, and Luca Rossetto. 2017. Interactive video search tools: a detailed analysis of the video browser showdown 2015. Multimedia Tools and Applications , Vol. 76, 4 (2017), 5539--5571. Google ScholarDigital Library
- Duc-Tien Dang-Nguyen, Klaus Schoeffmann, and Wolfgang Hurst. 2018. LSE2018 Panel - Challenges of Lifelog Search and Access. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge (LSC '18). ACM, New York, NY, USA, 1--2. Google ScholarDigital Library
- Aaron Duane, Cathal Gurrin, and Wolfgang Huerst. 2018. Virtual Reality Lifelog Explorer: Lifelog Search Challenge at ACM ICMR 2018. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge (LSC '18). ACM, New York, NY, USA, 20--23. Google ScholarDigital Library
- Christoph Kofler, Martha Larson, and Alan Hanjalic. 2016. User intent in multimedia search: a survey of the state of the art and future challenges. ACM Computing Surveys (CSUR) , Vol. 49, 2 (2016), 36. Google ScholarDigital Library
- Martha Larson, Mohammad Soleymani, Guillaume Gravier, Bogdan Ionescu, and Gareth JF Jones. 2017. The benchmarking initiative for multimedia evaluation: MediaEval 2016. IEEE MultiMedia , Vol. 24, 1 (2017), 93--96.Google ScholarCross Ref
- J. Lokovc, W. Bailer, K. Schoeffmann, B. Muenzer, and G. Awad. 2018a. On influential trends in interactive video retrieval: Video Browser Showdown 2015--2017. IEEE Transactions on Multimedia (2018), 1--1.Google Scholar
- Jakub Lokovc, Gregor Kovalvc'i k, and Tomá vs Souvcek. 2018b. Revisiting SIRET Video Retrieval Tool. In MultiMedia Modeling - 24th International Conference, MMM 2018, Bangkok, Thailand, February 5--7, 2018, Proceedings, Part II. 419--424.Google Scholar
- Jakub Lokovc , Tomá vs Souvcek, and Gregor Kovalvc'i k. 2018c. Using an Interactive Video Retrieval Tool for LifeLog Data. In Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge, LSC@ICMR 2018, Yokohama, Japan, June 11, 2018. 15--19. Google ScholarDigital Library
- Bernd Münzer, Andreas Leibetseder, Sabrina Kletz, Manfred Jürgen Primus, and Klaus Schoeffmann. 2018. lifeXplore at the Lifelog Search Challenge 2018. Proceedings of the 2018 ACM Workshop on The Lifelog Search Challenge (LSC '18). ACM, New York, NY, USA, 3--8. Google ScholarDigital Library
- Manfred Jürgen Primus, Bernd Münzer, Andreas Leibetseder, and Klaus Schoeffmann. 2018. The ITEC Collaborative Video Search System at the Video Browser Showdown 2018. In International Conference on Multimedia Modeling. Springer, 438--443.Google ScholarCross Ref
- Klaus Schoeffmann, Marco A. Hudelist, and Jochen Huber. 2015. Video Interaction Tools: A Survey of Recent Work. ACM Comput. Surv. , Vol. 48, 1, Article 14 (Sept. 2015), 34 pages. Google ScholarDigital Library
- Marcel Worring, Paul Sajda, Simone Santini, David A Shamma, Alan F Smeaton, and Qiang Yang. 2012. Where is the user in multimedia retrieval? IEEE MultiMedia , Vol. 19, 4 (2012), 6--10. Google ScholarDigital Library
Index Terms
- Interactive Video Search: Where is the User in the Age of Deep Learning?
Recommendations
Interactive Video Retrieval in the Age of Deep Learning
ICMR '19: Proceedings of the 2019 on International Conference on Multimedia RetrievalWe present a tutorial focusing on video retrieval tasks, where state-of-the-art deep learning approaches still benefit from interactive decisions of users. The tutorial covers general introduction to the interactive video retrieval research area, state-...
Interactive Video Search
MM '15: Proceedings of the 23rd ACM international conference on MultimediaWith an increasing amount of video data in our daily life, the need for content-based search in videos increases as well. Though a lot of research has been spent on video retrieval tools and methods which allow for automatic search in videos through ...
Exploring Effective Interactive Text-Based Video Search in vitrivr
MultiMedia ModelingAbstractvitrivr is a general purpose retrieval system that supports a wide range of query modalities. In this paper, we briefly introduce the system and describe the changes and adjustments made for the 2023 iteration of the video browser showdown. These ...
Comments