skip to main content
research-article

Complex event recognition in the big data era

Authors Info & Claims
Published:01 August 2017Publication History
Skip Abstract Section

Abstract

The concept of event processing is established as a generic computational paradigm in various application fields, ranging from data processing in Web environments, over maritime and transport, to finance and medicine. Events report on state changes of a system and its environment. Complex Event Recognition (CER) in turn, refers to the identification of complex/composite events of interest, which are collections of simple events that satisfy some pattern, thereby providing the opportunity for reactive and proactive measures. Examples include the recognition of attacks in computer network nodes, human activities on video content, emerging stories and trends on the Social Web, traffic and transport incidents in smart cities, fraud in electronic marketplaces, cardiac arrhythmias, and epidemic spread. In each scenario, CER allows to make sense of Big event Data streams and react accordingly. The goal of this tutorial is to provide a step-by-step guide for realizing CER in the Big Data era. To do so, it elaborates on major challenges and describes algorithmic toolkits for optimized manipulation of event streams characterized by high volume, velocity and/or lack of veracity, placing emphasis on distributed CER over potentially heterogeneous (data variety) event sources. Finally, we highlight future research directions in the field.

References

  1. Esper. http://esper.codehaus.org/.Google ScholarGoogle Scholar
  2. M. Akdere, U. Çetintemel, and N. Tatbul. Plan-based complex event detection across distributed sources. Proc. VLDB Endow., pages 66--77, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. A. Artikis, A. Margara, M. Ugarte, S. Vansummeren, and M. Weidlich. Complex event recognition languages. In DEBS, 2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A. Artikis, M. J. Sergot, and G. Paliouras. An event calculus for event recognition. IEEE Trans. Knowl. Data Eng., 27(4):895--908, 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. A. Artikis, A. Skarlatidis, F. Portet, and G. Paliouras. Logic-based event recognition. Knowledge Engineering Review, 27(4):469--506, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. Balkesen, N. Dindar, M. Wetter, and N. Tatbul. Rip: run-based intra-query parallelism for scalable complex event processing. In DEBS, pages 3--14, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. W. Brendel, A. Fern, and S. Todorovic. Probabilistic Event Logic for Interval-based Event Recognition. In CVPR, pages 3329--3336, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. L. Brenna, A. J. Demers, J. Gehrke, M. Hong, J. Ossher, B. Panda, M. Riedewald, M. Thatte, and W. M. White. Cayuga: a high-performance event processing engine. In SIGMOD, pages 1100--1102, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. G. Cugola and A. Margara. Complex event processing with t-rex. J. Syst. Softw., 85(8):1709--1728, aug 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. G. Cugola and A. Margara. Processing flows of information: From data stream to complex event processing. ACM Computing Surveys, 44(3):15:1--15:62, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. C. Dousson and P. L. Maigat. Chronicle recognition improvement using temporal focusing and hierarchisation. In IJCAI, pages 324--329, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. O. Etzion and P. Niblett. Event Processing in Action. Manning Publications Co., 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. I. Flouris, N. Giatrakos, A. Deligiannakis, M. Garofalakis, M. Kamp, and M. Mock. Issues in complex event processing: Status and prospects in the big data era. Journal of Systems and Software, 127:217 -- 236, 2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. I. Flouris, V. Manikaki, N. Giatrakos, A. Deligiannakis, M. Garofalakis, M. Mock, S. Bothe, I. Skarbovsky, F. Fournier, M. Stajcer, T. Krizan, J. Yom-Tov, and T. Curin. Ferari: A prototype for complex event processing over streaming multi-cloud platforms. In SIGMOD, pages 2093--2096, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. M. Garofalakis, J. Gehrke, and R. Rastogi. Data Stream Management: Processing High-Speed Data Streams. Springer, 2016.Google ScholarGoogle ScholarCross RefCross Ref
  16. N. Giatrakos, A. Deligiannakis, M. Garofalakis, I. Sharfman, and A. Schuster. Distributed geometric query monitoring using prediction models. ACM Trans. Database Syst., 39(2):16:1--16:42, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. T. Heinze, Y. Ji, Y. Pan, F. J. Grueneberger, Z. Jerzak, and C. Fetzer. Elastic complex event processing under varying query load. In BD3@ VLDB, 2013.Google ScholarGoogle Scholar
  18. M. Hirzel. Partition and compose: parallel complex event processing. In DEBS, pages 191--200, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. D. Keren, G. Sagy, A. Abboud, D. Ben-David, A. Schuster, I. Sharfman, and A. Deligiannakis. Geometric monitoring of heterogeneous streams. IEEE TKDE, 26(8):1890--1903, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  20. R. Mayer, C. Mayer, M. A. Tariq, and K. Rothermel. Graphcep: Real-time data analytics using parallel complex event and graph processing. In DEBS, pages 309--316, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Y. Mei and S. Madden. Zstream: a cost-based query processor for adaptively detecting composite events. In SIGMOD, pages 193--206, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. E. Michelioudakis, A. Skarlatidis, G. Paliouras, and A. Artikis. OSLα : Online structure learning using background knowledge axiomatization. In ECML PKDD, pages 232--247, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. C. Molinaro, V. Moscato, A. Picariello, A. Pugliese, A. Rullo, and V. S. Subrahmanian. PADUA: Parallel Architecture to Detect Unexplained Activities. ACM (TOIT), 14(1):3:1--3:28, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. B. Mozafari, K. Zeng, L. D'Antoni, and C. Zaniolo. High-performance complex event processing over hierarchical data. ACM Trans. Database Syst., 38(4):21:1--21:39, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. K. Patroumpas, E. Alevizos, A. Artikis, M. Vodas, N. Pelekis, and Y. Theodoridis. Online event recognition from moving vessel trajectories. GeoInformatica, 21(2):389--427, 2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. N. P. Schultz-Møller, M. Migliavacca, and P. Pietzuch. Distributed complex event processing with query rewriting. In DEBS, pages 4:1--4:12, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. I. Sharfman, A. Schuster, and D. Keren. A geometric approach to monitoring threshold functions over distributed data streams. In SIGMOD, pages 301--312, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. U. Srivastava, K. Munagala, and J. Widom. Operator placement for in-network stream query processing. In PODS, pages 250--258, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. S. Wasserkrug, A. Gal, O. Etzion, and Y. Turchin. Efficient processing of uncertain events in rule-based systems. IEEE TKDE, 24(1):45--58, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. L. Woods, J. Teubner, and G. Alonso. Complex event detection at wire speed with fpgas. Proc. VLDB Endow., 3(1-2):660--669, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. N. Zacheilas, V. Kalogeraki, N. Zygouras, N. Panagiotou, and D. Gunopulos. Elastic complex event processing exploiting prediction. In Big Data, pages 213--222, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. H. Zhang, Y. Diao, and N. Immerman. Recognizing patterns in streams with imprecise timestamps. Inf. Syst., 38(8):1187--1211, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. H. Zhang, Y. Diao, and N. Immerman. On complexity and optimization of expensive queries in complex event processing. In SIGMOD, pages 217--228, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in

Full Access

  • Published in

    cover image Proceedings of the VLDB Endowment
    Proceedings of the VLDB Endowment  Volume 10, Issue 12
    August 2017
    427 pages
    ISSN:2150-8097
    Issue’s Table of Contents

    Publisher

    VLDB Endowment

    Publication History

    • Published: 1 August 2017
    Published in pvldb Volume 10, Issue 12

    Qualifiers

    • research-article

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader