Mert Akdere
Nesime Tatbul
Abstract
Complex Event Detection (CED) is emerging as a key capability for many monitoring applications such as intrusion detection, sensor-based activity & phenomena tracking, and network monitoring. Existing CED solutions commonly assume centralized availability and processing of all relevant events, and thus incur significant overhead in distributed settings. In this paper, we present and evaluate communication efficient techniques that can efficiently perform CED across distributed event sources.
Our techniques are plan-based: we generate multi-step event acquisition and processing plans that leverage temporal relationships among events and event occurrence statistics to minimize event transmission costs, while meeting application-specific latency expectations. We present an optimal but exponential-time dynamic programming algorithm and two polynomial-time heuristic algorithms, as well as their extensions for detecting multiple complex events with common sub-expressions. We characterize the behavior and performance of our solutions via extensive experimentation on synthetic and real-world data sets using our prototype implementation.
- Eric N. Hanson, et al. Scalable Trigger Processing. ICDE 1999.Google Scholar
- S. Madden, M. J. Franklin, J. M. Hellerstein, and W. Hong. Tinydb. TODS 2005.Google Scholar
- Peter R. Pietzuch. "Hermes: A Scalable Event-Based Middleware". Ph.D. Thesis, University of Cambridge, 2004.Google Scholar
- S. Gatziu and K. R. Dittrich. Detecting composite events in active database systems using petri nets. In Proc. 4. Intl. Workshop on Research Issues in Data Engineering, 1994.Google ScholarCross Ref
- S. Chakravarthy, et al. Composite Events for Active Databases: Semantics, Contexts and Detection, VLDB 1994. Google ScholarDigital Library
- S. Chakravarthy and D. Mishra. Snoop: An Expressive Event Specification Language for Active Databases. Data and Knowledge Engineering, 14(10): 1--26, 1994. Google ScholarDigital Library
- Eugene Wu, et al. High-Performance Complex Event Processing over Streams. SIGMOD 2006 Google ScholarDigital Library
- N. Paton and O. Diaz, 'Active Database Systems', ACM Comp. Surveys, Vol. 31, No. 1, 1999. Google ScholarDigital Library
- Zimmer, D. and Unland, R. On the Semantics of Complex Events in Active Database Management Systems. ICDE '99. Google ScholarDigital Library
- The Power of Events. David Luckham, May 2002.Google Scholar
- Sellis, T. K. Multiple-query optimization. TODS Mar. 1988. Google ScholarDigital Library
- Zhou, J., et al. Efficient exploitation of similar subexpressions for query processing. SIGMOD '07. Google ScholarDigital Library
- Pattern Recognition and Machine Learning. Bishop, Christopher M. 2006, ISBN: 978-0-387-31073-2.Google Scholar
- Combinatorial optimization: algorithms and complexity. Christos H. Papadimitriou, Kenneth Steiglitz. 1998.Google Scholar
- S. V. Amaria and R. B. Misra, Closed-form expressions for distribution of sum of exponential random variables, IEEE Trans. Reliability, vol. 46, no. 4, pp. 519--522, Dec. 1997.Google ScholarCross Ref
- Amol Deshpande, et al. Model-based approximate querying in sensor networks. VLDB J. 14(4): 417--443 (2005)Google ScholarCross Ref
- Daniel Abadi, et al. The Design of the Borealis Stream Processing Engine. CIDR '05.Google Scholar
- S. Chandrasekaran, et al. TelegraphCQ: Continuous Dataflow Processing. In ACM SIGMOD Conference, June 2003. Google ScholarDigital Library
- R. Motwani, et al. Query Processing, Approximation, and Resource Management in a Data Stream Management System. In CIDR Conference, January 2003.Google Scholar
- http://planetflow.planet-lab.orgGoogle Scholar
- Selinger, P. G., et al. 1979. Access path selection in a relational database management system. SIGMOD '79. Google ScholarDigital Library
- SNORT Network Intrusion Detection. http://www.snort.orgGoogle Scholar
- S. Li, et al. Event Detection Services Using Data Service Middleware in Distributed Sensor Networks. IPSN 2003. Google ScholarDigital Library
Index Terms
- Plan-based complex event detection across distributed sources
Recommendations
Semantic Rule-Based Complex Event Processing
RuleML '09: Proceedings of the 2009 International Symposium on Rule Interchange and ApplicationsOne of the critical success factors of event-driven systems is the capability of detecting complex events from simple and ordinary event notifications. Complex events which trigger or terminate actionable situations can be inferred from large event ...
Adaptive distributed composite event detection
ARM '12: Proceedings of the 11th International Workshop on Adaptive and Reflective MiddlewareCommunication among devices, including sensors and actuators, forms the basis for ubiquitous computing scenarios such as smart homes. For such scenarios, an event-based tyle of interaction leveraging the publish/subscribe paradigm is well suited as it ...
Complex event processing over distributed probabilistic event streams
With the rapid development of Internet of Things (IoT), enormous events are produced every day. Complex Event Processing (CEP), which can be used to extract high level patterns from raw data, becomes the key part of the IoT middleware. In large-scale ...
Comments