Randy Smith
Abstract
Deep packet inspection is playing an increasingly important role in the design of novel network services. Regular expressions are the language of choice for writing signatures, but standard DFA or NFA representations are unsuitable for high-speed environments, requiring too much memory, too much time, or too much per-flow state. DFAs are fast and can be readily combined, but doing so often leads to state-space explosion. NFAs, while small, require large per-flow state and are slow.
We propose a solution that simultaneously addresses all these problems. We start with a first-principles characterization of state-space explosion and give conditions that eliminate it when satisfied. We show how auxiliary variables can be used to transform automata so that they satisfy these conditions, which we codify in a formal model that augments DFAs with auxiliary variables and simple instructions for manipulating them. Building on this model, we present techniques, inspired by principles used in compiler optimization, that systematically reduce runtime and per-flow state. In our experiments, signature sets from Snort and Cisco Systems achieve state-space reductions of over four orders of magnitude, per-flow state reductions of up to a factor of six, and runtimes that approach DFAs.
- A. V. Aho and M. Corasick. Efficient string matching: An aid to bibliographic search. In Communications of the ACM, June 1975. Google Scholar
Digital Library
- T. Ball and S. Rajamani. The SLAM project: Debugging system software via static analysis. January 2002. Google ScholarDigital Library
- M. Becchi and S. Cadambi. Memory-efficient regular expression search using state merging. In IEEE Infocom 2007.Google Scholar
- M. Becchi and P. Crowley. An improved algorithm to accelerate regular expression evaluation. In ANCS 2007. Google ScholarDigital Library
- B. Brodie, R., and D. Taylor. A scalable architecture for high-throughput regular-expression pattern matching. SIGARCH Comput. Archit. News, 34(2):191--202, 2006. Google ScholarDigital Library
- D. Brumley, J. Newsome, D. Song, H. Wang, and S. Jha. Towards automatic generation of vulnerability-based signatures. In IEEE Symposium on Security and Privacy, May 2006. Google ScholarDigital Library
- C. R. Clark and D. E. Schimmel. Scalable pattern matching for high-speed networks. In IEEE FCCM, April 2004. Google ScholarDigital Library
- E. M. Clarke, O. Grumberg, and D. Peled. Model Checking. The MIT Press, 1999. Google ScholarDigital Library
- S. Crosby and D. Wallach. Denial of service via algorithmic complexity attacks. In Usenix Security, August 2003. Google ScholarDigital Library
- S. Dharmapurikar and J. W. Lockwood. Fast and scalable pattern matching for network intrusion detection systems. IEEE Journal on Selected Areas in Comm., 24(10):1781--1792, 2006. Google ScholarDigital Library
- The Guardian. Trouble on the line. http://technology. guardian.co.uk/weekly/story/0,,1747343,00.html, 2006.Google Scholar
- M. Handley, V. Paxson, and C. Kreibich. Network intrusion detection: Evasion, traffic normalization, and end-to-end protocol semantics. In Usenix Security, August 2001. Google ScholarDigital Library
- S. W. Hawking. A brief history of time. From the Big Bang to Black Holes. Bantam Book, 1988.Google Scholar
- John L. Hennessy and David A. Patterson. Computer Architecture: A Quantitative Approach, 2nd Edition. Morgan Kaufmann, 1996. Google ScholarDigital Library
- J. Hopcroft, R. Motwani, and J. Ullman. Introduction to Automata Theory, Languages, and Computation. Addison Wesley, 2006. Google ScholarDigital Library
- Myles Jordan. Dealing with metamorphism. Virus Bulletin Weekly, 2002.Google Scholar
- C. Kachris and S. Vassiliadis. Design of a web switch in a reconfigurable platform. In ANCS 2006. Google ScholarDigital Library
- P. Kapustka. Vonage complaining of VoIP blocking. http://www.networkcomputing.com/channels/~networkinfrastructure/60400413, 2005.Google Scholar
- S. Kumar, B. Chandrasekaran, J. Turner, and G. Varghese. Curing regular expressions matching algorithms from insomnia, amnesia, and acalculia. In ANCS 2007, pages 155--164. Google ScholarDigital Library
- S. Kumar, S. Dharmapurikar, F. Yu, P. Crowley, and J. Turner. Algorithms to accelerate multiple regular expressions matching for deep packet inspection. In ACM SIGCOMM, September 2006. Google ScholarDigital Library
- S. Kumar, J. Turner, and J. Williams. Advanced algorithms for fast and scalable deep packet inspection. In ANCS 2006, pages 81--92. Google ScholarDigital Library
- R. Liu, N. Huang, C. Chen, and C. Kao. A fast string-matching algorithm for network processor--based intrusion detection system. Trans. on Embedded Computing Sys., 3(3):614--633, 2004. Google ScholarDigital Library
- H. McGhan. Niagara 2 opens the floodgates. In Microprocessor Report, November 2006.Google Scholar
- S. Muchnick. Advanced Compiler Design and Implementation. Morgan Kaufmann, 1997. Google ScholarDigital Library
- M. Neider. Deep packet inspection: A service provider's solution for secure VoIP. VoIP Magazine, Oct 2005.Google Scholar
- V. Paxson. Bro: a system for detecting network intruders in real-time. In Computer Networks, volume 31, pages 2435--2463, 1999. Google ScholarDigital Library
- T. Ptacek and T. Newsham. Insertion, evasion and denial of service: Eluding network intrusion detection. In Secure Networks, Inc., January 1998.Google Scholar
- M. Roesch. Snort -- lightweight intrusion detection for networks. In 13th Systems Administration Conference. USENIX, 1999. Google ScholarDigital Library
- U. Shankar and Vern Paxson. Active mapping: Resisting nids evasion without altering traffic. In IEEE Symp. on Security and Privacy, May 2003. Google ScholarDigital Library
- R. Smith, C. Estan, and S. Jha. Backtracking algorithmic complexity attacks against a NIDS. In ACSAC 2006, pages 89--98. Google ScholarDigital Library
- R. Smith, C. Estan, and S. Jha. XFA: Faster signature matching with extended automata. In IEEE Symposium on Security and Privacy, May 2008. Google ScholarDigital Library
- R. Sommer and V. Paxson. Enhancing byte--level network intrusion detection signatures with context. In ACM CCS, Oct. 2003. Google ScholarDigital Library
- I. Sourdis and D. Pnevmatikatos. Fast, large-scale string match for a 10gbps fpga-based network intrusion detection system. In Int. Conf. on Field Programmable Logic and Applications, sep. 2003.Google ScholarCross Ref
- L. Tan and T. Sherwood. A high throughput string matching architecture for intrusion detection and prevention. In ISCA, June 2005. Google ScholarDigital Library
- N. Tuck, T. Sherwood, B. Calder, and G. Varghese. Deterministic memory-efficient string matching algorithms for intrusion detection. In IEEE INFOCOM 2004, pages 333--340.Google ScholarCross Ref
- H. J. Wang, C. Guo, D. Simon, and A. Zugenmaier. Shield: Vulnerability-driven network filters for preventing known vulnerability exploits. In ACM SIGCOMM, August 2004. Google ScholarDigital Library
- F. Yu, Z. Chen, Y. Diao, T. V. Lakshman, and R. H. Katz. Fast and memory-efficient regular expression matching for deep packet inspection. In ANCS 2006. Google ScholarDigital Library
Index Terms
- Deflating the big bang: fast and scalable deep packet inspection with extended finite automata
Recommendations
A hybrid finite automaton for practical deep packet inspection
CoNEXT '07: Proceedings of the 2007 ACM CoNEXT conferenceDeterministic finite automata (DFAs) are widely used to perform regular expression matching in linear time. Several techniques have been proposed to compress DFAs in order to reduce memory requirements. Unfortunately, many real-world IDS regular ...
A-DFA: A Time- and Space-Efficient DFA Compression Algorithm for Fast Regular Expression Evaluation
Modern network intrusion detection systems need to perform regular expression matching at line rate in order to detect the occurrence of critical patterns in packet payloads. While Deterministic Finite Automata (DFAs) allow this operation to be ...
Deflating the big bang: fast and scalable deep packet inspection with extended finite automata
SIGCOMM '08: Proceedings of the ACM SIGCOMM 2008 conference on Data communicationDeep packet inspection is playing an increasingly important role in the design of novel network services. Regular expressions are the language of choice for writing signatures, but standard DFA or NFA representations are unsuitable for high-speed ...
Comments