Abstract
We present a payload-based anomaly detector, we call PAYL, for intrusion detection. PAYL models the normal application payload of network traffic in a fully automatic, unsupervised and very effecient fashion. We first compute during a training phase a profile byte frequency distribution and their standard deviation of the application payload flowing to a single host and port. We then use Mahalanobis distance during the detection phase to calculate the similarity of new data against the pre-computed profile. The detector compares this measure against a threshold and generates an alert when the distance of the new input exceeds this threshold. We demonstrate the surprising effectiveness of the method on the 1999 DARPA IDS dataset and a live dataset we collected on the Columbia CS department network. In once case nearly 100% accuracy is achieved with 0.1% false positive rate for port 80 traffic.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Armstrong, D., Carter, S., Frazier, G., Frazier, T.: A Controller-Based Autonomic Defense System. In: Proc. of DISCEX (2003)
Damashek, M.: Gauging similarity with n-grams: language independent categorization of text. Science 267(5199), 843–848 (1995)
Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A Sense of self for Unix Processes. In: Proc. of IEEE Symposium on Computer Security and Privacy (1996)
Ghosh, A.K., Schwartzbard, A.: A study in Using Neural Networks for Anomaly and Misuse Detection. In: Proc. 8th USENIX Security Symposium (1999)
Hoagland, J.: SPADE, Silican Defense, http://www.silicondefense.com/software/spice (2000)
Javits, H.S., Valdes, A.: The NIDES statistical component: Description and justification. Technical report, SRI International, Computer Science Laboratory (1993)
Knuth, D.E.: the Art of Computer Programming, 2nd edn. Fundamental Algorithms, vol. 1. Addison Wesley, Reading (1973)
Kruegel, C., Toth, T., Kirda, E.: Service Specific Anomaly Detection for Network Intrusion Detection. In: Symposium on Applied Computing (SAC), Spain (March 2002)
Lee, W., Stolfo, S.: A Framework for Constructing Features and Models for Intrusion Detection Systems. ACM Transactions on Information and System Security 3(4) (November 2000)
Lippmann, R., et al.: The 1999 DARPA Off-Line Intrusion Detection Evaluation. Computer Networks 34(4), 579–595 (2000)
Locasto, M., Parekh, J., Stolfo, S., Keromytis, A., Malkin, T., Misra, V.: Collaborative Distributed Intrusion Detection, Columbia University Tech Report, CUCS-012-04 (2004)
Mahoney, M.: Network Traffic Anomaly Detection Based on Packet Bytes. In: Proc. ACMSAC (2003)
Mahoney, M., Chan, P.K.: Learning Nonstationary Models of Normal Network Traffic for Detecting Novel Attacks. In: Proc. SIGKDD 2002, pp. 376–385 (2002)
Mahoney, M., Chan, P.K.: Learning Models of Network Traffic for Detecting Novel Attacks, Florida Tech, Technical report 2002-08, http://cs.fit.edu/~tr
Mahoney, M., Chan, P.K.: An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection. In: Vigna, G., Krügel, C., Jonsson, E. (eds.) RAID 2003. LNCS, vol. 2820, pp. 220–237. Springer, Heidelberg (2003)
Moore, D., Shannon, C., Voelker, G., Savage, S.: Internet Quarantine: Requirements for Containing Selp-Propagating Code. In: Proc. Infocom (2003)
V. Paxson, Bro: A system for detecting network intruders in real-time. In: USENIX Security Symposium (1998)
Porras, P., Neumann, P.: EMERALD: Event Monitoring Enabled Responses to Anomalous Live Disturbances. In: National Information Systems Security Conference (1997)
Robertson, S., Siegel, E., Miller, M., Stolfo, S.: Surveillance Detection in High Bandwidth Environments. In: Proceedings of the 2003 DARPA DISCEX III Conference (2003)
Roesch, M.: Snort: Lightweight intrusion detection for networks. In: USENIX LISA Conference (1999)
Staniford, S., Paxson, V., Weaver, N.: How to Own the Internet in Your Spare Time. In: Proceedings of the 11th USENIX Security Symposium (2002)
Stolfo, S.: Worm and Attack Early Warning: Piercing Stealthy Reconnaissance. IEEE Privacy and Security (May/June 2004) (to appear)
Taylor, C., Alves-Foss, J.: NATE – Network Analysis of Anomalous Traffic Events, A Low-Cost approach. In: New Security Paradigms Workshop (2001)
Vigna, G., Kemmerer, R.: NetSTAT: A Network-based intrusion detection approach. In: Computer Security Application Conference (1998)
Lane, T., Broadley, C.E.: Approaches to online learning and concept drift for user identification in computer security. In: 4th International Conference on Knowledge Discovery and Data Mining (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, K., Stolfo, S.J. (2004). Anomalous Payload-Based Network Intrusion Detection. In: Jonsson, E., Valdes, A., Almgren, M. (eds) Recent Advances in Intrusion Detection. RAID 2004. Lecture Notes in Computer Science, vol 3224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30143-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-30143-1_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23123-3
Online ISBN: 978-3-540-30143-1
eBook Packages: Springer Book Archive