Abstract
The ease of compiling malicious code from source code in higher programming languages has increased the volatility of malicious programs: The first appearance of a new worm in the wild is usually followed by modified versions in quick succession. As demonstrated by Christodorescu and Jha, however, classical detection software relies on static patterns, and is easily outsmarted. In this paper, we present a flexible method to detect malicious code patterns in executables by model checking. While model checking was originally developed to verify the correctness of systems against specifications, we argue that it lends itself equally well to the specification of malicious code patterns. To this end, we introduce the specification language CTPL (Computation Tree Predicate Logic) which extends the well-known logic CTL, and describe an efficient model checking algorithm. Our practical experiments demonstrate that we are able to detect a large number of worm variants with a single specification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Norman ASA. Norman sandbox whitepaper. Technical report (2003)
Bergeron, J., Debbabi, M., Desharnais, J., Erhioui, M.M., Lavoie, Y., Tawbi, N.: Static detection of malicious code in executable programs. In: Symposium on Requirements Engineering for Information Security (March 2001)
Christodorescu, M., Jha, S.: Static analysis of executables to detect malicious patterns. In: Proceedings of the 12th USENIX Security Symposium (Security 2003), August 2003, pp. 169–186. USENIX Association (2003)
Christodorescu, M., Jha, S.: Testing malware detectors. In: Proceedings of the International Symposium on Software Testing and Analysis, ISSTA 2004 (2004)
Clarke, E., Emerson, E.: Design and synthesis of synchronization skeletons using branching time temporal logic. In: Kozen, D. (ed.) Logic of Programs 1981. LNCS, vol. 131, pp. 52–71. Springer, Heidelberg (1982)
Clarke, E., Grumberg, O., Long, D.: Model Checking. MIT Press, Cambridge (1999)
Clarke, E., Schlingloff, B.: Model Checking. In: Handbook of Automated Reasoning, pp. 1637–1790. Elsevier, Amsterdam (2001)
Emerson, E.: Temporal and Modal Logic. In: Handbook of Theoretical Computer Science, vol. B, pp. 995–1072. Elsevier, Amsterdam (1990)
Fast Small Good, http://www.xtreeme.prv.pl (Last accessed: December16, 2004)
Huth, M., Ryan, M.: Logic in Computer Science: Modelling and Reasoning about Systems. Cambridge University Press, Cambridge (2000)
IDA Pro. http://www.datarescue.com/idabase/ (Last accessed: January 20, 2004)
IKARUS Software, http://www.ikarus-software.at/ (Last accessed: January 20, 2004)
Lakhotia, A., Singh, P.: Challenges in getting ’formal’ with viruses. Virus Bulletin (September 2003)
Singh, P., Lakhotia, A.: Static Verification of Worm and Virus Behavior in Binary Executables using Model Checking. In: 4th IEEE Information Assurance Workshop (June 2003)
Ultimate Packer for eXecutables, http://upx.sourceforge.net/ (Last accessed: December 16, 2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kinder, J., Katzenbeisser, S., Schallhart, C., Veith, H. (2005). Detecting Malicious Code by Model Checking. In: Julisch, K., Kruegel, C. (eds) Detection of Intrusions and Malware, and Vulnerability Assessment. DIMVA 2005. Lecture Notes in Computer Science, vol 3548. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11506881_11
Download citation
DOI: https://doi.org/10.1007/11506881_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-26613-6
Online ISBN: 978-3-540-31645-9
eBook Packages: Computer ScienceComputer Science (R0)