Rijnard van Tonder
ABSTRACT
Static analysis is a proven technique for catching bugs during software development. However, analysis tooling must approximate, both theoretically and in the interest of practicality. False positives are a pervading manifestation of such approximations---tool configuration and customization is therefore crucial for usability and directing analysis behavior. To suppress false positives, developers readily disable bug checks or insert comments that suppress spurious bug reports. Existing work shows that these mechanisms fall short of developer needs and present a significant pain point for using or adopting analyses. We draw on the insight that an analysis user always has one notable ability to influence analysis behavior regardless of analyzer options and implementation: modifying their program. We present a new technique for automated, generic, and temporary code changes that tailor to suppress spurious analysis errors. We adopt a rule-based approach where simple, declarative templates describe general syntactic changes for code patterns that are known to be problematic for the analyzer. Our technique promotes program transformation as a general primitive for improving the fidelity of analysis reports (we treat any given analyzer as a black box). We evaluate using five different static analyzers supporting three different languages (C, Java, and PHP) on large, real world programs (up to 800KLOC). We show that our approach is effective in sidestepping long-standing and complex issues in analysis implementations.
- 2019. Clang Static Analyzer. https://clang-analyzer.llvm.org/.Google Scholar
- 2019. CodeSonar. https://www.grammatech.com/products/codesonar.Google Scholar
- 2019. Coverity: suppressing asserts. https://community.synopsys.com/s/question/0D534000046YuzbCAC.Google Scholar
- 2019. Error Prone: Patching. https://errorprone.info/docs/patching.Google Scholar
- 2019. Infer. https://github.com/facebook/infer.Google Scholar
- 2019. NullAway: auto-suppressing. https://github.com/uber/NullAway/wiki/Suppressing-Warnings#auto-suppressing.Google Scholar
- 2019. PHPStan. https://github.com/phpstan/phpstan.Google Scholar
- 2019. Spotbugs. https://github.com/spotbugs/spotbugs.Google Scholar
- Cristiano Calcagno, Dino Distefano, Jérémy Dubreil, Dominik Gabi, Pieter Hooimeijer, Martino Luca, Peter W. O'Hearn, Irene Papakonstantinou, Jim Purbrick, and Dulma Rodriguez. 2015. Moving Fast with Software Verification. In NASA Formal Methods (NFM '15). 3--11.Google Scholar
- Maria Christakis and Christian Bird. 2016. What Developers Want and Need from Program Analysis: An Empirical Study. In International Conference on Automated Software Engineering (ASE '16). 332--343.Google ScholarDigital Library
- Maria Christakis, Peter Müller, and Valentin Wüstholz. 2015. An Experimental Evaluation of Deliberate Unsoundness in a Static Program Analyzer. In Verification, Model Checking, and Abstract Interpretation (VMCAI '15). 336--354.Google Scholar
- J. Robert M. Cornish, Graeme Gange, Jorge A. Navas, Peter Schachte, Harald Søndergaard, and Peter J. Stuckey. 2014. Analyzing Array Manipulating Programs by Program Transformation. In Logic-Based Program Synthesis and Transformation (LOPSTR '14). 3--20.Google Scholar
- Patrick Cousot and Radhia Cousot. 2002. Systematic Design of Program Transformation Frameworks by Abstract Interpretation. In Symposium on Principles of Programming Languages (POPL '02). 178--190.Google ScholarDigital Library
- Pascal Cuoq, Benjamin Monate, Anne Pacalet, Virgile Prevosto, John Regehr, Boris Yakobowski, and Xuejun Yang. 2012. Testing Static Analyzers with Randomly Generated Programs. In NASA Formal Methods (NFM '12). 120--125.Google Scholar
- Alastair F. Donaldson, Hugues Evrard, Andrei Lascu, and Paul Thomson. 2017. Automated testing of graphics shader compilers. PACMPL 1, OOPSLA (2017), 93:1--93:29.Google Scholar
- Pär Emanuelsson and Ulf Nilsson. 2008. A Comparative Study of Industrial Static Analysis Tools. Electr. Notes Theor. Comput. Sci. 217 (2008), 5--21.Google ScholarDigital Library
- Nikos Gorogiannis, Peter W. O'Hearn, and Ilya Sergey. 2019. A true positives theorem for a static race detector. PACMPL 3, POPL (2019), 57:1--57:29.Google Scholar
- Mark Harman. 2018. We Need a Testability Transformation Semantics. In Software Engineering and Formal Methods (SEFM '18). 3--17.Google ScholarCross Ref
- Mark Harman, Lin Hu, Robert M. Hierons, Joachim Wegener, Harmen Sthamer, André Baresel, and Marc Roper. 2004. Testability Transformation. IEEE Trans. Software Eng. 30, 1 (2004), 3--16.Google ScholarDigital Library
- Ciera Jaspan, I-Chin Chen, and Anoop Sharma. 2007. Understanding the Value of Program Analysis Tools. In Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA '07). 963--970.Google Scholar
- Brittany Johnson, Yoonki Song, Emerson R. Murphy-Hill, and Robert W. Bowdidge. 2013. Why Don't Software Developers use Static Analysis Tools to Find Bugs?. In International Conference on Software Engineering, 2013 ('13). 672--681.Google Scholar
- William Landi. 1992. Undecidability of Static Analysis. ACM Letters on Programming Languages and Systems 1, 4 (dec 1992), 323--337.Google ScholarDigital Library
- Chris Lattner and Vikram S. Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Code Generation and Optimization (CGO '04). 75--88.Google ScholarDigital Library
- Julia Lawall and Gilles Muller. 2018. Coccinelle: 10 Years of Automated Evolution in the Linux Kernel. In USENIX Annual Technical Conference. 601--614.Google Scholar
- Benjamin Livshits, Manu Sridharan, Yannis Smaragdakis, Ondrej Lhoták, José Nelson Amaral, Bor-Yuh Evan Chang, Samuel Z. Guyer, Uday P. Khedker, Anders Møller, and Dimitrios Vardoulakis. 2015. In Defense of Soundiness: A Manifesto. Commun. ACM 58, 2 (2015), 44--46.Google ScholarDigital Library
- Francesco Logozzo and Manuel Fähndrich. 2008. On the Relative Completeness of Bytecode Analysis Versus Source Code Analysis. In Compiler Construction (CC '08). 197--212.Google Scholar
- Eduardus A. T. Merks, J. Michael Dyck, and Robert D. Cameron. 1992. Language Design For Program Manipulation. IEEE Trans. Software Eng. 18, 1 (1992), 19--32.Google ScholarDigital Library
- Kedar S. Namjoshi and Zvonimir Pavlinovic. 2018. The Impact of Program Transformations on Static Program Analysis. In International Symposium on Static Analysis (SAS '18). 306--325.Google Scholar
- Hui Peng, Yan Shoshitaishvili, and Mathias Payer. 2018. T-Fuzz: Fuzzing by Program Transformation. In IEEE Symposium on Security and Privacy.Google ScholarCross Ref
- Martin C Rinard, Cristian Cadar, Daniel Dumitran, Daniel M Roy, Tudor Leu, and William S Beebee. 2004. Enhancing Server Availability and Security Through Failure-Oblivious Computing.. In OSDI, Vol. 4. 21--21.Google ScholarDigital Library
- Caitlin Sadowski, Edward Aftandilian, Alex Eagle, Liam Miller-Cushon, and Ciera Jaspan. 2018. Lessons from Building Static Analysis Tools at Google. Commun. ACM 61, 4 (2018), 58--66.Google ScholarDigital Library
- Rijnard van Tonder, John Kotheimer, and Claire Le Goues. 2018. Semantic Crash Bucketing. In International Conference on Automated Software Engineering (ASE '18). 612--622.Google Scholar
- Rijnard van Tonder and Claire Le Goues. 2018. Static Automated Program Repair for Heap Properties. In International Conference on Software Engineerinng (ICSE '18). 151--162.Google Scholar
- Rijnard van Tonder and Claire Le Goues. 2019. Lightweight Multi-Language Syntax Transformation with Parser Parser Combinators. In Conference on Programming language Design and Implementation (PLDI '19).Google Scholar
- Louis Wasserman. 2013. Scalable, Example-based Refactorings with Refaster. In Workshop on Refactoring Tools (WRT@SPLASH '13). 25--28.Google Scholar
- Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and Understanding Bugs in C Compilers. In Conference on Programming Language Design and Implementation (PLDI '11). 283--294.Google Scholar
- Tailoring programs for static analysis via program transformation
Recommendations
Static analysis of multi-staged programs via unstaging translation
POPL '11Static analysis of multi-staged programs is challenging because the basic assumption of conventional static analysis no longer holds: the program text itself is no longer a fixed static entity, but rather a dynamically constructed value. This article ...
Static program analysis of embedded executable assembly code
CASES '04: Proceedings of the 2004 international conference on Compilers, architecture, and synthesis for embedded systemsWe consider the problem of automatically checking if coding standards have been followed in the development of embedded applications. The problem arises from practical considerations because DSP chip manufacturers (in our case Texas Instruments) want ...
PMAF: an algebraic framework for static analysis of probabilistic programs
PLDI '18Automatically establishing that a probabilistic program satisfies some property ϕ is a challenging problem. While a sampling-based approach—which involves running the program repeatedly—can suggest that ϕ holds, to establish that the program satisfies ϕ,...
Comments