skip to main content
research-article
Open Access

Higher-order Demand-driven Program Analysis

Published:02 July 2019Publication History
Skip Abstract Section

Abstract

Developing accurate and efficient program analyses for languages with higher-order functions is known to be difficult. Here we define a new higher-order program analysis, Demand-Driven Program Analysis (DDPA), which extends well-known demand-driven lookup techniques found in first-order program analyses to higher-order programs.

This task presents several unique challenges to obtain good accuracy, including the need for a new method for demand-driven lookup of non-local variable values. DDPA is flow- and context-sensitive and provably polynomial-time. To efficiently implement DDPA, we develop a novel pushdown automaton metaprogramming framework, the Pushdown Reachability automaton. The analysis is formalized and proved sound, and an implementation is described.

References

  1. Ole Agesen. 1995. The cartesian product algorithm: Simple and precise type inference of parametric polymorphism. In Proceedings of the 9th European Conference on Object-Oriented Programming (ECOOP’95). Springer-Verlag, Berlin, 2--26. http://dl.acm.org/citation.cfm?id=646153.679533. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Alexander Aiken and Edward L. Wimmers. 1993. Type inclusion constraints and type inference. In Proceedings of the Conference on Functional Programming Languages and Computer Architecture (FPCA’93). ACM, New York, NY, 31--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Rajeev Alur, Ahmed Bouajjani, and Javier Esparza. 2012. Handbook of Model Checking. Springer, 541--572.Google ScholarGoogle Scholar
  4. Rajeev Alur and P. Madhusudan. 2004. Visibly pushdown languages. In Proceedings of the 36th Annual ACM Symposium on Theory of Computing. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Andrew W. Appel. 2007. Compiling with Continuations. Cambridge University Press, New York, NY. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Ahmed Bouajjani, Javier Esparza, and Oded Maler. 1997. Reachability analysis of pushdown automata: Application to model-checking. In Proceedings of the 8th International Conference on Concurrency Theory (CONCUR’97). Springer-Verlag, Berlin, 135--150. http://dl.acm.org/citation.cfm?id=646732.701281. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. José Castano. 2004. Global Index Languages. Ph.D. Dissertation. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Patrick Cousot and Radhia Cousot. 1977. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Proceedings of the 4th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (POPL’77). ACM, New York, NY, 238--252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Danny Dubé and Marc Feeley. 2002. A demand-driven adaptive type analysis. In Proceedings of the 7th ACM SIGPLAN International Conference on Functional Programming (ICFP’02). ACM, New York, NY, 84--97. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Evelyn Duesterwald, Rajiv Gupta, and Mary Lou Soffa. 1997. A practical framework for demand-driven interprocedural data flow analysis. ACM Trans. Program. Lang. Syst. 19, 6 (Nov. 1997), 992--1030. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Christopher Earl, Matthew Might, and David Van Horn. 2010. Pushdown control-flow analysis of higher-order programs. In Proceedings of the Workshop on Scheme and Functional Programming.Google ScholarGoogle Scholar
  12. Christopher Earl, Ilya Sergey, Matthew Might, and David Van Horn. 2012. Introspective pushdown analysis of higher-order programs. In Proceedings of the 17th ACM SIGPLAN International Conference on Functional Programming (ICFP’12). ACM, New York, NY, 177--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Leandro Facchinetti, Zachary Palmer, and Scott F. Smith. 2017. Relative store fragments for singleton abstraction. In Static Analysis, Francesco Ranzato (Ed.). Springer International Publishing, Cham, 106--127.Google ScholarGoogle Scholar
  14. Manuel Fähndrich, Jakob Rehof, and Manuvir Das. 2000. Scalable context-sensitive flow analysis using instantiation constraints. In Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design and Implementation (PLDI’00). ACM, New York, NY, 253--263. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Matthias Felleisen and Daniel P. Friedman. 1986. Control operators, the SECD-machine, and the lambda-calculus. In Proceeding of the 3rd Working Conference on the Formal Description of Programming Concepts.Google ScholarGoogle Scholar
  16. Matthias Felleisen and Robert Hieb. 1992. The revised report on the syntactic theories of sequential control and state. Theor. Comput. Sci. 103, 2 (Sept. 1992), 235--271. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Cormac Flanagan, Amr Sabry, Bruce F. Duba, and Matthias Felleisen. 1993. The essence of compiling with continuations. In Proceedings of the ACM SIGPLAN 1993 Conference on Programming Language Design and Implementation (PLDI’93). ACM, New York, NY, 237--247. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kimball Germane and Matthew Might. 2017. A posteriori environment analysis with pushdown delta CFA. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL’17). ACM, New York, NY, 19--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Thomas Gilray, Steven Lyde, Michael D. Adams, Matthew Might, and David Van Horn. 2016. Pushdown control-flow analysis for free. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’16). ACM, New York, NY, 691--704. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Nevin Heintze and Olivier Tardieu. 2001. Demand-driven pointer analysis. In Proceedings of the ACM SIGPLAN 2001 Conference on Programming Language Design and Implementation (PLDI’01). ACM, New York, NY, 24--34. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Susan Horwitz, Thomas Reps, and Mooly Sagiv. 1995. Demand interprocedural dataflow analysis. In Proceedings of the 3rd ACM SIGSOFT Symposium on Foundations of Software Engineering (SIGSOFT’95). ACM, New York, NY, 104--115. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Suresh Jagannathan and Stephen Weeks. 1995. A unified treatment of flow analysis in higher-order languages. In Proceedings of the 22nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’95). ACM, New York, NY, 393--407. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. J. Ian Johnson, Nicholas Labich, Matthew Might, and David Van Horn. 2013. Optimizing abstract abstract machines. In Proceedings of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP’13). ACM, New York, NY, 443--454. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Ian Johnson, Ilya Sergey, Christopher Earl, Matthew Might, and David Van Horn. 2014. Pushdown flow analysis with abstract garbage collection. J. Funct. Program. 24, 2-3 (2014), 218--283.Google ScholarGoogle ScholarCross RefCross Ref
  25. John Kodumal and Alex Aiken. 2004. The set constraint/CFL reachability connection in practice. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation (PLDI’04). ACM, New York, NY, 207--218. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Salvatore La Torre and Margherita Napoli. 2011. Reachability of multistack pushdown systems with scope-bounded matching relations. In Proceedings of the International Conference on Concurrency Theory (CONCUR’11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. John Lamping. 1990. An algorithm for optimal lambda calculus reduction. In Proceedings of the 17th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’90). ACM, New York, NY, 16--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. David Melski and Thomas Reps. 1997. Interconvertbility of set constraints and context-free language reachability. In Proceedings of the 1997 ACM SIGPLAN Symposium on Partial Evaluation and Semantics-based Program Manipulation (PEPM’97). ACM, New York, NY, 74--89. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Jan Midtgaard. 2012. Control-flow analysis of functional programs. ACM Comput. Surv. 44, 3, Article 10 (June 2012), 33 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Matthew Might. 2007. Environment Analysis of Higher-order Languages. Ph.D. Dissertation. Atlanta. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Matthew Might. 2010. Abstract interpreters for free. In Proceedings of the 17th International Conference on Static Analysis (SAS’10). Springer-Verlag, Berlin, 407--421. http://dl.acm.org/citation.cfm?id=1882094.1882119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Matthew Might and Olin Shivers. 2006. Environment analysis via ΔCFA. In Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’06). ACM, New York, NY, 127--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Matthew Might and Olin Shivers. 2006. Improving flow analyses via ΓCFA: Abstract garbage collection and counting. In Proceedings of the 11th ACM SIGPLAN International Conference on Functional Programming (ICFP’06). ACM, New York, NY, 13--25. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Matthew Might, Yannis Smaragdakis, and David Van Horn. 2010. Resolving and exploiting the k-CFA paradox: Illuminating functional vs. object-oriented program analysis. In Proceedings of the 31st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’10). ACM, New York, NY, 305--315. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Flemming Nielson, Hanne R. Nielson, and Chris Hankin. 1999. Principles of Program Analysis. Springer-Verlag, Berlin. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Zachary Palmer and Scott F. Smith. 2016. Higher-order demand-driven program analysis. In Proceedings of the 30th European Conference on Object-Oriented Programming (ECOOP’16), Shriram Krishnamurthi and Benjamin S. Lerner (Eds.), Leibniz International Proceedings in Informatics, Vol. 56. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 19:1--19:25.Google ScholarGoogle Scholar
  37. Jakob Rehof and Manuel Fähndrich. 2001. Type-base flow analysis: From polymorphic subtyping to CFL-reachability. In Proceedings of the 28th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’01). ACM, New York, NY, 54--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Thomas Reps. 1995. Shape analysis as a generalized path problem. In Proceedings of the 1995 ACM SIGPLAN Symposium on Partial Evaluation and Semantics-based Program Manipulation (PEPM’95). ACM, New York, NY, 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Thomas Reps. 2000. Undecidability of context-sensitive data-dependence analysis. ACM Trans. Program. Lang. Syst. 22, 1 (Jan. 2000), 162--186. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Thomas Reps, Susan Horwitz, and Mooly Sagiv. 1995. Precise interprocedural dataflow analysis via graph reachability. In Proceedings of the 22nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL’95). ACM, New York, NY, 49--61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Thomas Reps, Akash Lal, and Nick Kidd. 2007. Program analysis using weighted pushdown systems. In Proceedings of the 27th International Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS’07). Springer-Verlag, Berlin, 23--51. http://dl.acm.org/citation.cfm?id=1781794.1781799. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Thomas W. Reps. 1995. Demand Interprocedural Program Analysis Using Logic Databases. Springer US, Boston, MA, 163--196.Google ScholarGoogle Scholar
  43. Diptikalyan Saha and C. R. Ramakrishnan. 2005. Incremental and demand-driven points-to analysis using logic programming. In Proceedings of the 7th ACM SIGPLAN International Conference on Principles and Practice of Declarative Programming (PPDP’05). ACM, New York, NY, 117--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Olin Grigsby Shivers. 1991. Control-flow Analysis of Higher-order Languages. Ph.D. Dissertation. Pittsburgh, PA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jeffrey Mark Siskind. 1999. Flow-Directed Lightweight Closure Conversion. Technical Report.Google ScholarGoogle Scholar
  46. Yannis Smaragdakis and Martin Bravenboer. 2011. Using datalog for fast and easy program analysis. In Proceedings of the First International Conference on Datalog Reloaded (Datalog’10). Springer-Verlag, Berlin, 245--251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Johannes Späth, Lisa Nguyen Quang Do, Karim Ali, and Eric Bodden. 2016. Boomerang: Demand-driven flow- and context-sensitive pointer analysis for Java. In Proceedings of the 30th European Conference on Object-Oriented Programming (ECOOP’16), Shriram Krishnamurthi and Benjamin S. Lerner (Eds.), Leibniz International Proceedings in Informatics, Vol. 56. Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany, 22:1--22:26.Google ScholarGoogle Scholar
  48. S. Alexander Spoon and Olin Shivers. 2004. Demand-driven type inference with subgoal pruning: Trading precision for scalability. In European Conference on Object-Oriented Programming (ECOOP’04), Martin Odersky (Ed.). Springer, Berlin, 51--74.Google ScholarGoogle Scholar
  49. David Van Horn and Harry G. Mairson. 2007. Relating complexity and precision in control flow analysis. In Proceedings of the 12th ACM SIGPLAN International Conference on Functional Programming (ICFP’07). ACM, New York, NY, 85--96. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. David Van Horn and Harry G. Mairson. 2008. Deciding kCFA is complete for EXPTIME. In Proceedings of the 13th ACM SIGPLAN International Conference on Functional Programming (ICFP’08). ACM, New York, NY, 275--282. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Dimitrios Vardoulakis and Olin Shivers. 2010. CFA2: A context-free approach to control-flow analysis. In Proceedings of the 19th European Conference on Programming Languages and Systems (ESOP’10). Springer-Verlag, Berlin, 570--589. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Dimitrios Vardoulakis and Olin Shivers. 2011. Pushdown flow analysis of first-class control. In Proceedings of the 16th ACM SIGPLAN International Conference on Functional Programming (ICFP’11). ACM, New York, NY, 69--80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Christopher P. Wadsworth. 1971. Semantics and Pragmatics of the Lambda-calculus. Ph.D. Dissertation. University of Oxford.Google ScholarGoogle Scholar
  54. Stephen Weeks. 2006. Whole-program compilation in MLton. In Proceedings of the 2006 Workshop on ML (ML’06). ACM, New York, NY, 1--1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Qirun Zhang and Zhendong Su. 2017. Context-sensitive data-dependence analysis via linear conjunctive language reachability. In Proceedings of the 44th ACM SIGPLAN Symposium on Principles of Programming Languages (POPL’17). ACM, New York, NY, 344--358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Xin Zhang, Mayur Naik, and Hongseok Yang. 2013. Finding optimum abstractions in parametric dataflow analysis. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’13). ACM, New York, NY, 365--376. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Higher-order Demand-driven Program Analysis

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Programming Languages and Systems
        ACM Transactions on Programming Languages and Systems  Volume 41, Issue 3
        September 2019
        278 pages
        ISSN:0164-0925
        EISSN:1558-4593
        DOI:10.1145/3343145
        Issue’s Table of Contents

        Copyright © 2019 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 2 July 2019
        • Accepted: 1 January 2019
        • Revised: 1 August 2018
        • Received: 1 November 2017
        Published in toplas Volume 41, Issue 3

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format