Simon F. Goldsmith
Alex S. Aiken
ABSTRACT
The standard language for describing the asymptotic behavior of algorithms is theoretical computational complexity. We propose a method for describing the asymptotic behavior of programs in practice by measuring their empirical computational complexity. Our method involves running a program on workloads spanning several orders of magnitude in size, measuring their performance, and fitting these observations to a model that predicts performance as a function of workload size. Comparing these models to the programmer's expectations or to theoretical asymptotic bounds can reveal performance bugs or confirm that a program's performance scales as expected. Grouping and ranking program locations based on these models focuses attention on scalability-critical code. We describe our tool, the Trend Profiler (trend-prof), for constructing models of empirical computational complexity that predict how many times each basic block in a program runs as a linear (y = a + bx) or a powerlaw (y = axb) function of user-specified features of the program's workloads. We ran trend-prof on several large programs and report cases where a program scaled as expected, beat its worst-case theoretical complexity bound, or had a performance bug.
- A. Alexandrov, M. F. Ionescu, K. E. Schauser, and C. Scheiman. LogGP: Incorporating long messages into the LogP model - One step closer towards a realistic model for parallel computation. In SPAA '95: Proceedings of the 7th Annual ACM Symposium on Parallel Algorithms and Architectures, pages 95--105, New York, NY, USA, 1995. ACM Press. Google Scholar
Digital Library
- G. Ammons, J.-D. Choi, M. Gupta, and N. Swamy. Finding and removing performance bottlenecks in large systems. In ECOOP 2004. Springer Berlin / Heidelberg.Google ScholarCross Ref
- L.O.Andersen.Program Analysis and Specialization for the C Programming Language. Ph.d. thesis, DIKU, Unversity of Copenhagen, 1994.Google Scholar
- T. Ball and J. R. Larus. Optimally profiling and tracing programs. ACM Trans. Program. Lang. Syst., 16(4):1319--1360, 1994. Google ScholarDigital Library
- E. A. Brewer. High-level optimization via automated statistical modeling. In PPOPP '95: Proceedings of the 5th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pages 80--91, New York, NY, USA, 1995. ACM Press. Google ScholarDigital Library
- bzip2 project homepage. http://www.bzip.org/.Google Scholar
- gcov documentation. http://gcc.gnu.org/onlinedocs/gcc/Gcov.html.Google Scholar
- S. L. Graham, P. B. Kessler, and M. K. Mckusick. Gprof: A call graph execution profiler. In SIGPLAN '82: Proceedings of the 1982 SIGPLAN Symposium on Compiler Construction, pages 120--126, New York, NY, USA, 1982. ACM Press. Google ScholarDigital Library
- M. Kluge, A. Knüpfer, and W. E. Nagel. Knowledge based automatic scalability analysis and extrapolation for MPI programs. In Euro-Par 2005 Parallel Processing: 11th International Euro-Par Conference, Lecture Notes in Computer Science. Springer-Verlag. Google ScholarDigital Library
- J. Kodumal and A. Aiken. Banshee: A scalable constraint-based analysis toolkit. In SAS '05: Proceedings of the 12th International Static Analysis Symposium. London, United Kingdom, September 2005. Google ScholarDigital Library
- S. McPeak and G. C. Necula. Elkhound: A fast, practical GLR parser generator. In Conference on Compiler Construction (CC04), 2004.Google ScholarCross Ref
- D. L. Métayer. ACE: An automatic complexity evaluator. ACM Trans. Program. Lang. Syst., 10(2):248--266, 1988. Google ScholarDigital Library
- J. A. Rice. Mathematical Statistics and Data Analysis. Duxbury Press, 2006.Google Scholar
- M. Rosendahl. Automatic complexity analysis. Proceedings of the 4th International Conference on Functional Programming Languages and Computer Architecture, pages 144--156, 1989. Google ScholarDigital Library
- R. Rugina and K. Schauser. Predicting the running times of parallel programs by simulation. In Proceedings of the 12th International Parallel Processing Symposium and 9th Symposium on Parallel and Distributed Processing, 1998. Google ScholarDigital Library
- V. Sarkar. Determining average program execution times and their variance. In PLDI '89: Proceedings of the ACM SIGPLAN 1989 Conference on Programming Language Design and Implementation, pages 298--312, New York, NY, USA, 1989. ACM Press. Google ScholarDigital Library
- G. Sevitsky, W. de Pauw, and R. Konuru. An information exploration tool for performance analysis of Java programs. In TOOLS '01: Proceedings of the Technology of Object-Oriented Languages and Systems, page 85, Washington, DC, USA, 2001. IEEE Computer Society. Google ScholarDigital Library
- E. Ukkonen. A linear-time algorithm for finding approximate shortest common superstrings. In Algorithmica, volume 5, pages 313--323, 1990.Google ScholarCross Ref
- B. Wegbreit. Mechanical program analysis. Commun. ACM, 18(9):528--539, 1975. Google ScholarDigital Library
Index Terms
- Measuring empirical computational complexity
Recommendations
An overview of computational complexity
ACM Turing award lecturesAn historical overview of computational complexity is presented. Emphasis is on the fundamental issues of defining the intrinsic computational complexity of a problem and proving upper and lower bounds on the complexity of problems. Probabilistic and ...
An overview of computational complexity
An historical overview of computational complexity is presented. Emphasis is on the fundamental issues of defining the intrinsic computational complexity of a problem and proving upper and lower bounds on the complexity of problems. Probabilistic and ...
Comments