Tomas Kalibera
Richard Jones
Jan Vitek
ABSTRACT
Increasing levels of hardware parallelism are one of the main challenges for programmers and implementers of managed runtimes. Any concurrency or scalability improvements must be evaluated experimentally. However, application benchmarks available today may not reflect the highly concurrent applications we anticipate in the future. They may also behave in ways that VM developers do not expect. We provide a set of platform independent concurrency related metrics and an in-depth observational study of current state of the art benchmarks, discovering how concurrent they really are, how they scale the work and how they synchronise and communicate via shared memory.
- ASM project. http://asm.ow2.org, 2011.Google Scholar
- BTrace. http://kenai.com/projects/btrace, 2011.Google Scholar
- B. Alpern, C.R. Attanasio et al. Implementing Jalapeno in Java. In Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), 1999. Google ScholarDigital Library
- W. Binder, J. Hulaas and P. Moret. Reengineering standard Java runtime systems through dynamic bytecode instrumentation. In Source Code Analysis and Manipulation (SCAM), 2007. Google ScholarDigital Library
- S.M. Blackburn and K.S. McKinley. Ulterior reference counting: Fast garbage collection without a long wait. In Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), 2003. Google ScholarDigital Library
- S.M. Blackburn, R. Garner et al. The DaCapo benchmarks: Java benchmarking development and analysis. In Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), 2006. Google ScholarDigital Library
- A. Burns and A.J. WellingS. Real-Time Systems and Programming Languages: ADA 95, Real-Time Java and Real-Time POSIX. Addison-Wesley, 3rd edition, 2001. Google ScholarDigital Library
- K.-Y. Chen, J.M. Chang, and T.-W. Hou. Multithreading in Java: Performance and scalability on multicore systems. IEEE Transactions on Computers, 60 (11), 2011. Google ScholarDigital Library
- D. Doligez and G. Gonthier. Portable, unobtrusive garbage collection for multiprocessor systems. In Symposium on Principles of Programming Languages (POPL), 1994. Google ScholarDigital Library
- D. Doligez and X. Leroy. A concurrent generational garbage collector for a multi-threaded implementation of ML. In Symposium on Principles of Programming Languages (POPL), 1993. Google ScholarDigital Library
- T. Domani, E.K. Kolodner et al. Thread-local heaps for Java. In International Symposium on Memory Management (ISMM), 2002. Google ScholarDigital Library
- B. Dufour, K. Driesen et al. Dynamic metrics for Java. In Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), 2003. Google ScholarDigital Library
- H. Esmaeilzadeh, T. Cao et al. Looking back on the language and hardware revolutions: Measured power, performance and scaling. In Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2011. Google ScholarDigital Library
- L. Gidra, G. Thomas et al. Assessing the scalability of garbage collectors on many cores. In Programming Languages and Operating Systems (PLOS), 2011. Google ScholarDigital Library
- R.E. Jones and A.C. King. A fast analysis for thread-local garbage collection with dynamic class loading. In Source Code Analysis and Manipulation (SCAM), 2005. Google ScholarDigital Library
- R.E. Jones and C. Ryder. A study of Java object demographics. In International Symposium on Memory Management (ISMM), 2008. Google ScholarDigital Library
- R.E. Jones, A.L. Hosking, and J.E.B. Moss. The Garbage Collection Handbook: The Art of Automatic Memory Management. Chapman & Hall, 2011. Google ScholarDigital Library
- T. Liu and E.D. Berger. Sheriff: precise detection and automatic mitigation of false sharing. In Object Oriented Programming Systems, Languages and Applications (OOPSLA), 2011. Google ScholarDigital Library
- S. Marlow and S.L. Peyton Jones. Multicore garbage collection with local heaps. In International Symposium on Memory Management (ISMM), 2011. Google ScholarDigital Library
- K. Shiv, K. Chow et al. SPEC jvm2008 performance characterization. In SPEC Benchmark Workshop on Computer Performance Evaluation and Benchmarking, 2009. Google ScholarDigital Library
- B. Steensgaard. Thread-specific heaps for multi-threaded programs. In International Symposium on Memory Management (ISMM), 2000. Google ScholarDigital Library
- L. Valiant. A bridging model for parallel computation. CACM, 33 (8): 103--111, 1990. Google ScholarDigital Library
- P.H. Welch and J.B. Pedersen. Santa Claus: Formal analysis of a process-oriented solution. ACM Trans. Comput. Syst., 32 (4), 2010. Google ScholarDigital Library
- X. Yang, S.M. Blackburn et al. Why nothing matters: the impact of zeroing. In Object Oriented Programming Systems Languages and applications (OOPSLA), 2011. Google ScholarDigital Library
- X. Yang, S.M. Blackburn et al. Barriers reconsidered, friendlier still! In International Symposium on Memory Management (ISMM), 2012. Google ScholarDigital Library
- Y. Zhao, J. Shi et al. Allocation wall: A limiting factor of Java applications on emerging multi-core platforms. In Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), 2009. Google ScholarDigital Library
Index Terms
- A black-box approach to understanding concurrency in DaCapo
Recommendations
Renaissance: benchmarking suite for parallel applications on the JVM
PLDI 2019: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and ImplementationEstablished benchmark suites for the Java Virtual Machine (JVM), such as DaCapo, ScalaBench, and SPECjvm2008, lack workloads that take advantage of the parallel programming abstractions and concurrency primitives offered by the JVM and the Java Class ...
A Comprehensive Java Benchmark Study on Memory and Garbage Collection Behavior of DaCapo, DaCapo Scala, and SPECjvm2008
ICPE '17: Proceedings of the 8th ACM/SPEC on International Conference on Performance EngineeringBenchmark suites are an indispensable part of scientific research to compare different approaches against each another. The diversity of benchmarks is an important asset to evaluate novel approaches for effectiveness and weaknesses. In this paper, we ...
The DaCapo benchmarks: java benchmarking development and analysis
OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applicationsSince benchmarks drive computer science research and industry product development, which ones we use and how we evaluate them are key questions for the community. Despite complex runtime tradeoffs due to dynamic compilation and garbage collection ...
Comments