Deok-Jae Oh
Jung Ho Ahn
ABSTRACT
Hardware performance monitoring units (PMUs) are a standard feature in modern microprocessors for high-performance computing (HPC) and embedded systems, by providing a rich set of microarchitectural event samplers. Recently, many profile-guided optimization (PGO) frameworks have exploited them to feature much lower profiling overhead than conventional instrumentation-based frameworks. However, existing PGO frameworks mostly focus on optimizing the layout of binaries and do not utilize rich information provided by the PMU about data access behaviors over the memory hierarchy. Thus, we propose MaPHeA, a lightweight Memory hierarchy-aware Profile-guided Heap Allocation framework applicable to both HPC and embedded systems. MaPHeA improves application performance by guiding and applying the optimized allocation of dynamically allocated heap objects with very low profiling overhead and without additional user intervention. To demonstrate the effectiveness of MaPHeA, we apply it to optimizing heap object allocation in an emerging DRAM-NVM heterogeneous memory system (HMS), and to selective huge-page utilization. In an HMS, by identifying and placing frequently accessed heap objects to the fast DRAM region, MaPHeA improves the performance of memory-intensive graph-processing and Redis workloads by 56.0% on average over the default configuration that uses DRAM as a hardware-managed cache of slow NVM. Also, by identifying large heap objects that cause frequent TLB misses and allocating them to huge pages, MaPHeA increases the performance of read and update operations of Redis by 10.6% over the transparent huge-page implementation of Linux.
- A-R. Adl-Tabatabai, R. L. Hudson, M. J. Serrano, and S. Subramoney. 2004. Prefetch Injection Based on Hardware Monitoring and Object Metadata. In Proceedings of the ACM SIGPLAN 2004 Conference on Programming Language Design and Implementation. https://doi.org/10.1145/996841.996873 Google Scholar
Digital Library
- N. Agarwal and T. F. Wenisch. 2017. Thermostat: Application-Transparent Page Management for Two-Tiered Main Memory. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems. https://doi.org/10.1145/3037697.3037706 Google ScholarDigital Library
- S. Akram, J. B. Sartor, K. S. McKinley, and L. Eeckhout. 2018. Write-Rationing Garbage Collection for Hybrid Memories. In Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation. https://doi.org/10.1145/3296979.3192392 Google ScholarDigital Library
- AMD. 2017. AMD64 Architecture Programmer’s Manual Volume 2: System Programming. https://www.amd.com/system/files/TechDocs/24593.pdfGoogle Scholar
- M. Arafa, B. Fahim, S. Kottapalli, A. Kumar, L. P. Looi, S. Mandava, A. Rudoff, I. M. Steiner, B. Valentine, G. Vedaraman, and S. Vora. 2019. Cascade Lake: Next Generation Intel Xeon Scalable Processor. IEEE Micro, 39 (2019), https://doi.org/10.1109/MM.2019.2899330 Google ScholarCross Ref
- ARM. 2019. ARM® ARM Architecture Reference Manual Armv8, for Armv8-A Architecture Profile. https://documentation-service.arm.com/static/60119835773bb020e3de6fee?token=Google Scholar
- G. Ayers, J. H. Ahn, C. Kozyrakis, and P. Ranganathan. 2018. Memory Hierarchy for Web Search. In Proceedings of the 2018 IEEE International Symposium on High Performance Computer Architecture. https://doi.org/10.1109/HPCA.2018.00061 Google ScholarCross Ref
- T. W. Barr, A. L. Cox, and S. Rixner. 2011. SpecTLB: A Mechanism for Speculative Address Translation. In Proceedings of the 38th Annual International Symposium on Computer Architecture. https://doi.org/10.1145/2024723.2000101 Google ScholarDigital Library
- A. Basu, J. Gandhi, J. Chang, M. D. Hill, and M. M. Swift. 2013. Efficient Virtual Memory for Big Memory Servers. In Proceedings of the 40th Annual International Symposium on Computer Architecture. https://doi.org/10.1145/2508148.2485943 Google ScholarDigital Library
- S. Beamer, K. Asanović, and D. Patterson. 2015. The GAP Benchmark Suite. arXiv preprint arXiv:1508.03619.Google Scholar
- C. Cantalupo, V. Venkatesan, J. Hammond, K. Czurlyo, and S. D. Hammond. 2015. Memkind: An Extensible Heap Memory Manager for Heterogeneous Memory Platforms and Mixed Memory Policies.. United States. National Nuclear Security Administration.Google Scholar
- D. Chen, T. Moseley, and D. X. Li. 2017. AutoFDO: Automatic Feedback-Directed Optimization for Warehouse-Scale Applications. In Proceedings of the 2016 International Symposium on Code Generation and Optimization. https://doi.org/10.1145/2854038.2854044 Google ScholarDigital Library
- D. Chen, N. Vachharajani, R. Hundt, S. Liao, V. Ramasamy, P. Yuan, W. Chen, and W. Zheng. 2013. Taming Hardware Event Samples for Precise and Versatile Feedback Directed Optimizations. IEEE Trans. Comput., 62 (2013), https://doi.org/10.1109/TC.2011.233 Google ScholarDigital Library
- Y. Chen, I. B. Peng, Z. Peng, X. Liu, and B. Ren. 2020. ATMem: Adaptive Data Placement in Graph Applications on Heterogeneous Memories. In Proceedings of the 18th ACM/IEEE International Symposium on Code Generation and Optimization. https://doi.org/10.1145/3368826.3377922 Google ScholarDigital Library
- T. Chilimbi, M. D. Hill, and J. R. Larus. 1999. Cache-Conscious Structure Layout. In Proceedings of the ACM SIGPLAN’99 Conference on Programming Language Design and Implementation. https://doi.org/10.1145/301631.301633 Google ScholarDigital Library
- B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears. 2010. Benchmarking Cloud Serving Systems with YCSB. In Proceedings of the 1st ACM symposium on Cloud computing. https://doi.org/10.1145/1807128.1807152 Google ScholarDigital Library
- T. A. Davis and Y. Hu. 2011. The University of Florida Sparse Matrix Collection. ACM Trans. Math. Software, https://doi.org/10.1145/2049662.2049663 Google ScholarDigital Library
- A. C. de Melo. 2009. Performance counters on Linux. In Linux Plumbers Conference.Google Scholar
- S. R. Dulloor, A. Roy, Z. Zhao, N. Sundaram, N. Satish, R. Sankaran, J. Jackson, and K. Schwan. 2016. Data Tiering in Heterogeneous Memory Systems. In Proceedings of the Eleventh European Conference on Computer Systems. https://doi.org/10.1145/2901318.2901344 Google ScholarDigital Library
- D. Greenspan. 2019. LLAMA - Automatic Memory Allocations: An LLVM Pass and Library for Automatically Determining Memory Allocations. In Proceedings of the International Symposium on Memory Systems. https://doi.org/10.1145/3357526.3357534 Google ScholarDigital Library
- T. Hirofuchi and R. Takano. 2019. The Preliminary Evaluation of a Hypervisor-based Virtualization Mechanism for Intel Optane DC Persistent Memory Module. arXiv preprint arXiv:1907.12014.Google Scholar
- J. Hu, M. Xie, C. Pan, C. J. Xue, Q. Zhuge, and E. H. . Sha. 2015. Low Overhead Software Wear Leveling for Hybrid PCM + DRAM Main Memory on Embedded Systems. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 23 (2015), https://doi.org/10.1109/TVLSI.2014.2321571 Google ScholarDigital Library
- J. Hu, Q. Zhuge, C. J. Xue, W-C. Tseng, and E. H. . Sha. 2013. Software Enabled Wear-Leveling for Hybrid PCM Main Memory on Embedded Systems. In Proceedings of the Conference on Design, Automation and Test in Europe. https://doi.org/10.7873/DATE.2013.131 Google ScholarCross Ref
- J. Hubicka. 2005. Profile Driven Optimisations in GCC. In Proceedings of GCC Summit.Google Scholar
- IBM. 2018. POWER9 Performance Monitor Unit User’s Guide. https://wiki.raptorcs.com/w/images/6/6b/POWER9_PMU_UG_v12_28NOV2018_pub.pdfGoogle Scholar
- Intel. 2018. Memory Optimizer. https://github.com/intel/memory-optimizerGoogle Scholar
- Intel. 2018. Persistent Memory Documentation. https://docs.pmem.io/persistent-memory/Google Scholar
- Intel. 2019. Intel® 64 and IA-32 Architectures Software Developer’s Manual Combined Volumes 3B: System Programming Guide. https://software.intel.com/en-us/download/intel-64-and-ia-32-architectures-sdm-volume-3b-system-programming-guide-part-2Google Scholar
- JEDEC. 2012. JEDEC Standard: DDR4 SDRAM.Google Scholar
- JEDEC. 2015. High Bandwidth Memory (HBM) DRAM.Google Scholar
- S. Kanev, J. P. Darago, K. Hazelwood, P. Ranganathan, T. Moseley, G-Y. Wei, and D. Brooks. 2015. Profiling a Warehouse-Scale Computer. In Proceedings of the 42nd Annual International Symposium on Computer Architecture. 158–169. https://doi.org/10.1145/2749469.2750392 Google ScholarDigital Library
- S. Kanev, S. L. Xi, G-Y. Wei, and D. Brooks. 2017. Mallacc: Accelerating Memory Allocation. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems. https://doi.org/10.1145/3037697.3037736 Google ScholarDigital Library
- S. Kannan, A. Gavrilovska, V. Gupta, and K. Schwan. 2017. Heteroos: Os Design for Heterogeneous Memory Management in Datacenter. In Proceedings of the 44th Annual International Symposium on Computer Architecture. https://doi.org/10.1145/3079856.3080245 Google ScholarDigital Library
- D. Khaldi and B. Chapman. 2016. Towards Automatic HBM Allocation Using LLVM: A Case Study with Knights Landing. In Third Workshop on the LLVM Compiler Infrastructure in HPC. https://doi.org/10.5555/3018869.3018871Google Scholar
- H. Kwak, C. Lee, H. Park, and S. Moon. 2010. What is Twitter, a Social Network or a News Media? In Proceedings of the 19th international conference on World wide web. https://doi.org/10.1145/1772690.1772751 Google ScholarDigital Library
- Y. Kwon, H. Yu, S. Peter, C. J. Rossbach, and E. Witchel. 2016. Coordinated and Efficient Huge Page Management with Ingens. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation. https://doi.org/10.5555/3026877.3026931Google Scholar
- S. Lee, B. Jeon, K. Kang, D. Ka, N. Kim, Y. Kim, Y. Hong, M. Kang, J. Min, M. Lee, C. Jeong, K. Kim, D. Lee, J. Shin, Y. Han, Y. Shim, Y. Kim, Y. Kim, H. Kim, J. Yun, B. Kim, S. Han, C. Lee, J. Song, H. Song, I. Park, Y. Kim, J. Chun, and J. Oh. 2019. 23.4 A 512GB 1.1 V Managed DRAM Solution with 16GB ODP and Media Controller. In Proceedins of the 2019 IEEE International Solid-State Circuits Conference. https://doi.org/10.1109/ISSCC.2019.8662367 Google ScholarCross Ref
- J. Leidel and R. C. Murphy. 2015. Hybrid Memory Cube System Interconnect Directory-Based Cache Coherence Methodology. US Patent App. 14/706,516.Google Scholar
- Linux. 2009. Transparent Hugepages. https://lwn.net/ Articles/359158Google Scholar
- Linux. 2018. PMEM NUMA Node and Hotness Accounting/Migration. https://lkml.org/lkml/2018/12/26/138Google Scholar
- L. Looi and J. X. Jianping. 2019. Intel Optane Data Center Persistent Memory. In 2019 IEEE Hot Chips 31 Symposium.Google Scholar
- C-K. Luk, R. Muth, H. Patil, R. Cohn, and G. Lowney. 2004. Ispike: A Post-Link Optimizer for the Intel/spl reg/Itanium/spl reg/Architecture. In Proceedings of the 2004 International Symposium on Code Generation and Optimization. https://doi.org/10.1109/CGO.2004.1281660 Google ScholarCross Ref
- M. Maas, D. G. Andersen, M. Isard, M. M. Javanmard, K. S. McKinley, and C. Raffel. 2020. Learning-Based Memory Allocation for C++ Server Workloads. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems. https://doi.org/10.1145/3373376.3378525 Google ScholarDigital Library
- J. Magee and A. Qasem. 2009. A Case for Compiler-Driven Superpage Allocation. In Proceedings of the 47th Annual Southeast Regional Conference. https://doi.org/10.1145/1566445.1566553 Google ScholarDigital Library
- J. Marathe and F. Mueller. 2006. Hardware Profile-Guided Automatic Page Placement for ccNUMA Systems. In Proceedings of the Eleventh ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. https://doi.org/10.1145/1122971.1122987 Google ScholarDigital Library
- M. R. Meswani, S. Blagodurov, D. Roberts, J. Slice, M. Ignatowski, and G. H. Loh. 2015. Heterogeneous Memory Architectures: A HW/SW Approach for Mixing Die-Stacked and Off-Package Memories. In Proceedings of the 2015 IEEE 21st International Symposium on High Performance Computer Architecture. https://doi.org/10.1109/HPCA.2015.7056027 Google ScholarCross Ref
- Micron. 2016. 3D XPoint Technology. https://www.micron.com/products/advanced-solutions/3d-xpoint-technologyGoogle Scholar
- R. C. Murphy, K. B. Wheeler, B. W. Barrett, and J. A. Ang. 2010. Introducing the Graph 500. Sandia National Laboratories.Google Scholar
- L. Nai, Y. Xia, I. G. Tanase, H. Kim, and C-Y. Lin. 2015. GraphBIG: Understanding Graph Computing in the Context of Industrial Solutions. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. https://doi.org/10.1145/2807591.2807626 Google ScholarDigital Library
- A. Narayan, T. Zhang, S. Aga, S. Narayanasamy, and A. Coskun. 2018. MOCA: Memory Object Classification and Allocation in Heterogeneous Memory Systems. In Proceedings of the 2018 IEEE International Parallel and Distributed Processing Symposium. https://doi.org/10.1109/IPDPS.2018.00042 Google ScholarCross Ref
- J. Navarro, S. Iyer, P. Druschel, and A. Cox. 2002. Practical, Transparent Operating System Support for Superpages. In Proceedings of the USENIX Conference on Operating Systems Design and Implementation. https://doi.org/10.1145/844128.844138 Google ScholarDigital Library
- M. B. Olson, J. T. Teague, D. Rao, M. R. JANTZ, K. A. Doshi, and P. A. Kulkarni. 2018. Cross-Layer Memory Management to Improve DRAM Energy Efficiency. ACM Transactions on Architecture and Code Optimization, 15 (2018), https://doi.org/10.1145/3196886 Google ScholarDigital Library
- G. Ottoni and B. Maher. 2017. Optimizing Function Placement for Large-Scale Data-Center Applications. In Proceedings of the 2017 International Symposium on Code Generation and Optimization. https://doi.org/10.5555/3049832.3049858Google Scholar
- M. Panchenko, R. Auler, B. Nell, and G. Ottoni. 2019. Bolt: A Practical Binary Optimizer for Data Centers and Beyond. In Proceedings of the 2019 IEEE/ACM International Symposium on Code Generation and Optimization. https://doi.org/10.5555/3314872.3314876Google Scholar
- A. Panwar, S. Bansal, and K. Gopinath. 2019. HawkEye: Efficient Fine-Grained OS Support for Huge Pages. In Proceedings of the 24th International Conference on Architectural Support for Programming Languages and Operating Systems. https://doi.org/10.1145/3297858.3304064 Google ScholarDigital Library
- K. Pettis and R. C. Hansen. 1990. Profile Guided Code Positioning. In Proceedings of the ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation. https://doi.org/10.1145/93542.93550 Google ScholarDigital Library
- M. K. Qureshi and G. H. Loh. 2012. Fundamental Latency Trade-off in Architecting DRAM Caches: Outperforming Impractical SRAM-Tags with a Simple and Practical Design. In Proceedings of the 45th Annual IEEE/ACM International Symposium on Microarchitecture. https://doi.org/10.1109/MICRO.2012.30 Google ScholarDigital Library
- M. K. Qureshi, V. Srinivasan, and J. A. Rivers. 2009. Scalable High Performance Main Memory System Using Phase-Change Memory Technology. In Proceedings of the 36th Annual International Symposium on Computer Architecture. https://doi.org/10.1145/1555754.1555760 Google ScholarDigital Library
- Redis. 2020. redis.io. https://redis.ioGoogle Scholar
- A. Rudoff. 2017. Persistent Memory Programming. Login: The Usenix Magazine.Google Scholar
- J. Savage and T. M. Jones. 2020. HALO: Post-Link Heap-Layout Optimisation. In Proceedings of the 18th ACM/IEEE International Symposium on Code Generation and Optimization. https://doi.org/10.1145/3368826.3377914 Google ScholarDigital Library
- M. L. Seidl and B. G. Zorn. 1998. Segregating Heap Objects by Reference Behavior and Lifetime. In Proceedings of the Eighth International Conference on Architectural Support for Programming Languages and Operating Systems. https://doi.org/10.1145/384265.291012 Google ScholarDigital Library
- H. Servat, A. J. Peña, G. Llort, E. Mercadal, H. Hoppe, and J. Labarta. 2017. Automating the Application Data Placement in Hybrid Memory Systems. In Proceedings of the IEEE International Conference on Cluster Computing. https://doi.org/10.1109/CLUSTER.2017.50 Google ScholarCross Ref
- C. Wang, H. Cui, T. Cao, J. Zigman, H. Volos, O. Mutlu, F. Lv, X. Feng, and G. H. Xu. 2019. Panthera: Holistic Memory Management for Big Data Processing over Hybrid Memories. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. https://doi.org/10.1145/3314221.3314650 Google ScholarDigital Library
- L. Wang, J. Zhan, C. Luo, Y. Zhu, Q. Yang, Y. He, W. Gao, Z. Jia, Y. Shi, S. Zhang, C. Zheng, G. Lu, K. Zhan, X. Li, and B. Qiu. 2014. BigDataBench: A Big Data Benchmark Suite from Internet Services. In Proceedings of the 2014 IEEE 20th International Symposium on High Performance Computer Architecture. https://doi.org/10.1109/HPCA.2014.6835958 Google ScholarCross Ref
- S. Wen, L. Cherkasova, F. X. Lin, and X. Liu. 2018. Profdp: A Lightweight Profiler to Guide Data Placement in Heterogeneous Memory Systems. In Proceedings of the 2018 International Conference on Supercomputing. https://doi.org/10.1145/3205289.3205320 Google ScholarDigital Library
- K. Wu, T. Huang, and D. Li. 2017. Unimem: Runtime Data Management on Non-Volatile Memory-Based Heterogeneous Main Memory. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. https://doi.org/10.1145/3126908.3126923 Google ScholarDigital Library
- K. Wu, J. Ren, and D. Li. 2018. Runtime Data Management on Non-Volatile Memory-Based Heterogeneous Memory for Task-Parallel Programs. In Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis. https://doi.org/10.5555/3291656.3291698Google Scholar
- Z. Yan, D. Lustig, D. Nellans, and A. Bhattacharjee. 2019. Nimble Page Management for Tiered Memory Systems. In Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems. https://doi.org/10.1145/3297858.3304024 Google ScholarDigital Library
- A. M. Yang, E. Österlund, and T. Wrigstad. 2020. Improving Program Locality in the GC Using Hotness. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation. https://doi.org/10.1145/3385412.3385977 Google ScholarDigital Library
- Z. Zhang, Z. Jia, P. Liu, and L. Ju. 2016. Energy Efficient Real-Time Task Scheduling for Embedded Systems with Hybrid Main Memory. Journal of Signal Processing Systems, 84 (2016), https://doi.org/10.1109/RTCSA.2014.6910524 Google ScholarCross Ref
Index Terms
- MaPHeA: a lightweight memory hierarchy-aware profile-guided heap allocation framework
Recommendations
MaPHeA: A Framework for Lightweight Memory Hierarchy-aware Profile-guided Heap Allocation
Hardware performance monitoring units (PMUs) are a standard feature in modern microprocessors, providing a rich set of microarchitectural event samplers. Recently, numerous profile-guided optimization (PGO) frameworks have exploited them to feature much ...
Memory system performance of programs with intensive heap allocation
Heap allocation with copying garbage collection is a general storage management technique for programming languages. It is believed to have poor memory system performance. To investigate this, we conducted an in-depth study of the memory system ...
Heap data allocation to scratch-pad memory in embedded systems
Cache exploitation in embedded systemsThis paper presents the first-ever compile-time method for allocating a portion of the heap data to scratch-pad memory. A scratch-pad is a fast directly addressed compiler-managed SRAM memory that replaces the hardware-managed cache. It is motivated by ...
Comments