Moinuddin K. Qureshi
Joel Emer
ABSTRACT
The commonly used LRU replacement policy is susceptible to thrashing for memory-intensive workloads that have a working set greater than the available cache size. For such applications, the majority of lines traverse from the MRU position to the LRU position without receiving any cache hits, resulting in inefficient use of cache space. Cache performance can be improved if some fraction of the working set is retained in the cache so that at least that fraction of the working set can contribute to cache hits.
We show that simple changes to the insertion policy can significantly reduce cache misses for memory-intensive workloads. We propose the LRU Insertion Policy (LIP) which places the incoming line in the LRU position instead of the MRU position. LIP protects the cache from thrashing and results in close to optimal hitrate for applications that have a cyclic reference pattern. We also propose the Bimodal Insertion Policy (BIP) as an enhancement of LIP that adapts to changes in the working set while maintaining the thrashing protection of LIP. We finally propose a Dynamic Insertion Policy (DIP) to choose between BIP and the traditional LRU policy depending on which policy incurs fewer misses. The proposed insertion policies do not require any change to the existing cache structure, are trivial to implement, and have a storage requirement of less than two bytes. We show that DIP reduces the average MPKI of the baseline 1MB 16-way L2 cache by 21%, bridging two-thirds of the gap between LRU and OPT.
- L. A. Belady. A study of replacement algorithms for a virtual-storage computer. In IBM Systems journal, pages 78--101, 1966.Google Scholar
- W. fen Lin et al. Reducing dram latencies with an integrated memory hierarchy design. In HPCA-7, pages 301--312, 2001. Google ScholarDigital Library
- A. Gonzalez, C. Aliagas, and M. Valero. A data cache with multiple caching strategies tuned to different types of locality. In ICS-9, 1995. Google ScholarDigital Library
- F. Guo and Y. Solihin. An analytical model for cache replacement policy performance. SIGMETRICS Perform. Eval. Rev., 34(1):228--239, 2006. Google ScholarDigital Library
- E. G. Hallnor and S. K. Reinhardt. A fully associative software-managed cache design. In ISCA-27, 2000. Google ScholarDigital Library
- T. L. Johnson. Run-time adaptive cache management. PhD thesis, University of Illinois, Urbana, IL, May 1998. Google ScholarDigital Library
- S. Kaxiras et al. Cache decay: exploiting generational behavior to reduce cache leakage power. In ISCA-28, 2001. Google ScholarDigital Library
- A. Lai, C. Fide, and B. Falsafi. Dead-block prediction & dead-block correlating prefetchers. In ISCA-28, 2001. Google ScholarDigital Library
- W. Lin and S. Reinhardt. Predicting last-touch references under optimal replacement. In Technical Report CSE-TR-447-02, University of Michigan, 2002.Google Scholar
- S. McFarling. Cache replacement with dynamic exclusion. In ISCA-19, pages 191--200, 1992. Google ScholarDigital Library
- N. Megiddo and D. S. Modha. ARC: A self-tuning, low overhead replacement cache. In Proceeding of the 2nd USENIX Conference on File and Storage Technologies, 2003. Google ScholarDigital Library
- E. Perelman et al. Using simpoint for accurate and efficient simulation. SIGMETRICS Perform. Eval. Rev., 31(1):318--319, 2003. Google ScholarDigital Library
- M. K. Qureshi, D. N. Lynch, O. Mutlu, and Y. N. Patt. A case for MLP-aware cache replacement. In ISCA-33, 2006. Google ScholarDigital Library
- M. K. Qureshi, D. Thompson, and Y. N. Patt. The V-Way Cache: Demand Based Associativity via Global Replacement. In ISCA-32, pages 544--555, 2005. Google ScholarDigital Library
- J. T. Robinson and M. V. Devarakonda. Data cache management using frequency-based replacement. In SIGMETRICS '90, 1990. Google ScholarDigital Library
- S. Ross. A First Course in Probability. Pearson Prentice Hall, 2006.Google Scholar
- Y. Smaragdakis et al. The EELRU adaptive replacement algorithm. Performance Evaluation, 53(2):93--123, 2003. Google ScholarDigital Library
- R. Subramanian et al. Adaptive caches: Effective shaping of cache behavior to workloads. In MICRO-39, 2006. Google ScholarDigital Library
- G. Tyson, M. Farrens, J. Matthews, and A. R. Pleszkun. A modified approach to data cache management. In MICRO-28, 1995. Google ScholarDigital Library
- Z. Wang et al. Using the compiler to improve cache replacement decisions. In PACT, page 199, 2002. Google ScholarDigital Library
- W. A. Wong and J.-L. Baer. Modified LRU policies for improving second-level cache behavior. In HPCA-6, 2000.Google Scholar
Index Terms
- Adaptive insertion policies for high performance caching
Recommendations
SHiP: signature-based hit predictor for high performance caching
MICRO-44: Proceedings of the 44th Annual IEEE/ACM International Symposium on MicroarchitectureThe shared last-level caches in CMPs play an important role in improving application performance and reducing off-chip memory bandwidth requirements. In order to use LLCs more efficiently, recent research has shown that changing the re-reference ...
High performance cache replacement using re-reference interval prediction (RRIP)
ISCA '10Practical cache replacement policies attempt to emulate optimal replacement by predicting the re-reference interval of a cache block. The commonly used LRU replacement policy always predicts a near-immediate re-reference interval on cache hits and ...
Adaptive insertion policies for high performance caching
The commonly used LRU replacement policy is susceptible to thrashing for memory-intensive workloads that have a working set greater than the available cache size. For such applications, the majority of lines traverse from the MRU position to the LRU ...
Comments