Skip to main content
SpringerLink
Log in

A Prefetch-Adaptive Intelligent Cache Replacement Policy Based on Machine Learning

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Hardware prefetching and replacement policies are two techniques to improve the performance of the memory subsystem. While prefetching hides memory latency and improves performance, interactions take place with the cache replacement policies, thereby introducing performance variability in the application. To improve the accuracy of reuse of cache blocks in the presence of hardware prefetching, we propose Prefetch-Adaptive Intelligent Cache Replacement Policy (PAIC). PAIC is designed with separate predictors for prefetch and demand requests, and uses machine learning to optimize reuse prediction in the presence of prefetching. By distinguishing reuse predictions for prefetch and demand requests, PAIC can better combine the performance benefits from prefetching and replacement policies. We evaluate PAIC on a set of 27 memory-intensive programs from the SPEC 2006 and SPEC 2017. Under single-core configuration, PAIC improves performance over Least Recently Used (LRU) replacement policy by 37.22%, compared with improvements of 32.93% for Signature-based Hit Predictor (SHiP), 34.56% for Hawkeye, and 34.43% for Glider. Under the four-core configuration, PAIC improves performance over LRU by 20.99%, versus 13.23% for SHiP, 17.89% for Hawkeye and 15.50% for Glider.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Zangeneh S, Pruett S, Lym S, Patt Y N. BranchNet: A convolutional neural network to predict hard-to-predict branches. In Proc. the 53rd Annual IEEE/ACM International Symposium on Microarchitecture, Oct. 2020, pp.118–130. DOI: 10.1109/MICRO50266.2020.00022.

  2. Teran E, Wang Z, Jiménez D A. Perceptron learning for reuse prediction. In Proc. the 49th Annual IEEE/ACM International Symposium on Microarchitecture, Oct. 2016. DOI: 10.1109/MICRO.2016.7783705.

  3. Shi Z, Huang X R, Jain A, Lin C. Applying deep learning to the cache replacement problem. In Proc. the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, Oct. 2019, pp.413–425. DOI: 10.1145/3352460.3358319.

  4. Ipek E, Mutlu O, Martínez J F, Caruana R. Self-optimizing memory controllers: A reinforcement learning approach. ACM SIGARCH Computer Architecture News, 2008, 36(3): 39–50. DOI: https://doi.org/10.1145/1394608.1382172.

    Article  Google Scholar 

  5. Hallnor E G, Reinhardt S K. A fully associative softwaremanaged cache design. In Proc. the 27th Annual International Symposium on Computer Architecture, Jun. 2000, pp.107–116. DOI: 10.1145/339647.339660.

  6. Jaleel A, Theobald K B, Steely Jr S C, Emer J. High performance cache replacement using re-reference interval prediction (RRIP). ACM SIGARCH Computer Architecture News, 2010, 38(3): 60–71. DOI: https://doi.org/10.1145/1816038.1815971.

    Article  Google Scholar 

  7. Wu C J, Jaleel A, Hasenplaugh W, Martonosi M. SHiP: Signature-based hit predictor for high performance caching. In Proc. the 44th Annual IEEE/ACM International Symposium on Microarchitecture, Dec. 2011, pp.430–441. DOI: 10.1145/2155620.2155671.

  8. Jain A, Lin C. Back to the future: Leveraging Belady’s algorithm for improved cache replacement. In Proc. the 43rd Annual International Symposium on Computer Architecture, Jun. 2016, pp.78–89. DOI: 10.1109/ISCA.2016.17.

  9. Wu C J, Jaleel A, Martonosi M, Steely S C, Emer J. PACMan: Prefetch-aware cache management for high performance caching. In Proc. the 44th Annual IEEE/ACM International Symposium on Microarchitecture, Dec. 2011, pp.442–453. DOI: 10.1145/2155620.2155672.

  10. Heirman W, Bois K D, Vandriessche Y, Eyerman S, Hur I. Near-side prefetch throttling: Adaptive prefetching for high-performance many-core processors. In Proc. the 27th International Conference on Parallel Architectures and Compilation Techniques, Nov. 2018, Article No. 28. DOI: https://doi.org/10.1145/3243176.3243181.

  11. Ishii Y, Inaba M, Hiraki K. Unified memory optimizing architecture: Memory subsystem control with a unified predictor. In Proc. the 26th ACM International Conference on Supercomputing, Jun. 2012, pp.267–278. DOI: 10.1145/2304576.2304614.

  12. Kim J, Pugsley S H, Gratz P V, Reddy A L N, Wilkerson C, Chishti Z. Path confidence based lookahead prefetching. In Proc. the 49th Annual IEEE/ACM International Symposium on Microarchitecture, Oct. 2016. DOI: 10.1109/MICRO.2016.7783763.

  13. O'Neil E J, O'Neil P E, Weikum G. The LRU-K page replacement algorithm for database disk buffering. ACM SIGMOD Record, 1993, 22(2): 297–306. DOI: https://doi.org/10.1145/170036.170081.

    Article  Google Scholar 

  14. Srinath S, Mutlu O, Kim H, Patt Y N. Feedback directed prefetching: Improving the performance and bandwidth-efficiency of hardware prefetchers. In Proc. the 13th International Symposium on High Performance Computer Architecture, Feb. 2007, pp.63–74. DOI: 10.1109/HPCA.2007.346185.

  15. Jain A, Lin C. Rethinking Belady’s algorithm to accommodate prefetching. In Proc. the 45th Annual International Symposium on Computer Architecture, Jun. 2018, pp.110–123. DOI: 10.1109/ISCA.2018.00020.

  16. Kim J, Teran E, Gratz P V, Jiménez D A, Pugsley S H, Wilkerson C. Kill the program counter: Reconstructing program behavior in the processor cache hierarchy. ACM SIGPLAN Notices, 2017, 52(4): 737–749. DOI: https://doi.org/10.1145/3093336.3037701.

    Article  Google Scholar 

  17. Henning J L. SPEC CPU2006 benchmark descriptions. ACM SIGARCH Computer Architecture News, 2006, 34(4): 1–17. DOI: https://doi.org/10.1145/1186736.1186737.

    Article  Google Scholar 

  18. Bucek J, Lange K D, von Kistowski J. SPEC CPU2017: Next-generation compute benchmark. In Proc. the 2018 ACM/SPEC International Conference on Performance Engineering, Apr. 2018, pp.41–42. DOI: 10.1145/3185768.3185771.

  19. Perelman E, Hamerly G, Van Biesbrouck M, Sherwood T, Calder B. Using SimPoint for accurate and efficient simulation. ACM SIGMETRICS Performance Evaluation Review, 2003, 31(1): 318–319. DOI: https://doi.org/10.1145/885651.781076.

    Article  Google Scholar 

  20. Bhatia E, Chacon G, Pugsley S, Teran E, Gratz P V, Jiménez D A. Perceptron-based prefetch filtering. In Proc. the 46th International Symposium on Computer Architecture, Jun. 2019. DOI: 10.1145/3307650.3322207.

  21. Shevgoor M, Koladiya S, Balasubramonian R, Wilkerson C, Pugsley S H, Chishti Z. Efficiently prefetching complex address patterns. In Proc. the 48th International Symposium on Microarchitecture, Dec. 2015, pp.141–152. DOI: 10.1145/2830772.2830793.

  22. Seshadri V, Yedkar S, Xin H Y, Mutlu O, Gibbons P B, Kozuch M A, Mowry T C. Mitigating prefetcher-caused pollution using informed caching policies for prefetched blocks. ACM Transactions on Architecture and Code Optimization, 2015, 11(4): Article No. 51. DOI: https://doi.org/10.1145/2677956.

Download references

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Hui-Jing Yang, Juan Fang, Min Cai & Zhi Cai

Authors
  1. Hui-Jing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhi Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Fang.

Supplementary Information

ESM 1

(PDF 214 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, HJ., Fang, J., Cai, M. et al. A Prefetch-Adaptive Intelligent Cache Replacement Policy Based on Machine Learning. J. Comput. Sci. Technol. 38, 391–404 (2023). https://doi.org/10.1007/s11390-022-1573-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-022-1573-3

Keywords

Search

Navigation