Upma Gandhi
Ismail S. K. Bustany
Skip Abstract Section
Abstract
Physical designers typically employ heuristics to solve challenging problems in global routing. However, these heuristic solutions are not adaptable to the ever-changing fabrication demands, and the experience and creativity of designers can limit their effectiveness. Reinforcement learning (RL) is an effective method to tackle sequential optimization problems due to its ability to adapt and learn through trial and error. Hence, RL can create policies that can handle complex tasks. This work presents an RL framework for global routing that incorporates a self-learning model called RL-Ripper. The primary function of RL-Ripper is to identify the best nets that need to be ripped and rerouted in order to decrease the number of total short violations. In this work, we show that the proposed RL-Ripper framework’s approach can reduce the number of short violations for ISPD 2018 Benchmarks when compared to the state-of-the-art global router CUGR. Moreover, RL-Ripper reduced the total number of short violations after the first iteration of detailed routing over the baseline while being on par with the wirelength, VIA, and runtime. The proposed framework’s major impact is providing a novel learning-based approach to global routing that can be replicated for newer technologies.
- Charu C Aggarwal et al. 2018. Neural networks and deep learning. Springer 10, 978 (2018), 3.Google Scholar
- Erfan Aghaeekiasaraee, Aysa Fakheri Tabrizi, Tiago Augusto Fontana, Renan Netto, Sheiny Fabre Almeida, Upma Gandhi, José Luís Güntzel, David Westwick, and Laleh Behjat. 2023. CRP2. 0: a fast and robust cooperation between routing and placement in advanced technology nodes. ACM Transactions on Design Automation of Electronic Systems 28, 5(2023), 1–42.Google ScholarDigital Library
- Charles J. Alpert, Dinesh P. Mehta, and Sachin S. Sapatnekar. 2008. Handbook of Algorithms for Physical Design Automation (1st ed.). Auerbach Publications, USA.Google Scholar
- Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arxiv:1606.01540 [cs.LG]Google Scholar
- Yen-Jung Chang, Yu-Ting Lee, Jhih-Rong Gao, Pei-Ci Wu, and Ting-Chi Wang. 2010. NTHU-Route 2.0: A Robust Global Router for Modern Designs. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 29, 12(2010), 1931–1944. https://doi.org/10.1109/TCAD.2010.2061590Google ScholarDigital Library
- Minsik Cho, Katrina Lu, Kun Yuan, and David Pan. 2007. BoxRouter 2.0: Architecture and implementation of a hybrid and robust global router. In IEEE/ACM International Conference on Computer-Aided Design. IEEE/ACM, San Jose, USA, 503–508. https://doi.org/10.1109/ICCAD.2007.4397314Google ScholarCross Ref
- Xinguo Deng, Sijin Ye, Jiarui Chen, and Chuandong Chen. 2021. Ordered Escape Routing Combining Improved A* Algorithm with Rip-up and Reroute. Journal of Electronics & Information Technology 43, 201033(2021), 1609. https://doi.org/10.11999/JEIT201033Google ScholarCross Ref
- Wang Fangzhou and Wanying Zheng. 2020. To Rip or not to Rip: A Reinforcement Learning-based Rip-up and Reroute Algorithm for Global Routing.Google Scholar
- Upma Gandhi, Erfan Aghaeekiasaraee, Ismail S. K. Bustany, Payam Mousavi, Matthew E. Taylor, and Laleh Behjat. 2023. RL-Ripper: A Framework for Global Routing Using Reinforcement Learning and Smart Net Ripping Techniques. In Proceedings of the Great Lakes Symposium on VLSI 2023 (Knoxville, TN, USA) (GLSVLSI ’23). Association for Computing Machinery, New York, NY, USA, 197–201. https://doi.org/10.1145/3583781.3590312Google ScholarDigital Library
- Upma Gandhi, Ismail Bustany, William Swartz, and Laleh Behjat. 2019. A reinforcement learning-based framework for solving physical design routing problem in the absence of large test sets. In 2019 ACM/IEEE 1st Workshop on Machine Learning for CAD (MLCAD). IEEE, ACM/IEEE, Banff, Canada, 1–6.Google ScholarCross Ref
- Rahul Ghose, Soummyo Priyo Chattopadhyay, Tiyasha Das, Tejes Das, and Ayoshna Saha. 2015. An overview of global routing. In 2015 International Conference and Workshop on Computing and Communication (IEMCON). IEMCON, Vancouver, Canada, 1–5. https://doi.org/10.1109/IEMCON.2015.7344530Google ScholarCross Ref
- Stèphano M. M. Gonçalves, Leomar S. da Rosa, and Felipe De S. Marques. 2017. A survey of path search algorithms for VLSI detailed routing. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, Baltimore, USA, 1–4. https://doi.org/10.1109/ISCAS.2017.8050432Google ScholarCross Ref
- Peter E. Hart, Nils J. Nilsson, and Bertram Raphael. 1968. A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Transactions on Systems Science and Cybernetics 4, 2(1968), 100–107. https://doi.org/10.1109/TSSC.1968.300136Google ScholarCross Ref
- Pieter Hintjens. 2013. ZeroMQ: messaging for many applications. ” O’Reilly Media, Inc.”, Sebastopol, USA.Google Scholar
- Haishan Huang, Weijie Chen, Zhijie Cai, Jiarong Huang, and Jianli Chen. 2022. An Efficient Global Router for Large-scale Congestion-driven Routing. In 2022 IEEE 16th International Conference on Solid-State & Integrated Circuit Technology (ICSICT). IEEE, Nangjing,China, 1–3. https://doi.org/10.1109/ICSICT55466.2022.9963327Google ScholarCross Ref
- R. Hyndman, A.B. Koehler, J.K. Ord, and R.D. Snyder. 2008. Forecasting with Exponential Smoothing: The State Space Approach. Springer Berlin Heidelberg, Heidelberg, Germany. https://books.google.ca/books?id=GSyzox8Lu9YCGoogle ScholarCross Ref
- Cadence Design Systems Inc. 2018. LEF/DEF Language Reference. https://www.ispd.cc/contests/18/lefdefref.pdfGoogle Scholar
- Andrew B. Kahng, Jens Lienig, Igor L. Markov, and Jin Hu. 2011. VLSI Physical Design: From Graph Partitioning to Timing Closure (1st ed.). Springer Publishing Company, Incorporated, Cham, Switzerland.Google Scholar
- Andrew B Kahng, Lutong Wang, and Bangqi Xu. 2018. Tritonroute: An initial detailed router for advanced vlsi technologies. In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, IEEE/ACM, San Diego, USA, 1–8.Google ScholarDigital Library
- Vijay Konda and John Tsitsiklis. 1999. Actor-Critic Algorithms. In Advances in Neural Information Processing Systems, S. Solla, T. Leen, and K. Müller (Eds.), Vol. 12. MIT Press, Denver, USA,. https://proceedings.neurips.cc/paper_files/paper/1999/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdfGoogle Scholar
- Maxim Lapan. 2018. Deep Reinforcement Learning Hands-On: Apply modern RL methods, with deep Q-networks, value iteration, policy gradients, TRPO, AlphaGo Zero and more. Packt Publishing, Birmingham, UK). https://books.google.ca/books?id=xKdhDwAAQBAJGoogle Scholar
- C. Y. Lee. 1961. An Algorithm for Path Connections and Its Applications. IRE Transactions on Electronic Computers EC-10, 3 (1961), 346–365. https://doi.org/10.1109/TEC.1961.5219222Google ScholarCross Ref
- Haiguang Liao, Wentai Zhang, Xuliang Dong, Barnabas Poczos, Kenji Shimada, and Levent Burak Kara. 2020. A deep reinforcement learning approach for global routing. Journal of Mechanical Design 142, 6 (2020), 061701.Google ScholarCross Ref
- Jinwei Liu, Chak-Wa Pui, Fangzhou Wang, and Evangeline F. Y. Young. 2020. CUGR: Detailed-Routability-Driven 3D Global Routing with Probabilistic Resource Model. In Proceedings of the 57th ACM/EDAC/IEEE Design Automation Conference (DAC ’20). IEEE Press, Virtual Event, USA, Article 128, 6 pages.Google Scholar
- Wen-Hao Liu, Wei-Chun Kao, Yih-Lang Li, and Kai-Yuan Chao. 2013. NCTU-GR 2.0: Multithreaded Collision-Aware Global Routing With Bounded-Length Maze Routing. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 32, 5(2013), 709–722. https://doi.org/10.1109/TCAD.2012.2235124Google ScholarDigital Library
- Stefanus Mantik, Gracieli Posser, Wing-Kai Chow, Yixiao Ding, and Wen-Hao Liu. 2018. ISPD 2018 initial detailed routing contest and benchmarks. In Proceedings of the 2018 International Symposium on Physical Design. ISPD, Monterey, USA, 140–143.Google Scholar
- Azalia Mirhoseini, Anna Goldie, Mustafa Yazgan, Joe Wenjie Jiang, Ebrahim Songhori, Shen Wang, Young-Joon Lee, Eric Johnson, Omkar Pathak, Azade Nazi, et al. 2021. A graph placement methodology for fast chip design. Nature 594, 7862 (2021), 207–212.Google Scholar
- Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. In International conference on machine learning (ICLML). PMLR, ICML, New York City, USA, 1928–1937.Google Scholar
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. arxiv:1312.5602 [cs.LG]Google Scholar
- Muddasar Naeem, Syed Tahir Hussain Rizvi, and Antonio Coronato. 2020. A Gentle Introduction to Reinforcement Learning and its Application in Different Fields. IEEE Access 8(2020), 209320–209344. https://doi.org/10.1109/ACCESS.2020.3038605Google ScholarCross Ref
- Zhongdong Qi, Jingchong Zhang, Gengjie Chen, and Hailong You. 2022. Effective and Efficient Detailed Routing with Adaptive Rip-up Scheme and Pin Access Refinement. In Proceedings of the Great Lakes Symposium on VLSI 2022 (Irvine, CA, USA) (GLSVLSI ’22). Association for Computing Machinery, New York, NY, USA, 165–168. https://doi.org/10.1145/3526241.3530361Google ScholarDigital Library
- Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. 2021. Stable-baselines3: Reliable reinforcement learning implementations. The Journal of Machine Learning Research 22, 1 (2021), 12348–12355.Google ScholarDigital Library
- Ulf Schlichtmann, Sabya Das, Chao Lin, and Mark Po-Hung Lin. 2019. Overview of 2019 CAD Contest at ICCAD. In 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). IEEE, IEEE/ACM, Westminster, USA, 1–2.Google ScholarCross Ref
- Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al. 2020. Mastering atari, go, chess and shogi by planning with a learned model. Nature 588, 7839 (2020), 604–609.Google Scholar
- Naveed A. Sherwani. 1995. Algorithms for VLSI Physical Design Automation (2nd ed.). Kluwer Academic Publishers, USA.Google Scholar
- Umair F. Siddiqi, Sadiq M. Sait, and Yoichi Shiraishi. 2015. A Game Theory-Based Heuristic for the Two-Dimensional VLSI Global Routing Problem. Journal of Circuits, Systems and Computers 24, 06 (2015), 1550082. https://doi.org/10.1142/S0218126615500826 arXiv:https://doi.org/10.1142/S0218126615500826Google ScholarCross Ref
- Richard S Sutton, Andrew G Barto, et al. 1998. Introduction to reinforcement learning. Vol. 135. MIT press Cambridge, Cambridge, USA.Google Scholar
- Gerald Tesauro. 1995. Temporal Difference Learning and TD-Gammon. Commun. ACM 38, 3 (mar 1995), 58–68. https://doi.org/10.1145/203330.203343Google ScholarDigital Library
- Geetanjali Udgirkar and G Indumathi. 2016. VLSI global routing algorithms: A survey. In 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom). INDIACom, New Delhi,India, 2528–2533.Google Scholar
- Peter R Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, et al. 2022. Outracing champion Gran Turismo drivers with deep reinforcement learning. Nature 602, 7896 (2022), 223–228.Google Scholar
- Peng Zou, Xiqiong Bai, Yingjie Wu, Lifeng Wu, and Jianli Chen. 2019. An Effective Detailed Routing Algorithm Considering Advanced VLSI Technologies. In 2019 IEEE 13th International Conference on ASIC (ASICON). IEEE, Chongqing, China, 1–4. https://doi.org/10.1109/ASICON47005.2019.8983638Google ScholarCross Ref
Index Terms
- Applying reinforcement learning to learn best net to rip and re-route in global routing
Recommendations
RL-Ripper:: A Framework for Global Routing Using Reinforcement Learning and Smart Net Ripping Techniques
GLSVLSI '23: Proceedings of the Great Lakes Symposium on VLSI 2023Physical designers have been using heuristics to solve challenging problems in routing. However, these heuristic solutions are not adaptable to the ever-changing fabrication demands and their effectiveness is limited by the experience and creativity of ...
V-GR: 3D Global Routing with via Minimization and Multi-Strategy Rip-Up and Rerouting
ASPDAC '24: Proceedings of the 29th Asia and South Pacific Design Automation ConferenceIn VLSI, a large number of vias may reduce manufacturability, degrade circuit performance, and increase layout area required for interconnection. In this paper, we propose a 3D global router V-GR, which considers minimizing the number of vias. V-GR uses ...
Multi-threaded collision-aware global routing with bounded-length maze routing
DAC '10: Proceedings of the 47th Design Automation ConferenceModern global routers use various routing methods to improve routing speed and the quality. Maze routing is the most time-consuming process for existing global routing algorithms. This paper presents two bounded-length maze routing (BLMR) algorithms (...
Comments