Shawn T. Brown
ABSTRACT
Today’s landscape of computational science is evolving rapidly, with a need for new, flexible, and responsive supercomputing platforms for addressing the growing areas of artificial intelligence (AI), data analytics (DA) and convergent collaborative research. To support this community, we designed and deployed the Bridges-2 platform. Building on our highly successful Bridges supercomputer, which was a high-performance computing resource supporting new communities and complex workflows, Bridges-2 supports traditional and nontraditional research communities and applications; integrates new technologies for converged, scalable high-performance computing (HPC), AI, and data analytics; prioritizes researcher productivity and ease of use; and provides an extensible architecture for interoperation with complementary data intensive projects, campuses, and clouds. In this report, we describe Bridges-2’s hardware and configuration, user environments, and systems support and present the results of the successful Early User Program.
- Bridges-2 EUP Seminar 2021. Bridges-2 Early User Program Seminar. https://www.psc.edu/wp-content/uploads/2021/01/Bridges-Early-User-Workshop.pdfGoogle Scholar
- Bridges-2 User Guide 2021. Bridges-2 User Guide. https://www.psc.edu/resources/bridges-2/user-guide-2/Google Scholar
- Paola A. Buitrago and Nicholas A. Nystrom. 2021. Neocortex and Bridges-2: A High Performance AI+HPC Ecosystem for Science, Discovery, and Societal Good. In High Performance Computing, Sergio Nesmachnow, Harold Castro, and Andrei Tchernykh(Eds.). Springer International Publishing, Cham, 205–219.Google Scholar
- Paola A. Buitrago, Nicholas A. Nystrom, Rajarsi Gupta, and Joel Saltz. 2020. Delivering Scalable Deep Learning to Research with Bridges-AI. In High Performance Computing: 6th Latin American Conference, CARLA 2019: Turrialba, Costa Rica, September 25–27, 2019: Revised Selected Papers(Communications in Computer and Information Science, Vol. 1087), Juan Luis Crespo-Mariño and Esteban Meneses-Rojas (Eds.). Springer International Publishing, Switzerland, 200–214. https://doi.org/10.1007/978-3-030-41005-6_14Google ScholarCross Ref
- T. Gamblin, M. LeGendre, M. R. Collette, G. L. Lee, A. Moody, B. R. de Supinski, and S. Futral. 2015. The Spack package manager: bringing order to HPC software chaos. In SC15: International Conference for High-Performance Computing, Networking, Storage and Analysis. IEEE Computer Society, Los Alamitos, CA, USA, 1–12. https://doi.org/10.1145/2807591.2807623Google ScholarDigital Library
- Grafana 2021. Grafana website. https://grafana.com/.Google Scholar
- InfluxDB 2021. InfluxDB product website. https://www.influxdata.com/products/influxdb/.Google Scholar
- Robert McLay, Karl W. Schulz, William L. Barth, and Tommy Minyard. 2011. Best Practices for the Deployment and Management of Production HPC Clusters. In State of the Practice Reports (Seattle, Washington) (SC ’11). Association for Computing Machinery, New York, NY, USA, Article 9, 11 pages. https://doi.org/10.1145/2063348.2063360Google Scholar
- Nicholas A Nystrom, Paola A Buitrago, and Philip D Blood. 2019. Bridges: Converging HPC, AI, and Big Data for Enabling Discovery. In Contemporary High Performance Computing: From Petascale toward Exascale, Volume Three, Jeffrey S. Vetter (Ed.). CRC Press, Boca Raton, FL.Google Scholar
- Statuscake 2021. Statuscake website. https://www.statuscake.com.Google Scholar
- J. Towns, T. Cockerill, M. Dahan, I. Foster, K. Gaither, A. Grimshaw, V. Hazlewood, S. Lathrop, D. Lifka, G. D. Peterson, R. Roskies, J. R. Scott, and N. Wilkins-Diehr. 2014. XSEDE: Accelerating Scientific Discovery. Computing in Science Engineering 16, 5 (2014), 62–74. https://doi.org/10.1109/MCSE.2014.80Google ScholarCross Ref
- Warewulf3 2021. Warewulf3 website. https://warewulf.lbl.gov/.Google Scholar
Index Terms
- Bridges-2: A Platform for Rapidly-Evolving and Data Intensive Research
Recommendations
Data intensive applications on clouds
DataCloud-SC '11: Proceedings of the second international workshop on Data intensive computing in the cloudsThe cyberinfrastructure supporting science appears will include large-scale simulation systems headed to exascale combined with cloud like systems supporting data intensive and high throughput computing, pleasingly parallel jobs and the long tail of ...
Research of Grid Platform Oriented to Intensive Data Processing
GCC '08: Proceedings of the 2008 Seventh International Conference on Grid and Cooperative ComputingAiming at intensive data processing in grid environment, a grid platform IDPGP is constructed based on Globus Toolkit 4, OGSA-DAI, Condor and PBS. This platform mainly provides grid information acquirement service, grid resource management service, data ...
Enabling dynamic and intelligent workflows for HPC, data analytics, and AI convergence
AbstractThe evolution of High-Performance Computing (HPC) platforms enables the design and execution of progressively larger and more complex workflow applications in these systems. The complexity comes not only from the number of elements ...
Highlights- Analysis of the HPC, Big Data and AI convergence in complex scientific workflows.
Comments