Jesus Carretero
Marco Aldinucci
Jean Baptiste Besnard
Felix Wolf
ABSTRACT
The main objective of the ADMIRE project1 is the creation of an active I/O stack that dynamically adjusts computation and storage requirements through intelligent global coordination, the elasticity of computation and I/O, and the scheduling of storage resources along all levels of the storage hierarchy, while offering quality-of-service (QoS), energy efficiency, and resilience for accessing extremely large data sets in very heterogeneous computing and storage environments. We have developed a framework prototype that is able to dynamically adjust computation and storage requirements through intelligent global coordination, separated control, and data paths, the malleability of computation and I/O, the scheduling of storage resources along all levels of the storage hierarchy, and scalable monitoring techniques. The leading idea in ADMIRE is to co-design applications with ad-hoc storage systems that can be deployed with the application and adapt their computing and I/O behaviour on runtime, using malleability techniques, to increase the performance of applications and the throughput of the applications.
- Jean Luca Bez, Alberto Miranda, Ramon Nou, Francieli Zanon Boito, Toni Cortes, and Philippe O. A. Navaux. 2021. Arbitration Policies for On-Demand User-Level I/O Forwarding on HPC Platforms. In 35th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2021, Portland, OR, USA, May 17-21, 2021. IEEE, 577--586.Google Scholar
Cross Ref
- André Brinkmann, Kathryn Mohror, Weikuan Yu, Philip H. Carns, Toni Cortes, Scott Klasky, Alberto Miranda, Franz-Josef Pfreundt, Robert B. Ross, and Marc-Andre Vef. 2020. Ad Hoc File Systems for High-Performance Computing. J. Comput. Sci. Technol. 35, 1 (2020), 4--26.Google ScholarDigital Library
- Félix Garcia-Carballeira, Alejandro Calderon, Jesus Carretero, Javier Fernandez, and Jose M Perez. 2003. The design of the Expand parallel file system. The International Journal of High Performance Computing Applications 17, 1 (2003), 21--37.Google ScholarDigital Library
- Florin Isaila, Javier Garcia-Blas, Jesus Carretero, Rob Ross, and Dries Kimpe. 2017. Making the case for reforming the I/O software stack of extreme-scale systems. Advances in Engineering Software 111 (2017), 26--31. Advances in High Performance Computing: on the path to Exascale software.Google ScholarCross Ref
- Raffaele Montella, Diana Di Luccio, Pasquale Troiano, Angelo Riccio, Alison Brizius, and Ian Foster. 2016. WaComM: A parallel Water quality Community Model for pollutant transport and dispersion operational predictions. In 2016 12th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS). IEEE, 717--724.Google ScholarCross Ref
- Francisco José Rodrigo Duro, Fabrizio Marozzo, Javier García Blas, Jesús Carretero Pérez, Domenico Talia, and Paolo Trunfio. 2015. Evaluating data caching techniques in DMCF workflows using Hercules. (2015).Google Scholar
- M. Vef, N. Moti, T. Süß, T. Tocci, R. Nou, A. Miranda, T. Cortes, and A. Brinkmann. 2018. GekkoFS - A Temporary Distributed File System for HPC Applications. In 2018 IEEE International Conference on Cluster Computing. 319--324.Google Scholar
- Chen Wang. 2022. Detecting Data Races on Relaxed Systems Using Recorder.Google Scholar
- Chen Wang, Kathryn Mohror, and Marc Snir. 2021. File System Semantics Requirements of HPC Applications. In HPDC '21: The 30th International Symposium on High-Performance Parallel and Distributed Computing, Virtual Event. ACM.Google Scholar
Index Terms
- Adaptive multi-tier intelligent data manager for Exascale
Recommendations
A Multi-Tier RAID Storage System with RAID1 and RAID5
IPDPS '00: Proceedings of the 14th International Symposium on Parallel and Distributed ProcessingRedundant Arrays of Inexpensive Disks (RAID) is a popular technique used to improve the reliability and performance of secondary storage. Of various levels of RAID discussed in \cite {bib:raid}, RAID1 and RAID5 have become more popular. Mirroring or ...
Malleable and Adaptive Ad-Hoc File System for Data Intensive Workloads in HPC Applications
High Performance ComputingAbstractAdvancement in storage technologies, such as NVMe and persistent memory, enables the acceleration of I/O operations in HPC systems. However, relying solely on ultra-fast storage devices is not cost-effective, leading to the need for multi-tier ...
Evaluation of Exclusive Data Allocation Between SSD Tier and SSD Cache in Storage Systems
ICEIS 2014: Proceedings of the 16th International Conference on Enterprise Information Systems - Volume 1We proposed and evaluated the storage I/O response time with the exclusive allocation method between SSD for tiered volume and SSD for cache in the storage system utilizing SSD and HDD. In the proposed method, the SSD cache function with exclusive ...
Comments