Published January 31, 2023 | Version 1.0.0
Dataset Open

M100 dataset 2: from 21-01 to 21-06

Description

This entry is a part of a larger data set collected from the most recent Tier-0 supercomputer hosted at CINECA (Marconi100, https://www.hpc.cineca.it/hardware/marconi100). The data covers the entirety of the system, ranging from the computing nodes (980+ computing nodes) internal information such as core loads, temperatures, frequencies, memory write/read operations, CPU power consumption, fan speed, GPU usage details, etc., to the system-wide information, including the liquid cooling infrastructure, the air conditioning system, the power supply units, workload manager statistics, and job-related information, system status alerts, and weather forecast.
It comprises hundreds of metrics measured on each computing node, in addition to hundreds of other metrics gathered from sensors monitored along all system components.
The whole data set is stored as a collection of Zenodo entries; this particular entry corresponds to the period: 21-01, 21-06.

The dataset is stored as a partitioned Parquet dataset, with this partitioning hierarchy: year_month ("YY-MM"), plugin, metric. The data is distributed as tarball files, each corresponding to one month of data (first-level partitioning, year_month).
The collected data is generated by a monitoring infrastructure working on unstructured data (to improve efficiency and scalability); however, this data has been organized in a structured manner to facilitate its fruition. The simplest way to understand how the access the data is to refer to the companion software modules released together with the dataset itself, which can be found at: https://gitlab.com/ecs-lab/exadata.

Files

Files (45.3 GB)

Name Size Download all
md5:23010c76b0fa8a88b980afe0e53da323
6.8 GB Download
md5:10eed30aa685300d811a42d8feb504b6
6.4 GB Download
md5:3ce2cf1131024aeb22caf9cd87bdacb4
601.9 MB Download
md5:5890f5c75368da66302b61be12d6aedc
6.9 GB Download
md5:eed96a2d5b26e3d6f87c603d8104e6fd
9.9 GB Download
md5:1dc58f8cd27c80c54886d742bc82e3d9
14.6 GB Download

Additional details

Funding

REGALE – An open architecture to equip next generation HPC applications with exascale capabilities 956560
European Commission
EUPEX – EUROPEAN PILOT FOR EXASCALE 101033975
European Commission
The European PILOT – Pilot using Independent Local & Open Technologies 101034126
European Commission
DECICE – Device-Edge-Cloud Intelligent Collaboration framEwork 101092582
European Commission
Graph-Massivizer – Extreme and Sustainable Graph Processing for Urgent Societal Challenges in Europe 101093202
European Commission