Abstract
The ATLAS experiment at the LHC processes, analyses and stores vast amounts of data, which is either recorded by the detector or simulated worldwide using Monte Carlo methods. ATLAS Computing metadata is generated at very high rates and volumes. The necessity to analyze this metadata is constantly increasing, since the heterogeneous, distributed and dynamically changing computing infrastructure requires sophisticated optimization decisions, made by human or/and by machines. Visual analytics is one of the methods facilitating the analysis of massive amounts of data (structured, semi-structured, and unstructured) which leverages human judgement by means of interactive visual representations. Given the huge number of ATLAS computing jobs that need to be visualized simultaneously for error investigations or other optimization processes, resources of the client application responsible for such visualization may reach its limits. Data objects that share similar feature values can be represented and visualized as a single group, thus initial large data sample would be represented at different levels of detail. This approach will also avoid client overload. In this paper we evaluate implementations of k-means-based Level-of-Detail generator method applied to the metadata of ATLAS jobs. This method is used in the visual analytics application InVEx (Interactive Visual Explorer) that is under development, and which is based on 3-dimensional interactive visualization of multidimensional data.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
The ATLAS Collab., “The ATLAS experiment at the CERN Large Hadron Collider,” J. Instrum. 3 (08), S08003 (2008).
ATLAS Fact Sheet: To raise awareness of the ATLAS detector and collaboration on the LHC, ATLAS-Brochure-2010-002-Eng. https://cds.cern.ch/record/1457044/. Accessed 2019.
F. H. Barreiro et al., “The ATLAS production system evolution: new data processing and analysis paradigm for the LHC Run2 and high-luminosity,” J. Phys.: Conf. Ser. 898, 052016 (2017).
F. H. Barreiro et al., “PanDA: exascale federation of resources for the ATLAS experiment at the LHC,” EPJ Web Conf. 108, 01001 (2016).
M. Titov, M. Borodin, D. Golubkov, and A. Klimentov, “The ATLAS production system predictive analytics service: an approach for intelligent task analysis,” CEUR Workshop Proc. 2267, 124–128 (2018).
P. C. Wong and J. Thomas, “Visual analytics,” IEEE Comput. Graph. Appl. 24 (5), 20–21 (2004).
A. Alekseev, A. Klimentov, T. Korchuganova, S. Padolski, and T. Wenaus, “ATLAS BigPanDA monitoring,” J. Phys.: Conf. Ser. 1085, 032043 (2018).
J. Andreeva et al., “Experiment dashboard for monitoring of the LHC distributed computing systems,” J. Phys.: Conf. Ser. 331, 072001 (2011).
J. Nielsen, Usability Engineering (Morgan Kaufmann, San Francisco, 1993).
J. Bejar, “K-means vs Mini Batch K-means: a comparison,” UPCommons, Res. Report (Polytech. Univ. of Catalonia, 2013). http://hdl.handle.net/2117/23414. Accessed 2019.
M. Chavan, As. Patil, L. Dalvi, and Aj. Patil, “KMini batch K-means clustering on large dataset,” Int. J. Sci. Eng. Technol. Res. 004, 1356–1358 (2015).
Scikit-learn: Machine Learning in Python, Online documentation: Clustering - Mini Batch K-Means, https://scikit-learn.org/stable/modules/clustering.html#mini-batch-kmeans. Accessed 2019.
Funding
We gratefully acknowledge the financial support from the Russian Science Foundation grant no. 18-71-10003.
Author information
Authors and Affiliations
Corresponding authors
Additional information
Submitted by E. E. Tyrtyshnikov
Rights and permissions
About this article
Cite this article
Grigorieva, M.A., Titov, M.A., Alekseev, A.A. et al. Evaluation of the Level-of-Detail Generator for Visual Analysis of the ATLAS Computing Metadata. Lobachevskii J Math 40, 1788–1798 (2019). https://doi.org/10.1134/S199508021911012X
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S199508021911012X