Abstract
In this research a deep convolutional Generative Adversarial Network (GAN) model is post-training quantized to a reduced precision arithmetic for a complex High Energy Physics (HEP) use-case. This research is motivated by the aim to decrease the necessary model size and computing time for reducing the required hardware resources for future Large Hadron Collider (LHC) detector simulations at CERN. However, in order to interpret the measured physics results, the detector simulations have to maintain the highest possible accuracy. Therefore, the quantized model is not only in detail analyzed in terms of hardware resource consumption but additionally comprehensively evaluated in terms of the achieved physics accuracy. We report that we achieve with the quantized model a 3.0x speed-up versus the initial model on modern CPUs. Furthermore, we investigate several new physics accuracy metrics to demonstrate that the accuracy does not significantly decrease due to the quantization process. Reduced precision computing for classification problems is already adequately studied, however, this is not the case for more complex image generation problems as we require for our use-case of detector simulations in this research. By using the Intel Neural Compressor, the quantization is performed in an iterative process. Neural Compressor automatically quantizes only the parameters of the neural network which do not decrease the accuracy of the model regarding a predefined accuracy metric. In our research we post-training quantize the GAN model from the 32-bit format down to 8-bit format.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Intel® neural compressor (2021). https://github.com/intel/neural-compressor
Agostinelli, S., et al.: GEANT4-a simulation toolkit. Nucl. Instrum. Meth. A 506, 250–303 (2003). https://doi.org/10.1016/S0168-9002(03)01368-8
Elsen, E.: A roadmap for HEP software and computing R &D for the 2020s. Comput. Softw. Big Sci. 3(1), 1–2 (2019). https://doi.org/10.1007/s41781-019-0031-6
Apollinari, G., et al.: High-luminosity large hadron collider (HL-LHC): technical design report V. 0.1 4/2017 (11 2017). https://doi.org/10.23731/CYRM-2017-004
Banner, R., Nahshan, Y., Hoffer, E., Soudry, D.: Post-training 4-bit quantization of convolution networks for rapid-deployment (2019)
Borji, A.: Pros and cons of GAN evaluation measures (2018)
Buhmann, E.: Getting high: high fidelity simulation of high granularity calorimeters with high speed (5 2020)
Feng Tian, Haihao Shen, J.G., Abidi, H.: Intel® lpot key takeaways (2021), https://www.intel.com/content/www/us/en/artificial-intelligence/posts/intel-low-precision-optimization-tool.html
Goodfellow, I.J., et al.: Generative adversarial networks (2014)
Gupta, R., Ranga, V.: Comparative study of different reduced precision techniques in deep neural network, pp. 123–136 (2021). https://doi.org/10.1007/978-981-15-8377-3-11
IEEE: IEEE standard for floating-point arithmetic. IEEE Std 754-2008, pp. 1–70 (2008)
Intel: oneAPI deep neural network library (oneDNN). https://github.com/oneapi-src/oneDNN
Itay Hubara, Yury Nahshan, Y.H., Banner, R.: Accurate post training quantization with small calibration sets (2021)
Jain, A., Bhattacharya, S., Masuda, M., Sharma, V., Wang, Y.: Efficient execution of quantized deep learning models: a compiler approach (2020)
Lu, L.: Dying relu and initialization: theory and numerical examples. Communications in Computational Physics 28(5), 1671–1706 (2020). https://doi.org/10.4208/cicp.oa-2020-0165
Micikevicius, P., et al.: Mixed precision training (2017)
Nandakumar, S.R., Le Gallo, M., Piveteau, C., Joshi, V., Mariani, G., Boybat, I., et al.: Mixed-precision deep learning based on computational memory. Front. Neurosci. 14, 406 (2020). https://doi.org/10.3389/fnins.2020.00406
Nasr, G.E., Badr, E., Joun, C.: Cross entropy error function in neural networks: forecasting gasoline demand. In: FLAIRS Conference (2002)
de Oliveira, L., Paganini, M., Nachman, B.: Learning particle physics by example: location-aware generative adversarial networks for physics synthesis. Comput. Softw. Big Sci. 1(1), 1–24 (2017). https://doi.org/10.1007/s41781-017-0004-6
Osorio, J.: Evaluating mixed-precision arithmetic for 3D generative adversarial networks to simulate high energy physics detectors
Paganini, M., de Oliveira, L., Nachman, B.: CaloGAN: simulating 3D high energy particle showers in multilayer electromagnetic calorimeters with generative adversarial networks. Phys. Rev. D 97(1), 014021 (2018). https://doi.org/10.1103/physrevd.97.014021
Pierini, M., Zhang, M.: CLIC Calorimeter 3D images: electron showers at fixed angle (2020). https://doi.org/10.5281/zenodo.3603122
PyTorch: Introduction to quantization on pyTorch (2020). https://pytorch.org/blog/introduction-to-quantization-on-pytorch/
Rehm, F., Vallecorsa, S., Borras, K., Krücker, D.: Validation of deep convolutional generative adversarial networks for high energy physics calorimeter simulations (2021)
Rehm, F., et al.: Reduced precision strategies for deep learning: a high energy physics generative adversarial network use case. In: Proceedings of the 10th International Conference on Pattern Recognition Applications and Methods (2021). https://doi.org/10.5220/0010245002510258
Swamidass, P.M. (ed.): MAPE (mean absolute percentage error). In: Swamidass, P.M. (ed.) Encyclopedia of Production and Manufacturing Management, p. 462. Springer, Boston (2000). https://doi.org/10.1007/1-4020-0612-8_580
TensorFlow Lite: Post training quantization. https://www.tensorflow.org/lite/performance/post_training_quantization
Vallecorsa, S., Carminati, F., Khattak, G.: 3D convolutional GAN for fast simulation. EPJ Web of Conferences 214, 02010 (2019). https://doi.org/10.1051/epjconf/201921402010
Wang, N., Choi, J., Brand, D., Chen, C.Y., Gopalakrishnan, K.: Training deep neural networks with 8-bit floating point numbers (2018)
Wu, H.: Inference at reduced precision on GPUs (2019). https://developer.download.nvidia.com/video/gputechconf/gtc/2019/presentation/s9659-inference-at-reduced-precision-on-gpus.pdf
Wu, H., Judd, P., Zhang, X., Isaev, M., Micikevicius, P.: Integer quantization for deep learning inference: Principles and empirical evaluation (2020)
Acknowledgements
This work has been sponsored by the Wolfgang Gentner Programme of the German Federal Ministry of Education and Research.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Springer Nature Switzerland AG
About this paper
Cite this paper
Rehm, F., Saletore, V., Vallecorsa, S., Borras, K., Krücker, D. (2023). Reduced Precision Research of a GAN Image Generation Use-case. In: De Marsico, M., Sanniti di Baja, G., Fred, A. (eds) Pattern Recognition Applications and Methods. ICPRAM ICPRAM 2021 2022. Lecture Notes in Computer Science, vol 13822. Springer, Cham. https://doi.org/10.1007/978-3-031-24538-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-24538-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-24537-4
Online ISBN: 978-3-031-24538-1
eBook Packages: Computer ScienceComputer Science (R0)
