Abstract
This paper proposes a standard way to represent sparse tensors. A broad theoretical framework for tensor data scattering methods used in various deep learning frameworks is established. This paper presents a theorem that is very important for performance analysis and accelerator optimization for implementing data scattering. The theorem shows how the impossibility of slicing happens in tenser data scattering. A sparsity measuring formula is provided, which can effectively indicate the storage efficiency of sparse tensor and the possibility of parallelly using it. The source code, including CUDA code, is provided in a related open-source project.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Soyata, T.: GPU Parallel Program Development using CUDA. CRC Press, Boca Raton (2018)
Child, R., Gray, S., Radford, A., Sutskever, I.: Generating long sequences with sparse variators. CoRR abs/1904.10509 (2019). http://arxiv.org/abs/1904.10509
TensorFlow API: tf.tensor_scatter_nd_update, https://www.tensorflow.org/api_docs/python/tf/tensor_scatter_nd_update
PyTorch Docs: torch.Tensor.scatter. https://pytorch.org/docs/stable/tensors.html?Highlight=scatter#torch.Tensor.scatter
Harris, C.R., Millman, K.J., van der Walt, S.J., et al.: Array programming with NumPy. Nature 585, 357–362 (2020)
Zhang, T., Liu, X., Wang, X., Walid, A.: cuTensor-tubal: efficient primitives for tubal-rank tensor learning operations on GPUs. IEEE Trans. Parallel Distrib. Syst. 31(3), 595–610 (2020)
Algebraic Tensor Project. https://github.com/wmpan/AlgebraicTensor
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pan, W. (2022). Tensor Data Scattering and the Impossibility of Slicing Theorem. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2021. Lecture Notes in Networks and Systems, vol 294. Springer, Cham. https://doi.org/10.1007/978-3-030-82193-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-82193-7_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-82192-0
Online ISBN: 978-3-030-82193-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)