Ivan Šimeček
ABSTRACT
Computations with tensors are widespread in many scientific areas. Usually, the used tensors are very large but sparse, i.e., the vast majority of their elements are zero. The space complexity of sparse tensor storage formats varies significantly. For overall efficiency, it is important to reduce the execution time and additional space requirements of the initial preprocessing (i.e., converting tensors from common storage formats to the given internal format).
The main contributions of this paper are threefold. Firstly, we present a new storage format for sparse tensors, which we call the succinct k-d tree-based tensor (SKTB) format. We compare the space complexity of common tensor storage formats and of the SKTB format and demonstrate the viability of using a tree as a data structurefor sparse tensors. Secondly, we present a parallel space-efficient algorithm for converting tensors to the SKTB format. Thirdly, we demonstrate the computational efficiency of the proposed format in sparse tensor-vector multiplication.
Supplemental Material
- Mart'in Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.Google Scholar
- Evrim Acar, Seyit Ahmet Çamtepe, and Bülent Yener. 2006. Collective Sampling and Analysis of High Order Tensors for Chatroom Communications. In ISI (Lecture Notes in Computer Science), Sharad Mehrotra, Daniel Dajun Zeng, Hsinchun Chen, Bhavani M. Thuraisingham, and Fei-Yue Wang (Eds.), Vol. 3975. Springer, 213--224. http://dblp.uni-trier.de/db/conf/isi/isi2006.html#AcarCY06Google ScholarDigital Library
- D. Michal Aristotle. 2008. Matrix and Tensor Calculus: With Applications to Mechanics, Elasticity and Aeronautics .Dover Publications.Google Scholar
- Brett W. Bader and Tamara G. Kolda. 2007. Efficient MATLAB Computations with Sparse and Factored Tensors. SIAM Journal on Scientific Computing, Vol. 30, 1 (December 2007), 205--231. https://doi.org/10.1137/060676489Google Scholar
- R. Barrett, M. Berry, T. F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine, and H. Van der Vorst. 1994. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods 2nd ed.). SIAM, Philadelphia, PA.Google Scholar
- JL Bentley. 1975. Multidimensional binary search trees used for associative searching. Commun. ACM, Vol. 18, 9 (1975), 517.Google ScholarDigital Library
- Iain Duff and John Reid. 1979. Some Design Features of a Sparse Matrix Code. ACM Trans. Math. Software, Vol. 5 (03 1979). https://doi.org/10.1145/355815.355817Google ScholarDigital Library
- Fred G. Gustavson. 1972. Some Basic Techniques for Solving Sparse Systems of Linear Equations .Springer US, Boston, MA, 41--52. https://doi.org/10.1007/978--1--4615--8675--3_4Google Scholar
- G. Jacobson. 1989. Space-Efficient Static Trees and Graphs. In Proceedings of the 30th Annual Symposium on Foundations of Computer Science (SFCS '89). IEEE Computer Society, USA, 549--554. https://doi.org/10.1109/SFCS.1989.63533Google ScholarDigital Library
- T. Kolda and B. Bader. 2009. Tensor Decompositions and Applications. SIAM Rev., Vol. 51, 3 (2009), 455--500. https://doi.org/10.1137/07070111XGoogle ScholarDigital Library
- Jiajia Li, Yuchen Ma, Chenggang Yan, and Richard Vuduc. 2016. Optimizing Sparse Tensor Times Matrix on Multi-core and Many-core Architectures. In Proceedings of the Sixth Workshop on Irregular Applications: Architectures and Algorithms (IA$^3$ '16). IEEE Press, Piscataway, NJ, USA, 26--33.Google ScholarDigital Library
- Jiajia Li, Jimeng Sun, and Richard Vuduc. 2018. HiCOO: Hierarchical Storage of Sparse Tensors. In Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC '18). IEEE Press, Piscataway, NJ, USA, Article 19, 15 pages. http://dl.acm.org/citation.cfm?id=3291656.3291682Google ScholarDigital Library
- Chun-Yuan Lin, Yeh-Ching Chung, and Jen-Shiuh Liu. 2003. Efficient Data Compression Methods for Multidimensional Sparse Array Operations Based on the EKMR Scheme. IEEE Trans. Comput., Vol. 52, 12 (Dec. 2003), 1640--1646. https://doi.org/10.1109/TC.2003.1252859Google Scholar
- Bangtian Liu, Chengyao Wen, Anand D. Sarwate, and Maryam Mehri Dehnavi. 2017. A Unified Optimization Approach for Sparse Tensor Operations on GPUs. 2017 IEEE International Conference on Cluster Computing (CLUSTER) (May 2017), 47--57.Google Scholar
- G. M. Morton. 1966. A computer Oriented Geodetic Data Base; and a New Technique in File Sequencing. IBM Ltd.Google Scholar
- Shaden Smith, Jee W. Choi, Jiajia Li, Richard Vuduc, Jongsoo Park, Xing Liu, and George Karypis. 2017. FROSTT: The Formidable Repository of Open Sparse Tensors and Tools. http://frostt.io/Google Scholar
- Shaden Smith, Niranjay Ravindran, Nicholas D. Sidiropoulos, and George Karypis. 2015. SPLATT: Efficient and Parallel Sparse Tensor-Matrix Multiplication. In Proceedings of the 2015 IEEE International Parallel and Distributed Processing Symposium (IPDPS '15). IEEE Computer Society, Washington, DC, USA, 61--70. https://doi.org/10.1109/IPDPS.2015.27Google ScholarDigital Library
- Jimeng Sun, Dacheng Tao, and Christos Faloutsos. 2006. Beyond Streams and Graphs: Dynamic Tensor Analysis. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '06). Association for Computing Machinery, New York, NY, USA, 374--383. https://doi.org/10.1145/1150402.1150445Google ScholarDigital Library
- Parker Allen Tew. 2016. An Investigation of Sparse Tensor Formats for Tensor Libraries. http://groups.csail.mit.edu/commit/papers/2016/parker-thesis.pdfGoogle Scholar
- Ivan Simevcek. 2009. Sparse Matrix Computations Using the Quadtree Storage Format. In Proceedings of the 2009 11th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC '09). IEEE Computer Society, Washington, DC, USA, 168--173. http://dx.doi.org/10.1109/SYNASC.2009.55Google ScholarDigital Library
- I. Simevcek, D. Langr, and J. Trdlivcka. 2014. Efficient Converting of Large Sparse Matrices to Quadtree Format. In Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), 2014 16th International Symposium on. 122--129. https://doi.org/10.1109/SYNASC.2014.25Google ScholarCross Ref
- I. Simevcek, D. Langr, and P. Tvrdik. 2012. Minimal Quadtree Format for Compression of Sparse Matrices Storage. In 14th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC'2012) (SYNASC'2012). Timisoara, Romania, 359--364. https://doi.org/10.1109/SYNASC.2012.30Google Scholar
Index Terms
- Space-Efficient k-d Tree-Based Storage Format for Sparse Tensors
Recommendations
Sparse tensor storage by tensor unfolding
SAC '22: Proceedings of the 37th ACM/SIGAPP Symposium on Applied ComputingLarge scale high dimensional multi-aspect data are naturally represented as tensors or multi-way arrays for scientific and engineering computations. But in practical applications such as signal processing, data mining, computer vision, and graph ...
Computing Eigenvalues of Large Scale Sparse Tensors Arising from a Hypergraph
The spectral theory of higher-order symmetric tensors is an important tool for revealing some important properties of a hypergraph via its adjacency tensor, Laplacian tensor, and signless Laplacian tensor. Owing to the sparsity of these tensors, we propose ...
A Krylov-Schur-like method for computing the best rank-(r1,r2,r3) approximation of large and sparse tensors
AbstractThe paper is concerned with methods for computing the best low multilinear rank approximation of large and sparse tensors. Krylov-type methods have been used for this problem; here block versions are introduced. For the computation of partial ...
Comments