Dimitris Achlioptas
Abstract
Given a matrix A, it is often desirable to find a good approximation to A that has low rank. We introduce a simple technique for accelerating the computation of such approximations when A has strong spectral features, that is, when the singular values of interest are significantly greater than those of a random matrix with size and entries similar to A. Our technique amounts to independently sampling and/or quantizing the entries of A, thus speeding up computation by reducing the number of nonzero entries and/or the length of their representation. Our analysis is based on observing that the acts of sampling and quantization can be viewed as adding a random matrix N to A, whose entries are independent random variables with zero-mean and bounded variance. Since, with high probability, N has very weak spectral features, we can prove that the effect of sampling and quantization nearly vanishes when a low-rank approximation to A + N is computed. We give high probability bounds on the quality of our approximation both in the Frobenius and the 2-norm.
- Alon, N., Krivelevich, M., and Vu, V. H. 2002. Concentration of eigenvalues of random matrices. Israel Math. J. 131, 259--267.Google Scholar
Cross Ref
- Azar, Y., Fiat, A., Karlin, A., McSherry, F., and Saia, J. 2001. Data mining through spectral analysis. In Proceedings of the 33rd Annual Symposium on Theory of Computing (Heraklion, Crete, Greece). ACM, New York, 619--626. Google ScholarDigital Library
- Bellhumer, P. N., Hespanha, J., and Kriegman, D. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. (Special Issue on Face Recognition) 17, 7, 711--720. Google ScholarDigital Library
- Berry, M., Drmač, Z., and Jessup, E. 1999. Matrices, vector spaces, and information retrieval. SIAM Rev. 41, 2, 335--362. Google ScholarDigital Library
- Berry, M., Dumais, S., and O'Brien, G. 1995. Using linear algebra for intelligent information retrieval. SIAM Rev. 41, 4, 573--595. Google ScholarDigital Library
- Deerwester, S., Dumais, S., Landauer, T., Furnas, G., and Harshman, R. 1990. Indexing by latent semantic analysis. J. Soc. Inf. Sci. 41, 6, 391--407.Google ScholarCross Ref
- Drineas, P., Frieze, A., Kannan, R., Vempala, S., and Vinay, V. 2004. Clustering large graphs via the singular value decomposition. Mach. Learn. 56, 1--3, 9--33. Google ScholarDigital Library
- Drineas, P., and Kannan, R. 2003. Pass efficient algorithms for approximating large matrices. In Proceedings of the 14th Annual Symposium on Discrete Algorithms (Baltimore, MD). ACM, New York, 223--232. Google ScholarDigital Library
- Frieze, A., Kannan, R., and Vempala, S. 2004. Fast Monte-Carlo algorithms for finding low-rank approximations. J. ACM 51, 6, 1025--1041. Google ScholarDigital Library
- Füredi, Z., and Komlós, J. 1981. The eigenvalues of random symmetric matrices. Combinatorica 1, 3, 233--241.Google ScholarCross Ref
- Georghiades, A., Belhumeur, P., and Kriegman, D. 2001. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23, 6, 643--660. Google ScholarDigital Library
- Golub, G., and Van Loan, C. 1996. Matrix computations, 3rd ed. Johns Hopkins University Press, Baltimore, MD. Google ScholarDigital Library
- Papadimitriou, C., Raghavan, P., Tamaki, H., and Vempala, S. 2000. Latent semantic indexing: A probabilistic analysis. J. Comput. Syst. Sci. 61, 2, 217--235. Google ScholarDigital Library
- Vu, V. H. 2005. Spectral norm of random matrices. In Proceedings of the 37th Annual Symposium on Theory of Computing (Baltimore, MD). ACM, New York, 423--430. Google ScholarDigital Library
Index Terms
- Fast computation of low-rank matrix approximations
Recommendations
Extended Lanczos bidiagonalization algorithm for low rank approximation and its applications
We propose an extended Lanczos bidiagonalization algorithm for finding a low rank approximation of a given matrix. We show that this method can yield better low-rank approximations than standard Lanczos bidiagonalization algorithm, without increasing ...
On the Compression of Low Rank Matrices
A procedure is reported for the compression of rank-deficient matrices. A matrix A of rank k is represented in the form $A = U \circ B \circ V$, where B is a $k\times k$ submatrix of A, and U, V are well-conditioned matrices that each contain a $k\times k$...
A fast and efficient algorithm for low-rank approximation of a matrix
STOC '09: Proceedings of the forty-first annual ACM symposium on Theory of computingThe low-rank matrix approximation problem involves finding of a rank k version of a m x n matrix A, labeled Ak, such that Ak is as "close" as possible to the best SVD approximation version of A at the same rank level. Previous approaches approximate ...
Comments