Alexandr Andoni
ABSTRACT
We undertake a systematic study of sketching a quadratic form: given an n x n matrix A, create a succinct sketch sk(A) which can produce (without further access to A) a multiplicative (1+ε)-approximation to xT A x for any desired query x ∈ Rn. While a general matrix does not admit non-trivial sketches, positive semi-definite (PSD) matrices admit sketches of size θ(ε{-2 n), via the Johnson-Lindenstrauss lemma, achieving the "for each" guarantee, namely, for each query x, with a constant probability the sketch succeeds. (For the stronger "for all" guarantee, where the sketch succeeds for all x's simultaneously, again there are no non-trivial sketches.)
We design significantly better sketches for the important subclass of graph Laplacian matrices, which we also extend to symmetric diagonally dominant matrices. A sequence of work culminating in that of Batson, Spielman, and Srivastava (SIAM Review, 2014), shows that by choosing and reweighting O(ε{-2 n) edges in a graph, one achieves the "for all" guarantee. Our main results advance this front.
For the "for all" guarantee, we prove that Batson et al.'s bound is optimal even when we restrict to "cut queries" x ∈ (0,1}n. Specifically, an arbitrary sketch that can (1+ε)-estimate the weight of all cuts (S, bar S) in an n-vertex graph must be of size Ω(ε{-2 n) bits. Furthermore, if the sketch is a cut-sparsifier (i.e., itself a weighted graph and the estimate is the weight of the corresponding cut in this graph), then the sketch must have Ω(ε{-2 n) edges.
In contrast, previous lower bounds showed the bound only for spectral-sparsifiers.
For the "for each" guarantee, we design a sketch of size Õ(ε{-1 n) bits for "cut queries" x ∈{0,1}n. We apply this sketch to design an algorithm for the distributed minimum cut problem. We prove a nearly-matching lower bound of Ω(ε{-1 n) bits. For general queries x ∈ Rn, we construct sketches of size Õ(ε{-1.6 n) bits.
Our results provide the first separation between the sketch size needed for the "for all" and "for each" guarantees for Laplacian matrices.
- K. J. Ahn, S. Guha, and A. McGregor. Analyzing graph structure via linear measurements. In 23rd Annual ACM-SIAM Symposium on Discrete Algorithms, pages 459--467, 2012. Google Scholar
Digital Library
- K. J. Ahn, S. Guha, and A. McGregor. Graph sketches: sparsification, spanners, and subgraphs. In 31st ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, pages 5--14, 2012. Google ScholarDigital Library
- S. Arora, E. Hazan, and S. Kale. Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. In 46th Annual IEEE Symposium on Foundations of Computer Science, FOCS '05, pages 339--348. IEEE Computer Society, 2005.href http://dx.doi.org/10.1109/SFCS.2005.35pathdoi:10.1109/SFCS.2005.35. Google ScholarDigital Library
- N. Alon. On the edge-expansion of graphs. Comb. Probab. Comput., 6(2):145--152, June 1997.href http://dx.doi.org/10.1017/S096354839700299Xpathdoi:10.1017/S096354839700299X. Google ScholarDigital Library
- S. Arora, S. Rao, and U. Vazirani. Expander flows, geometric embeddings and graph partitioning. J. ACM, 56(2):1--37, 2009.href http://dx.doi.org/10.1145/1502793.1502794pathdoi:10.1145/1502793.1502794. Google ScholarDigital Library
- J. Blocki, A. Blum, A. Datta, and O. Sheffet. Differentially private data analysis of social networks via restricted sensitivity. In Innovations in Theoretical Computer Science, ITCS 2013, pages 87--96, 2013. Google ScholarDigital Library
- A. A. Benczúr and D. R. Karger. Approximating $\rm s$-$\rm t$ minimum cuts in Õ(n2) time. In 28th Annual ACM Symposium on Theory of Computing, pages 47--55. ACM, 1996.href http://dx.doi.org/10.1145/237814.237827pathdoi:10.1145/237814.237827. Google ScholarDigital Library
- J. D. Batson, D. A. Spielman, and N. Srivastava. Twice-ramanujan sparsifiers. SIAM Review, 56(2):315--334, 2014.href http://dx.doi.org/10.1137/130949117pathdoi:10.1137/130949117.Google ScholarDigital Library
- K. L. Clarkson and D. P. Woodruff. Numerical linear algebra in the streaming model. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing, STOC 2009, pages 205--214, 2009. Google ScholarDigital Library
- W. S. Fung, R. Hariharan, N. J. Harvey, and D. Panigrahi. A general framework for graph sparsification. In Proceedings of the Symposium on Theory of Computing (STOC), pages 71--80. ACM, 2011.href http://dx.doi.org/10.1145/1993636.1993647pathdoi:10.1145/1993636.1993647. Google ScholarDigital Library
- A. Gupta, A. Roth, and J. Ullman. Iterative constructions and private data release. In 9th International Conference on Theory of Cryptography, TCC'12, pages 339--356. Springer-Verlag, 2012.href http://dx.doi.org/10.1007/978-3-642-28914-9_19pathdoi:10.1007/978--3--642--28914--9_19. Google ScholarDigital Library
- M. R. Henzinger and D. P. Williamson. On the number of small cuts in a graph. Inf. Process. Lett., 59(1):41--44, 1996. Google ScholarDigital Library
- P. Jain and A. Thakurta. Mirror descent based database privacy. In 15th International Workshop, APPROX 2012, and 16th International Workshop, RANDOM 2012, pages 579--590, 2012.Google ScholarCross Ref
- D. R. Karger. Minimum cuts in near-linear time. J. ACM, 47(1):46--76, 2000.href http://dx.doi.org/10.1145/331605.331608pathdoi:10.1145/331605.331608. Google ScholarDigital Library
- M. Kapralov, Y. T. Lee, C. Musco, C. Musco, and A. Sidford. Single pass spectral sparsification in dynamic streams. In 55th Annual Symposium on Foundations of Computer Science, FOCS '14, pages 561--570. IEEE Computer Society, 2014.href http://arxiv.org/abs/1407.1289patharXiv:1407.1289,href http://dx.doi.org/10.1109/FOCS.2014.66pathdoi:10.1109/FOCS.2014.66. Google ScholarDigital Library
- H. Klauck, D. Nanongkai, G. Pandurangan, and P. Robinson. Distributed computation of large-scale graph problems. In 26th Annual ACM-SIAM Symposium on Discrete Algorithms, SODA '15, pages 391--410. SIAM, 2015.href http://arxiv.org/abs/1311.6209patharXiv:1311.6209,href http://dx.doi.org/10.1137/1.9781611973730.28pathdoi:10.1137/1.9781611973730.28. Google ScholarDigital Library
- M. Kapralov and R. Panigrahy. Spectral sparsification via random spanners. In 3rd Innovations in Theoretical Computer Science Conference, pages 393--398. ACM, 2012.href http://dx.doi.org/10.1145/2090236.2090267pathdoi:10.1145/2090236.2090267. Google ScholarDigital Library
- A. Madry. Fast approximation algorithms for cut-based problems in undirected graphs. In Proceedings of the Symposium on Foundations of Computer Science (FOCS), pages 245--254. IEEE, 2010. Google ScholarDigital Library
- A. McGregor. Graph stream algorithms: a survey. SIGMOD Record, 43(1):9--20, 2014. Google ScholarDigital Library
- A. Nilli. On the second eigenvalue of a graph. Discrete Math, 91:207--210, 1991.href http://dx.doi.org/10.1016/0012-365X(91)90112-Fpathdoi:10.1016/0012-365X(91)90112-F. Google ScholarDigital Library
- J. Sherman. Breaking the multicommodity flow barrier for O(√log n)-approximations to sparsest cut. In Proceedings of the Symposium on Foundations of Computer Science (FOCS), pages 363--372, 2009. Google ScholarDigital Library
- D. A. Spielman and N. Srivastava. Graph sparsification by effective resistances. SIAM J. Comput., 40(6):1913--1926, December 2011.href http://dx.doi.org/10.1137/080734029pathdoi:10.1137/080734029. Google ScholarDigital Library
- D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In Proceedings of the Symposium on Theory of Computing (STOC), pages 81--90. ACM, 2004.href http://dx.doi.org/10.1145/1007352.1007372pathdoi:10.1145/1007352.1007372. Google ScholarDigital Library
- D. A. Spielman and S.-H. Teng. Spectral sparsification of graphs. SIAM J. Comput., 40(4):981--1025, 2011.href http://dx.doi.org/10.1137/08074489Xpathdoi:10.1137/08074489X. Google ScholarDigital Library
- J. Upadhyay. Random projections, graph sparsification, and differential privacy. In 19th International Conference on Advances in Cryptology, ASIACRYPT 2013, pages 276--295. Springer-Verlag, 2013.href http://dx.doi.org/10.1007/978-3-642-42033-7_15pathdoi:10.1007/978-3-642-42033-7_15. Google ScholarDigital Library
- J. Upadhyay. Circulant matrices and differential privacy. CoRR, abs/1410.2470, 2014.href http://arxiv.org/abs/1410.2470patharXiv:1410.2470.Google Scholar
- D. P. Woodruff and Q. Zhang. When distributed computation is communication expensive. In Distributed Computing - 27th International Symposium, DISC 2013, pages 16--30, 2013.href http://dx.doi.org/10.1007/978-3-642-41527-2_2pathdoi:10.1007/978-3-642-41527-2_2.Google Scholar
Index Terms
- On Sketching Quadratic Forms
Recommendations
Super-linear time-space tradeoff lower bounds for randomized computation
FOCS '00: Proceedings of the 41st Annual Symposium on Foundations of Computer ScienceWe prove the first time-space lower bound tradeoffs for randomized computation of decision problems. The bounds hold even in the case that the computation is allowed to have arbitrary probability of error on a small fraction of inputs. Our techniques ...
Time-space trade-off lower bounds for randomized computation of decision problems
We prove the first time-space lower bound trade-offs for randomized computation of decision problems. The bounds hold even in the case that the computation is allowed to have arbitrary probability of error on a small fraction of inputs. Our techniques ...
Sketching user experiences tutorial: stories, strategies, surfaces
ITS '13: Proceedings of the 2013 ACM international conference on Interactive tabletops and surfacesPaper-pencil sketches are a valuable tool during different stages of experience design in human-computer interaction. This hands-on tutorial will demonstrate how to integrate sketching into researchers' and interaction designers' everyday practice -- ...
Comments