Abstract
Motivated by statistical learning theoretic treatment of principal component analysis, we are concerned with the set of points in ℝd that are within a certain distance from a k-dimensional affine subspace. We prove that the VC dimension of the class of such sets is within a constant factor of (k+1)(d−k+1), and then discuss the distribution of eigenvalues of a data covariance matrix by using our bounds of the VC dimensions and Vapnik’s statistical learning theory. In the course of the upper bound proof, we provide a simple proof of Warren’s bound of the number of sign sequences of real polynomials.
Article PDF
Similar content being viewed by others
References
Basu, S., Pollack, R., Roy, M.-F.: On the Betti numbers of sign conditions. Proc. Am. Math. Soc. 133(4), 965–974 (2005) (electronic)
Basu, S., Pollack, R., Roy, M.-F.: An asymptotically tight bound on the number of connected components of realizable sign conditions. Combinatorica (to appear)
Blumer, A., Ehrenfeucht, A., Haussler, D., Warmuth, M.K.: Learnability and the Vapnik–Chervonenkis dimension. J. Assoc. Comput. Mach. 36, 929–965 (1989)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley-Interscience, New York (2001)
Dudley, R.M.: Balls in R k do not cut all subsets of k+2 points. Adv. Math. 31(3), 306–308 (1979)
Johnstone, I.M.: On the distribution of the largest eigenvalue in principal components analysis. Ann. Stat. 29(2), 295–327 (2001)
Massart, P.: Concentration Inequalities and Model Selection. Lecture Notes in Mathematics, vol. 1896. Springer, Berlin (2007). Lectures from the 33rd Summer School on Probability Theory Held in Saint-Flour, 6–23 July 2003, With a Foreword by Jean Picard
Matoušek, J.: Lectures on Discrete Geometry. Graduate Texts in Mathematics, vol. 212. Springer, New York (2002)
Mendelson, S., Vershynin, R.: Entropy and the combinatorial dimension. Invent. Math. 152(1), 37–55 (2003)
Milnor, J.W.: On the Betti numbers of real varieties. Proc. Am. Math. Soc. 15, 275–280 (1964)
Milnor, J.W.: Topology from the Differentiable Viewpoint. Princeton Landmarks in Mathematics. Princeton University Press, Princeton (1997)
Vapnik, V.N.: Statistical Learning Theory. Adaptive and Learning Systems for Signal Processing, Communications, and Control. Wiley, New York (1998)
Warren, H.E.: Lower bounds for approximation by nonlinear manifolds. Trans. Am. Math. Soc. 133, 167–178 (1968)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Akama, Y., Irie, K., Kawamura, A. et al. VC Dimensions of Principal Component Analysis. Discrete Comput Geom 44, 589–598 (2010). https://doi.org/10.1007/s00454-009-9236-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00454-009-9236-5