Abstract
Concept classes can canonically be represented by matrices with entries 1 and −1. We use the singular value decomposition of this matrix to determine the optimal margins of embeddings of the concept classes of singletons and of half intervals in homogeneous Euclidean half spaces. For these concept classes the singular value decomposition can be used to construct optimal embeddings and also to prove the corresponding best possible upper bounds on the margin. We show that the optimal margin for embedding n singletons is \(\tfrac{n}{{3n - 4}}\) and that the optimal margin for half intervals over {1,...,n} is \(\tfrac{\pi }{{2\ln n}} + \Theta (\tfrac{1}{{(\ln n)^2 }})\). For the upper bounds on the margins we generalize a bound by Forster (2001). We also determine the optimal margin of some concept classes defined by circulant matrices up to a small constant factor, and we discuss the concept classes of monomials to point out limitations of our approach.
Article PDF
Similar content being viewed by others
References
Ben-David, S. (2000). Personal Communication.
Ben-David, S., Eiron, N., & Simon, H. U. (2001). imitations of learning via embeddings in Euclidean half-spaces. To appear in the Journal of Machine Learning Research. A preliminary version appeared in the Proceedings of the 14th Annual Conference on Computational Learning Theory (pp. 385–401). Berlin: Springer.
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1989). Learnability and the Vapnik-Chervonenkis dimension. Journal of the ACM, 100, 157–184.
Cristianini, N., & Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines. Cambridge, United Kingdom: Cambridge University Press.
Forster, J. (2001). A linear lower bound on the unbounded error probabilistic communication complexity. In Proceedings of the 16th Annual IEEE Conference on Computational Complexity (pp. 100–106). IEEE Computer Society Press.
Forster, J., Krause, M., Lokam, S. V., Mubarakzjanov, R., Schmitt, N., & Simon, H. U. (2001). Relations between communication complexity, linear arrangements and computational complexity. In Proceedings of the 21st Conference on Foundations of Software Technology and Theoretical Computer Science (pp. 171–182). Berlin: Springer.
Horn, R. A.,& Johnson, C. R. (1985). Matrix Analysis. Cambridge, United Kingdom: Cambridge University Press.
Kearns, M. J., & Vazirani, U. V. (1994). An Introduction to Computational Learning Theory. Cambridge, Massachusetts: Massachusetts Institute of Technology.
Krause, M. (1996). Geometric arguments yield better bounds for threshold circuits and distributed computing. Theoretical Computer Science, 156, 99–117.
Maass,W., & Turan, G. (1992). Lower bound methods and separation results for on-line learning models. Journal of Machine Learning, 9, 107–145.
Novikoff, A. B. (1962). On convergence proofs on perceptrons. Symposium on the Mathematical Theory of Automata, 12 (pp. 615–622). Polytechnic Institute of Brooklyn.
Vapnik, V. (1998). Statistical Learning Theory. New York: John Wiley & Sons, Inc.
Vapnik, V. N., & Chervonenkis, A. Y. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16, 264–280.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Forster, J., Schmitt, N., Simon, H.U. et al. Estimating the Optimal Margins of Embeddings in Euclidean Half Spaces. Machine Learning 51, 263–281 (2003). https://doi.org/10.1023/A:1022905618164
Issue Date:
DOI: https://doi.org/10.1023/A:1022905618164