Abstract
Although a landmark work, version spaces have proven fundamentally limited by being constrained to only consider candidate classifiers that are strictly consistent with data. This work generalizes version spaces to partially overcome this limitation. The main insight underlying this work is to base learning on version-space intersection, rather than the traditional candidate-elimination algorithm. The resulting learning algorithm, incremental version-space merging (IVSM), allows version spaces to contain arbitrary sets of classifiers, however generated, as long as they can be represented by boundary sets. This extends version spaces by increasing the range of information that can be used in learning; in particular, this paper describes how three examples of very different types of information—ambiguous data, inconsistent data, and background domain theories as traditionally used by explanation-based learning—can each be used by the new version-space approach.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
J. R. Anderson (1983). The Architecture of Cognition, Harvard University Press, Cambridge, MA.
F. Bergadano and A. Giordana (1983). Guiding induction with domain theories. In Y. Kodratoff and R. S. Michalski, editors, Machine Learning: An Artificial Intelligence Approach, Volume III, pages 474–492, Morgan Kaufmann, Los Altos, CA.
A. Borgida, R. J. Brachman, D. L. McGuinness, and L. Resnick (1989) CLASSIC: A structural data model for objects, In Proceedings of SIGMOD-89, Portland, Oregon.
B. G. Buchanan and T. M. Mitchell (1978). Model-directed learning of production rules, In D. A. Waterman and F. Hayes-Roth, editors, Pattern-Directed Inference Systems, pages 297–312. Academic Press, New York.
S. Burgess (1991) Hierarchical and range data in induction over explanation, Unpublished Master's essay, Computer Science Department, Rutgers University.
W. W. Cohen and H. Hirsh (1992). Learnability of description logics, In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, Pittsburgh, PA.
W. W. Cohen (1990) Explanation-Based Generalization as an Abstraction Mechanism in Concept Learning, PhD thesis, Rutgers University.
A. P. Danyluk (1987). The use of explanations for similarity-based learning, In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, Milan, Italy.
T. G. Dietterich, B. London, K. Clarkson, and G. Dromey (1982). Learning and inductive inference, In P. Cohen and E. A. Feigenbaum, editors, The Handbook of Artificial Intelligence, Volume III. William Kaufmann, Los Altos, CA.
G. Drastal, R. Meunier, and S. Raatz (1989). Error correction in constructive induction, In Proceedings of the Sixth International Workshop on Machine Learning, pages 81–83, Ithaca, New York.
R. A. Fisher (1950). The use of multiple measurements in taxonomic problems, Annual Eugenics, 7:179–188, 1936, Also in Contributions to Mathematical Statistics, John Wiley & Sons, NY.
N. S. Flann and T. G. Dietterich (1990). A study of explanation-based methods for inductive learning, Machine Learning, 4 (2).
C. A. Gunter, T.-H. Ngair, P. Panangaden, and D. Subramanian (1991). The common order-theoretic structure of version spaces and ATMS's (extended abstract), In Proceedings of the National Conference on Artificial Intelligence, pages 500–505, Anaheim, CA.
A. Gupta (1987). Explanation-based failure recovery, In Proceedings of the National Conference on Artificial Intelligence, Seattle, Washington.
D. Haussler (1988). Quantifying inductive bias: AI learning algorithms and Valiant's learning framework, Artificial Intelligence, 26 (2):177–221.
H. Hirsh (1989). Incremental Version-Space Merging: A General Framework for Concept Learning, PhD thesis, Stanford University.
H. Hirsh (1990). Incremental Version-Space Merging: A General Framework for Concept Learning, Kluwer, Boston, MA.
H. Hirsh (1991). Theoretical underpinnings of version spaces, In Proceedings of the Twelfth Joint International Conference on Artificial Intelligence, pages 665–670, Sydney, Australia.
H. Hirsh (1992). The computational complexity of version spaces, Technical Report ML-TR-36, Department of Computer Science, Rutgers University.
H. Hirsh (1992). Polynomial-time learning with version spaces, In Proceedings of the National Conference on Artificial Intelligence, San Jose, CA.
P. Idestam-Almquist (1989). Demand networks: An alternative representation of version spaces, SYSLAB Report 75, Department of Computer and Systems Sciences, The Royal Institute of Technology and Stockholm University.
M. Lebowitz (1986). Integrated learning: Controlling explanation, Cognitive Science, 10 (2).
R. S. Michalski and J. B. Larson (1978). Selection of most representative training examples and incremental generation of v11 hypotheses: The underlying methodology and description of programs ESEL and AQ11, Report 867, University of Illinois.
R. S. Michalski (1983). A theory and methodology of inductive learning, In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, pages 83–134. Morgan Kaufmann, Los Altos, CA.
S. N. Minton (1988). Learning Effective Search Control Knowledge: An Explanation-Based Approach, PhD thesis, Carnegie Mellon University.
T. M. Mitchell, P. E. Utgoff, and R. B. Banerji (1983). Learning by experimentation: Acquiring and refining problem-solving heuristics, In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, pages 163–190. Morgan Kaufmann, Los Altos, CA.
T. M. Mitchell, R. M. Keller, and S. T. Kedar-Cabelli (1986). Explanation-based generalization: A unifying view, Machine Learning, 1 (1):47–80.
T. M. Mitchell (1978). Version Spaces: An Approach to Concept Learning, PhD thesis, Stanford University.
T. M. Mitchell (1982). Generalization as search, Artificial Intelligence, 18 (2):203–226.
T. M. Mitchell (1984). Toward combining empirical and analytic methods for learning heuristics, In A. Elithorn and R. Banerji, editors, Human and Artificial Intelligence. Erlbaum.
D. J. Mostow and N. Bhatnagar (1987). Failsafe—A floor planner that uses ebg to learn from its failures, In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, Milan, Italy.
M. Pazzani (1988). Learning Causal Relationships: An Integration of Empirical and Explanation-Based Learning Methods, PhD thesis, University of California, Los Angeles.
M. Pazzani and D. Kibler (1992). The utility of knowledge in inductive learning, T. M. Mitchell, R. M. Keller, and S. T. Kedar-Cabelli, Machine Learning, 9 (1):57–94.
J. R. Quinlan (1986). The effect of noise on concept learning, In R. S. Michalski, J. G. Carbonell, and T. M. Mitchell, editors, Machine Learning: An Artificial Intelligence Approach, Volume II, pages 149–166. Morgan Kaufmann, Los Altos, CA.
R. L. Rivest (1987). Learning decision lists, Machine Learning, 2 (3).
S. J. Russell and B. N. Grosof (1987). A declarative approach to bias in concept learning, In Proceedings of the National Conference on Artificial Intelligence, Seattle, Washington.
H. Simon and G. Lea (1974). Problem solving and rule induction, In H. Simon, editor, Models of Thought. Yale University Press.
B. D. Smith and P. S. Rosenbloom (1990). Incremental non-backtracking focusing: A polynomially bounded generalization algorithm for version spaces. In Proceedings of the National Conference on Artificial Intelligence, pages 848–853, Boston, MA.
P. E. Utgoff (1986). Machine Learning of Inductive Bias, Kluwer, Boston, MA.
K. VanLehn and W. Ball (1987). A version space approach to learning context-free grammars, Machine Learning, 2 (1):39–74.
S. M. Weiss and C. A. Kulikowski (1991). Computer Systems That Learn, Morgan Kaufmann, Los Altos, CA.
P. H. Winston (1975). Learning structural descriptions from examples, In P. H. Winston, editor, The Psychology of Computer Vision, chapter 5. McGraw Hill, New York.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hirsh, H. Generalizing Version Spaces. Machine Learning 17, 5–46 (1994). https://doi.org/10.1023/A:1022600917598
Issue Date:
DOI: https://doi.org/10.1023/A:1022600917598