Abstract
While the decision tree is an effective representation that has been used in many domains, a tree can often encode a concept inefficiently. This happens when the tree has to represent a subconcept multiple times in different parts of the tree. In this paper we introduce a new representation based on trees, the linked decision forest, that does not need to repeat internal structure. We also introduce a supervised learning algorithm, Lumberjack, that uses the new representation. We then show empirically that Lumberjack improves generalization accuracy on hierarchically decomposable concepts.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Leo Breiman, Jerome H. Friedman, Richard A. Olshen, and Charles J. Stone. Classification And Regression Trees. Wadsworth and Brooks/Cole Advanced Books and Software, Monterey, CA, 1984.
Randal E. Bryant. Symbolic boolean manipulation with ordered binary decision diagrams. ACM Computing Surveys, 24(3):293–318, 1992.
David Chapman and Leslie Pack Kaelbling. Input generalization in delayed reinforcement learning: An algorithm and performance comparisons. In Proceedings of the Twelfth International Joint Conference on Artificial Intelligence (IJCAI-91), pages 726–731, Sydney, Australia, 1991.
Jesse Hoey, Robert St-Aubin, Alan Hu, and Craig Boutilier. Spudd: Stochastic planning using decision diagrams. In Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI-99), Stockholm, Sweden, 1999. Morgan Kaufmann.
Ron Kohavi. Wrappers for Performance Enhancement and Oblivious Decision Graphs. Ph. d. thesis, Department of Computer Science, Stanford University, 1995.
Andrew Kachites McCallum. Reinforcement Learning with Selective Perception and Hidden State. PhD thesis, Department of Computer Science, University of Rochester, 1995.
Patrick M. Murphy and Michael J. Pazzani. Exploring the decision forest: An empirical invesitgation of Occam’s razor in decision tree induction. Journal of Artificial Intelligence Research, 1:257–275, 1994.
Craig G. Nevill-Manning and Ian H. Witten. Identifying hierarchical structures in sequences: A linear-time algorithm. Journal of Artificial Intelligence Research, 7:67–82, 1997.
Craig G. Nevill-Manning. Inferring Sequential Structure. Ph. d. thesis, Computer Science, University of Waikato, Hamilton, New Zealand, 1996.
J. Oliver and C. S. Wallace. Inferring decision graphs. Technical Report 91/170, Department of Computer Science, Monash University, November 1992.
Giulia Pagallo and David Haussler. Boolean feature discovery in empirical learning. Machine Learning, 5:71–99, 1990.
J. R. Quinlan and R. L. Rivest. Inferring decision trees using the minimum description length principle. Information and Computation, 80(3):227–248, 1989.
J. Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, CA, 1992.
Jorma Rissanen. A universal prior for integers and estimation by minimum description length. The Annals of Statistics, 11(2):416–431, 1983.
William T. B. Uther and Manuela M. Veloso. Tree based discretization for continuous state space reinforcement learning. In Proceedings of the Fifteenth National Conference on Artificial Intelligence (AAAI-98), pages 769–774, Madison, WI, 1998.
William T. B. Uther and Manuela M. Veloso. The lumberjack algorithm for learning linked decision forests. In Symposium on Abstraction, Reformulation and Approximation (SARA-2000), volume 1864 of Lecture Notes in Artificial Intelligence. Springer Verlag, 2000.
C. S. Wallace and D. M. Boulton. An information measure for classification. Computer Journal, 11(2):185–194, 1968.
C. S. Wallace and J. D. Patrick. Coding decision trees. Machine Learning, 11:7–22, 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Uther, W.T.B., Veloso, M.M. (2000). The Lumberjack Algorithm for Learning Linked Decision Forests. In: Mizoguchi, R., Slaney, J. (eds) PRICAI 2000 Topics in Artificial Intelligence. PRICAI 2000. Lecture Notes in Computer Science(), vol 1886. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44533-1_19
Download citation
DOI: https://doi.org/10.1007/3-540-44533-1_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67925-7
Online ISBN: 978-3-540-44533-3
eBook Packages: Springer Book Archive