Alin Dobra
Johannes Gehrke
ABSTRACT
Developing regression models for large datasets that are both accurate and easy to interpret is a very important data mining problem. Regression trees with linear models in the leaves satisfy both these requirements, but thus far, no truly scalable regression tree algorithm is known. This paper proposes a novel regression tree construction algorithm (SECRET) that produces trees of high quality and scales to very large datasets. At every node, SECRET uses the EM algorithm for Gaussian mixtures to find two clusters in the data and to locally transform the regression problem into a classification problem based on closeness to these clusters. Goodness of split measures, like the gini gain, can then be used to determine the split variable and the split point much like in classification tree construction. Scalability of the algorithm can be achieved by employing scalable versions of the EM and classification tree construction algorithms. An experimental evaluation on real and artificial data shows that SECRET has accuracy comparable to other linear regression tree algorithms but takes orders of magnitude less computation time for large datasets.
- W. P. Alexander and S. D. Grimshaw. Treed regression. Journal of Computational and Graphical Statistics, (5):156--175, 1996.Google Scholar
- P. S. Bradley, U. M. Fayyad, and C. Reina. Scaling clustering algorithms to large databases. In Knowledge Discovery and Data Mining, pages 9--15, 1998.Google Scholar
- L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, Belmont, 1984.Google Scholar
- P. Chaudhuri, M.-C. Huang, W.-Y. Loh, and R. Yao. Piecewise-polynomial regression trees. Statistica Sinica, 4:143--167, 1994.Google Scholar
- J. H. Friedman. Multivariate adaptive regression splines. The Annals of Statistics, 19:1--141 (with discussion), 1991.Google ScholarCross Ref
- K. Fukanaga. Introduction to Statistical Pattern Recognition, Second edition. Academic Press, 1990. Google ScholarDigital Library
- J. Gehrke, R. Ramakrishnan, and V. Ganti. Rainforest -- a framework for fast decision tree construction of large datasets. In Proceedings of the 24th International Conference on Very Large Databases, pages 416--427. Morgan Kanfmarn, August 1998. Google ScholarDigital Library
- G. H. Golub and C. F. V. Loan. Matrix Computations. Johns Hopkins, 1996.Google Scholar
- A. Karalic. Linear regression in regression tree leaves. In International School for Synthesis of Expert Knowledge, Bled, Slovenia, 1992. Google ScholarDigital Library
- K.-C. Li, H.-H. Lue, and C.-H. Chen. Interactive tree-structured regression via principal hessian directions. journal of the American Statistical Association, (95):547--560, 2000.Google Scholar
- W.-Y. Loh. Regression trees with unbiased variable selection and interaction detection. Statistica Sinica, 2002. in press.Google Scholar
- W.-Y. Loh and Y.-S. Shih. Split selection methods for classification trees. Statistica Sinica, 7(4), October 1997.Google Scholar
- S. K. Murthy. Automatic construction of decision trees from data: A multi-disciplinary survey. Data Mining and Knowledge Discovery, 1997. Google ScholarDigital Library
- J. R. Quinlan. Learning with Continuous Classes. In 5th Australian Joint Conference on Artificial Intelligence, pages 343--348, 1992.Google Scholar
- J. R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufman, 1993. Google ScholarDigital Library
- L. Torgo. Functional models for regression tree leaves. In Proc. l4th International Conference on Machine Learning, pages 385--393. Morgan Kaufmann, 1997. Google ScholarDigital Library
- L. Torgo. Kernel regression trees. In European Conference on Machine Learning, 1997. Poster paper.Google Scholar
- L. Torgo. A comparative study of reliable error estimators for pruning regression trees. Iberoamerican Conf. on Artificial Intelligence. Springer-Verlag, 1998. Google ScholarDigital Library
Index Terms
- SECRET: a scalable linear regression tree algorithm
Recommendations
Optimal Information Rate of Secret Sharing Schemes on Trees
The information rate for an access structure is the reciprocal of the load of the optimal secret sharing scheme for this structure. We determine this value for all trees: it is $(2-1/c)^{-1}$, where $c$ is the size of the largest core of the tree. A ...
Hierarchical Scheme for Secret Separation Based on Computable Access Labels
AbstractIn this paper, we propose a threshold scheme for sharing a secret between tree-like graph nodes. Nodes whose descendants cover the entire set of leaf nodes can recover the secret. Otherwise, the nodes do not receive any information about the ...
Comments