João Mendes-Moreira
Alípio Mário Jorge
Skip Abstract Section
Abstract
The goal of ensemble regression is to combine several models in order to improve the prediction accuracy in learning problems with a numerical target variable. The process of ensemble learning can be divided into three phases: the generation phase, the pruning phase, and the integration phase. We discuss different approaches to each of these phases that are able to deal with the regression problem, categorizing them in terms of their relevant characteristics and linking them to contributions from different fields. Furthermore, this work makes it possible to identify interesting areas for future research.
- Aggarwal, N., Prakash, N., and Sofat, S. 2010. Content management system effort estimation using bagging predictors. In Proceedings of the International Joint Conference on Computer Information Systems Sciences and Engineering Technological Developments in Education and Automation, M. Iskander, V. Kapila, and M. Karim, Eds. 19--24.Google Scholar
- Aha, D. W. and Bankert, R. L. 1996. A comparative evaluation of sequential feature selection algorithms. In Learning from Data, D. Fisher and H.-J. Lenz, Eds. Springer, Chapter 4, 199--206.Google Scholar
- Aksela, M. 2003. Comparison of classifier selection methods for improving committee performance. In Proceedings of the International Workshop on Multiple Classifier Systems. Lecture Notes in Computer Science, vol. 2709. Springer, 84--93. Google ScholarDigital Library
- Al-Ani, A. 2005. Feature subset selection using ant colony optimization. Int. J. Comput. Intell. 2, 1, 53--58.Google Scholar
- Avnimelech, R. and Intrator, N. 1999. Boosting regression estimators. Neural Comput. 11, 499--520. Google ScholarDigital Library
- Azevedo, P. J. and Jorge, A. M. 2007. Iterative reordering of rules for building ensembles without relearning. In Proceedings of the International Conference on Discovery Science. Lecture Notes in Computer Science, vol. 4755. Springer, 56--67. Google ScholarDigital Library
- Azevedo, P. J. and Jorge, A. M. 2010. Ensembles of jittered association rule classifiers. Data Min. Knowl. Discov. 21, 1, 91--129. Google ScholarDigital Library
- Bakker, B. and Heskes, T. 2003. Clustering ensembles of neural network models. Neural Netw. 16, 2, 261--269. Google ScholarDigital Library
- Bentley, J. L. 1975. Multidimensional binary search trees used for associative searching. Comm. ACM 18, 9, 509--517. Google ScholarDigital Library
- Bezerra, G. B., Barra, T. V., Castro, L. N., and von Zuben, F. J. 2005. Adaptive radius immune algorithm for data clustering. In Proceedings of the International Conference on Artificial Immune Systems (ICARIS'05). Lecture Notes in Computer Science, vol. 3627. Springer, 290--303. Google ScholarDigital Library
- Bifet, A., Holmes, G., Pfahringer, B., Kirkby, R., and Gavaldà, R. 2009. New ensemble methods for evolving data streams. In Proceedings of the Annual ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'09). ACM, New York, 139--148. Google ScholarDigital Library
- Blockeel, H. and Vanschoren, J. 2007. Experiment databases: Towards an improved experimental methodology in machine learning. In Proceedings of the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD'07). Lecture Notes in Computer Science, vol. 4702. Springer, 6--17. Google ScholarDigital Library
- Borra, S. and Ciaccio, A. D. 2002. Improving nonparametric regression methods by bagging and boosting. Comput. Statist. Data Anal. 38, 4, 407--420. Google ScholarDigital Library
- Brazdil, P., Giraud-Carrier, C., Soares, C., and Vilalta, R. 2009. Metalearning: Applications to Data Mining. Springer. Google ScholarCross Ref
- Breiman, L. 1996a. Bagging predictors. Mach. Learn. 26, 123--140. Google ScholarDigital Library
- Breiman, L. 1996b. Heuristics of instability and stabilization in model selection. Ann. Statist. 24, 6, 2350--2383.Google ScholarCross Ref
- Breiman, L. 1996c. Stacked regressions. Mach. Learn. 24, 49--64. Google ScholarDigital Library
- Breiman, L. 2000. Randomizing outputs to increase prediction accuracy. Mach. Learn. 40, 3, 229--242. Google ScholarDigital Library
- Breiman, L. 2001a. Random forests. Mach. Learn. 45, 5--32. Google ScholarDigital Library
- Breiman, L. 2001b. Using iterated bagging to debias regressions. Mach. Learn. 45, 3, 261--277. Google ScholarDigital Library
- Brown, G. 2004. Diversity in neural network ensembles. Ph.D. thesis, University of Birmingham.Google Scholar
- Brown, G., Wyatt, J. L., Harris, R., and Yao, X. 2005a. Diversity creation methods: A survey and categorisation. Inf. Fusion 6, 5--20.Google ScholarCross Ref
- Brown, G., Wyatt, J. L., and Tino, P. 2005b. Managing diversity in regression ensembles. J. Mach. Learn. Res. 6, 1621--1650. Google ScholarDigital Library
- Buhlmann, P. 2010. Bagging, Boosting and Ensemble Methods. Springer.Google Scholar
- Buja, A. and Stuetzle, W. 2006. Observations on bagging. Statistica Sinica 16, 323--351.Google Scholar
- Cai, T. and Wu, X. 2008. Research on ensemble learning based on discretization method. In Proceedings of the 9th International Conference on Signal Processing (ICSP'08). 1528--1531.Google Scholar
- Caruana, R., Niculescu-Mozil, A., Crew, G., and Ksikes, A. 2004. Ensemble selection from libraries of models. In International Conference on Machine Learning. Google ScholarDigital Library
- Coelho, G. P. and von Zuben, F. J. 2006. The influence of the pool of candidates on the performance of selection and combination techniques in ensembles. In Proceedings of the International Joint Conference on Neural Networks. 10588--10595.Google Scholar
- Delve. 2002. Delve: Data for evaluating learning in valid experiments. http://www.cs.toronto.edu/~delve/Google Scholar
- Didaci. L. and Giacinto, G. 2004. Dynamic classifier selection by adaptive k-nearest neighbourhood rule. In Proceedings of the International Workshop on Multiple Classifier Systems, F. Roli, J. Kittler, and T. Windeatt, Eds. Lecture Notes in Computer Science, vol. 3077. Springer, 174--183.Google Scholar
- Dietterich, T. G. 1997. Machine-Learning research: Four current directions. AI Mag. 18, 4, 97--136.Google Scholar
- Domeniconi, C. and Yan, B. 2004. Nearest neighbor ensemble. In Proceedings of the International Conference on Pattern Recognition. Vol. 1. 228--231. Google ScholarDigital Library
- Domingos, P. 1997. Why does bagging work? A Bayesian account and its implications. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. AAAI Press, 155--158.Google Scholar
- Drucker, H. 1997. Improving regressors using boosting techniques. In Proceedings of the 14th International Conference on Machine Learning (ICML'97). Morgan Kaufmann Publishers, San Fransisco, CA, 107--115. Google ScholarDigital Library
- Duffy, N. and Helmbold, D. 2002. Boosting methods for regression. Mach. Learn. 47, 153--200. Google ScholarDigital Library
- Ferrer, L., Sönmez, K., and Shriberg, E. 2009. An anticorrelation kernel for subsystem training in multiple classifier systems. J. Mach. Learn. Res. 10, 2079--2114. Google ScholarDigital Library
- Flannagan, S. and Sperber, L. 2008. Datamob/datasets. http://datamob.org/datsetsGoogle Scholar
- Frank, A. and Asuncion, A. 2010. UCI machine learning repository. http://archive.ics.uci.edu/mlGoogle Scholar
- Frank, E. and Pfahringer, B. 2006. Improving on bagging with input smearing. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 97--106. Google ScholarDigital Library
- Freund, Y. and Schapire, R. 1996. Experiments with a new boosting algorithm. In Proceedings of the International Conference on Machine Learning. 148--156.Google Scholar
- Freund, Y. and Schapire, R. E. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119--139. Google ScholarDigital Library
- Friedman, J. H. 1991. Multivariate adaptive regression splines. The Ann. Statist. 19, 1, 1--141.Google ScholarCross Ref
- Friedman, J. H. 1996. Local learning based on recursive covering. Tech. rep.Google Scholar
- Friedman, J. H. 2001. Greedy function approximation: A gradient boosting machine. Ann. Statist. 29, 5, 1189--1232.Google ScholarCross Ref
- Friedman, J. H. 2002. Stochastic gradient boosting. Comput. Statist. Data Anal. 38, 4, 367--378. Google ScholarDigital Library
- Friedman, H. H. and Stuetzle, W. 1981. Projection pursuit regression. J. Amer. Statist. Regress. 76, 376, 817--823.Google Scholar
- García-Pedrajas, N., Hervás-Martínez, C., and Ortiz-Boyer, D. 2005. Cooperative coevolution of artificial neural network ensembles for pattern classification. IEEE Trans. Evolut. Comput. 9, 3, 271--302. Google ScholarDigital Library
- Geman, S., Bienenstock, E., and Doursat, R. 1992. Neural networks and the bias/variance dilemma. Neural Comput. 4, 1, 1--58. Google ScholarDigital Library
- Giacinto, G. and Roli, F. 1997. Adaptive selection of image classifiers. In Proceedings of the International Conference on Image Analysis and Processing. Lecture Notes in Computer Science, vol. 1310. Springer, 38--45. Google ScholarDigital Library
- Granitto, P., Verdes, P., and Ceccatto, H. 2005. Neural network ensembles: Evaluation of aggregation algorithms. Artif. Intell. 163, 2, 139--162. Google ScholarDigital Library
- Guvenir, H. A, and Uysal, I. 2000. Function approximation repository. http://funapp.cs.bilkent.edu. tr/DataSets/Google Scholar
- Hashem, S. 1993. Optimal linear combinations of neural networks. Ph.D. thesis, Purdue University. Google ScholarDigital Library
- Hastie, T. and Tibshirani, R. 1996. Discriminant adaptive nearest neighbor classification. IEEE Trans. Pattern Anal. Mach. Intell. 18, 6, 607--616. Google ScholarDigital Library
- Hastie, T., Tibshirani, R., and Friedman, J. H. 2001. The Elements of Statistical Learning: Data Mining, Inference, and Predictions. Springer Series in Statistics. Springer.Google Scholar
- Hernández-Lobato, D., Martínez-Muñoz, G., and Suárez, A. 2006. Pruning in ordered regression bagging ensembles. In Proceedings of the International Joint Conference on Neural Networks. 1266--1273.Google Scholar
- Ho, T. K. 1998. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20, 8, 832--844. Google ScholarDigital Library
- Islam, M. M., Yao, X., and Murase, K. 2003. A constructive algorithm for training cooperative neural network ensembles. IEEE Trans. Neural Netw. 14, 4, 820--834. Google ScholarDigital Library
- Jain, A. and Zongker, D. 1997. Feature selection: Evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. Mach. Intell. 19, 2, 153--158. Google ScholarDigital Library
- Jorge, A. M. and Azevedo, P. J. 2005. An experiment with association rules and classification: Post-Bagging and conviction. In Discovery Science. Lecture Notes in Computer Science, vol. 3735. Springer, 137--149. Google ScholarDigital Library
- Kim, H.-C., Pang, S., Je, H.-M., Kim, D., and Bang, S.-Y. 2003. Constructing support vector machine ensemble. Pattern Recogn. 36, 12, 2757--2767.Google ScholarCross Ref
- Kivinen, J. and Warmuth, M. K. 1997. Exponentiated gradient versus gradient descent for linear predictors. Inf. Comput. 132, 1, 1--63. Google ScholarDigital Library
- Ko, A. H.-R., Sabourin, R., and Britto Jr., A. D. S. 2008. From dynamic classifier selection to dynamic ensemble selection. Pattern Recogn. 41, 1718--1731. Google ScholarDigital Library
- Kolen, J. F. and Pollack, J. B. 1990. Back propagation is sensitive to initial conditions. Tech. rep. TR-90-JK-BPSIC, The Ohio State University.Google Scholar
- Kolter, J. Z. and Maloof, M. A. 2007. Dynamic weighted majority: An ensemble method for drifting concepts. J. Mach. Learn. Res. 8, 2755--2790. Google ScholarDigital Library
- Kotsiantis, S. and Pintelas, P. 2005. Selective averaging of regression models. Ann. Math. Comput. Teleinf. 1, 3, 65--74.Google Scholar
- Krogh, A. and Vedelsby, J. 1995. Neural network ensembles, cross validation, and active learning. Adv. Neural Inf. Process. Syst. 7, 231--238.Google ScholarDigital Library
- Kuncheva, L. I. 2002. Switching between selection and fusion in combining classifiers: An experiment. IEEE Trans. Syst. Man Cybernet. B32, 2, 146--156. Google ScholarDigital Library
- Kuncheva, L. I. 2004. Combining Pattern Classifiers. Wiley. Google ScholarDigital Library
- Lasota, T., Telec, Z., Trawinski, B., and Trawinsky, K. 2009. A multi-agent system to assist with real estate appraisals using bagging ensembles. In Computational Collective Intelligence: Semantic Web, Social Networks and Multiagent Systems, N. Nguyen, R. Kowalczyk, and S. Chen, Eds. Lecture Notes in Artificial Intelligence, vol. 5796. Springer, 813--824. Google ScholarDigital Library
- Lazarevic, A. 2001. Effective pruning of neural network classifier ensembles. In Proceedings of the International Joint Conference on Neural Networks. 796--801.Google ScholarCross Ref
- LeBlanc, M. and Tibshirani, R. 1996. Combining estimates in regression and classification. J. Amer. Statist. Assoc. 91, 1641--1650.Google Scholar
- Lehmann, E. 1998. Theory of Point Estimation. Springer.Google Scholar
- Lin, H.-T. and Li, L. 2005. Infinite ensemble learning with support vector machines. In Proceedings of the European Conference on Machine Learning. Lecture Notes in Artificial Intelligence, vol. 3720. Springer, 242--254. Google ScholarDigital Library
- Liu, Y. and Yao, X. 1999. Ensemble learning via negative correlation. Neural Netw. 12, 1399--1404. Google ScholarDigital Library
- Liu, Y., Yao, X., and Higuchi, T. 2000. Evolutionary ensembles with negative correlation learning. IEEE Trans. Evolut. Comput. 4, 4, 380--387. Google ScholarDigital Library
- Loughrey, J. and Cunningham, P. 2005. Using early stopping to reduce overfitting in wrapper-based feature weighting. Tech. rep. TCD-CS-2005-41, Trinity College Dublin, Computer Science Department.Google Scholar
- Margineantu, D. D. and Dietterich, T. G. 1997. Pruning adaptive boosting. In Proceedings of the International Conference on Machine Learning. 211--218. Google ScholarDigital Library
- Martinez-Munoz, G., Hernandez-Lobato, D., and Suarez, A. 2009. An analysis of ensemble pruning techniques based on ordered aggregation. IEEE Trans. Pattern Anal. Mach. Intell. 31, 245--259. Google ScholarDigital Library
- Martinez-Munoz, G. and Suarez, A. 2006. Pruning in ordered bagging ensembles. In Proceedings of the International Conference on Machine Learning. 609--616. Google ScholarDigital Library
- Martinez-Munoz, G. and Suarez, A. 2007. Using boosting to prune bagging ensembles. Pattern Recogn. Lett. 28, 1, 156--165. Google ScholarDigital Library
- Matignon, R. 2007. Data Mining Using SAS Enterprise Miner. Wiley. Google ScholarDigital Library
- Mendes-Moreira, J. 2008. Travel time prediction for the planning of mass transit companies: A machine learning approach. Ph.D. thesis, Faculty of Engineering, University of Porto.Google Scholar
- Mendes-Moreira, J., Jorge, A. M., Freire de Sousa, J., and Soares, C. 2012. Comparing state-of-the-art regression methods for long term travel time prediction. Intell. Data Anal. 16, 3.Google ScholarCross Ref
- Mendes-Moreira, J., Jorge, A. M., Soares, C., and Freire de Sousa, J. 2009. Ensemble learning: A study on different variants of the dynamic selection approach. In Proceedings of the 6th International Conference on Machine Learning and Data Mining, P. Perner, Ed. Lecture Notes in Computer Science, vol. 5632. Springer, 191--205. Google ScholarDigital Library
- Meng, G., Pan, C., Zheng, N.,, and Sun, C. 2010. Skew estimation of document images using bagging. IEEE Trans. Image Process. 19, 7, 1837--1846. Google ScholarDigital Library
- Merz, C. J. 1996. Dynamical selection of learning algorithms. In Proceedings of the International Workshop on Artificial Intelligence and Statistics, D. Fisher and H.-J. Lenz, Eds. Springer.Google ScholarCross Ref
- Merz, C. J. 1998. Classification and regression by combining models. Ph.D. thesis, University of California Irvine. Google ScholarDigital Library
- Merz, C. J. and Pazzani, M. J. 1999. A principal components approach to combining regression estimates. Mach. Learn. 36, 9--32. Google ScholarDigital Library
- Meyer, D., Leisch, F., and Hornik, K. 2003. The support vector machine under test. Neurocomput. 55, 1--2, 169--186.Google ScholarCross Ref
- MLG, U. D. 2011. http://mlg.ucd.ie/Google Scholar
- Molina, L. C., Belanche, L., and Nebot, A. 2002. Feature selection algorithms: A survey and experimental evaluation. In Proceedings of the IEEE International Conference on Data Mining. 306--313. Google ScholarDigital Library
- Monti, S., Tamayo, P., Mesirov, J., and Golub, T. 2003. Consensus clustering: A resampling-based method for class discovery and visualization of gene expression microarray data. Mach. Learn., 91--118. Google ScholarDigital Library
- Moreira, J. M., Sousa, J. F., Jorge, A. M., and Soares, C. 2006. An ensemble regression approach for bus trip time prediction. In Proceedings of the Meeting of the EURO Working Group on Transportation. 317--321. http://www.liaad.up.pt/~amjorge/docs/Triana/Moreira06b.pdf.Google Scholar
- Oh, I.-S., Lee, J.-S., and Moon, B.-R. 2004. Hybrid genetic algorithms for feature selection. IEEE Trans. Pattern Anal. Mach. Intell. 26, 11, 1424--1437. Google ScholarDigital Library
- Opitz, D. W. 1999. Feature selection for ensembles. In Proceedings of the National Conference on Artificial Intelligence. AAAI Press, 379--384. Google ScholarDigital Library
- Opitz, D. W. and Shavlik, J. W. 1996. Generating accurate and diverse members of a neural-network ensemble. Adv. Neural Inf. Process. Syst. 8, 535--541.Google Scholar
- Ortiz-Boyer, D., Hervas-Martinez, C., and Garcia-Pendrajas, N. 2005. Cixl2: A crossover operator for evolutionary algorithms based on population features. J. Artif. Intell. Res. 24, 1--48. Google ScholarDigital Library
- Parmanto, B., Munro, P, W., and Doyle, H. R. 1996. Reducing variance of committee prediction with resampling techniques. Connect. Sci. 8, 3--4, 405--425.Google ScholarCross Ref
- Partridge, D. and Yates, W. B. 1996. Engineering multiversion neural-net systems. Neural Comput. 8, 4, 869--893. Google ScholarDigital Library
- Perrone, M. P. and Cooper, L. N. 1993. When networks disagree: Ensemble methods for hybrid neural networks. In Neural Networks for Speech and Image Processing, R. Mammone, Ed. Chapman-Hall.Google Scholar
- Polikar, R. 2009. Ensemble learning. Scholarpedia 4, 1, 2776.Google ScholarCross Ref
- Pudil, P., Ferri, F., Novovicova, J., and Kittler, J. 1994. Floating search methods for feature selection with nonmonotonic criterion functions. In Proceedings of the IEEE International Conference on Pattern Recognition. Vol. 11. 279--283.Google Scholar
- Puuronen, S., Terziyan, V., and Tsymbal, A. 1999. A dynamic integration algorithm for an ensemble of classifiers. In Proceedings of the International Symposium on Methodologies for Intelligent Systems. Lecture Notes in Computer Science, vol. 1609. Springer, 592--600. Google ScholarDigital Library
- Ranawana, R. and Palade, V. 2006. Multi-Classifier systems: Review and a roadmap for developers. Int. J. Hybrid Intell. Syst. 3, 1, 35--61. Google ScholarDigital Library
- Ratsch, G., Demiriz, A., and Bennett, K. P. 2002. Sparse regression ensembles in infinite and finite hypothesis spaces. Mach. Learn. 48, 189--218. Google ScholarDigital Library
- Raviv, Y. and Intrator, N. 1996. Bootstrapping with noise: An effective regularization technique. Connect. Sci. 8, 3--4, 355--372.Google ScholarCross Ref
- Robnik-Sikonja, M. 2004. Improving random forests. In Proceedings of the European Conference on Machine Learning. Lecture Notes in Artificial Intelligence, vol. 3201. Springer, 359--370.Google Scholar
- Robnik-Sikonja, M. and Kononenko, I. 2003. Theoretical and empirical analysis of relieff and rrelieff. Mach. Learn. 53, 1-2, 23--69. Google ScholarDigital Library
- Rodriguez, J. J., Kuncheva, L. I., and Alonso, C. J. 2006. Rotation forest: A new classifier ensemble. IEEE Trans. Pattern Anal. Mach. Intell. 28, 10, 1619--1630. Google ScholarDigital Library
- Rokach, L. 2009a. Collective-Agreement-Based pruning of ensembles. Comput. Statist. Data Anal. 53, 1015--1026. Google ScholarDigital Library
- Rokach, L. 2009b. Pattern Classification Using Ensemble Methods. World Scientific. Google ScholarDigital Library
- Rokach, L. 2009c. Taxonomy for characterizing ensemble methods in classification tasks: A review and annotated bibliography. Comput. Statist. Data Anal. 53, 12, 4046--4072. Google ScholarDigital Library
- Rokach, L. 2010. Ensemble-Based classifiers. Artif. Intell. Rev. 33, 1--39. Google ScholarDigital Library
- Roli, F., Giacinto, G,, and Vernazza, G. 2001. Methods for designing multiple classifier systems. In Proceedings of the International Workshop on Multiple Classifier Systems. Lecture Notes in Computer Science, vol. 2096. Springer, 78--87. Google ScholarDigital Library
- Rooney, N. and Patterson, D. 2007. A weighted combination of stacking and dynamic integration. Pattern Recogn. 40, 4, 1385--1388. Google ScholarDigital Library
- Rooney, N., Patterson, D., Anand, S., and Tsymbal, A. 2004. Dynamic integration of regression models. In Proceedings of the International Workshop on Multiple Classifier Systems. Lecture Notes in Computer Science, vol. 3181. Springer, 164--173.Google Scholar
- Rosen, B. E. 1996. Ensemble learning using decorrelated neural networks. Connect. Sci. 8, 3--4, 373--383.Google ScholarCross Ref
- Ruta, D. and Gabrys, B. 2001. Application of the evolutionary algorithms for classifier selection in multiple classifier systems with majority voting. In Proceedings of the International Workshop on Multiple Classifier Systems. Lecture Notes in Computer Science, vol. 2096. Springer, 399--408. Google ScholarDigital Library
- Schapire, R. 1990. The strength of weak learnability. Mach. Learn. 5, 2, 197--227. Google ScholarDigital Library
- Schclar, A. and Rokach, L. 2009. Random projection ensemble classifiers. In Proceedings of the International Conference on Enterprise Information Systems (ICEIS'09). Springer.Google Scholar
- Schclar, A., Tsinkinovsky, A., Rokach, L., Meisels, A., and Antwarg, L. 2009. Ensemble methods for improving the performance of neighborhood-based collaborative filtering. In Proceedings of the 3rd ACM Conference on Recommender Systems (RecSys'09). ACM Press, New York, 261--264. Google ScholarDigital Library
- Shrestha, D. L. and Solomatine, D. P. 2006. Experiments with adaboost.rt, an improved boosting scheme for regression. Neural Comput. 18, 1678--1710. Google ScholarDigital Library
- Skalak, D. B. 1994. Prototype and feature selection by sampling and random mutation hill climbing algorithms. In Proceedings of the International Conference on Machine Learning. Morgan Kaufmann, 293--301.Google ScholarCross Ref
- Skomoroch, P. 2008. Some datasets available on the web. http://www.datawrangling.com/some-datasets-available-on-the-webGoogle Scholar
- Stark, P. and Parker, R. 1995. Bounded-Variable least squares: An algorithm and applications. Comput. Statist. 10, 2, 129--141.Google Scholar
- Stone, M. 1974. Cross-Validatory choice and assessment of statistical predictions. J. Roy. Statist. Soc. B36, 2, 111--147.Google Scholar
- Strehl, A. and Ghosh, J. 2003. Cluster ensembles: A knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583--617. Google ScholarDigital Library
- Tamon, C. and Xiang, J. 2000. On the boosting pruning problem. In Proceedings of the European Conference on Machine Learning. Lecture Notes in Computer Science, vol. 1810. Springer, 404--412. Google ScholarDigital Library
- Thompson, S. K. and Seber, G. A. 1986. Adaptive Sampling. John Wiley & Sons.Google Scholar
- Tian, J., Wu, N., Chu, X., and Fan, Y. 2010. Predicting changes in protein thermostability brought about by single- or multi-site mutations. BMC Bioinf. 11, 370.Google ScholarCross Ref
- Todorovski, L. and Dzeroski, S. 2003. Combining classifiers with meta decision trees. Mach. Learn. 50, 3, 223--249. Google ScholarDigital Library
- Torgo, L. Regression datasets. http://www.liaad.up.pt/ltorgo/Regression/Datasets.html.Google Scholar
- Tresp, V. and Taniguchi, M. 1995. Combining estimators using non-constant weighting functions. Adv. Neural Inf. Process. Syst. 7, 419--426.Google Scholar
- Tsang, I. W., Kocsor, A., and Kwok, J. T. 2006. Diversified svm ensembles for large data sets. In Proceedings of the International Conference on Machine Learning. Lecture Notes in Artificial Intelligence, vol. 4212. Springer, 792--800. Google ScholarDigital Library
- Tsang, I. W., Kwok, J. T., and Lai, K. T. 2005. Core vector regression for very large regression problems. In Proceedings of the International Conference on Machine Learning. 912--919. Google ScholarDigital Library
- Tsoumakas, G., Partalas, I., and Vlahavas, I. 2008. A taxonomy and short review of ensemble selection. In Proceedings of the Workshop on Supervised and Unsupervised Ensemble Methods and Their Applications.Google Scholar
- Tsymbal, A., Pechenizkiy, M., and Cunningham, P. 2006a. Dynamic integration with random forests. In Proceedings of the European Conference on Machine Learning (ECML'06). Lecture Notes in Artificial Intelligence, vol. 4212. Springer, 801--808. Google ScholarDigital Library
- Tsymbal, A., Pechenizkiy, M., and Cunningham, P. 2006b. Dynamic integration with random forests. Tech. rep. TCD-CS-2006-23, The University of Dublin, Trinity College.Google Scholar
- Tsymbal, A., Pechenizkiy, M., Cunningham, P., and Puuronen, S. 2008. Dynamic integration of classifiers for handling concept drift. Inf. Fusion 9, 1, 56--68. Google ScholarDigital Library
- Ueda, N. and Nakano, R. 1996. Generalization error of ensemble estimators. In Proceedings of the IEEE Conference on Neural Networks. Vol. 1. 90--95.Google Scholar
- Vafaie, H. and Jong, K. D. 1993. Robust feature selection algorithms. In Proceedings of the IEEE Conference on Tools for Artificial Intelligence. 356--363.Google Scholar
- Verikas, A., Lipnickas, A., Malmqvist, K., Becauskiene, M., and Gelzinis, A. 1999. Soft combining of neural classifiers: A comparative study. Pattern Recogn. Lett. 20, 4, 429--444. Google ScholarDigital Library
- Vlachos, P. 2005. Statlib: Datasets archive. http://lib.stat.cmu.edu/datasets/.Google Scholar
- Wang, H., Fan, W., Yu, P. S., and Han, J. 2003. Mining concept-drifting data streams using ensemble classifiers. In ACM International Conference on Knowledge Discovery and Data Mining. Google ScholarDigital Library
- Webb, G. I. and Zheng, Z. 2004. Multistrategy ensemble learning: Reducing error by combining ensemble learning techniques. IEEE Trans. Knowl. Data Engin. 16, 8, 980--991. Google ScholarDigital Library
- Weyuker, E. J., Ostrand, T. J., and Bell, R. M. 2010. Comparing the effectiveness of several modeling methods for fault prediction. Empir. Softw. Engin. 15, 277--295. Google ScholarDigital Library
- Wichard, J., Merkwirth, C., and Ogorzalek, M. 2003. Building ensembles with heterogeneous models. In Course of the International School on Neural Nets.Google Scholar
- Wilson, D. R. and Martinez, T. R. 1997. Improved heterogeneous distance functions. J. Artif. Intell. Res. 6, 1--34. Google ScholarCross Ref
- Witten, I. H. and Frank, E. 2011. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann. Google ScholarDigital Library
- Wolpert, D. H. 1992. Stacked generalization. Neural Netw. 5, 2, 241--259. Google ScholarDigital Library
- Woods, K. 1997. Combination of multiple classifiers using local accuracy estimates. IEEE Trans. Pattern Anal. Mach. Intell. 19, 4, 405--410. Google ScholarDigital Library
- Yang, J. and Honavar, V. 1997. Feature subset selection using a genetic algorithm. IEEE Trans. Intell. Syst. 13, 2, 44--49. Google ScholarDigital Library
- Yankov, D., DeCoste, D., and Keogh, E. 2006. Ensembles of nearest neighbor forecasts. In Proceedings of the European Conference on Machine Learning. Lecture Notes in Artificial Intelligence, vol. 4212. Springer, 545--556. Google ScholarDigital Library
- Yao, X., Fischer, M., and Brown, G. 2001. Neural network ensembles and their application to traffic flow prediction in telecommunications networks. Neural Netw. 1, 693--698.Google Scholar
- Yu, Y., Zhou, Z.-H., and Ting, K. M. 2007. Cocktail ensemble for regression. In Proceedings of the IEEE International Conference on Data Mining. 721--726. Google ScholarDigital Library
- Zemel, R. S. and Pitassi, T. 2001. Advances in NeuralInformation Processing Systems. Vol. 13. MIT Press, Chapter A gradient-based boosting algorithm for regression problems, 696--702.Google Scholar
- Zenobi, G. and Cunningham, P. 2001. Using diversity in preparing ensembles of classifiers based on different feature subsets to minimize generalization error. In Proceedings of the European Conference on Machine Learning. Lecture Notes in Computer Science, vol. 2167. Springer, 576--587. Google ScholarDigital Library
- Zhang, C.-X., Zhang, J.-S., and Wang, G.-W. 2008. An empirical study of using rotation forest to improve regressors. Appl. Math. Comput. 195, 2, 618--629.Google ScholarCross Ref
- Zhang, J., Zou, Y., and Fan, Y. 2009. Embedded neural network to model-based permanent magnet synchronous motor diagnostics. In Proceedings of the Power Electronics and Motion Control Conference. 1813--1817.Google Scholar
- Zhao, Q.-L., Jiang, Y.-H., and Xu, M. 2009. A fast ensemble pruning algorithm based on pattern mining process. Data Min. Knowl. Discov. 19, 277--292. Google ScholarDigital Library
- Zhou, Z.-H. and Tang, W. 2003. Selective ensemble of decision trees. In Proceedings of the International Conference on Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing. Lecture Notes in Artificial Intelligence, vol. 2639. Springer, 476--483. Google ScholarDigital Library
- Zhou, Z.-H., Wu, J., and Tang, W. 2002. Ensembling neural networks: Many could be better than all. Artif. Intell. 137, 239--263. Google ScholarDigital Library
Index Terms
- Ensemble approaches for regression: A survey
Recommendations
Boosting regression methods based on a geometric conversion approach: Using SVMs base learners
Boosting is one of the most important developments in ensemble learning during the past decade. Among different types of boosting methods, AdaBoost is the earliest and the most prevailing one that receives lots of attention for its effectiveness and ...
Building boosted classification tree ensemble with genetic programming
GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference CompanionAdaptive boosting (AdaBoost) is a method for building classification ensemble, which combines multiple classifiers built in an iterative process of reweighting instances. This method proves to be a very effective classification method, therefore it was ...
Novel ensemble methods for regression via classification problems
Regression via classification (RvC) is a method in which a regression problem is converted into a classification problem. A discretization process is used to covert continuous target value to classes. The discretized data can be used with classifiers as ...
Comments