Skip to main content
SpringerLink
Log in

Dynamic Classifier Chain with Random Decision Trees

  • Conference paper
  • First Online:
Discovery Science (DS 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11198))

Included in the following conference series:

Abstract

Classifiers chains (CC) is an effective approach in order to exploit label dependencies in multi-label data. However, it has the disadvantages that the chain is chosen at total random or relies on a pre-specified ordering of the labels which is expensive to compute. Moreover, the same ordering is used for every test instance, ignoring the fact that different orderings might be best suited for different test instances. We propose a new approach based on random decision trees (RDT) which can choose the label ordering for each prediction dynamically depending on the respective test instance. RDT are not adapted to a specific learning task, but in contrast allow to define a prediction objective on the fly during test time, thus offering a perfect test bed for directly comparing different prediction schemes. Indeed, we show that dynamically selecting the next label improves over using a static ordering of the labels under an otherwise unchanged RDT model and experimental environment.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We assume, w.l.o.g., that \(y_1, y_2, \ldots \) is the ordering of the predicted labels.

References

  1. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  2. Dembczyński, K., Cheng, W., Hüllermeier, E.: Bayes optimal multilabel classification via probabilistic classifier chains. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, pp. 279–286 (2010)

    Google Scholar 

  3. Dembczyński, K., Waegeman, W., Cheng, W., Hüllermeier, E.: On label dependence and loss minimization in multi-label classification. Mach. Learn. 88(1–2), 5–45 (2012)

    Article  MathSciNet  Google Scholar 

  4. Fan, W.: On the Optimality of probability estimation by random decision trees. In: Proceedings of the 19th National Conference on Artificial Intelligence, pp. 336–341 (2004)

    Google Scholar 

  5. Fan, W., Greengrass, E., McCloskey, J., Yu, P.S., Drammey, K.: Effective estimation of posterior probabilities: explaining the accuracy of randomized decision tree approaches. In: Proceedings of the 5th International Conference on Data Mining, pp. 154–161 (2005)

    Google Scholar 

  6. Fan, W., Wang, H., Yu, P.S., Ma, S.: Is random model better? On its accuracy and efficiency. In: Proceedings of the 3rd IEEE International Conference on Data Mining, pp. 51–58 (2003)

    Google Scholar 

  7. Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)

    Article  Google Scholar 

  8. Goncalves, E.C., Plastino, A., Freitas, A.A.: A genetic algorithm for optimizing the label ordering in multi-label classifier chains. In: Proceedings of the IEEE 25th International Conference on Tools with Artificial Intelligence, pp. 469–476 (2013)

    Google Scholar 

  9. Kong, X., Yu, P.S.: An ensemble-based approach to fast classification of multi-label data streams. In: Proceedings of the 7th International Conference on Collaborative Computing: Networking, Applications and Worksharing, pp. 95–104 (October 2011)

    Google Scholar 

  10. Kumar, A., Vembu, S., Menon, A.K., Elkan, C.: Beam search algorithms for multilabel learning. Mach. Learn. 92(1), 65–89 (2013)

    Article  MathSciNet  Google Scholar 

  11. Li, N., Zhou, Z.-H.: Selective ensemble of classifier chains. In: Zhou, Z.-H., Roli, F., Kittler, J. (eds.) MCS 2013. LNCS, vol. 7872, pp. 146–156. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38067-9_13

    Chapter  Google Scholar 

  12. Malerba, D., Semeraro, G., Esposito, F.: A multistrategy approach to learning multiple dependent concepts. Mach. Learn. Stat. Interface chap. 4, 87–106 (1997)

    Google Scholar 

  13. Mena, D., Montañés, E., Quevedo, J.R., Coz, J.J.d.: Using A* for inference in probabilistic classifier chains. In: Proceedings of the 24th International Conference on Artificial Intelligence, pp. 3707–3713 (2015)

    Google Scholar 

  14. Mena, D., Montañés, E., Quevedo, J.R., Coz, J.J.: An overview of inference methods in probabilistic classifier chains for multilabel classification. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 6(6), 215–230 (2016)

    Article  Google Scholar 

  15. Nam, J., Loza Mencía, E., Kim, H.J., Fürnkranz, J.: Maximizing subset accuracy with recurrent neural networks in multi-label classification. In: Advances in Neural Information Processing Systems 30 (NIPS-17). pp. 5419–5429 (2017)

    Google Scholar 

  16. Quevedo, J.R., Luaces, O., Bahamonde, A.: Multilabel classifiers with a probabilistic thresholding strategy. Pattern Recognit. 45(2), 876–883 (2012)

    MATH  Google Scholar 

  17. Read, J., Martino, L., Luengo, D.: Efficient Monte Carlo methods for multi-dimensional learning with classifier chains. Pattern Recognit. 47(3), 1535–1546 (2014)

    Article  Google Scholar 

  18. Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Mach. Learn. 85(3), 333–359 (2011)

    Article  MathSciNet  Google Scholar 

  19. Senge, R., del Coz, J.J., Hüllermeier, E.: On the problem of error propagation in classifier chains for multi-label classification. In: Spiliopoulou, M., Schmidt-Thieme, L., Janning, R. (eds.) Data Analysis, Machine Learning and Knowledge Discovery. SCDAKO, pp. 163–170. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-01595-8_18

    Chapter  Google Scholar 

  20. da Silva, P.N., Gonçalves, E.C., Plastino, A., Freitas, A.A.: Distinct chains for different instances: an effective strategy for multi-label classifier chains. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014. LNCS (LNAI), vol. 8725, pp. 453–468. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-44851-9_29

    Chapter  Google Scholar 

  21. Sucar, L.E., Bielza, C., Morales, E.F., Hernandez-Leal, P., Zaragoza, J.H., Larrañaga, P.: Multi-label classification with Bayesian network-based chain classifiers. Pattern Recognit. Lett. 41, 14–22 (2014)

    Article  Google Scholar 

  22. Tsoumakas, G., Katakis, I., Vlahavas, I.: Mining Multi-label data. Data Mining and Knowledge Discovery Handbook, pp. 667–685 (2010)

    Chapter  Google Scholar 

  23. Tsoumakas, G., Spyromitros-Xioufis, E., Vilcek, J., Vlahavas, I.: MULAN: a java library for multi-label learning. J. Mach. Learn. Res. 12, 2411–2414 (2011)

    MathSciNet  MATH  Google Scholar 

  24. Vens, C., Struyf, J., Schietgat, L., Džeroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Mach. Learn. 73(2), 185 (2008)

    Article  Google Scholar 

  25. Zhang, X., Fan, W., Du, N.: Random decision hashing for massive data learning. In: Proceedings of the 4th International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications, pp. 65–80 (2015)

    Google Scholar 

  26. Zhang, X., Yuan, Q., Zhao, S., Fan, W., Zheng, W., Wang, Z.: Multi-label classification without the multi-label cost. In: Proceedings of the Society for Industrial and Applied Mathematics International Conference on Data Mining, pp. 778–789 (2010)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Knowledge Engineering Group, Technische Universtität Darmstadt, Darmstadt, Germany

    Moritz Kulessa & Eneldo Loza Mencía

Authors
  1. Moritz Kulessa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eneldo Loza Mencía
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eneldo Loza Mencía .

Editor information

Editors and Affiliations

  1. Goldsmiths University of London, London, UK

    Larisa Soldatova

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Joaquin Vanschoren

  3. University of Cyprus, Nicosia, Cyprus

    George Papadopoulos

  4. Università degli Studi di Bari Aldo Moro, Bari, Italy

    Michelangelo Ceci

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kulessa, M., Loza Mencía, E. (2018). Dynamic Classifier Chain with Random Decision Trees. In: Soldatova, L., Vanschoren, J., Papadopoulos, G., Ceci, M. (eds) Discovery Science. DS 2018. Lecture Notes in Computer Science(), vol 11198. Springer, Cham. https://doi.org/10.1007/978-3-030-01771-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-01771-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-01770-5

  • Online ISBN: 978-3-030-01771-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation