Skip to main content
SpringerLink
Log in

Online streaming feature selection: a minimum redundancy, maximum significance approach

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

All the traditional feature selection methods assume that the entire input feature set is available from the beginning. However, online streaming features (OSF) are integral part of many real-world applications. In OSF, the number of training examples is fixed while the number of features grows with time as new features stream in. A critical challenge for online streaming feature selection (OSFS) is the unavailability of the entire feature set before learning starts. OS-NRRSAR-SA is a successful OSFS algorithm that controls the unknown feature space in OSF by means of the rough sets-based significance analysis. This paper presents an extension to the OS-NRRSAR-SA algorithm. In the proposed extension, the redundant features are filtered out before significance analysis. In this regard, a redundancy analysis method based on functional dependency concept is proposed. The result is a general OSFS framework containing two major steps, (1) online redundancy analysis that discards redundant features, and (2) online significance analysis, which eliminates non-significant features. The proposed algorithm is compared with OS-NRRSAR-SA algorithm, in terms of compactness, running time and classification accuracy during the features streaming. The experiments demonstrate that the proposed algorithm achieves better results than OS-NRRSAR-SA algorithm, in every way.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Beaubouef T, Petry F (2013) Incorporating rough data in database design for imprecise information representation. In: Skowron A, Suraj Z (eds) Rough sets and intelligent systems—Professor Zdzisław Pawlak in Memoriam, intelligent systems reference library, vol 43. Springer, Berlin, pp 137–155

    Google Scholar 

  2. Blake C, Merz CJ (1998) UCI Repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html. Accessed 06 March 2015

  3. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol 2(3):27:1–27:27

    Article  Google Scholar 

  4. Chao S, Cai J, Yang S, Wang S (2016) A clustering based feature selection method using feature information distance for text data. Springer, Cham, pp 122–132

    Google Scholar 

  5. Clopinet: feature selection challenge, NIPS 2003. http://clopinet.com/isabelle/Projects/NIPS2003/ (2003). Accessed 6 March 2015

  6. Clopinet: performance prediction challenge, WCCI 2006. http://clopinet.com/isabelle/Projects/modelselect/ (2006). Accessed 6 March 2015

  7. Clopinet: causation and prediction challenge, WCCI 2008. http://www.causality.inf.ethz.ch (2008). Accessed 6 March 2015

  8. Dubois D, Prade H (1992) Putting rough sets and fuzzy sets together. In: SÅowiÅki R (ed) Intelligent decision support, theory and decision library, vol 11. Springer, Dordrecht, pp 203–232

    Chapter  MATH  Google Scholar 

  9. Eskandari S, Akbas E (2017) Supervised infinite feature selection. CoRR abs/1704.02665. http://arxiv.org/abs/1704.02665

  10. Eskandari S, Javidi M (2016) Online streaming feature selection using rough sets. Int J Approx Reason 69:35–57

    Article  MathSciNet  MATH  Google Scholar 

  11. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html

  12. Glocer K, Eads D, Theiler J (2005) Online feature selection for pixel classification. In: Proceedings of the 22nd international conference on machine learning, Bonn, Germany

  13. Gosztolya G, Tóth L (2017) A feature selection-based speaker clustering method for paralinguistic tasks. Pattern Anal Appl 1–12

  14. Herbert JP, Yao J (2011) Game-theoretic rough sets. Fundam Inform 108(3–4):267–286

    Article  MathSciNet  MATH  Google Scholar 

  15. Javidi MM, Eskandari S (2016) Streamwise feature selection: a rough set method. Int J Mach Learn Cybern 1–10

  16. Javidi MM, Eskandari S (2017) A noise resistant dependency measure for rough set-based feature selection. J Intell Fuzzy Syst 33(3):1–14

    Google Scholar 

  17. Jensen R, Shen Q (2001) A rough set-aided system for sorting www bookmarks. In: Proceedings of the first Asia-Pacific conference on web intelligence: research and development, WI’01, London, UK

  18. Jensen R, Shen Q (2005) Fuzzy-rough data reduction with ant colony optimization. Fuzzy Sets Syst 149(1):5–20

    Article  MathSciNet  MATH  Google Scholar 

  19. Jensen R, Shen Q (2008) Computational intelligence and feature selection: rough and fuzzy approaches. Wiley, London

    Book  Google Scholar 

  20. Jensen R (2004) Qiang Shen: semantics-preserving dimensionality reduction: rough and fuzzy-rough based approaches. IEEE Trans Knowl Data Eng 16(16):1457–1471

    Article  Google Scholar 

  21. Jensen R, Tuson A, Shen Q (2014) Finding rough and fuzzy-rough set reducts with SAT. Inf Sci 255:100–120

    Article  MathSciNet  MATH  Google Scholar 

  22. Liu W, Zha ZJ, Wang Y, Lu K, Tao D (2016) p-Laplacian regularized sparse coding for human activity recognition. IEEE Trans Ind Electron 63(8):5120–5129

    Google Scholar 

  23. Li T, Ruan D, Geert W, Song J, Xu Y (2007) A rough sets based characteristic relation approach for dynamic attribute generalization in data mining. Knowl Based Syst 20(5):485–494

    Article  Google Scholar 

  24. Li Z, Liu J, Tang J, Lu H (2015) Robust structured subspace learning for data representation. IEEE Trans Pattern Anal Mach Intell 37(10):2085–2098

    Article  Google Scholar 

  25. Li Z, Liu J, Yang Y, Zhou X, Lu H (2014) Clustering-guided sparse structural learning for unsupervised feature selection. IEEE Trans Knowl Data Eng 26(9):2138–2150

    Article  Google Scholar 

  26. Li Z, Tang J (2015) Unsupervised feature selection via nonnegative spectral analysis and redundancy control. IEEE Trans Image Process 24(12):5343–5355

    Article  MathSciNet  MATH  Google Scholar 

  27. Parthalain N, Shen Q, Jensen R (2010) A distance measure approach to exploring the rough set boundary region for attribute reduction. IEEE Trans Knowl Data Eng 22(3):305–317

    Article  Google Scholar 

  28. Pawlak Z (1982) Rough sets. Int J Comput Inform Sci 11(5):341–356

    Article  MATH  Google Scholar 

  29. Perkins S, Theiler J (2003) Online feature selection using grafting. In: International conference on machine learning. ACM Press, pp 592–599

  30. Pudil P, Novovičová J, Kittler J (1994) Floating search methods in feature selection. Pattern Recogn Lett 15(11):1119–1125

    Article  Google Scholar 

  31. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco

    Google Scholar 

  32. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Preprint arXiv:1409.1556

  33. Skowron A, Stepaniuk J (1996) Tolerance approximation spaces. Fundam Inf 27(2–3):245–253

    Article  MathSciNet  MATH  Google Scholar 

  34. Swiniarski RW, Skowron A (2003) Rough set methods in feature selection and recognition. Pattern Recogn Lett 24(6):833–849

    Article  MATH  Google Scholar 

  35. Ungar L, Zhou J, Foster D, Stine B (2005) Streaming feature selection using IIC. In: Proceedings of the 10th international conference on artificial intelligence and statistics

  36. Walczak B, Massart D (1999) Rough sets theory. Chemometr Intell Lab Syst 47(1):1–16

    Article  Google Scholar 

  37. Wang F, Liang J, Qian Y (2013) Attribute reduction: a dimension incremental strategy. Knowl Based Syst 39:95–108

    Article  Google Scholar 

  38. Wang J, Zhao P, Hoi S, Jin R (2014) Online feature selection and its applications. IEEE Trans Knowl Data Eng 26(3):698–710. https://doi.org/10.1109/TKDE.2013.32

    Article  Google Scholar 

  39. Witten IH, Frank E (2005) Data mining: practical machine learning tools and techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems). Morgan Kaufmann Publishers Inc., San Francisco

    Google Scholar 

  40. Wu X, Yu K, Ding W, Wang H, Zhu X (2013) Online feature selection with streaming features. IEEE Trans Pattern Anal Mach Intell 35:1178–1192

    Article  Google Scholar 

  41. Yang X, Liu W, Tao D, Cheng J (2017) Canonical correlation analysis networks for two-view image recognition. Inf Sci 385–386:338–352

    Article  Google Scholar 

  42. Yao Y (2007) Decision-theoretic rough set models. Springer, Berlin, pp 1–12

    Google Scholar 

  43. Yu K, Ding W, Simovici DA, Wang H, Pei J, Wu X (2015) Classification with streaming features: an emerging-pattern mining approach. ACM Trans Knowl Discov Data (TKDD) 9(4):30

    Google Scholar 

  44. Yu K, Ding W, Wu X (2016) Lofs: a library of online streaming feature selection. Knowl Based Syst 113:1–3

    Article  Google Scholar 

  45. Yu K, Wang D, Ding W, Pei J, Small DL, Islam S, Wu X (2015) Tornado forecasting with multiple markov boundaries. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 2237–2246

  46. Yu K, Wu X, Ding W, Pei J (2014) Towards scalable and accurate online feature selection for big data. In: IEEE international conference on data mining (ICDM). IEEE, pp 660–669

  47. Zhou J, Foster D, Stine R, Ungar L (2005) Streaming feature selection using alpha-investing. In: Proceedings of the 11th ACM SIGKDD international conference on knowledge discovery in data mining, Chicago, IL, USA

  48. Ziarko W (1993) Variable precision rough set model. J Comput Syst Sci 46(1):39–59

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors would like to thank professor Mahbano Tata for her comments that greatly improved the manuscript.

Author information

Authors and Affiliations

  1. Shahid Bahonar University of Kerman, Kerman, Iran

    Mohammad Masoud Javidi & Sadegh Eskandari

  2. Faculty of Mathematical Sciences, University of Guilan, Rasht, Iran

    Sadegh Eskandari

Authors
  1. Mohammad Masoud Javidi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sadegh Eskandari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Masoud Javidi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Javidi, M.M., Eskandari, S. Online streaming feature selection: a minimum redundancy, maximum significance approach. Pattern Anal Applic 22, 949–963 (2019). https://doi.org/10.1007/s10044-018-0690-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-018-0690-7

Keywords

Search

Navigation