Skip to main content
SpringerLink
Log in

A Clustering-Based Approach to Reduce Feature Redundancy

  • Conference paper
  • First Online:
Knowledge, Information and Creativity Support Systems: Recent Trends, Advances and Solutions

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 364))

  • 876 Accesses

Abstract

Research effort has recently focused on designing feature weighting clustering algorithms. These algorithms automatically calculate the weight of each feature, representing their degree of relevance, in a data set. However, since most of these evaluate one feature at a time they may have difficulties to cluster data sets containing features with similar information. If a group of features contain the same relevant information, these clustering algorithms set high weights to each feature in this group, instead of removing some because of their redundant nature. This paper introduces an unsupervised feature selection method that can be used in the data pre-processing step to reduce the number of redundant features in a data set. This method clusters similar features together and then selects a subset of representative features for each cluster. This selection is based on the maximum information compression index between each feature and its respective cluster centroid. We present an empirical validation for our method by comparing it with a popular unsupervised feature selection on three EEG data sets. We find that our method selects features that produce better cluster recovery, without the need for an extra user-defined parameter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. de Amorim, R.C.: An empirical evaluation of different initializations on the number of k-means iterations. Lect. Notes Comput. Sci. 7629, 15–26 (2013)

    Article  Google Scholar 

  2. de Amorim, R.C.: Feature relevance in Ward’s hierarchical clustering using the Lp norm. J. Classif. 32(1) (to appear in 2015)

    Google Scholar 

  3. de Amorim, R.C., Komisarczuk, P.: On initializations for the Minkowski weighted k-means. Lect. Notes Comput. Sci. 7619, 45–55 (2012)

    Article  Google Scholar 

  4. de Amorim, R.C., Mirkin, B.: Minkowski metric, feature weighting and anomalous cluster initializing in k-means clustering. Pattern Recogn. 45(3), 1061–1075 (2012)

    Article  Google Scholar 

  5. de Amorim, R.C., Mirkin, B.: Removing redundant features via clustering: preliminary results in mental task separation. In: Proceedings of the 8th International Conference on Knowledge, Information and Creativity Support Systems (KICSS), November, pp. 7–9. Krakow, Poland (2013)

    Google Scholar 

  6. de Amorim, R.C., Mirkin, B., Gan, J.Q.: A method for classifying mental tasks in the space of EEG transforms. Technical report, Technical Report BBKS-10-01, Birkbeck University of London, London (2010)

    Google Scholar 

  7. de Amorim, R.C., Mirkin, B., Gan, J.Q.: Anomalous pattern based clustering of mental tasks with subject independent learning-some preliminary results. Artif. Intell. Res. 1(1), 46–54 (2012)

    Google Scholar 

  8. Ball, G.H., Hall, D.J.: A clustering technique for summarizing multivariate data. Behav. Sci. 12(2), 153–155 (1967)

    Article  Google Scholar 

  9. Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic Publishers, Norwell, MA (1981)

    Book  MATH  Google Scholar 

  10. Celebi, M.E., Kingravi, H.A., Vela, P.A.: A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl. 40(1), 200–210 (2013)

    Article  Google Scholar 

  11. Chan, E.Y., Ching, W.K., Ng, M.K., Huang, J.Z.: An optimization algorithm for clustering using weighted dissimilarity measures. Pattern Recogn. 37(5), 943–952 (2004)

    Article  MATH  Google Scholar 

  12. Chiang, M.M.T., Mirkin, B.: Intelligent choice of the number of clusters in k-means clustering: an experimental study with different cluster spreads. J. Classif. 27(1), 3–40 (2010)

    Article  MathSciNet  Google Scholar 

  13. Chiappa, S., Bengio, S.: HMM and IOHMM modeling of EEG rhythms for asynchronous BCI systems. In: European Symposium on Artificial Neural Networks, ESANN, pp. 193–204 (2004)

    Google Scholar 

  14. De Soete, G.: Optimal variable weighting for ultrametric and additive tree clustering. Qual. Quant. 20(2–3), 169–180 (1986)

    Article  Google Scholar 

  15. De Soete, G.: OVWTRE: a program for optimal variable weighting for ultrametric and additive tree fitting. J. Classif. 5(1), 101–104 (1988)

    Article  Google Scholar 

  16. DeSarbo, W.S., Carroll, J.D., Linda, C.A., Green, P.E.: Synthesized clustering: a method for amalgamating alternative clustering bases with differential weighting of variables. Psychometrika 49(1), 57–78 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  17. Frigui, H., Nasraoui, O.: Unsupervised learning of prototypes and attribute weights. Pattern Recogn. 37(3), 567–581 (2004)

    Article  Google Scholar 

  18. Gan, J.Q.: Self-adapting BCI based on unsupervised learning. In: 3rd International Workshop on Brain-Computer Interfaces, pp. 50–51 (2006)

    Google Scholar 

  19. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  20. Huang, J.Z., Ng, M.K., Rong, H., Li, Z.: Automated variable weighting in k-means type clustering. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 657–668 (2005)

    Article  Google Scholar 

  21. Huang, J.Z., Xu, J., Ng, M., Ye, Y.: Weighting method for feature selection in k-means. In: Computational Methods of Feature Selection, pp. 193–209 (2008)

    Google Scholar 

  22. Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)

    Article  Google Scholar 

  23. MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297. California, USA (1967)

    Google Scholar 

  24. Maitra, R., Peterson, A.D., Ghosh, A.P.: A systematic evaluation of different methods for initializing the k-means clustering algorithm. Trans. Knowl. Data Eng. 522–537 (2010)

    Google Scholar 

  25. Makarenkov, V., Legendre, P.: Optimal variable weighting for ultrametric and additive trees and k-means partitioning: methods and software. J. Classif. 18(2), 245–271 (2001)

    MathSciNet  MATH  Google Scholar 

  26. Millan, J., Mouriño, J.: Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project. IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 159–161 (2003)

    Article  Google Scholar 

  27. Milligan, G.W., Cooper, M.C.: A study of standardization of variables in cluster analysis. J. Classif. 5(2), 181–204 (1988)

    Article  MathSciNet  Google Scholar 

  28. Mirkin, B.: Clustering for Data Mining: A Data Recovery Approach, vol. 3. CRC Press (2005)

    Google Scholar 

  29. Mitra, P., Murthy, C.A., Pal, S.K.: Unsupervised feature selection using feature similarity. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 301–312 (2002)

    Article  Google Scholar 

  30. Pena, J.M., Lozano, J.A., Larranaga, P.: An empirical comparison of four initialization methods for the k-means algorithm. Pattern Recogn. Lett. 20(10), 1027–1040 (1999)

    Article  Google Scholar 

  31. Steinbach, M., Karypis, G., Kumar, V.: A comparison of document clustering techniques. In: KDD Workshop on Text Mining, pp. 525–526. Boston (2000)

    Google Scholar 

  32. Steinley, D.: Standardizing variables in k-means clustering. In: Classification, Clustering, and Data Mining Applications, pp. 53–60. Springer (2004)

    Google Scholar 

  33. Steinley, D., Brusco, M.J.: Initializing k-means batch clustering: a critical evaluation of several techniques. J. Classif. 24(1), 99–121 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  34. Svetlova, L., Mirkin, B., Lei, H.: MFWK-Means: Minkowski metric fuzzy weighted k-means for high dimensional data clustering. In: IEEE 14th International Conference on Information Reuse and Integration (IRI), pp. 692–699. IEEE (2013)

    Google Scholar 

  35. Tibshirani, R., Walther, G., Hastie, T.: Estimating the number of clusters in a data set via the gap statistic. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 63(2), 411–423 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  36. Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, College Lane Campus, University of Hertfordshire, Hatfield, AL10 9AB, UK

    Renato Cordeiro de Amorim

  2. Department of Computer Science and Information Systems, Birkbeck University of London, Malet Street, London, WC1E 7HX, UK

    Boris Mirkin

Authors
  1. Renato Cordeiro de Amorim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Boris Mirkin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Renato Cordeiro de Amorim .

Editor information

Editors and Affiliations

  1. Department of Automatic Control and Biomedical Engineering, AGH University of Science and Technology, Kraków, Poland, and International Centre for Decision Sciences and Forecasting, Kraków, Poland

    Andrzej M.J. Skulimowski

  2. Polish Academy of Sciences, Systems Research Institute, Warszawa, Poland, and AGH University of Science and Technology, Kraków, Poland

    Janusz Kacprzyk

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

de Amorim, R.C., Mirkin, B. (2016). A Clustering-Based Approach to Reduce Feature Redundancy. In: Skulimowski, A., Kacprzyk, J. (eds) Knowledge, Information and Creativity Support Systems: Recent Trends, Advances and Solutions. Advances in Intelligent Systems and Computing, vol 364. Springer, Cham. https://doi.org/10.1007/978-3-319-19090-7_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19090-7_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19089-1

  • Online ISBN: 978-3-319-19090-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation