Skip to main content
SpringerLink
Log in

Pattern classification with missing data: a review

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Pattern classification has been successfully applied in many problem domains, such as biometric recognition, document classification or medical diagnosis. Missing or unknown data are a common drawback that pattern recognition techniques need to deal with when solving real-life classification tasks. Machine learning approaches and methods imported from statistical learning theory have been most intensively studied and used in this subject. The aim of this work is to analyze the missing data problem in pattern classification tasks, and to summarize and compare some of the well-known methods used for handling missing values.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Henceforth, the terms pattern, input vector, case, observation, sample, and example are used as synonyms.

  2. http://www.isical.ac.in/~sushmita/patterns/.

  3. http://www.archive.ics.uci.edu/ml/.

References

  1. Duda RO, Hart PE, Stork DG (2000) Pattern classification. Wiley, New York

    Google Scholar 

  2. Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, New York

    MATH  Google Scholar 

  3. Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, Oxford

    Google Scholar 

  4. Watanabe S (1985) Pattern recognition: human and mechanical. Wiley, New York

    Google Scholar 

  5. Jain AK, Duin RPW, Mao J (2000) Statistical pattern recognition: a review. IEEE Trans Pattern Anal Mach Intell 22(1):4–37

    Article  Google Scholar 

  6. Little RJA, Rubin DB (2002) Statistical analysis with missing data, 2nd edn. Wiley, New Jersey

    MATH  Google Scholar 

  7. Schafer JL (1997) Analysis of incomplete multivariate data. Chapman & Hall, Florida

    MATH  Google Scholar 

  8. Allison PD (2001) Missing data. Sage university papers series on quantitative applications in the social sciences. Thousan Oaks, California

    Google Scholar 

  9. Rubin DB (1987) Multiple imputation for nonresponse in surveys. Wiley, New York

    Book  Google Scholar 

  10. Wang L, Fan X (2004) Missing data in disguise and implications for survey data analysis. Field Methods 16(3):332–351

    Article  Google Scholar 

  11. Nguyen LN, Scherer WT (2003) Imputation techniques to account for missing data in support of intelligent transportation systems applications. Tech. Rep., University of Virginia, USA

  12. Lakshminarayan K, Harp SA, Samad T (2004) Imputation of missing data in industrial databases. Eng Appl Artif Intell 11(3):259–275

    Google Scholar 

  13. Ji C, Elwalid A (2000) Measurement-based network monitoring: missing data formulation and scalability analysis. IEEE Int Symp Inf Theory, Sorrento, Italy, p 78

  14. Halatchev M, Gruenwald L (2005) Estimating missing values in related sensor data streams. In Int Conf Manage Data, pp 83–94

  15. Mohammed HS, Stepenosky N, Polikar R (2006) An ensemble technique to handle missing data from sensors. In: IEEE Sens Appl Symp, Houston, Texas, USA, pp 101–105

  16. Cooke M, Green P, Crawford M (1994) Handling missing data in speech recognition. Int Conf Spoken Lang Process, pp 1555–1558

  17. Parveen S, Green P (2004) Speech enhancement with missing data techniques using recurrent neural networks. In: IEEE ICASSP, vol 1, pp 733–736

  18. DiCesare G (2006) Imputation, estimation and missing data in finance. Ph.D. dissertation, University of Waterloo

  19. Sharpe IG, Kofman P (2003) Using multiple imputation in the analysis of incomplete observations in finance. J Financ Econ 1(2):216–249

    Google Scholar 

  20. Troyanskaya O, Cantor M, Alter O, Sherlock G, Brown P, Botstein D, Tibshirani R, Hastie T, Altman R (2001) Missing value estimation methods for DNA microarrays. Bioinformatics 17(6):520–525

    Article  Google Scholar 

  21. Kim H, Golub GH, Park H (2004) Imputation of missing values in DNA microarray gene expression data. In: Proc IEEE Comput Syst Bioinform Conf

  22. Liu P, El-Darzi E, Lei L, Vasilakis C, Chountas P, Huang W (2005) An analysis of missing data treatment methods and their application to health care dataset. In: Li X et al (eds) ADMA, LNCS 3584, Springer, pp 583–590

  23. Markey MK, Patel A (2004) Impact of missing data in training artificial neural networks for computer-aided diagnosis. In: Proc Int Conf Mach Learn Appl, pp 351–354

  24. Proschan MA, McMahon RP, Shih JH, Hunsberger SA, Geller N, Knatterud G, Wittes J (2001) Sensitivity analysis using an imputation method for missing binary data in clinical trials. J Stat Plan Inference 96(1):155–165

    Article  MATH  Google Scholar 

  25. Jerez JM, Molina I, Subirats JL, Franco L (2006) Missing data imputation in breast cancer prognosis. In BioMed’06. ACTA Press Anaheim, CA, pp 323–328

    Google Scholar 

  26. Batista G, Monard MC (2003) Experimental comparison of K-nearest neighbour and mean or mode imputation methods with the internal strategies used by C4.5 and CN2 to treat missing data. Tech. Rep., University of Sao Paulo

  27. Batista G, Monard MC (2002) A study of K-nearest neighbour as an imputation method. In: Abraham A et al (eds) Hybrid Intell Syst, Ser Front Artif Intell Appl 87, IOS Press, pp 251–260

  28. Kohonen T (2006) Self-organizing maps, 3rd edn. Springer

  29. Samad T, Harp SA (1992) Self-organization with partial data. Netw Computat Neural Syst 3(2):205–212

    Article  Google Scholar 

  30. Fessant F, Midenet S (2002) Self-organizing map for data imputation and correction in surveys. Neural Comput Appl 10(4):300–310

    Article  MATH  Google Scholar 

  31. Piela P (2002) Introduction to self-organizing maps modelling for imputation—techniques and technology. Res Stat Note Health Care Financ Adm Off Policy Plan Res 2:5–19

    Google Scholar 

  32. Sharpe PK, Solly RJ (1995) Dealing with missing values in neural network-based diagnostic systems. Neural Comput Appl 3(2):73–77

    Article  Google Scholar 

  33. Nordbotten S (1996) Neural network imputation applied to the Norwegian 1990 population census data. J Off Stat 12:385–401

    Google Scholar 

  34. Gupta A, Lam MS (1996) Estimating missing values using neural networks. J Oper Res Soc 47(2):229–238

    MATH  Google Scholar 

  35. Yoon SY, Lee SY (1999) Training algorithm with incomplete data for feed-forward neural networks. Neural Process Lett 10:171–179

    Article  Google Scholar 

  36. Kallin L (2002) Missing data and the preprocessing perceptron. Tech. Rep., Umeaå University

  37. Bengio Y, Gingras F (1995) “Recurrent neural networks for missing or asynchronous data. In: Touretzky DS et al (eds) Adv Neural Inf Process Syst 8. MIT Press, pp 395–401

  38. Parveen S (2003) Connectionist approaches to the deployment of prior knowledge for improving robustness in automatic speech recognition. Ph.D. dissertation, University of Sheffield

  39. Pyle D (1999) Data preparation for data mining. Morgan Kaufmann Publishers Inc., San Francisco

    Google Scholar 

  40. Narayanan S, Vian JL, Choi J, El-Sharkawi M, Thompson BB (2002) Set constraint discovery: missing sensor data restoration using auto-associative regression machines. In: Proc Int Jt Conf Neural Netw, Honolulu, pp 2872–2877

  41. Chung D, Merat FL (1996) Neural network based sensor array signal processing. In: Proc Int Conf Multisens Fusion Integr Intell Syst, Washington, USA, pp 757–764

  42. Marseguerra M, Zoia A (2005) The autoassociative neural network in signal analysis. II. Application to on-line monitoring of a simulated BWR component. Ann Nuclear Energy 32(11):1207–1223

    Article  Google Scholar 

  43. Marwala T, Chakraverty S (2006) Fault classification in structures with incomplete measured data using autoassociative neural networks and genetic algorithm. Curr Sci India 90(4):542–548

    Google Scholar 

  44. Caruana R (1997) Multitask learning. Ph.D. dissertation, Carnegie Mellon University

  45. Silver DL (2000) Selective transfer of neural network task knowledge, Ph.D. dissertation, University of Western Ontario

  46. García-Laencina PJ, Figueiras-Vidal AR, Serrano-García J, Sancho-Gómez JL (2005) Exploiting multitask learning schemes using private subnetworks. In: Cabestany J et al (eds) Comput Intell Bioinsp Syst, Lect Notes Comput Sci 3512, Springer, pp 233–240

  47. García-Laencina PJ, Serrano J, Figueiras-Vidal AR, Sancho-Gómez JL (2007) Multi-task neural networks for dealing with missing inputs. In: Mira J, Álvarez JR (eds) IWINAC 2007, part I, Lect Notes Comput Sci 4527, Springer, pp 282–291

  48. Ghahramani Z, Jordan MI (1994) Supervised learning from incomplete data via an EM approach. In: Cowan JD et al (eds) Adv Neural Inf Process Syst 6, Morgan Kaufmann Publishers Inc., pp 120–127

  49. Ghahramani Z, Jordan MI (1994) Learning from incomplete data. Tech. Rep. AIM-1509, Massachusetts Institute of Technology, Cambridge, MA, USA

  50. McLachlan GJ, Krishnan T (1997) The EM algorithm and extensions. Wiley, New York

    MATH  Google Scholar 

  51. Ahmad S, Tresp V (1993) Some solutions to the missing feature problem in vision. In: Adv Neural Inf Process Syst 5, Morgan Kaufmann Publishers Inc., San Mateo, CA, USA, pp 393–400

  52. Tresp V, Ahmad S, Neuneier R (1993) Training neural networks with deficient data. In: Cowan JD et al (eds) Adv Neural Inf Process Syst 6. Morgan Kaufmann Publishers Inc., San Francisco, pp 128–135

    Google Scholar 

  53. Tresp V, Neuneier R, Ahmad S (1994) Efficient methods for dealing with missing data in supervised learning. In: Tesauro G et al (eds) Adv Neural Inf Process Syst 7, The MIT Press, pp 689–696

  54. Williams D, Liao X, Xue Y, Carin L, Krishnapuram B (2007) On classification with incomplete data. IEEE Trans Pattern Anal Mach Intell 29(3):427–436

    Article  Google Scholar 

  55. Ramoni M, Sebastiani P (2001) Robust learning with missing data. Mach Learn 45:147–170

    Article  MATH  Google Scholar 

  56. Krause S, Polikar R (2003) An ensemble of classifiers for the missing feature problem. In: Proc Intl Jt Conf Neural Netw, Portland, USA, pp 553–558

  57. Jian K, Chen H, Yuan S (2005) Classification for incomplete data using classifier ensembles. In: Proc Intl Conf Neural Netw Brain, pp 559–563

  58. Juszczak P, Duin RPW (2004) Combining one-class classifiers to classify missing data. In: Roli F et al (eds) Mult Classif Syst, Lect Notes Comput Sci 3077, Springer, pp 92–101

  59. Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106

    Google Scholar 

  60. Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann (Series in Machine Learning)

  61. Quinlan JR (1989) Unknown attribute values in induction. In: Proc Intl Workshop Mach Learn, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp 164–168

  62. Webb GI (1998) The problem of missing values in decision tree grafting. In: Proc Aust Jt Conf Artif Intell, Springer, pp 273–283

  63. Zheng Z, Low BT (1999) Classifying unseen cases with many missing values. In: Zhong N, Zhou L (eds) Pac Asia Conf Knowl Discov Data Min, Lect Notes Art Intell 1574, Springer, pp 370–374

  64. Clark P, Niblett T (1989) The CN2 induction algorithm. Mach Learn 3(4):261–283

    Google Scholar 

  65. Ishibuchi H, Miyazaki A, Kwon K, Tanaka H (1993) Learning from incomplete training data with missing values and medical application. In: Proc IEEE Intl Jt Conf Neural Netw, pp 1871–1874

  66. Ishibuchi H, Moriola K (1995) Classification of fuzzy input patterns by neural networks. In: Proc IEEE Intl Conf Neural Netw, Perth, WA, Australia, pp 3118–3123

  67. Ishibuchi H, Tanaka H (1991) An extension of the BP-algorithm to interval input vectors-learning from numerical data and expert’s knowledge. In: Proc IEEE Intl Jt Conf Neural Netw, pp 1588–1593

  68. Petit-Renaud S, Denux T (1998) A neuro-fuzzy model for missing data reconstruction. In: Proc IEEE Workshop Emerg Technol, St. Paul, MN, USA

  69. Gabrys B (2000) Pattern classification for incomplete data. In: Proc Intl Conf Knowl Based Intell Eng Syst Allied Technol, Brightom, UK, pp 454–457

  70. Gabrys B (2002) Neuro-fuzzy approach to processing inputs with missing values in pattern recognition problems. Int J Approx Reason 30(3):149–179

    Article  MATH  MathSciNet  Google Scholar 

  71. Berthold MR, Huber KP (1998) Missing values and learning of fuzzy rules. Intl J Uncertain Fuzzy Knowl Based Syst 6(2):171–178

    Article  MATH  Google Scholar 

  72. Berthold MR, Huber KP (1997) Missing values and learning of fuzzy rules. In: Proc Workshop Fuzzy Neuro Syst, 1997

  73. Nauck D, Kruse R (1999) Learning in neuro-fuzzy systems with symbolic attributes and missing values. In: Proc 6th Intl Conf Neural Inf Process, Perth, WA, Australia, pp 142–147

  74. Hathaway RJ, Bezdek JC (2001) Fuzzy C-means clustering of incomplete data. IEEE Trans Syst Man Cybern B Cybern 31(5):735–744

    Article  Google Scholar 

  75. Ichihashi H, Honda K (2005) Fuzzy c-means classifier for incomplete data sets with outliers and missing values. In: Proc Intl Conf Comput Intell Modell Control Autom, IEEE Computer Society, Washington, DC, USA, pp 457–464

  76. Sarkar M, Leong TY (2001) Fuzzy k-means clustering with missing values. In: Proc AMIA Annu Symp, pp 588–592

  77. Lim CP, Leong JH, Kuan MM (2005) A hybrid neural network system for pattern classification tasks with missing features. IEEE Trans Pattern Anal Mach Intell 27(4):648–653

    Article  Google Scholar 

  78. Bhattacharyya C, Shivaswamy PK, Smola AJ (2004) A second order cone programming formulation for classifying missing data. In: Saul LK et al (eds) Adv Neural Inf Process Syst 17. MIT Press, Cambridge, pp 153–160

    Google Scholar 

  79. Smola AJ, Vishwanathan S, Hofmann T (2005) Kernel methods for missing variables. In: Ghahramani Z, Cowell R (eds) Proc AISTATS’05. Society for artificial intelligence and statistics, pp 325–332

  80. Pelckmans K, Brabanter JD, Suykens JAK, Moor BD (2005) Handling missing values in support vector machine classifiers. Neural Netw 18(5–6):684–692

    Article  MATH  Google Scholar 

  81. Bi J, Zhang T (2005) Support vector classification with input data uncertainty. In: Saul LK et al (eds) Adv Neural Inf Process Syst 17. MIT Press, Cambridge, pp 161–168

    Google Scholar 

  82. Chechik G, Heitz G, Elidan H, Abbeel P, Koller D (2007) Max-margin classification with incomplete data. In: Schölkopf B et al (eds) Adv Neural Inf Process Syst 19. MIT Press, Cambridge, pp 233–240

    Google Scholar 

  83. Kwak N, Choi C-H (2002) Input feature selection by mutual information based on Parzen window. IEEE Trans Pattern Anal Mach Intell 24(12):1667–1671

    Article  Google Scholar 

Download references

Acknowledgments

This work is partially supported by Ministerio de Educación y Ciencia under grants TEC2005-00992 and TEC2006-13338/TCM, and also by Consejería de Educación y Cultura de Murcia under grant 03122/PI/05.

Author information

Authors and Affiliations

  1. Dpto. Tecnologías de la Información y las Comunicaciones, Universidad Politécnica de Cartagena, Plaza del Hospital 1, 30202, Cartagena (Murcia), Spain

    Pedro J. García-Laencina & José-Luis Sancho-Gómez

  2. Dpto. Teoría de Señal y Comunicaciones, Universidad Carlos III de Madrid, Avda. de la Universidad, 30, 28911, Leganés (Madrid), Spain

    Aníbal R. Figueiras-Vidal

Authors
  1. Pedro J. García-Laencina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José-Luis Sancho-Gómez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aníbal R. Figueiras-Vidal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro J. García-Laencina.

Rights and permissions

Reprints and permissions

About this article

Cite this article

García-Laencina, P.J., Sancho-Gómez, JL. & Figueiras-Vidal, A.R. Pattern classification with missing data: a review. Neural Comput & Applic 19, 263–282 (2010). https://doi.org/10.1007/s00521-009-0295-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-009-0295-6

Keywords

Search

Navigation