Abstract
Though impressive classification accuracy is often obtained via discrimination-based learning techniques such as Multi-Layer Perceptrons (DMLP), these techniques often assume that the underlying training sets are optimally balanced (in terms of the number of positive and negative examples). Unfortunately, this is not always the case. In this paper, we look at a recognition-based approach whose accuracy in such environments is superior to that obtained via more conventional mechanisms. At the heart of the new technique is a modified auto-encoder that allows for the incorporation of a recognition component into the conventional MLP mechanism. In short, rather than being associated with an output value of “1”, positive examples are fully reconstructed at the network output layer while negative examples, rather than being associated with an output value of “0”, have their inverse derived at the output layer. The result is an auto-encoder able to recognize positive examples while discriminating against negative ones by virtue of the fact that negative cases generate larger reconstruction errors. A simple technique is employed to exaggerate the impact of training with these negative examples so that reconstruction errors can be more reliably established. Preliminary testing on both seismic and sonar data sets has demonstrated that the new method produces lower error rates than standard connectionist systems in imbalanced settings. Our approach thus suggests a simple and more robust alternative to commonly used classification mechanisms.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Batti, R., 1992. “First and Second Order Methods for Learning: Between Steepest Descent and Newton’s Method”, Neural Computation, vol. 4, no. 2, pp. 141–166.
Cottrell, G. W., Munro, P. and Zipser, D., 1987. “Learning Internal Representations from Gray-Scale Images: An Example of Extensional Programming”, Proceedings of the 1987 Conference of the Cognitive Science Society, pp. 462–473.
Ezawa, K.J., Singh, M. and Norton, S.W., 1996. “Learning Goal Oriented Bayesian Networks for Telecommunications Management. Proceedings of the Thirteenth International Conference on Machine Learning, pp. 139–147.
Fawcett, T. E. and Provost, F., 1997. “Adaptive Fraud Detection”, Data Mining and Knowledge Discovery, 1(3):291–316.
Gluck, M.A. and Myers, C., 1993. “Hippocampal Mediation of Stimulus Representation: A Computational Theory”, Hippocampus, 4(3): 491–516.
Japkowicz, N. Myers, C. and Gluck, M.A., 1995. “A Novelty Detection Approach to Classification”, Proceedings of the Fourteenth Joint Conference on Artificial Intelligence, pp. 518–523.
Kubat, M. and Matwin, S., 1997. “Addressing the Curse of Imbalanced Data Sets: One-sided Sampling”, Proceedings of the Fourteenth International Conference on Machine Learning, pp. 179–186.
Kubat, M., Holte, R. and Matwin, S., 1998. “Machine Learning for the Detection of Oil Spills in Satellite Radar Images”, Machine Learning, 30:195–215.
Ling, C. and Li, C., 1998. “Data Mining for Direct Marketing: Problems and Solutions”, Proceedings of KDD-98.
Pazzani, M. and Merz, C. and Murphy, P. and Ali, K. and Hume, T. and Brunk, C., “Reducing Misclassification Costs”, Proceedings of the Eleventh International Conference on Machine Learning, pp. 217–225.
Stainvas, I., Intrator, N. and Moshaiov, A., “Blurred Face Recognition via a Hybrid Network Architecture”, Proceedings of the conference on Neural Computation in Science and Technology 99.
Schwenk, H. and Milgram, M., 1995. “Transformation Invariant Autoassociation with Application to Handwritten Character Recognition”, Proceedings of the Seventh Conference on Neural Information Processing Systems, pp. 991–998.
Riddle, P., Segal, R. and Etzioni, O., 1994. “Representation Design and Brute-Force Induction in a Boeing Manufacturing Domain. Applied Artificial Intelligence, 8:125–147.
Riedmiller, M., and H. Braun, “A direct adaptive method for faster backpropagation learning: The RPROP algorithm,” Proceedings of the IEEE International Conference on Neural Networks, 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Eavis, T., Japkowicz, N. (2000). A Recognition-Based Alternative to Discrimination-Based Multi-layer Perceptrons. In: Hamilton, H.J. (eds) Advances in Artificial Intelligence. Canadian AI 2000. Lecture Notes in Computer Science(), vol 1822. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45486-1_23
Download citation
DOI: https://doi.org/10.1007/3-540-45486-1_23
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67557-0
Online ISBN: 978-3-540-45486-1
eBook Packages: Springer Book Archive