Abstract
In speech recognition, spectro-temporal feature extraction and the training of the acoustical model are usually performed separately. To improve recognition performance, we present a combined model which allows the training of the feature extraction filters along with a neural net classifier. Besides expecting that this joint training will result in a better recognition performance, we also expect that such a neural net can generate coefficients for spectro-temporal filters and also enhance preexisting ones, such as those obtained with the two-dimensional Discrete Cosine Transform (2D DCT) and Gabor filters. We tested these assumptions on the TIMIT phone recognition task. The results show that while the initialization based on the 2D DCT or Gabor coefficients is better in some cases than with simple random initialization, the joint model in practice always outperforms the standard two-step method. Furthermore, the results can be significantly improved by using a convolutional version of the network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aertsen, A.M., Johannesma, P.I.: The spectro-temporal receptive field. A functional characteristic of auditory neurons. Biological Cybernetics 42(2), 133–143 (1981)
Bouvrie, J., Ezzat, T., Poggio, T.: Localized Spectro-Temporal Cepstral Analysis of Speech. In: Proc. ICASSP, pp. 4733–4736 (2008)
Kovács, G., Tóth, L.: Localized Spectro-Temporal Features for Noise-Robust Speech Recognition. In: Proc. ICCC-CONTI 2010, pp. 481–485 (2010)
Kovács, G., Tóth, L.: Phone Recognition Experiments with 2D-DCT Spectro-Temporal Features. In: Proc. SACI 2011, pp. 143–146 (2011)
Meyer, B.T., Kollmeier, B.: Optimization and evaluation of Gabor feature sets for ASR. In: Proc. Interspeech 2008, pp. 906–909 (2008)
Kleinschmidt, M.: Localized Spectro-Temporal Features for Automatic Speech Recognition. In: Proc. EuroSpeech 2003, pp. 2573–2576 (2003)
Kleinschmidt, M.: Methods for capturing spectrotemporal modulations in automatic speech recognition. Acta Acustica United With Acustica 88(3), 416–422 (2002)
Greenberg, S.: Understanding Speech Understanding: Towards A Unified Theory Of Speech Perception. In: Proceedings of the ESCA Tutorial and Advanced Research Workshop on the Auditory Basis of Speech Perception, pp. 1–8 (1996)
Ezzat, T., Bouvrie, J., Poggio, T.: Spectro-Temporal Analysis of Speech Using 2-D Gabor Filters. In: Proc. Interspeech 2007, pp. 506–509 (2007)
Kleinschmidt, M., Gelbart, D.: Improving Word Accuracy with Gabor Feature Extraction. In: Proc. ICSLP 2002, pp. 25–28 (2002)
Bourlard, H., Morgan, N.: Connectionist speech recognition: A hybrid approach. Kluwer Academic Pub. (1994)
Abdel-Hamid, O., Mohamed, A., Jiang, H., Penn, G.: Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition. In: Proc. ICASSP 2012, pp. 4277–4280 (2012)
Vesely, K., Karafiat, M., Grezl, F.: Convolutive Bottleneck Network features for LVCSR. In: Proc. ASRU 2011, pp. 42–47 (2011)
Lee, K.-F., Hon, H.-W.: Speaker-independent phone recognition using Hidden Markov models. IEEE Trans. Acoust., Speech Signal Processing 37, 1641–1648 (1989)
Young, S., et al.: PC The HTK book version 3.4. Cambridge University Engineering Department, Cambridge (2006)
Vinyals, O., Deng, L.: Are sparse representations enough for acoustic modeling? In: Proc. INTERSPEECH (2012)
Huang, G.-B., Wang, D.H., Lan, Y.: Extreme learning machines: A survey. International Journal of Machine Learning and Cybernetics 2(2), 107–122 (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kovács, G., Tóth, L. (2013). The Joint Optimization of Spectro-Temporal Features and Neural Net Classifiers. In: Habernal, I., Matoušek, V. (eds) Text, Speech, and Dialogue. TSD 2013. Lecture Notes in Computer Science(), vol 8082. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40585-3_69
Download citation
DOI: https://doi.org/10.1007/978-3-642-40585-3_69
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40584-6
Online ISBN: 978-3-642-40585-3
eBook Packages: Computer ScienceComputer Science (R0)