Recurrent Neural Network Language Models (RNN-LMs) have recently shown exceptional performance across a variety of applications. In this paper, we modify the architecture to perform Language Understanding, and advance the state-of-the-art for the widely used ATIS dataset. The core of our approach is to take words as input as in a standard RNN-LM, and then to predict slot labels rather than words on the output side. We present several variations that differ in the amount of word context that is used on the input side, and in the use of non-lexical features. Remarkably, our simplest model produces state-of-the-art results, and we advance state-of-the-art through the use of bag-of-words, word embedding, named-entity, syntactic, and word-class features. Analysis indicates that the superior performance is attributable to the task-specific word representations learned by the RNN.
Cite as: Yao, K., Zweig, G., Hwang, M.-Y., Shi, Y., Yu, D. (2013) Recurrent neural networks for language understanding. Proc. Interspeech 2013, 2524-2528, doi: 10.21437/Interspeech.2013-569
@inproceedings{yao13b_interspeech,
author={Kaisheng Yao and Geoffrey Zweig and Mei-Yuh Hwang and Yangyang Shi and Dong Yu},
title={{Recurrent neural networks for language understanding}},
year=2013,
booktitle={Proc. Interspeech 2013},
pages={2524--2528},
doi={10.21437/Interspeech.2013-569}
}