Abstract
In previous research a model for thematic role assignment (θRARes) was proposed, using the Reservoir Computing paradigm. This language comprehension model consisted of a recurrent neural network (RNN) with fixed random connections which models distributed processing in the prefrontal cortex, and an output layer which models the striatum. In contrast to this previous batch learning method, in this paper we explored a more biological learning mechanism. A new version of the model (i-θRARes) was developed that permitted incremental learning, at each time step. Learning was based on a stochastic gradient descent method. We report here results showing that this incremental version was successfully able to learn a corpus of complex grammatical constructions, reinforcing the neurocognitive plausibility of the model from a language acquisition perspective.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bates, E., McNew, S., MacWhinney, B., Devescovi, A., Smith, S.: Functional constraints on sentence processing: A cross-linguistic study. Cognition 11, 245–299 (1982)
Dominey, P.F., Voegtlin, T.: Learning word meaning and grammatical constructions from narrated video events. In: Proc HLT-NAACL (2003)
Dominey, P.F., Hoen, M., Inui, T.: A neurolinguistic model of grammatical construction processing. J. Cogn. Neurosci. 18, 2088–2107 (2006)
Farkas, I., Crocker, M.W.: Syntactic systematicity in sentence processing with a recurrent self-organizing network. Neurocomputing 71, 1172–1179 (2008)
Goldberg, A.E.: Constructions: a new theoretical approach to language. Trends. Cogn. Sci. 7, 219–224 (2003)
Hinaut, X., Dominey, P.F.: A three-layered model of primate prefrontal cortex encodes identity and abstract categorical structure of behavioral sequences. J. Physiol. - Paris 105, 16–24 (2011)
Hinaut, X., Dominey, P.F.: On-Line Processing of Grammatical Structure Using Reservoir Computing. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds.) ICANN 2012, Part I. LNCS, vol. 7552, pp. 596–603. Springer, Heidelberg (2012)
Hinaut, X., Dominey, P.F.: Real-Time Parallel Processing of Grammatical Structure in the Fronto-Striatal System: A Recurrent Network Simulation Study Using Reservoir Computing. PloS ONE 8(2), e52946 (2013)
Hinaut, X., Petit, M., Pointeau, G., Dominey, P.F.: Exploring the Acquisition and Production of Grammatical Constructions Through Human-Robot Interaction with Echo State Networks. Front. Neurorobot. 8, 16 (2014)
Hoerzer, G.M., Legenstein, R., Maass, W.: Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cereb Cortex, Advance online publication (November 11, 2012) (retrieved)
Jaeger, H., Haas, H.: Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004)
Lukoševičius, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3, 127–149 (2009)
Miikkulainen, R.: Subsymbolic case-role analysis of sentences with embedded clauses. Cognitive Sci. 20, 47–73 (1996)
Tong, M.H., Bickett, A.D., Christiansen, E.M., Cottrell, G.W.: Learning grammatical structure with Echo State Networks. Neural Networks 20, 424–432 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Hinaut, X., Wermter, S. (2014). An Incremental Approach to Language Acquisition: Thematic Role Assignment with Echo State Networks. In: Wermter, S., et al. Artificial Neural Networks and Machine Learning – ICANN 2014. ICANN 2014. Lecture Notes in Computer Science, vol 8681. Springer, Cham. https://doi.org/10.1007/978-3-319-11179-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-11179-7_5
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11178-0
Online ISBN: 978-3-319-11179-7
eBook Packages: Computer ScienceComputer Science (R0)