Abstract
An important property of Reservoir Computing, and signal processing techniques in general, is generalization and noise robustness. In trajectory generation tasks, we don’t want that a small deviation leads to an instability. For forecasting and system identification we want to avoid over-fitting. In prior work on Reservoir Computing, the addition of noise to the dynamic reservoir trajectory is generally used. In this work, we show that high-performing reservoirs can be trained using only the commonly used ridge regression. We experimentally validate these claims on two very different tasks: long-term, robust trajectory generation and system identification of a heating tank with variable dead-time.
This research is partially funded by FWO Flanders project G.0317.05 and the Photonics@be Interuniversity Attraction Poles program (IAP 6/10), initiated by the Belgian State, Prime Minister’s Services, Sience Policy Office.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14(11), 2531–2560 (2002)
Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks. Technical Report GMD Report 148, German National Research Center for Information Technology (2001)
Verstraeten, D., Schrauwen, B., D’Haene, M., Stroobandt, D.: An experimental unification of reservoir computing methods. Neural Networks 20, 391–403 (2007)
Jaeger, H.: Tutorial on training recurrent neural networks, covering BPTT, RTRL, EKF and the “echo state network” approach. Technical Report GMD Report 159, German National Research Center for Information Technology (2002)
Hoerl, A., Kennard, R.: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12, 55–67 (1970)
Tikhonov, A., Arsenin, V.: Solution of Ill-posed Problems. Winston & Sons (1977)
Jaeger, H., Lukosevicius, M., Popovici, D., Siewert, U.: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Networks 20, 335–352 (2007)
Feldkamp, L., Prokhorov, D., Eagen, C., Yuan, F.: Enhanced multi-stream Kalman filter training for recurrent neural networks. In: Nonlinear modeling: Advanced black-box techniques edn. Kluwer, Dordrecht (1998)
Moody, J., Hanson, S., Lippmann, R.: The effective number of parameters: An analysis of generalization and regularization in nonlinear systems. Advances in Neural Information Processing Systems 4, 847–854 (1992)
Zegers, P., Sundareshan, M.K.: Trajectory generation and modulation using dynamic neural networks. IEEE Transactions on Neural Networks 14, 520–533 (2003)
Antonelo, E., Schrauwen, B., Dutoit, X., Stroobandt, D., Nuttin, M.: Event detection and localization in mobile robot navigation using reservoir computing. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D.P. (eds.) ICANN 2007. LNCS, vol. 4669, pp. 660–669. Springer, Heidelberg (2007)
Antonelo, E.A., Schrauwen, B., Stroobandt, D.: Experiments with reservoir computing on the road sign problem. In: Brazilian Congress on Neural Networks (CBRN) (2007)
Wyffels, F., Schrauwen, B., Verstraeten, D., Stroobandt, D.: Band-pass reservoir computing. In: Proceedings of the International Joint Conference on Neural Networks (accepted, 2008)
Cristea, S., de Prada, C., De Keyser, R.: Predictive control of a process with variable dead-time. In: CD-Proceedings of the 16th IFAC World Congress (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wyffels, F., Schrauwen, B., Stroobandt, D. (2008). Stable Output Feedback in Reservoir Computing Using Ridge Regression. In: KĹŻrková, V., Neruda, R., KoutnĂk, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9_83
Download citation
DOI: https://doi.org/10.1007/978-3-540-87536-9_83
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87535-2
Online ISBN: 978-3-540-87536-9
eBook Packages: Computer ScienceComputer Science (R0)