Abstract
Keepaway soccer has been previously put forth as a testbed for machine learning. Although multiple researchers have used it successfully for machine learning experiments, doing so has required a good deal of domain expertise. This paper introduces a set of programs, tools, and resources designed to make the domain easily usable for experimentation without any prior knowledge of RoboCup or the Soccer Server. In addition, we report on new experiments in the Keepaway domain, along with performance results designed to be directly comparable with future experimental results. Combined, the new infrastructure and our concrete demonstration of its use in comparative experiments elevate the domain to a machine learning benchmark, suitable for use by researchers across the field.
Chapter PDF
Similar content being viewed by others
Keywords
- Radial Basis Function
- Multiagent System
- Radial Basis Function Network
- Function Approximator
- Episode Duration
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Albus, J.S.: Brains, Behavior, and Robotics. Byte Books, Peterborough (1981)
Balch, T.: Teambots (2000), http://www.teambots.org
Balch, T.: Teambots domain: Soccerbots (2000), http://www-2.cs.cmu.edu/~trb/TeamBots/Domains/SoccerBots
Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998)
Bradtke, S.J., Duff, M.O.: Reinforcement learning methods for continuous-time Markov decision problems. In: Leen, T., Tesauro, G., Touretzky, D. (eds.) Advances in Neural Information Processing Systems, San Mateo, CA, vol. 7, pp. 393–400. Morgan Kaufmann, San Francisco (1995)
Chen, M., Foroughi, E., Heintz, F., Kapetanakis, S., Kostiadis, K., Kummeneje, J., Noda, I., Obst, O., Riley, P., Steffens, T., Wang, Y., Yin, X.: Users manual: RoboCup soccer server manual for soccer server version 7.07 and later (2003), available at: http://sourceforge.net/projects/sserver/
Crites, R.H., Barto, A.G.: Improving elevator performance using reinforcement learning. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems, vol. 8. MIT Press, Cambridge (1996)
de Boer, R., Kok, J.R.: The incremental development of a synthetic multi-agent system: The uva trilearn 2001 robotic soccer simulation team. Master’s thesis, University of Amsterdam, The Netherlands (February 2002)
Hsu, W.H., Gustafson, S.M.: Genetic programming and multi-agent layered learning by reinforcements. In: Genetic and Evolutionary Computation Conference, New York (July 2002)
Kuhlmann, G., Stone, P.: Progress in learning 3 vs. 2 keepaway. In: Polani, D., Browning, B., Bonarini, A., Yoshida, K. (eds.) RoboCup 2003. LNCS, vol. 3020, pp. 694–702. Springer, Heidelberg (2004)
Noda, I., Matsubara, H., Hiraki, K., Frank, I.: Soccer server: A tool for research on multiagent systems. Applied Artificial Intelligence 12, 233–250 (1998)
Di Pietro, A., While, L., Barone, L.: Learning in RoboCup keepaway using evolutionary algorithms. In: Langdon, W.B., Cantú-Paz, E., Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M.A., Schultz, A.C., Miller, J.F., Burke, E., Jonoska, N. (eds.) GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1065–1072. Morgan Kaufmann, San Francisco (2002)
Puterman, M.L.: Markov Decision Processes. Wiley, Chichester (1994)
Rummery, G.A., Niranjan, M.: On-line Q-learning using connectionist systems. Technical Report CUED/F-INFENG/TR 166, Cambridge University Engineering Department (1994)
Stone, P., Sutton, R.S.: Keepaway soccer: A machine learning testbed. In: Birk, A., Coradeschi, S., Tadokoro, S. (eds.) RoboCup 2001. LNCS, vol. 2377, pp. 214–223. Springer, Heidelberg (2002)
Stone, P., Sutton, R.S., Kuhlmann, G.: Reinforcement learning for RoboCup-soccer keepaway. In: Adaptive Behavior (to appear, 2005)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Taylor, M.E., Stone, P.: Behavior transfer for value-function-based reinforcement learning. In: The Fourth International Joint Conference on Autonomous Agents and Multiagent Systems (July 2005) (to appear)
Tesauro, G.: TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation 6(2), 215–219 (1994)
Walker, T., Shavlik, J., Maclin, R.: Relational reinforcement learning via sampling the space of first-order conjunctive features. In: Proceedings of the ICML Workshop on Relational Reinforcement Learning, Banff, Canada (July 2004)
Whiteson, S., Kohl, N., Miikkulainen, R., Stone, P.: Evolving keepaway soccer players through task decomposition. Machine Learning 59(1), 5–30 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stone, P., Kuhlmann, G., Taylor, M.E., Liu, Y. (2006). Keepaway Soccer: From Machine Learning Testbed to Benchmark. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds) RoboCup 2005: Robot Soccer World Cup IX. RoboCup 2005. Lecture Notes in Computer Science(), vol 4020. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11780519_9
Download citation
DOI: https://doi.org/10.1007/11780519_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35437-6
Online ISBN: 978-3-540-35438-3
eBook Packages: Computer ScienceComputer Science (R0)