Abstract
Reinforcement learning plays an important role in Multi-Agent Systems. The reasoning and learning ability of agents is the key for autonomous agents. Autonomous agents are required to be able to adapt and learn in uncertain environments via communication and collaboration (in both competitive and cooperative situations). For real-time, non-deterministic and dynamic systems, it is often extremely complex and difficult to formally verify their properties a priori. In this paper, we adopt the reinforcement learning algorithms to verify goal-oriented agents’ competitive and cooperative learning abilities for decision making. In doing so, a simulation testbed is applied to test the learning algorithms in the specified scenarios. In addition, the function approximation technique known as tile coding (TC), is used to generate value functions, which can avoid the value function growing exponentially with the number of the state values.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
InfoGrames Epic Games and Digital Entertainment. Technical report, Unreal tournament manual (2000)
Teambots (2000), http://www.cs.cmu.edu/trb/Teambots/Domains/SoccerBots
Humaniod Kid and Medium Size League, Rules and Setup for Osaka 2005. Technical report, Robocup (2005)
Dayan, P., Sejnowski, T.J.: TD(λ) Converges with Probability 1. Machine Learning 14(1), 295–301 (1994)
Jennings, N.R., Wooldridge, M.: Applications of Intelligent Agents. Agent Technology: Foundations, Applications, and Markets, 3–28 (1998)
Jennings, N.R., Sycara, K., Wooldridge, M.: A Roadmap of Agent Research and Development. Autonomous Agents and Multi-Agent Systems 1(1), 7–38 (1998)
Kuhlmann, G., Stone, P., Mooney, R., Shavlik, J.: Guiding a Reinforcement Learner with Natural Language Advice: Initial Results in RoboCup Soccer. In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS (LNAI), vol. 4020, pp. 30–35. Springer, Heidelberg (2006)
Leng, J., Fyfe, C., Jain, L.: Teamwork and Simulation in Hybrid Cognitive Architecture. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds.) KES 2006. LNCS (LNAI), vol. 4252, pp. 472–478. Springer, Heidelberg (2006)
Riedmiller, M.A., Merke, A., Meier, D., Hoffman, A., Sinner, A., Thate, O., Ehrmann, R.: Karlsruhe Brainstormers - A Reinforcement Learning Approach to Robotic Soccer. In: Martin, A. (ed.) RoboCup 2000: Robot Soccer World Cup IV, London, UK, pp. 367–372. Springer, Heidelberg (2001)
Sherstov, A.A., Stone, P.: Function Approximation via Tile Coding: Automating Parameter Choice. In: Zucker, J.-D., Saitta, L. (eds.) SARA 2005. LNCS (LNAI), vol. 3607, pp. 194–205. Springer, Heidelberg (2005)
Singh, S.P., Sutton, R.S.: Reinforcement Learning with Replacing Eligibility Traces. Machine Learning 22(1–3), 123–158 (1996)
Stankevich, L., Serebryakov, S., Ivanov, A.: Data Mining Techniques for RoboCup Soccer Agents. In: Gorodetsky, V., Liu, J., Skormin, V.A. (eds.) AIS-ADM 2005. LNCS (LNAI), vol. 3505, pp. 289–301. Springer, Heidelberg (2005)
Stone, P., Veloso, M.: TPOT-RL: Team-partitioned, opaque-transition reinforcement learning. In: RoboCup 98: Robot Soccer World Cup II, p. 221. Springer, Berlin (1998)
Stone, P.: Layered Learning in Multiagent Systems: A Winning Approach to Robotic Soccer. MIT Press, Cambridge (2000)
Stone, P., Kuhlmann, G., Taylor, M.E., Liu, Y.: Keepaway Soccer: From Machine Learning Testbed to Benchmark. In: RoboCup-2005: Robot Soccer World Cup IX, vol. 4020, pp. 93–105. Springer, Heidelberg (2006)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Tsitsiklis, J.N.: Asynchronous Stochastic Approximation and Q-learning. Machine Learning 16(3), 185–202 (1994)
Wooldridge, M., Jennings, N.: Intelligent Agents: Theory and Practice. Knowledge Engineering Review 10(2), 115–152 (1995)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Leng, J., Fyfe, C., Jain, L. (2007). Reinforcement Learning of Competitive Skills with Soccer Agents. In: Apolloni, B., Howlett, R.J., Jain, L. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2007. Lecture Notes in Computer Science(), vol 4692. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74819-9_71
Download citation
DOI: https://doi.org/10.1007/978-3-540-74819-9_71
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74817-5
Online ISBN: 978-3-540-74819-9
eBook Packages: Computer ScienceComputer Science (R0)