Abstract
Parallel rollout is a formal method of combining multiple heuristic policies available to a sequential decision maker in the framework of Markov Decision Processes (MDPs). The method improves the performances of all of the heuristic policies adapting to the different stochastic system trajectories. From an inherent multi-level parallelism in the method, in this paper we propose a parallelized version of parallel rollout algorithm, and evaluate its performance on a multi-class task scheduling problem by using OpenMP and MPI programming model. We analyze and compare the performance in two versions of parallelized codes, e.g., OpenMP and MPI on several execution environment. We show that the performance using OpenMP API is higher than MPI due to lower overhead in data synchronization across processors.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, 1994.
D. P. Bertsekas and J. N. Tsitsiklis. Neuro dynamic programming. Athena Scientific, 1996.
R. Sutton and A. Barto. Reinforcement Learning. MIT Press, 2000.
A. Printista, M. Errecalde, and C. Montoya. A parallel implementation of Qlearning based on communication with cache. Journal of Computer Science and Technology, 1(6), 2002.
H. S. Chang, R. Givan, and E. K. P. Chong. Parallel rollout for on-line solution of partially observable markov decision processes. Discrete Event Dynamic Systems (Revised), 2002.
M. Littman, T. Dean, and L. Kaelbling. On the complexity of solving markov decision problems. In Proc. 11th Annual Conf. on Uncertainty in Artificial Intelligence, pages 394–402, 1995.
A. M. Law and W. D. Kelton. Simulation Modeling and Analysis, 3rd Ed. McGraw-Hill, New York, 2000.
D. P. Bertsekas. Differential training of rollout policies. In Proc. 35th Allerton Conf. on Comm., Control, and Computing, 1997.
R. Givan, E. K. P. Chong, and H. S. Chang. Scheduling multiclass packet streams to minimize weighted loss. Queueing Systems, 41:241–270, 2002.
W. Fischer and K. Meier-Hellstern. The markov-modulated poisson process (mmpp) cookbook. Performance Evaluation, 18:149–171, 1992.
Wolfgang E. Nagel, Alfred Arnold, Michael Weber, and Hans-Christian Hoppe. VAMPIR: Visualization and analysis of MPI resources. Supercomputer, (1):69–80, January 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, S.W., Chang, H.S. (2003). Parallelizing Parallel Rollout Algorithm for Solving Markov Decision Processes. In: Voss, M.J. (eds) OpenMP Shared Memory Parallel Programming. WOMPAT 2003. Lecture Notes in Computer Science, vol 2716. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45009-2_10
Download citation
DOI: https://doi.org/10.1007/3-540-45009-2_10
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40435-4
Online ISBN: 978-3-540-45009-2
eBook Packages: Springer Book Archive