Abstract
We consider a system of two identical rectangular shaped water tanks. A source of constant water inflow is available, which may only be directed to one tank at a time. The objective is to find a control policy to maximize the final sum of the water levels at some terminal time T, subject to minimum water level constraints on each tank. Water exits each tank corresponding to Toricelli’s law (i.e., the velocity depends on the current water level). We derive a closed form dynamic programming solution in discrete time to this problem without the water-level threshold constraints. Subsequently, we implement the value iteration algorithm on a set of support points to find a control policy with the threshold constraints, where a random forest regressor is iteratively used to update the value function. Our results show consistency between the dynamic programming solution and the value iteration solution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Aliyu, M.D.S.: Solving the Hamilton-jacobi equation, pp. 333–357. Nonlinear H infinity -Control, Hamiltonian Systems and Hamilton-Jacobi Equations (2017)
Hamilton-Jacobi equations. Semiconcave Functions, Hamilton-Jacobi Equations, and Optimal Control, pp. 97–139 (2004)
Bertsekas, D.P.: Dynamic programming and Optimal Control. Athena Scientific, Belmont, Mass (2018)
Powell, W.B.: What you should know about approximate dynamic programming. Naval Res. Logist. (NRL). 56, 239–249 (2009)
Sutton, R.S., Bach, F., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press Ltd, Massachusetts (2018)
Agostini, A., Celaya, E.: Reinforcement learning with a gaussian mixture model. In: The 2010 International Joint Conference on Neural Networks (IJCNN) (2010)
Ho, F., Kamel, M.: Reinforcement learning using a recurrent neural network. In: Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN’94) (1994)
Lecture notes on hybrid systems - university of California, Berkeley, https://people.eecs.berkeley.edu/sastry/ee291e/lygeros.pdf
Atesmen, M.K.: Understanding the world around through simple mathematics: From credit card interest, basal metabolic rate to earthquake magnitude. Infinity Publishing, West Conshohocken, PA (2011)
Vojtesek, J., Dostal, P.: Adaptive control of water level in real model of water tank. 2015 20th International Conference on Process Control (PC). (2015)
Xiao, G., Liu, F.: PID- explicit predictive dual mode control for double-water tank. In: 2018 Chinese Control And Decision Conference (CCDC) (2018)
Godwin Premi, M.S., Malakar, J.: Automatic water tank level and Pump Control System. In: 2019 International Conference on Intelligent Computing and Control Systems (ICCS) (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 IFIP International Federation for Information Processing
About this paper
Cite this paper
Lalvani, S., Katsaggelos, A. (2023). Control of a Water Tank System with Value Function Approximation. In: Maglogiannis, I., Iliadis, L., MacIntyre, J., Dominguez, M. (eds) Artificial Intelligence Applications and Innovations. AIAI 2023. IFIP Advances in Information and Communication Technology, vol 675. Springer, Cham. https://doi.org/10.1007/978-3-031-34111-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-34111-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-34110-6
Online ISBN: 978-3-031-34111-3
eBook Packages: Computer ScienceComputer Science (R0)
