Skip to main content
SpringerLink
Log in

ANFIS Based Reinforcement Learning Strategy for Control A Nonlinear Coupled Tanks System

  • Original Article
  • Published:
Journal of Electrical Engineering & Technology Aims and scope Submit manuscript

Abstract

In this paper, a novel algorithm based machine learning technique for control nonlinear coupled tanks system is presented. An intelligent controller using adaptive neuro-fuzzy inference system (ANFIS) based reinforcement learning is proposed (ANFIS-RL) by representing the nonlinear coupled tanks system as a Markov decision process. A model-free learning algorithm has been used to train a policy that controls the liquid level of the tanks system without the need to determine the dynamic model of the controlled system. Based on the optimal learned policy, which is approximated by ANFIS, the controlled system can perform the best action quickly based on the states of the system. Simulation results demonstrated the feasibility of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Abbreviations

ANFIS:

Adaptive Neuro-Fuzzy Inference System

RL:

Reinforcement Learning

MDP:

Markov decision process

MPC:

Model Predictive Control

IAE:

Integral Absolute Error

ITAE:

Integral Time Absolute Error

\(L_{i}\) :

Liquid level of tank i

\(S_{i}\) :

Cross-section area of tank i

\(s_{i}\) :

Outflow orifice at the bottom of tank i

\(g\) :

The gravitational acceleration

\(f\left( t \right)\) :

Inflow rate

\(k_{p}\) :

Pump constant

\(u\left( t \right)\) :

The voltage applied to the pump.

\(O_{1,i}\) :

Membership function of fuzzification layer

\(O_{2,i}\) :

Function of product layer

\(O_{3,i}\) :

Normalization layer function

\(O_{4,i}\) :

Defuzzification layer function

\(p_{i}\) :

Consequent parameter

\(q_{i}\) :

Consequent parameter

\(r_{i}\) :

Consequent parameter

\(O_{5,i}\) :

Output layer function

S :

The environment states

r :

The reward value

a :

The policy

\(R_{t}\) :

Accumulated reward

\(\gamma\) :

Discount factor

S :

States set

U :

Actions set

R :

Reward function

F :

State transition function

\(V^{\pi } \left( s \right)\) :

The value function

\(\pi^{*}\) :

Best policy

\(V^{*} \left( s \right)\) :

Optimal value function

\(\beta\) :

Positive real number

References

  1. Başçi A, Derdiyok A (2016) Implementation of an adaptive fuzzy compensator for coupled tank liquid level control system. Measurement 91:12–18

    Article  Google Scholar 

  2. Gouta H, Said SH, Barhoumi N, M’Sahli F (2015) Observer-based backstepping controller for a state-coupled two-tank system. IETE J Res 61(3):259–268

    Article  Google Scholar 

  3. Biswas PP, Srivastava R, Ray S, Samanta AN (2009) Sliding mode control of quadruple tank process. Mechatronics 19(4):548–561

    Article  Google Scholar 

  4. Hamoudi AK (2017) Design an Integral Sliding Mode Controller for a Nonlinear System. Al-Khwarizmi Eng J 13(1):138–147

    Article  Google Scholar 

  5. Kara T, Mary AH (2018) Robust trajectory tracking control of robotic manipulators based on model-free PID-SMC approach. J Eng Res 6(3):170–188

    Google Scholar 

  6. Mary AH, Kara T (2016) Robust proportional control for trajectory tracking of a nonlinear robotic manipulator: LMI optimization approach. Arab J Sci Eng 41(12):5027–5036

    Article  Google Scholar 

  7. Xu J, Li C, He X, Huang T (2016) Recurrent neural network for solving model predictive control problem in application of four-tank benchmark. Neurocomputing 19(190):172–178

    Article  Google Scholar 

  8. Mrugalski M, Luzar M, Pazera M, Witczak M, Aubrun C (2016) Neural network-based robust actuator fault diagnosis for a non-linear multi-tank system. ISA Trans 61:318–328

    Article  Google Scholar 

  9. Patel HR, Shah VA (2019) Passive fault tolerant control system using feed-forward neural network for two-tank interacting conical level control system against partial actuator failures and disturbances. IFAC-PapersOnLine 52(14):141–146

    Article  Google Scholar 

  10. Jegatheesh A, Kumar CA (2020) Novel fuzzy fractional order PID controller for non linear interacting coupled spherical tank system for level process. Microprocess Microsyst 72:102948

    Article  Google Scholar 

  11. Boubakir A, Boudjema F, Labiod S (2009) A neuro-fuzzy-sliding mode controller using nonlinear sliding surface applied to the coupled tanks system. Int J Autom Comput 6(1):72–80

    Article  Google Scholar 

  12. Li F, Jiang Q, Zhang S, Wei M, Song R (2019) Robot skill acquisition in assembly process using deep reinforcement learning. Neurocomputing 345:92–102

    Article  Google Scholar 

  13. García J, Shafie D (2020) Teaching a humanoid robot to walk faster through Safe Reinforcement Learning. Eng Appl Artif Intell 88:103360

    Article  Google Scholar 

  14. Shipman WJ, Coetzee LC (2019) Reinforcement learning and deep neural networks for PI controller tuning. IFAC-PapersOnLine 52(14):111–116

    Article  Google Scholar 

  15. Haznedar B, Kalinli A (2018) Training ANFIS structure using simulated annealing algorithm for dynamic systems identification. Neurocomputing 302:66–74

    Article  Google Scholar 

  16. Wu YH, Yu ZC, Li CY, He MJ, Hua B, Chen ZM (2020) Reinforcement learning in dual-arm trajectory planning for a free-floating space robot. Aerosp Sci Technol 98:105657

    Article  Google Scholar 

  17. Pane YP, Nageshrao SP, Kober J, Babuška R (2019) Reinforcement learning based compensation methods for robot manipulators. Eng Appl Artif Intell 78:236–247

    Article  Google Scholar 

  18. Johannes D, Norbert L, Peter G (2020) Model-free adaptive optimal control of episodic fixed-horizon manufacturing processes using reinforcement learning. Int J Control Autom Syst 18(1):1–12

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Mechatronics Engineering Department, Al-Khwarizmi College of Engineering, University of Baghdad, Baghdad, Iraq

    Ali Hussien Mary

  2. Electrical Engineering Department, Mustansiriyah University, Baghdad, Iraq

    Abbas H. Miry

  3. Communication Engineering Department, University of Technology, Baghdad, Iraq

    Mohammed H. Miry

Authors
  1. Ali Hussien Mary
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abbas H. Miry
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammed H. Miry
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Hussien Mary.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mary, A.H., Miry, A.H. & Miry, M.H. ANFIS Based Reinforcement Learning Strategy for Control A Nonlinear Coupled Tanks System. J. Electr. Eng. Technol. 17, 1921–1929 (2022). https://doi.org/10.1007/s42835-021-00753-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42835-021-00753-1

Keywords

Search

Navigation