Picking Robot Visual Servo Control Based on Modified Fuzzy Neural Network Sliding Mode Algorithms

Abstract

Through an analysis of the kinematics and dynamics relations between the target positioning of manipulator joint angles of an apple-picking robot, the sliding-mode control (SMC) method is introduced into robot servo control according to the characteristics of servo control. However, the biggest problem of the sliding-mode variable structure control is chattering, and the speed, inertia, acceleration, switching surface, and other factors are also considered when approaching the sliding die surface. Meanwhile, neural network has the characteristics of approaching non-linear function and not depending on the mechanism model of the system. Therefore, the fuzzy neural network control algorithm can effectively solve the chattering problem caused by the variable structure of the sliding mode and improve the dynamic and static performances of the control system. The comparison experiment is carried out through the application of the PID algorithm, the sliding mode control algorithm, and the improved fuzzy neural network sliding mode control algorithm on the picking robot system in the laboratory environment. The result verified that the intelligent algorithm can reduce the complexity of parameter adjustments and improve the control accuracy to a certain extent.

1. Introduction

With the development of science and technology, robot technology has acquired unprecedented achievements. In agriculture, the development of fruit-picking robots has become a hotpot project due to the rising labor costs and the huge output of apples and citrus. The apple-picking robot is an integrated system that integrates environmental perception, dynamic decision-making, and behavior control; motion control is therein the most basic and important link. At present, there are two main kinds of visual servo control for picking robots: one is image-based visual servo control, and the other is a position-based visual servo control system [1]. The picking robot manipulator is a typical non-linear system, because it is composed of multiple joints and has time-varying uncertainty during the picking process. This indicates that traditional control methods would not meet the control requirements. Image feedback-based visual servo control is a real-time feedback control for the manipulator according to the visual acquisition image, and it also shows strong self-adaptability to the change of external environment and system parameters so that it reduces the complexity of manipulator in the control process.

At present, most visual servo control systems based on image feedback adopt traditional control schemes such as adaptive control [2], PID control [3], and sliding model control [4], etc., or intelligent control algorithms [5] such as neural network and fuzzy theory [6], etc. There are some defects in adopting a single control method. Adaptive control is a strategy to deal with structural uncertainties, which provides good performance to a certain extent [7]. PID control has better control performance for the position and joint angle parameters of the manipulator end-effector [8]. Sliding-mode control (SMC) has excellent robustness and good anti-jamming ability to the uncertainties of the system, especially in the control of non-linear systems [9]. Neural network (NN) has the ability to learn and approximate non-linear functions [10]. It is often used to compensate unstructured uncertainties in robot controller modeling. Fuzzy control [11] imitates human reasoning and decision-making processes on the basis of not depending on the precise mathematical model of the controlled object. Fuzzy logic is adopted in robot controllers to achieve good control performance under uncertain conditions. In the reference [12], a robust adaptive control method based on the dynamic structure of the fuzzy wavelet neural network is proposed, which is used to track the trajectory of the robot with uncertainties and disturbances by the adaptive sliding-mode control. Mehta and Ton et al. introduced a continuous terminal sliding mode visual servo controller to adjust the camera to the desired position, but this has many requirements for the camera, and requires the correction of the camera position [13]. In the field of robotic citrus picking, there is a non-linear robust image visual servo controller that can adjust the end effector to the fruit position under the condition of unknown fruit movement, but the speed of the picking process is slow, and is not conducive to agricultural application [14]. The fuzzy neural network (FNN) learning algorithm [15,16] is a superior system identification method that utilizes non-linear control. There is no need to get the prior knowledge about the uncertainty and a sufficient amount of observed data. Efe and Devesh et al. used the novel particle swarm optimization (PSO)–SMC hybrid learning algorithm to train the FNN effectively, but instead of using the visual servo control of picking robots, they realized the flattening and rotation control of the four-rotor configuration for agricultural purposes [17].

A great deal of research has been devoted to these intelligent algorithm scholars, but there are few applications in the field of agricultural picking robots. In this paper, a joint visual servo control algorithm based on an improved sliding mode and kinematics and dynamics equation is presented that appears to have good robustness in a non-linear system. However, sliding-mode control always requires detailed system information and corresponding uncertainty bounds to ensure stability [18]. Even though the stability requirements are met, there will still be problems such as jitter [19]. Therefore, fuzzy and adaptive neural network control algorithms are introduced and combined with sliding-mode control to suppress the sliding-mode jitter problem. However, the traditional neural network model is complex and computationally intensive, with a high probability of over-fitting. To prevent over-fitting, we increased the dropout rate for the dropout layer [20].

According to the characteristics of apple-picking robot motion control, the image-based robot visual servo control method is combined with the improved fuzzy neural network sliding mode control algorithm to make use of the non-linear and non-system-dependent mechanism model of the fuzzy neural network. Combined with sliding mode variable structure control, the sliding mode chattering is eliminated, and the stability of the control system is improved. In this paper, the algorithm is applied for the first time in the field of agricultural fruit-picking robots, which improves the extraction speed of apple-picking robots.

The research aim of this paper is mainly to verify the visual servo control of the sliding-mode control algorithm based on the improved fuzzy neural network through the apple-picking robot. The article is divided into three parts, Firstly, we establish the kinematics and dynamics equations of the picking robot. Secondly, visual positioning is introduced to calculate the position of the target point, and an image-based visual control algorithm is used. Finally, the sliding-mode control algorithm and the improved fuzzy neural network control algorithm are combined to carry out simulation analysis, and the algorithm is verified on the picking robot. It is concluded that the improved fuzzy neural network sliding mode control algorithm can improve the efficiency of the robot arm servo control and has higher stability.

2. Picking Robot Structure and Model Establishment

2.1. Manipulator Structure Design

Since the apple trees are relatively high and the fruits are disperse, the higher degree of freedom of the picking robot manipulator brings not only the kinematics algorithm complexity, but also the waste of joint freedom degree, which is disadvantageous to the practicality of agriculture. This reduces the requirements of the manipulator and minimizes the usage of freedom degrees. So, this paper introduces the self-designed five degrees of freedom (DOF) manipulator, which is shown in Figure 1. Figure 2 is a schematic view of the picking robot manipulator. The manipulator gripping device adopts a curved design in order to reduce the damage of the apple during the process of picking up the apple. Figure 3 shows a wheeled picking robot.

Fruit-picking robots are mainly composed of two parts. The first part is a chassis platform, and the second part is a manipulator. Therein, the five DOF manipulator is a PRRRP joint structure. The first degree of freedom is the lifting platform. Its main function is to lift the whole manipulator. The second degree of freedom is the waist rotation joint of the manipulator. The third degree of freedom is the swing axis of the arm, the fourth degree of freedom is the swing axis of the arm, and the fifth degree of freedom is the rotating axis of the manipulator with the function of adjusting the grasping posture according to the picking requirement.

2.2. Kinematics Equation of Manipulator

The manipulator is a complex system, since it includes various disciplines such as kinematics and dynamics. Kinematics analysis is used to model according to the relationship between the spatial posture and the angle of each joint of the manipulator. The designed structural kinematics parameters of the manipulator are shown in

Table 1. Kinematic parameters of manipulators.

Joints i	${\mathit{\alpha }}_{\mathit{i}}$/(°)	${\mathit{a}}_{\mathit{i}}$/(°)	${\mathit{\theta }}_{\mathit{i}}$/(°)	${\mathit{d}}_{\mathit{i}}$/(°)	σ	Variable Range
1	0	0	90	$d_1$	1	$d_1$(0.63 to 1.3)
2	−90	0.133	${\theta }_2$	0	0	${\theta }_2$ (−180° to 180°)
3	0	1	${\theta }_3$	0	0	${\theta }_3$ (–142° to −36°)
4	−90	0	${\theta }_4$	0	0	${\theta }_4$ (−180° to 180°)
5	0	0	${\theta }_5$	0	0	${\theta }_5$ (0° to 360°)

. In the table, ${\alpha }_i$ is the torsion angle, $a_i$ is the length of the connecting rod, ${\theta }_i$ is the joint angle, $d_i$ is the offset of the connecting rod, σ = 0 is the rotating joint, σ = 1 is the direct acting joint, and $d_i$ is the other constant. The use of the D–H rule to determine the coordinate position of the connecting rod is shown in Figure 4.

The homogeneous coordinate transformation matrix is used in the above-mentioned establishment of connecting the rod D–H coordinate system and kinematics mode.

$$\begin{array}{c}{}_1{}^{0}T=\left[\begin{array}{cccc}0& -1& 0& 0\\ 1& 0& 0& 0\\ 0& 0& 1& d_1\\ 0& 0& 0& 1\end{array}\right]\\ {}_2{}^{1}T=\left[\begin{array}{cccc}cos{\theta }_2& 0& -sin{\theta }_2& 0.133 cos{\theta }_2\\ sin{\theta }_2& 0& cos{\theta }_2& 0.133 sin{\theta }_2\\ 0& -1& 0& 0\\ 0& 0& 0& 1\end{array}\right]\\ {}_3{}^{2}T=\left[\begin{array}{cccc}cos{\theta }_3& -sin{\theta }_3& 0& cos{\theta }_3\\ 1& cos{\theta }_3& 0& sin{\theta }_3\\ 0& 0& 1& 0\\ 0& 0& 0& 1\end{array}\right]\\ {}_4{}^{3}T=\left[\begin{array}{cccc}cos{\theta }_4& 0& -sin{\theta }_4& 0\\ sin{\theta }_4& 0& cos{\theta }_4& 0\\ 0& -1& 0& 0\\ 0& 0& 0& 1\end{array}\right]\\ {}_5{}^{4}T=\left[\begin{array}{cccc}1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& d_5\\ 0& 0& 0& 1\end{array}\right]\end{array}$$

(1)

Therein, ${}_i{}^{i-1}T$ presents the homogeneous matrix of translation and rotation from the i − 1 joint coordinate system to the i joint coordinate system. The total transformation matrix of the end effector relative to the robot coordinate system is:

$${}_5{}^{0}T={}_1{}^{0}T{}_2{}^{1}T{}_3{}^{2}T{}_4{}^{3}T{}_5{}^{4}T=\left[\begin{array}{cccc}{n}_x& o_x& a_x& p_x\\ n_y& o_y& a_y& p_y\\ n_z& o_z& a_z& p_z\\ 0& 0& 0& 1\end{array}\right]$$

(2)

The end effector position equation is:

$$\begin{array}{c}{p}_x=d_5 sin{\theta }_{2}sin\left({\theta }_3+{\theta }_4\right)-sin{\theta }_{2}cos{\theta }_3-0.133 sin{\theta }_2\\ p_y=-d_5 cos{\theta }_{2}sin\left({\theta }_3+{\theta }_4\right)+cos{\theta }_{2}cos{\theta }_3+0.133 cos{\theta }_2\\ p_z=-d_5 cos\left({\theta }_3+{\theta }_4\right)-sin{\theta }_3+d_1\end{array}$$

(3)

The end effector posture equation is:

$$\begin{array}{c}{p}_x=d_5 sin{\theta }_{2}sin\left({\theta }_3+{\theta }_4\right)-sin{\theta }_{2}cos{\theta }_3-0.133 sin{\theta }_2\\ p_y=-d_5 cos{\theta }_{2}sin\left({\theta }_3+{\theta }_4\right)+cos{\theta }_{2}cos{\theta }_3+0.133 cos{\theta }_2\\ p_z=-d_5 cos\left({\theta }_3+{\theta }_4\right)-sin{\theta }_3+d_1\end{array}$$

(4)

2.3. Dynamic Model of Manipulator

According to the analysis of manipulator joints [21], the five degrees of freedom of the manipulator are respectively the lifting base, waist rotation axis, arm joint, and wrist rotation. The lifting base only works when the length of the manipulator is insufficient, while the waist rotation and wrist rotation axis are tentatively defined as not playing a key role in the visual recognition and picking process; therefore, the manipulator is simplified as a two-joint manipulator with only a big arm and a small arm. Meanwhile, a dynamic model is established, as shown in Figure 5.

$$M\left(q\right)\ddot{q}+C\left(q,\dot{q}\right)\dot{q}+G\left(q\right)=\tau$$

(5)

Herein, $\mathit{q}={\left[{\mathit{q}}_{\mathbf{1}}{\mathit{q}}_{\mathbf{2}}\right]}^T$, $\mathit{\tau }={\left[{\mathit{\tau }}_{\mathbf{1}}{\mathit{\tau }}_{\mathbf{2}}\right]}^{\mathit{T}}$. In the formula, $\mathit{q}$, $\dot{\mathit{q}}$, $\ddot{\mathit{q}}$ are respectively the joint angle position, velocity, and acceleration vectors of the manipulator. $\mathit{M}\left(\mathit{q}\right)\in {\mathit{R}}^{\mathit{n}\times \mathit{n}}$ is a positive definite mass inertia matrix. $\mathit{C}\left(\mathit{q},\dot{\mathit{q}}\right)\in {\mathit{R}}^{\mathit{n}\times \mathit{n}}$ represents the centrifugal force and Coriolis force. $\mathit{G}\left(\mathit{q}\right)\in {\mathit{R}}^{\mathit{n}}$ is gravity vector. $\mathit{\tau }\in {\mathit{R}}^{\mathit{n}}$ is the control moment input from each part of the joint. ${\mathit{\tau }}_{\mathrm{d}}\mathit{\in }{\mathit{R}}^{\mathit{n}}$ is the unknown external disturbances. Therein, Formula (1) satisfies the following characteristics:

(1) Positive definite symmetry: For any $\mathit{q}$ matrix, $\mathit{M}\left(\mathit{q}\right)$ is a positive definite symmetric.

(2) Boundedness: Matrix functions $\mathit{M}\left(\mathit{q}\right)$ and $\mathit{C}\left(\mathit{q},\dot{\mathit{q}}\right)$ are uniformly bounded for all $\mathit{q}$,$\dot{\mathit{q}}$.

(3) Skew symmetry: The matrix function $\mathit{M}\left(\mathit{q}\right)-2\mathit{C}\left(\mathit{q},\dot{\mathit{q}}\right)$ is a skew symmetric matrix, i.e., for any vector $\mathsf{\alpha }$, there is ${\mathsf{\alpha }}^T\left(\mathit{M}\left(\mathit{q}\right)-2\mathit{C}\left(\mathit{q},\dot{\mathit{q}}\right)\right)\mathsf{\alpha }=0$.

3. Visual Servo Control

The visual servo control structure can be divided into position-based control and image-based control according to different feedback signals. The feedback signal of position-based control is defined in coordinate form in three-dimensional task space. Its basic principle is to estimate the target in a three-dimensional Cartesian coordinate system by extracting the image features and combining the geometric model of the known target, and then planning the trajectory and calculating the control quantity according to the position comparison between the end effector and the target of the manipulator. For the image-based control, the control quantity is calculated directly from the error signal in the image, and the task is completed by driving the manipulator [22]. In this paper, the image-based control method is used to output control signals after image calculation, and the sliding-mode control parameters are adaptively adjusted by the fuzzy neural network algorithm so as to improve the robustness and stability of the manipulator grasping.

3.1. Visual Target Location

According to the visual servo control system working process of fruit picking robots, the relationship between the position information of the robot and its joints should be determined first. The mapping relationship between the image plane coordinates and the three-dimensional coordinates in the real space can be established by the perspective model of the small hole so that the coordinate position of the target can be calculated [23]. Due to the vision camera adopted in this paper being the target localization of the monocular robot, its camera is installed at the end of the manipulator. According to the principle of the optical imaging of the camera, the model of monocular vision localization is established as shown in Figure 6, in which the mapping relation of two-dimensional image coordinates and coordinates in three-dimensional space is established by the transformed geometric coordinates.

Here, we assume that the target fruit coordinate in the coordinate system of the manipulator is ($X_a$, $Y_a$, $Z_a$) and its coordinate in the camera coordinate system is ($X_c$, $Y_c$, $Z_c$). The relationship between the 3D coordinates ($X_c$, $Y_c$, $Z_c$) of the target in the camera coordinate system and its two-dimensional (2D) coordinates ($X_1$, $Y_1$) on the camera imaging plane is obtained.

$$\{\begin{array}{c}{X}_1=\frac{X_{c}f}{Z_c}\\ Y_1=\frac{Y_{c}f}{Z_c}\end{array}$$

(6)

In the formula, $f$ is the camera focal length. The coordinate transformation formula between the camera coordinate system and manipulator coordinate system can be obtained by the geometric analytic method shown in Figure 2.

$$\{\begin{array}{c}X=\sqrt{\frac{L_2^2+L_3^2-2L_2{L}_3\cos \left(\pi -{\theta }_3\right)-{\left(L_2\sin {\theta }_2+L_3\sin \left({\theta }_2+{\theta }_3\right)\right)}^2}{1+{\tan }^2{\theta }_1}}\\ Y=X\tan {\theta }_1\\ Z=L_1+L_2\sin {\theta }_2+L_3\sin \left({\theta }_2+{\theta }_3\right)\end{array}$$

(7)

In the formula, $L_1$—Waist length

• $L_2$—Big arm length
• $L_3$—Small arm length
• ${\theta }_1$—one DOF joint angle of manipulator
• ${\theta }_2$—two DOF joint angle of manipulator
• ${\theta }_3$—three DOF joint angle of manipulator

From Formula (3), the relationship between the target fruit coordinate in the manipulator coordinate system ($X_a$, $Y_a$, $Z_a$), and the fruit coordinate in the camera coordinate system ($X_c$, $Y_c$, $Z_c$) can be obtained:

$$\{\begin{array}{c}{X}_c=X_a-X\\ Y_c=Y_a-Y\\ Z_c=Z_a-Z\end{array}$$

(8)

The relationship between the robot angle change of each joint and the plane coordinate change of the target fruit imaging is obtained.

3.2. Visual Servo Control

Since the fruit picking robot works in unknown and uncertain environments and the visual servo control system has the characteristics of time-varying, strong coupling, and non-linearity, therefore, the visual servo control system based on images is used due to its advantages regarding insensitivity to errors. The structure of the image-based visual servo system is shown in Figure 7.

In this system, the picking robot captures the target image with a camera installed by eye-in-hand mode, and the position information of the target fruit is obtained by an image processing system. The angle of each joint of the robot can be obtained by coordinate transformation. In order to improve the motion performance of the manipulator, a visual servo algorithm based on fuzzy neural network adaptive sliding mode control is adopted to overcome the uncertainty of the system so as to improve the robustness of the servo control of the manipulator.

4. Complex Fuzzy Neural Network Sliding Mode Control

4.1. Model Design of Sliding Mode Algorithms

Sliding-mode [24] variable structure control mainly uses a high-speed control rate under non-linear conditions. It can reach the predetermined sliding mode control track in a short time and move along the track according to the variable dynamic control system. The sliding mode control scheme is shown in Figure 8.

First, we set the ideal angle instruction to ${\mathit{q}}_d\left(t\right)$, and track the error of the sliding-mode controller:

$$e\left(t\right)=q\left(t\right)-q_d\left(t\right)$$

(9)

Then, we design the sliding surface as:

$$s\left(t\right)=\ddot{e}+{\Lambda }_1\dot{e}+{\Lambda }_{2}e$$

(10)

Therein, $\mathit{\Lambda }=\mathit{d}\mathit{i}\mathit{a}\mathit{g}\left({\mathit{\lambda }}_{\mathit{i}\mathbf{1}},{\mathit{\lambda }}_{\mathit{i}\mathbf{2}},\dots ,{\mathit{\lambda }}_{\mathit{i}\mathit{n}}\right)$, ${\mathit{\lambda }}_{\mathit{i}\mathit{j}}>\mathbf{0}$, $\mathit{i}=\mathbf{1},\mathbf{2}$, $\mathit{j}=\mathbf{1},\mathbf{2},\dots ,\mathit{n}$.

Then, we set the Lyapunov function to:

$$V=\frac{1}{2}{s}^{T}Ms$$

(11)

Therefore:

$$\dot{V}=\frac{1}{2}{\dot{s}}^{T}Ms+\frac{1}{2}{s}^T\dot{M}s+\frac{1}{2}{s}^{T}M\dot{s}$$

(12)

Bring the characteristics of the manipulator dynamics equation into the formula:

$$\dot{V}=s^{T}M\dot{s}+s^{T}C\dot{s}=s^T\left(M\left(\stackrel{⃛}{q}-{\stackrel{⃛}{q}}_d+{\Lambda }_1\ddot{e}+{\Lambda }_2\dot{e}\right)+C\left(\ddot{e}+{\Lambda }_1\dot{e}+{\Lambda }_{2}e\right)\right)$$

(13)

According to:

$$\mathsf{\tau }\dot{}+\Lambda \mathsf{\tau }=\Lambda U_{SMC}$$

(14)

Find:

$$\dot{V}=s^T\left(H+\Lambda U_{SMC}\right)$$

(15)

Therein:

$$\begin{array}{c}H=M\left({\Lambda }_1\ddot{e}+{\Lambda }_2\dot{e}-{\stackrel{⃛}{q}}_d\right)+C\left({\Lambda }_1\dot{e}+{\Lambda }_{2}e-{\stackrel{⃛}{q}}_d\right)-\left(\dot{M}+\Lambda M\right)\ddot{q}\\ -\left(\dot{C}+\Lambda C\right)\dot{q}-\left(\dot{G}+\Lambda G\right)\end{array}$$

(16)

Then, we design the SMC system as:

$$U_{SMC}=-{\Lambda }^{-1}\left(H+\eta sgn\left(s\right)\right)$$

(17)

Find:

$$\dot{V}=-\eta s^{T}sgn\left(s\right)=-\eta ||s||\le 0$$

(18)

According to the LaSalle invariant set theorem, when $\mathit{t}\to \infty$, s $\to 0$, then $\tilde{\mathit{q}}\to 0$, $\dot{\tilde{\mathit{q}}}\to 0$.

4.2. Sliding-Mode Control Based on Complex Fuzzy Neural Network

The fuzzy neural network is a kind of special neural network that is a hybrid intelligent system formed by the combination of neural network and fuzzy logic. It combines the two kinds of techniques by combining the human-like reasoning of fuzzy systems with the learning and connection structure of neural networks [25,26]. In a nutshell, the fuzzy neural network (FNN) assigns a conventional neural network to fuzzy input signals and fuzzy weights. Its function is to use the fuzzy neural network structure to implement fuzzy logic reasoning.

In order to improve the control accuracy of the two-joint manipulator, a fuzzy neural network control algorithm, the fuzzy neural network (FNN) is constructed to optimize the conventional sliding mode control because of the chattering phenomenon in the conventional sliding-mode control [27,28,29,30]. The system consists of an input layer, an adaptive fuzzy rule layer, a rule layer, and an output layer. The structure of the system is shown in Figure 9. The input is a sliding surface, and the output is a control quantity. The following is the main flow chart of the FNN algorithm:

Step 1: Input the layer data initialization of the fuzzy neural network, and transfer the sliding-mode vector variable $s_i\left(i=1,2,\dots ,n\right)$ to the next layer.

Step 2: The nodes in each layer represent the membership functions of the members, which use Gauss function as an input:

$${\mu }_i^j\left(s_i\right)=exp\left[-\frac{{\left(s_i-x_i^j\right)}^2}{{\left({\sigma }_i^j\right)}^2}\right], \left(i=1,2,\dots ,n; j=1,2,\dots ,m\right)$$

(19)

In the formula, $n$ represents the number of input variables, and $m$ is the number of member functions. $x_i^j$ and ${\sigma }_i^j$ represent the j level node of the i input from $s_i$ to the Gauss function.

Step 3: The rule layer represents the fuzzy reasoning mechanism of the neural network, and the data results are obtained by multiplying the input signals of the layer. The formula is as follows:

$$g_k={\displaystyle \prod }_{i=1}^n{\omega }_{ji}^k{\mu }_i^j\left(s_i\right)$$

(20)

Herein, ${\mathit{g}}_k$ represents the Kth output of the rule layer.

Step 4: The fourth layer is the output layer, in which each node represents an output variable. It integrates the output into the sum of all the input signals. The execution output of this layer deblurring is as follows:

$$y_0={\displaystyle \sum }_{k=1}^{N_y}{\omega }_k^0{g}_k$$

(21)

By adding FNN online cyclic training, inheriting the traditional sliding mode control system, and relaxing the requirement of system information, the SMC jitter problem can be eliminated. However, the traditional neural network training method mentioned above makes the learning speed of the algorithm slow and may fall into local minimum problem, etc. Therefore, a dropout layer is added on this basis to prevent over-fitting by randomly hiding layer nodes. The improved fuzzy neural network is expressed in vector form:

$$y=U_{NFNN}\left(e,x,\sigma ,W,l,p\right)=Wg$$

(22)

In the formula, $\mathit{W}$ is the weight matrix of the hidden layer and the output layer. $\mathit{e}$ is the tracking error vector and the input vector of the system. $\mathit{l}$ represents the fuzzy rules. $\mathit{p}$ represents the dropout layer parameters of Bernoulli distribution.

Therefore, the equivalent control law of the optimal output estimation sliding surface is obtained.

$$U_{SMC}=U_{NN}^{*}\left(e,x^{*},{\sigma }^{*},W^{*},l^{*},p^{*}\right)+\epsilon =W^{*}{g}^{*}+\epsilon$$

(23)

Herein, $\epsilon$ represents the error vectors of the network. ${\mathit{W}}^{*},{\mathit{x}}^{*},{\mathit{\sigma }}^{*},{\mathit{l}}^{*},{\mathit{p}}^{*}$ are the optimal parameters of network. Then, the sliding mode control law of the improved fuzzy neural network can be expressed as:

$$U\left(\mathrm{t}\right)={\widehat{U}}_{NN}\left(e,\widehat{x},\widehat{\sigma },\widehat{W},\widehat{l},\widehat{p}\right)+U_r$$

(24)

Herein, ${\widehat{\mathit{U}}}_{NN}$ is the improved fuzzy neural network controller, which is used to estimate the sliding mode control law. ${\mathit{U}}_r$ is an additional value of robustness, which ensures that the system would be on the sliding surface when $\mathrm{t}\ge 0$. $\widehat{\mathit{x}},\widehat{\mathit{\sigma }},\widehat{\mathit{W}},\widehat{\mathit{l}},\widehat{\mathit{p}}$ represent the neural network parameter estimation.

$$\tilde{U}=U_{SMC}-U=W^{*}{g}^{*}+\epsilon -\widehat{W}\widehat{g}-U_r=\tilde{W}{g}^{*}+\widehat{W}\tilde{g}+\epsilon -U_r$$

(25)

In the formula, $\tilde{\mathit{W}}={\mathit{W}}^{*}-\widehat{\mathit{W}}$, $\tilde{\mathit{g}}={\mathit{g}}^{*}-\widehat{\mathit{g}}$, $\tilde{\mathit{g}}$ can be expressed by Taylor expansion due to g being a non-linear activation function.

$$\tilde{g}=g_x\tilde{x}e+g_{\sigma }\tilde{\sigma }e+o_m$$

(26)

Herein, $\tilde{\mathit{x}}={\mathit{x}}^{*}-\widehat{\mathit{x}}$, $\tilde{\mathit{\sigma }}={\mathit{\sigma }}^{*}-\widehat{\mathit{\sigma }}$ can be expressed as:

$$g^{*}=\widehat{g}+g_x\tilde{x}e+g_{\sigma }\tilde{\sigma }e+o_m$$

(27)

Bringing Formula (22) into Formula (20), then we find:

$$\begin{array}{c}\tilde{U}=W^{*}{g}^{*}+\epsilon -\widehat{W}\widehat{g}-U_r\hfill \\ =W^{*}\left(\widehat{g}+g_x\tilde{x}e+g_{\sigma }\tilde{\sigma }e+o_m\right)+\epsilon -\widehat{W}\widehat{g}-U_r\\ =\left(W^{*}-\widehat{W}\right)\widehat{g}+\left(\tilde{W}+\widehat{W}\right)g_x\tilde{x}e+\left(\tilde{W}+\widehat{W}\right)g_{\sigma }\tilde{\sigma }e+\epsilon -U_r+W^{*}{o}_m\\ =\tilde{W}\widehat{g}+\widehat{W}{g}_x\tilde{x}e+\widehat{W}{g}_{\sigma }\tilde{\sigma }e-U_r+P\end{array}$$

(28)

Herein: $\mathit{P}=\tilde{\mathit{W}}{\mathit{g}}_{\mathit{x}}\tilde{\mathit{x}}\mathit{e}+\tilde{\mathit{W}}{\mathit{g}}_{\mathit{\sigma }}\tilde{\mathit{\sigma }}\mathit{e}+{\mathit{W}}^{*}{\mathit{o}}_{\mathit{m}}+\mathit{\epsilon }$, $||\mathit{P}||<\mathit{r},$ r is a minimal positive number.

4.3. Stability Analysis

Next, we improve the fuzzy neural network sliding mode control algorithm according to the dynamics and kinematics equation of the manipulator to make the manipulator reach the prescribed sliding surface in the closed-loop control of the visual servo system, and reduce and eliminate the chattering problem of sliding-mode control, so as to ensure the long-term stability of the system.

Set:

$$\widehat{W}={\eta }_1{\left(\widehat{g}{S}^T\right)}^T$$

(29)

$$\widehat{x}={\eta }_2{\left(e{S}^T\widehat{W}{g}_x\right)}^T$$

(30)

$$\widehat{\sigma }={\eta }_3{\left(e{S}^T\widehat{W}{g}_{\sigma }\right)}^T$$

(31)

$$U_r=\frac{{\tau }_{0}sgn\left(S\right)}{{{\displaystyle \sum }}_{i=1}^n{S}_i^2}$$

(32)

Define the Lyapunov function as:

$$V=\frac{1}{2}{S}^{T}S+\frac{1}{2{\eta }_1}tr\left(\tilde{W}{\tilde{W}}^T\right)+\frac{1}{2{\eta }_2}tr\left(\tilde{x}{\tilde{x}}^T\right)+\frac{1}{2{\eta }_3}tr\left(\tilde{\sigma }{\tilde{\sigma }}^T\right)$$

(33)

Find the derivation:

$$\dot{\mathrm{V}}=\frac{1}{2}{S}^T\dot{S}-\frac{1}{{\mathsf{\eta }}_1}\mathrm{tr}\left(\tilde{W}{\tilde{W}}^T\right)-\frac{1}{{\mathsf{\eta }}_2}\mathrm{tr}\left(\tilde{x}{\tilde{x}}^T\right)-\frac{1}{{\mathsf{\eta }}_3}\mathrm{tr}\left(\tilde{\sigma }{\tilde{\sigma }}^T\right)$$

(34)

Since $\dot{\mathit{S}}=\tilde{\mathit{U}}$, we introduce Formulas (6)–(15) and (6)–(17) and find:

$$\begin{array}{c}\dot{\mathrm{V}}=S^T\left[\tilde{W}\widehat{g}+\widehat{W}{g}_x\tilde{x}e+\widehat{W}{g}_{\sigma }\tilde{\sigma }e-U_r+P\right]\\ -\frac{1}{{\mathsf{\eta }}_1}\mathrm{tr}\left(\tilde{W}{\tilde{W}}^T\right)-\frac{1}{{\mathsf{\eta }}_2}\mathrm{tr}\left(\tilde{x}{\tilde{x}}^T\right)-\frac{1}{{\mathsf{\eta }}_3}\mathrm{tr}\left(\tilde{\sigma }{\tilde{\sigma }}^T\right)\\ =\mathrm{tr}\left(\tilde{W}\left(\widehat{g}{S}^T-\frac{1}{{\mathsf{\eta }}_1}{\tilde{W}}^T\right)\right)+\mathrm{tr}\left(e{S}^T\widehat{W}{g}_x-\frac{1}{{\mathsf{\eta }}_2}{\tilde{x}}^T\right)+\mathrm{tr}\left(e{S}^T\widehat{W}{g}_{\sigma }-\frac{1}{{\mathsf{\eta }}_3}{\tilde{\sigma }}^T\right)+S^T\left(-U_r+P\right)\end{array}$$

(35)

According to Formulas (24), (27) and (30), we transform $\dot{\mathbf{V}}$ to:

$$\dot{\mathrm{V}}=S^T\left(-U_r+P\right)\le ||S^T||\times ||-U_r+P||=-\alpha ||S^T||\le 0$$

(36)

Herein, $\alpha$ is a positive, sliding-mode control system based on the improved fuzzy neural network, and is asymptotically stable according to the Lyapunov stability principle. When $t\to \infty$, s $\to 0$.

5. Simulation Analysis and Experiment

The adaptive sliding-mode controller is simulated by matlab. The AC servo motor is used in the robot joint. The corresponding parameters are set according to the physical parameters of the picking robot arm as shown in

Table 2. Structure parameters of picking robot.

${\mathit{m}}_{\mathbf{1}}$	10.8 kg	${\mathit{l}}_{\mathit{c}\mathit{e}}$	0.42 m
${\mathit{l}}_{\mathbf{1}}$	0.58 m	${\mathit{I}}_{\mathit{e}}$	0.4 kg
${\mathit{l}}_{\mathit{c}\mathbf{1}}$	0.29 m	${\mathit{\delta }}_{\mathit{e}}$	0
${\mathit{I}}_{\mathbf{1}}$	0.9 kg	${\mathit{e}}_{\mathbf{1}}$	–0.58
${\mathit{m}}_{\mathit{e}}$	8.2 kg	${\mathit{e}}_{\mathbf{2}}$	9.81

.

$$H=\left[\begin{array}{cc}\alpha +2\epsilon \cos (q_2)+2\eta \sin \left(q_2\right)& \beta +\epsilon \cos \left(q_2\right)+\eta \sin \left(q_2\right)\\ \beta +\epsilon \cos \left(q_2\right)+\eta \sin \left(q_2\right)& \beta \end{array}\right]$$

$$C=\left[\begin{array}{cc}\left(-2\epsilon \sin (q_2\right)+2\eta \cos \left(q_2\right))\dot{q_2}& (-\epsilon \sin (q_2)+\eta \cos \left(q_2\right))\dot{q_2}\\ (\epsilon \sin (q_2)-\eta \cos \left(q_2\right))\dot{q_1}& 0\end{array}\right]$$

$$G=\left[\begin{array}{c}\epsilon e_2\cos \left(q_1+q_2\right)+\eta e_2\sin (q_1+q_2)+\left(\alpha -\beta +e_1\right)e_1\cos q_1\\ \epsilon e_2\cos \left(q_1+q_2\right)+\eta e_2\sin \left(q_1+q_2\right)\end{array}\right]$$

Herein, $\alpha$, $\beta$, $\eta$, $\epsilon$ is constant. $\alpha =I_1+m_1{l}_{c1}^2+I_e+m_e{l}_{ce}^2+m_e{l}_1^2$, $\beta =I_1+m_e{l}_{ce}^2$, $\epsilon =m_e{l}_1{l}_{ce}cos\left({\delta }_e\right)$, $\eta =m_e{l}_1{l}_{ce}sin\left({\delta }_e\right)$. Take $\mathit{a}={\left[\begin{array}{cccc}\alpha & \beta & \epsilon & \eta \end{array}\right]}^{\mathrm{T}}$, $\widehat{\mathit{a}}$ is an estimation of $\mathit{a}$, set $\tilde{\mathit{a}}$ = $\widehat{\mathit{a}}$ − $\mathit{a}$, and $\mathit{a}$ is constant, so $\dot{\tilde{\mathit{a}}}\approx \dot{\widehat{\mathit{a}}}$.

Set the angle instruction of joint 2 as $q_{d1}=sin\left(2\pi t\right)$, $q_{d2}=sin\left(2\pi t\right)$, ${\dot{q}}_{d1}$ and ${\dot{q}}_{d2}$ are respectively the angular velocities of joints 1 and 2. Take $\mathsf{\Lambda }=\left[\begin{array}{cc}15& 0\\ 0& 15\end{array}\right]$, ${\mathsf{\Lambda }}_1=\left[\begin{array}{cc}5& 0\\ 0& 5\end{array}\right]$, ${\mathsf{\Lambda }}_2=\left[\begin{array}{cc}50& 0\\ 0& 50\end{array}\right]$, ${\mathit{K}}_{\mathrm{d}}=\left[\begin{array}{cc}100& 0\\ 0& 100\end{array}\right]$. Set the gain of the adaptive fuzzy neural network to ${\eta }_1=20$; ${\eta }_2=5$; ${\eta }_3=2$.

Firstly, the tracking curves of angular position and velocity of joints 1 and 2 are obtained by using sliding mode control without adding the fuzzy neural network algorithm, as shown in Figure 10, Figure 11 and Figure 12.

From Figure 10, it can be seen that the angular position switching motion of common SMC joints without the fuzzy neural network is continuous and smooth. From Figure 12, it can be seen that joint 2 is more likely to be disturbed due to the need to deal with the conductive force from joint 1, so the output buffeting is relatively large.

The simulation results of a manipulator system with an improved fuzzy neural network sliding-mode control algorithm are shown in Figure 13, Figure 14 and Figure 15. From Figure 15, it can be seen that compared with the traditional SMC scheme, the proposed fuzzy neural network sliding-mode control algorithm effectively eliminates the control chattering phenomenon. Meanwhile, the angular velocity tracking of joint 1 and joint 2 is more robust. It realizes high-precision control without detailed system information, and embodies the superiority of the improved fuzzy neural sliding-mode control algorithm.

In order to further verify the effectiveness of the sliding-mode algorithm of the fuzzy neural network system, the self-developed picking robot system is shown in Figure 16. The control system of the manipulator that is used in the experiment is an industrial computer with an Intel processor. The motion control system of the manipulator is connected with the host computer through serial communication, and the C control code generated by Matlab is imported into the control system.

The control instruction is generated according to the results of image recognition, so we process the control instructions by the sliding-mode algorithm of the fuzzy neural network and dynamic joint angle, output the dynamic joint angle, and bring into the kinematics equations related to the manipulator and output manipulator motor control instruction so as to control the manipulator gripper to reach the target position. Intensive experiments are performed on the apple-picking robot using the conventional PID algorithm, the sliding-mode control algorithm, and the improved fuzzy neural network sliding mode control on the robot arm servo control in the same experimental environment. Comparing the above three different algorithms on the picking-robot system on the grab speed and the grab accuracy,

Table 3. Experimental results.

Algorithm	Success Rate	Starting Position to the End of the Crawl Time(s)
Conventional PID algorithm	88.2%	15.8
Sliding mode control algorithm	74.0%	13.0
improved fuzzy neural network sliding mode control algorithm	91.2%	13.8

is the data comparison using different algorithms.

Through the comparison of experimental data, it can be known that the conventional PID algorithm captures a long time, while the sliding mode control algorithm can effectively reduce the grab time, but there is a lower catch success rate, and the improved fuzzy neural network algorithm is adopted. The sliding mode control algorithm not only improves the grabbing efficiency, but also greatly improves the success rate of the capture. It is shown in the result of the visual servo motion of the manipulator that the fruit-picking robot based on the improved fuzzy neural network sliding-mode algorithm has good stability and robustness. Figure 14 shows that the fruit-picking robot completes the visual servo control process of the manipulator using an improved algorithm.

6. Conclusions

In this paper, based on the self-developed apple-picking robot, the kinematics and dynamics equations of the robot are established. The image-based visual servo control method is adopted to control the manipulator in order to improve the grasping accuracy in the picking process. Then, the joint control performance of the control system has been improved by the adaptive fuzzy neural network sliding-mode control algorithm. Finally, the superiority of the algorithm is verified by simulation and experiment. The results show that the method improves the shortcomings of the common sliding mode control for the training control jitter in the actual robotic arm visual servo system to a certain extent, and improves the stability of the servo control.

Due to the low stability of the mechanical design of the current picking robot end effector, the existing mechanical structure of the picking robot will be improved in future work, and continue to verify the superiority of the current algorithm on the basis of the new terminal actuator, and complete the picking experiment in the real environment of the apple-picking robot.