Information System for Diagnosing the Condition of the Complex Structures Based on Neural Networks

Abstract

In this paper, we describe the relevance of diagnosing the lining condition of steel ladles in metallurgical facilities. Accidents with steel ladles lead to losses and different types of damage in iron and steel works. We developed an algorithm for recognizing thermograms of steel ladles to identify burnout zones in the lining based on the technology and design of neural networks. A diagnostic system structure for automated evaluating of the technical conditions of steel ladles without taking them out of service has been developed and described.

1. Introduction

The operation of lined equipment (torpedo ladle cars, steel ladles, etc.) in metallurgical production is associated with the need for constant monitoring and diagnostics of the technical condition of its lining. Particular attention to the diagnostics of the technical condition of the lining of critical production equipment is justified by the fact that wear of the lining can be the cause of accidents with this equipment, leading to significant material damage. In practice, in metallurgical production, monitoring and diagnostics of the technical condition of torpedo ladle cars or steel ladles are carried out with the help of qualified personnel based on their personal experience, using measuring instruments that are characterized by significant measurement error [1]. Therefore, during the operation of this type of production equipment, it becomes necessary to solve the problem of increasing the objectivity and quality of decision making when diagnosing its technical condition. Thus, conducting scientific research to solve this problem is relevant.

Currently, there are various systems for diagnosing and controlling technological equipment and processes [2,3]. Existing systems have a large functional set. However, these complexes have disadvantages. From the analysis papers [2,4,5], it can be concluded that the existing systems do not provide diagnostics of the lined objects under consideration. The systems do not have the ability to assess the state of the lining. In addition, all existing systems are focused on a specific production and their adaptation at another facility requires significant time and money.

The causes of accidents can be completely different, ranging from equipment failures to human factors. Ferrous metallurgy is associated primarily with rivers of molten metal. Blast furnace and steel smelting redistribution include tens of millions of tons of iron and steel. Metallurgical and auxiliary units are designed taking into account the constant or variable influence of temperatures, aggressive substances, and other negative factors. Despite this, in the practice of ferrous metal production, there are cases of emergency equipment failure with large-scale consequences.

Breakdowns can overtake not only crane equipment. For example, the steel-pouring ladle itself is also a rather complex mechanism.

In general, the main factor causing an increase in accident and injury rates in metallurgical and coke-chemical facilities is non-compliance with the full range of requirements in the field of industrial safety. The use of automation of the industry allows controlling and minimizing the accident rate.

2. Literature Review

There are several studies in the field of monitoring and diagnostics of the steel ladles state—one can single out papers [6,7,8,9]—as well as software for diagnosing steel ladle conditions such as “FLIR”, “SICK”, “PIEPER”, and others [10,11,12].

In the paper [13], the authors proposed guidelines for the monitoring and diagnostics of the state of steel ladles. In addition, the insufficient level of automation to monitor and diagnose the steel ladles in metallurgical facilities is mentioned, as well. A special method was created in the study [14] based on the use of the thermographic results in combination with mathematical models. Authors of the study [15] present a laser meter and thermography at the steel ladle as assessment methods. In paper [16], the model of steel ladle monitoring is based on the thermal expansion coefficient. In paper [17], a model has been proposed for assessing the defect criticality to diagnose the technical state of the steel ladles. A decision support system for monitoring the steel ladle’s condition is described.

To maintain the steel ladles, leading companies developed specialized software and automated systems to diagnose individual facilities [18,19]. Nowadays, there are several diagnostic systems [10,11,12,20,21] to assess the state of steel ladles. In addition, the digital twin in paper [22] is another direction for evaluating the state of steel ladles. The company “Hatch” developed a system for the nondestructive monitoring of the residual lining in blast furnaces. This system is based on an acoustic ultrasound echo signal [23,24,25,26,27].

Among the most acute problems in the metallurgical and coke-chemical industries, experts emphasize the slow pace of replacement of equipment and technical safety equipment, and the slow introduction of modern technologies. The issues of compliance with industrial safety requirements are quite acute, in terms of personnel certification (there is no professional training, there are people who, having not received proper training, are allowed to work); untimely examination of technical devices, buildings, and structures; and issues of organizing production control.

However, the developed methods do not allow the diagnosis of the state of steel ladles without taking them out of service and do not provide the opportunity for preventive diagnostics.

3. Algorithm for Recognizing Thermograms of Steel Ladles to Identify Burnout Zones in the Lining

A feature of the diagnostics of the steel ladles is that the change in their lining state can be recorded using the thermal control method [10] based on the analysis and processing of images of thermograms obtained with a thermal imager. In this case, the task of studying the technical condition of the steel ladle lining based on a thermogram is reduced to recognizing the dynamic sequence of a multicolor spot-like image by specific features.

In this paper, we propose an algorithm for recognizing thermograms of steel ladles to identify burnout zones in the lining. An IDEF0 diagram presenting the operations of the proposed algorithm is shown in Figure 1.

According to the development concept, each IDEF0 diagram consists of blocks and arcs. Blocks show the functions of the system being modeled. The arcs connect the blocks to each other and show the display of interaction and the levels of interconnection between them. Function blocks in diagrams are shown as rectangles and denote named processes, functions, or tasks that take place over a period of time and have recognizable results.

IDEF0 provides the developer with rules about the presence of at least three and no more than six blocks in the diagram. These rules keep the complexity of diagrams and models at a level that is easy to read, understand, and use.

The algorithm consists of the following actions:

• Increasing the contrast of the steel ladle thermogram.
• Separation of informative segments of the thermogram from the background by filtering methods.
• Vectorization of steel ladle thermogram segments.
• Segmentation of the steel ladle thermogram.
• Classification of steel ladle thermogram segments using the proposed neural network.

For the sake of contrasting, we propose using the method of adaptive transformation [6].

We propose using the H_S as a characteristic of the local neighborhood of a pixel. For each image element f(i,j), the local contrast value C(i,j) is calculated in the current neighborhood S centered at the element with coordinates (i,j). After that, we calculate the local statistics for the current moving neighborhood S using the histogram length function:

$$H_S=\frac{L_{max}-L_{min}}{H_{max}},$$

(1)

where L_max and L_min are the maximum and minimum values, respectively, of the brightness of the elements of the sliding neighborhood centered in the element with coordinates (i,j), and H_max is the maximum value of the histogram of the brightness levels of the neighborhood elements, centered on the element with coordinates (i,j).

After that, local contrast amplification is performed using non-linear functions and taking into account the local statistics of the current moving neighborhood S. The next step is to calculate the power law transformation of the local contrast, which, due to the use of the length function of the moving neighborhood histogram, has an adaptive character:

$$C^{*}\left(i,j\right)=C{\left(i,j\right)}^{{\alpha }_{adapt}},$$

(2)

where

$${\alpha }_{adapt}={\alpha }_{min}+\left({\alpha }_{max}-{\alpha }_{min}\right)\frac{H_S-H_{Smin}}{H_{Smax}-H_{Smin}},$$

(3)

H_Smax and H_Smin are the maximum and minimum values of the histogram length function for the neighborhood, respectively, and α_max and α_min are maximum and minimum values of the power law function of local contrast transformation, respectively.

Next, the brightness value of the image is restored f*(i, j) with the amplified local contrast.

From expression (1), it follows that thermogram fragments with constant brightness have H_S = 0, because L_max ≈ L_min. Local contrasts of such areas of the image are not amplified, since this will lead to additional distortions due to the amplification of the noise component of the image of the thermogram. Fragments of a thermogram with a uniform histogram and the maximum possible brightness range have H_S = 255. For such a neighborhood, we will assume that it is high-contrast and does not need to be enhanced. For fragments with a bimodal histogram, H_S depends on the brightness range and the ratio of the number of pixels in the vicinity of the minimum and maximum brightness.

The result of contrasting the thermogram of the steel ladle is shown in Figure 2.

The next step of the algorithm is to separate informative segments of the steel ladle thermogram from the background, which indicate a possible burnout of the lining.

We propose this operation to be carried out not by the process of changing the contrast, but by a new approach as the use of data flow filtering methods. We propose the use of filtering methods: Prewitt, Sobel, and Roberts filters and the Canny method. The rationale for the choice of identifiable filters for solving the problem is based on their high efficiency in solving boundary detection problems [6,7].

The filtering results are shown in Figure 3.

When using the considered filters, “lost” points on the curve may appear, as well as small deviations from the ideal shape of a straight line, circle, or ellipse. To eliminate this effect, additional image transformations are required. To solve this problem, an algorithm for thermogram image vectorization is proposed, the essence of which is to group the selected boundaries into a corresponding set of straight lines.

Based on the fact that the thermogram coordinates x and y can lie on the entire numerical range from −∞ to +∞, the straight line on the image is represented in the form:

$$r=x\cdot \cos \left(\theta \right)+y\cdot \sin \left(\theta \right),$$

(4)

where r is the magnitude of the vector that comes out of the origin, and θ is the angle of inclination between the vector and the coordinate axis.

Let us assume that the vectorized thermogram image will be represented as a function f(x,y), where the background value is defined as f(x,y) = 0, and the analyzed element as f(x,y) = 1. We introduce one more assumption—each pixel can contain a maximum of two adjacent elements, i.e., the objects under consideration are presented as closed lines or segments. Let all segmented elements contain at least two or more pixels, and the rest are removed as noise. Let us select individual objects from the image and write them into the function object O_i(X,Y), where i is the number of objects with the matrix size X and Y. As the initial vectorization position, we choose an arbitrary point A₀(x,y) on the object Oi that belongs to the space O_i(X,Y) and is not equal to 0. To detect the next point, we set the search radius r, which will describe the distance of two neighboring points from each other, as well as a positive bypass. This segment search scheme is shown in Figure 4.

In case of an unsuccessful search for the point A_i+1 (A_i at the initial moment of the algorithm execution is equal to A₀), the segment search radius is reduced by one until the function gives a positive result. If the searching index i is greater than one (at least two pixels belonging to the line are found) and the value of the search radius is r = 0, the algorithm is terminated.

The found point A_i+1 must satisfy the condition that the neighboring pixels of the point must lie in the range between A₀ and A_i. If the condition is not met (the found point lies in the traversed range), the search radius decreases, and the algorithm starts execution from function (4). A successfully found point is written to the Mas(i) structure, which contains data about the next point. If the value A_i+1 = A₀, then the segment is closed. If all segments belonging to this object on the thermogram are found, then the thermogram vectorization algorithm terminates.

The next step is to check A_i+1 for neighbors. If a point contains one neighbor, then it is finite and the segment under consideration is not closed. We complete the execution of the algorithm. If two neighboring points are found, set A_i = A_i+1 and proceed to the function execution.

As an output value, the vector Mas(i) is formed as a result, the structural image of which is shown in Figure 5. Each element of the structure has data on the relative position of the point and a pointer to the next element. In case the curve is closed, then the last element points to the first one.

Performing vectorization makes it possible to link disparate parts of one element, as well as reduce the amount of data for further analysis. The result of vectorization of the thermogram of the steel ladle is shown in Figure 6.

After determining the base points and vectorization of the thermogram, the algorithm assumes the determination of numerical values that will characterize the image segments. The one illustrated in Figure 7 is formed according to the method in [8].

We propose to describe the illustrated thermogram segment with the following set of values:

$$p=\left\{G_p,\sin \left(A\right),\cos (A)\right\},p\in P.,$$

(5)

where p is a set of values.

As basic examples of parametric models, machine learning researchers consider perceptron, logistic and linear regression, and linear support vector machine. Non-parametric models cannot be characterized by a fixed set of parameters, and their number grows along with the size of the training sample, thereby increasing the dimension of the model and computing resources. An example of a non-parametric algorithm is nearest neighbor regression. Unlike linear regression, the nearest neighbor regression model retains the target and independent variables from the training set. When a similar one makes a prediction for a test point x, it finds the nearest points in the training sample and returns the weighted average of the values of the target variable for these points. Other examples of non-parametric models are presented as a classifier based on a decision tree or random forest and a kernel decision support vector machine.

The proposed set of values (5) is the input of the neural network. To solve the problems of classification of thermograms of the steel ladles, the following types of neural networks were investigated: multilayer perceptron (MLP) and radial basis function (RBF) network.

The process of calculating the number of neurons in the input layer is based on multiplying by 3 the number of base points in the image sample volume. The dimension of the hidden layer is expressed through the dependence of the number of processing segments that identify burnout zones (the structure of the neural network of the steel-pouring ladle is shown in Figure 8).

The following backpropagation algorithm varieties with a sigmoidal activation function [26,27,28] to train the neural networks were chosen: gradient descent backpropagation, gradient descent with adaptive learning rate backpropagation, conjugate gradient backpropagation with Polak-Ribiére updates, etc.

Among the basic and innovative properties of artificial neural networks, scientists name their ability to learn. Networks are trained in a variety of ways. Most teaching methods are based on general postulates and have many identical characteristics for the system (The results are summarized in

Table 1. Output neurons with the temperature of the steel ladle.

Output Neuron Value Range	Temperature Range	The Entered Designation of the Temperature Range
-	0–30 °C	Tnormal
-	30–40 °C	Tpre
0.0001–0.13	40–50 °C	T1
…	…	…
0.78–0.86	300–350 °C	T9
0.87–0.99	350–400 °C	T10

).

Having studied the tables and the results of processing based on the neural network obtained images of the thermogram, it is possible to build a matrix of states of the sections of the lining of the steel-pouring ladle:

$$Places=\left(\begin{array}{cccccccc}4& 0& 0& 5& 5& 5& 0& 0\\ 0& 0& 0& 0& 4& 5& 0& 0\\ 0& 0& 0& 0& 0& 0& 0& 0\\ 4& 0& 0& 0& 3& 4& 0& 0\\ 4& 0& 0& 0& 0& 0& 0& 0\\ 4& 0& 0& 0& 3& 0& 0& 3\end{array}\right)$$

(6)

Each structural matrix element characterizes the process of the base section of the steel-pouring ladle lining. In this case, zero values indicate that there is no damage to the section. The temperature map is built on the basis of the matrix of conditional sections. The software we developed uses the GetPixel() function to determine the color palette on the original thermogram image (Figure 9).

4. The Structure of the Information System for Monitoring and Diagnostics of 50 t Steel Ladles

The proposed information system for monitoring and diagnostics of 50 t steel ladles includes:

• FLIR GF309 thermal imager, providing data collection and sampling on a 50 t steel ladle.
• Portable metallographic microscope MICROKON MPM-2U-KS for forming metallographic images of a 50 t steel ladle.
• Technologist’s computer with specialized software for ladle diagnostics includes:

  –

  Software analyzer of images of a steel-pouring ladle, which determines the state of the ladle lining and identifies burnout zones based on computer processing of ladle thermograms according to the method proposed by the authors [26]; and

  –

  A software expert system for assessing the technical condition of the ladle and forming recommendations regarding the type of repair of the steel-pouring ladle and the mode of its operation [29].

The structure of the information system for monitoring and diagnostics of 50 t steel-pouring ladles is shown in Figure 10.

With the help of our developed software, we conducted an experiment to recognize images of thermograms of steel-pouring ladles of 50 t, in order to identify the areas that need to be replaced with a lining. An example of recognizable thermograms of a steel-pouring ladle is shown in Figure 11.

5. Experimental Studies of the Developed Tools

During the experiment, the optimal number of learning epochs of the created neural networks was determined. The results of the functioning of the developed neural networks are summarized in

Table 2. Comparative results of the functioning of the developed neural networks.

NN Type and Its Structure		An Evaluated Optimal Number of Learning Epochs	Total Number of Recognizable Images of Thermograms for Steel Ladle	Percentage of Correctly Recognized Images of Thermograms among the Total Number of Thermograms
Multi-segment MLP network	210-70-20	280	130	89.7
	400-100-30	320	130	90.2
	300-120-20	500	130	91.7
	200-100-15	550	130	94.3
Multi-segment RBF network	210-70-20	250	130	90.8
	400-100-30	300	130	92.6
	300-120-20	600	130	93.5
	200-100-15	620	130	93.8

.

The neural network was trained based on reference images of thermograms obtained experimentally at PJSC Alchevsk Iron and Steel Works. The training sample consisted of 380 images of thermograms, with 190 “correct” and 190 “incorrect” ones. For control and test samples, 200 images of thermograms of 50 t steel ladles used at Alchevsk Iron and Steel Works were used for each sample (Figure 12).

During the experiment with the developed software, the probability of correct analysis was determined. The results of the analysis of images of 50 t ladles are summarized in

Table 3. Results of recognition of images of thermograms of steel-pouring ladles.

No.	Total Number of Thermograms of Ladles	Number of Correctly Recognized Thermograms	Correct Recognition Probability
50 t Ladle #10	112	103	91.9%
50 t Ladle #15	108	98	90.7%
50 t Ladle #22	89	78	87.6%
50 t Ladle #23	140	131	93.6%
50 t Ladle #25	135	124	91.9%

.

Let us consider the effectiveness of the proposed information system for diagnostics of steel-pouring ladles as compared to the standard system [12] for technical diagnostics of ladles used at PJSC Alchevsk Iron and Steel Works. The calculation of the error in determining the permissible pouring is according to Formula (7) (

Table 4. Experimental determination of the number of permissible pouring of liquid metal.

No. of the Experiment	Standard System [12] for Steel Ladles			Information System
	ncount	nreal	δw	ncount	nreal	δw
#1	10	11	9.1%	11	11	0
#2	35	37	5.4%	42	43	2.3%
#3	30	33	9.1%	40	41	2.4%
#4	30	35	14.3%	20	21	4.8%
#5	40	43	7%	25	27	7.4%
…	…	…	…	…	…	…
#25	15	21	28.6%	24	25	4%
#26	20	24	16.7%	20	20	0
#27	10	14	28.6%	21	22	4.6%
#28	10	8	25%	14	14	0
#29	25	26	3.9%	11	11	0
#30	40	42	4.8%	48	41	7.9%

).

$${\delta }_w=\frac{\Delta w}{w_{real}}\cdot 100\%,$$

(7)

where Δw is the difference between the real value of the diagnosed parameter of the steel-pouring ladle and the calculated value using the proposed information system, and w_real is the real value of the diagnosed parameter of the steel-pouring ladle.

Table 2 suggests that in the standard system of technical diagnostics of steel ladles, the error in determining the number of permissible pouring of liquid metal into steel ladles can reach 28.6%. In our proposed information system, this error does not exceed 7.9%.

6. Discussion

We determine the errors of representativeness and the number of degrees of freedom as:

$$m_1=\sqrt{\frac{p_1\cdot q_1}{n_1}}=\sqrt{\frac{96.7\cdot \left(100-96.7\right)}{30}}=3.26$$

(8)

$$m_2=\sqrt{\frac{p_2\cdot q_2}{n_2}}=\sqrt{\frac{82.5\cdot \left(100-82.5\right)}{40}}=3.93$$

(9)

$$k=n_1+n_2-2=30+40-2=68$$

(10)

We determine the reliability of the difference between systems P1 and P2 as:

$$t=\frac{P_1-P_2}{\sqrt{m_1^2+m_2^2}}=\frac{96.7-82.5}{\sqrt{{3.26}^2+{3.93}^2}}=2.78$$

(11)

The value of t obtained in the experiment is greater than the tabular value of t_0.01; therefore, the differences between P₁ and P₂ can be considered significant at p < 0.01. Thus, the reliability of the differences between options P₁ and P₂ suggests that the developed information system for technical diagnostics and monitoring of steel-pouring ladles turned out to be more effective than the standard system in determining the main operational characteristic—the number of permissible pouring of liquid metal into ladles [30].

7. Conclusions

From this study, the following results were obtained.

1.

An information system for monitoring and diagnostics of 50 t steel ladles condition is proposed. The experimental data indicate that in the proposed diagnostic information system, in 96.7% of cases, the value of the number of permissible pouring operations was correctly calculated.

2.

The analysis of technologies and adaptive methods is carried out. We developed a concept and algorithmic process diagram. The method of recognition and identification of the structure of components has been improved.