Implementation of Predictive Models in Industrial Machines with Proposed Automatic Adaptation Algorithm

Abstract

This article describes in more detail the issue of using predictive models of NAR neural networks to predict the course of certain quantities, which may indicate a problem with the industrial machines or their major failures. It is very important to find sufficient size of the structure and values of parameters that directly affect the output accuracy of the model. This article presents the way in which it is possible to automatically find the settings of these NAR models so that the required final accuracy metric is achieved. This presented algorithm was tested on simulation data samples collected by using the M5StickC microcontroller device. This collected dataset presented in this article contains accelerometer and gyroscopic data only, but there is a possibility to expand and add some other sensors to this microcontroller, to collect some other relevant data. This M5-StickC microcontroller device can be used for gathering data in the first phase of the machine state analysis without interfering with the mechanical construction and electrical connections of the machine. Testing of proposed algorithm was carried out in MATLAB environment. The article also describes the way in which these predictive NAR neural network models can be implemented directly in control systems, specifically PLCs from the manufacturer SIEMENS without the use of 3rd party analytical platforms. This application can be helpful in the area of predictive maintenance tasks, especially in avoiding critical failures of industrial machines and devices, or some of their specific parts.

1. Introduction

In the conditions of the impermanence of the modern world and the economy, the issue of ensuring the safety and continuity of production is acute in all spheres [1]. For these purposes, strategies for the development and transformation of the relevant industries have been adopted in Russia and Slovakia. In Slovakia, the main document is ⟪The 2030 Strategy for Digital Transformation of Slovakia⟫ [2]. In Russia, this is the action plan (“a road map”) “TECHNET” (advanced production technologies) of the National Technology Initiative for 2020–2036 years [3]. The listed regulatory documents are supplemented with specialized industry ones depending on the specific industry sector.

The basis of modern industries consists of complex and multifactorial technical systems that have a certain structural and functional organization and have all the parameters of complex systems [4].

In Refs. [5,6,7] a technical system is an artificially created object that interacts with the external environment and has a complex internal structure, a large number of components and elements. It can exist as [8]:

• product of manufacture;
• device or equipment potentially ready to produce a beneficial effect;
• technological process of transformation or interaction with environmental components resulting in a beneficial effect;

A part of every more complex industrial machine is a PLC system, which has output-input peripherals—whether they are PLC cards with digital or analog inputs or more complex output devices such as, e.g., frequency converters and devices for controlling various other drives and actuators [9]. It is some sensed quantities that may indicate problems and incorrect operation of the device, e.g., low values sensed by the pressure sensor may indicate a problem with the hydraulic system, in particular, the hydraulic hose may be ruptured or the hydraulic pump may be damaged. Knowing such behavior of the equipment in advance is important in terms of cost savings for repairing the equipment, especially if such damage can be prevented by specific maintenance actions. In this article, we have decided to avoid available commercial analytical platforms such as Microsoft Power BI, Thingworx Analytics, or others that require frequent interventions by a data analyst in the form of setting up structures, types of models designed for predictive purposes, etc. [10,11].

One of the possible solutions is to implement a predictive module in the control systems, which will estimate the expected development of specific quantities based on their historical data. We decided to use nonlinear autoregressive models of NAR neural networks for such prediction [12]. The reason why we decided to use nonlinear autoregressive networks is that their structure is almost identical with the feed-forward neural networks. They differ only in the input layer, which is in this case called time delay layer. Therefore, there is a possibility to use open-sourced libraries which are offering only feed-forward neural network models. The time delay input number can be created in a script or a program. Other reason is that for forecasting the desired quantity or parameter the model only needs a historical record only of this one quantity or parameter. Training data set is created from this time series data [13,14,15]. In general, neural network models require higher computational demands, especially the training process of models. Therefore, the industrial machine must be supplemented with IPC, which has a higher computing power than most PLCs used in industrial machines.

2. Control and Management Systems for Technical Systems

The functioning of technical systems is largely determined by changes in the environment. Their stochastic nature requires the development and implementation of control and management systems to ensure their uninterrupted operation and avoid the occurrence of critical situations [16].

Design, implementation and improvement of control and management systems at enterprises, as a rule, are carried out by technologists with the participation of involved specialists—electrical engineers, mechanics, programmers and economists. At the same time, the solution of several important tasks is achieved [17]:

-

reducing of production costs;

-

ensuring the safety of life;

-

improving labor productivity;

-

improving the quality of finished products;

-

ensuring production flexibility.

A wide range of automatic and automated systems are used in various areas of production to control and manage technical systems [18].

The controls in an automated system can be either closed-loop or open-loop [19]. A closed-loop control system, also known as a feedback control system, is one in which the output variable is compared with an input parameter, and any difference between the two is used to drive the output into an agreement with the input. As shown in Figure 1, a closed-loop control system consists of six basic elements: (1) input parameter, (2) process, (3) output variable, (4) feedback sensor, (5) controller, and (6) actuator.

A typical example of such an advanced control system with a feedback loop are controllers for positioning the servo drive axis. These devices use encoders to constantly measure and control the rotation position of the servo motor rotor. Thanks to this kind of feedback, it is possible to calculate and control the rotor speed in real-time.

The SINAMICS control unit from SIEMENS, which can control the positioning of several servo drives, is a typical example of the use of a feedback loop system in practice (Figure 2). The entire actuator control device consists of multiple modules. The SINAMICS control unit is connected to the LM module, which converts the AC supply voltage 380 V to the DC supply voltage 500 V. The converted supply voltage is further adjusted in the motor modules from where it is further fed directly to specific actuators.

This actuator control unit communicates with the ET200-SP PLC system via a communication protocol called PROFINET by exchanging SW blocks called Telegram or via Technology Objects. This device is superior to the SINAMICS control unit and sends the required positions to which all the axes need to be positioned. In the figure below you can see PLC which sends the required positions to the SINAMICS Control unit and acts as a master device.

In Figure 3 there is a given hardware configuration of the control unit SINAMICS CU320-2 for controlling 4 servo drives.

In contrast to a closed-loop control system, an open-loop control system operates without the feedback loop, as in Figure 4. In this case, the controls operate without measuring the output variable, so no comparison is made between the actual value of the output and the desired input parameter. With an open-loop system, there is always the risk that the actuator will not have the intended effect on the process, and that is the disadvantage of an open-loop system. Its advantage is that it is generally simpler and less expensive than a closed-loop system.

Within the framework of this research, closed-loop control systems and their constituent elements are of the greatest interest.

3. Materials and Methods

3.1. Nonlinear Autoregressive Neural Networks NAR

The main part of this module is a nonlinear auto-regression artificial neural network NAR NN. These models are specified by the time delay input layer [12], which is representing time-series data of some physical quantity. This layer represents and stores the values of a variable that has the character of a time series. NAR neural networks are solving a function y given in equation below:

$$y=x_{t+1}=f\left(x_t,x_{t-1},x_{t-2}\dots x_{t-h}\right) ,$$

(1)

where x is the variable that needs to be predicted, t is actual time and parameter, h is a history window size. This window defines based on what amount of historical data will be the prediction estimated. These models also use a recurring loop to calculate multi-step ahead prediction. The architecture of this kind of NAR neural network model is given in Figure 5 below.

In this research, there was used a supervised machine learning method called Scaled Conjugate Gradient Method in every model created in this article. The advantage of this method is that it has fewer parameters, that need to be set by a user in comparison with other gradient descent methods such as Levenberg-Marquardt, Gauss-Newton, and other methods [20,21]. This SCG optimization method was used to minimize the global error function. In this research, there was chosen the MSE function for the global error function. The formula of MSE is given below:

$$\mathrm{MSE}=\frac{1}{n}{\sum }_{i=1}^n{\left(t_i-y_i\right)}^2 ,$$

(2)

where n is the actual size of the training data set, t are the targets samples from the training dataset and y are the predicted values by NAR neural network model. The SCG algorithm has a step size scaling mechanism and thus doesn’t require to use of a time-consuming line search for every training iteration. In general, this algorithm is well optimized and faster than other training methods [21].

3.2. Predictive Module Implementation into an Existing Industrial Machine

Figure 6 shows the proposed way to implement a predictive analytics module in an existing device. Such a module was created in a .NET console application that has access to a local or remote database in which historical data from the PLC is stored. Likewise, this module has access to the currently scanned PLC data through the used functions from the S7.Net Plus library [22]. The exchange of information takes place via data blocks to which this predictive module has access.

In this case, the PLC program must be supplemented by the above-mentioned data blocks (objects in the program, which can consist of several variables of different data types—structures). In our case, these data blocks are used to exchange data between the PLC device and the .NET predictive module. This service has the right to write and read the necessary variables from specific data blocks, as well as has access to a local database, which is constantly filled with current and predicted data. In our case, we opted for the open-source PostgreSQL object-relational database [23], which can be located locally on a device for analytical calculations, or it can be located remotely on a server.

With the help of such a proposed solution, it is possible to implement conditional monitoring, detection of anomalies on a given device [24], or prediction of the development of specific parameters that are directly related to a frequent error or failure of the machine or its specific part. Before the actual deployment of specific analytical models, it is necessary to perform failure analysis [25] and prepare equipment for data collection—e.g., connect the device to a database for data storage or supplement the device with various other sensors that will sense other important, significant quantities. Preprocessing and initial analysis can be performed using available software such as, e.g., MATLAB, or scripts are written in various programming languages as needed (e.g., R, Python, or other programming languages).

If the collected data contains information from the nonfaulty operation of the machine for a very long period, it is possible to create an anomaly detector [26]. As a result, it would be a model that is trained on the given data and the accuracy is constantly checked on the estimated data from the AI models and the actual incoming data. In the event of a decrease in the values of statistical metrics that describe the accuracy of the model, it is clear that the device is in a state in which it was not once during the data collection and therefore it can be argued that this is an anomalous phenomenon of the device or machine. This information can be displayed to the operator, who can check the individual problem parts of the device in time to prevent a major failure.

If the behavior of the device in a faulty or faulty state is also included in the data during the initial data collection, the trained model can be used as a simulation tool for the processes that may occur on the device. Using other tools, it is possible to select the recommended setting of parameters in which it is assumed that the error will not occur. Such parameter settings must be approved by users with higher authority and knowledge of the processes—technologists, data analysts, or other specialists in the field [27].

3.3. Data Acquisition for the Experiment Testing and Evaluation

M5StickC is a microcontroller that has a built-in 6-axis accelerometer MPU6886, WiFi module, LCD, and various other peripherals. The data presented in this article was collected using this integrated IMU accelerometer and written Winforms application (Microsoft .NET Framework Version 4.8.04084) in Visual Studio 2019 version 16.10.2.

M5StickC was programmed in ArduinoIDE version 1.8.13. Data on 3 measured accelerations in the X, Y, Z axes are sent to the web server running on this microcontroller. In addition, gyroscopic data (also in the X, Y, Z axes), temperature values, current battery voltage, and electrical current consumption are also read. All this information is stored in a .csv file. Our testing data set consists of 4394 samples that were collected for algorithm testing purposes. The Figure 7 below shows a part of the measured data—specifically accelerations in X axis.

The measured data was gathered by the external sensor only for the testing predictive module performance and the collected data were preprocessed in MATLAB R2020b. The imported data has been converted to timetable objects, which offer several useful functions. These data were cleared or filtered for outliers and erroneous values, respectively. Outlying data were replaced by a linear interpolation method. The measured data were resampled using the same method using the retime() function. This measured and processed signal was then smoothed using a smoothData() filter. In addition, it was necessary to create functions for creating training data from specific parameters.

3.4. Proposed Algorithm for Setting up the Parameters of the Predictive Analytical Module

The proposed analytical module has implemented its algorithm for the automatic setting of NAR NN model parameters. The authorized user determines the initial parameters such as—the initial size of the input layer, the critical size of the input layer and other critical parameters, minimum number of training data, the minimum number of prediction steps, size of the evaluated sample, number of hidden layers and number of neurons in them.

This adaptation algorithm starts by checking the number of stored data (samples) in the DB. If the number of samples in the database is equal to or greater than defined in the minTrainingDataAmount parameter, it is possible to call the function for creating training data. On this data, the newly created NAR model is trained and after training, an estimate of the number of steps to the future is calculated, which is defined in the stepsAheadParam parameter. Our proposed algorithm can be seen in the Figure 8 below.

The algorithm is constantly checking if there is a new sample in the database. If there is a new sample, the input layer of the model is filled by help of a SQL query command. When the input layer of the model is filled, there is possible to estimate a prediction on a time which is defined by the parameter stepsAheadParam and the sampling interval. After the prediction is estimated the result is stored back to the PostgreSQL DB to a table, where the results of the predictions should be stored. If there is more or equal samples than defined in the parameter evalSize, it is possible to finally calculate the accuracy metrics. If the calculated accuracy is sufficient, the algorithm waits for new samples so the prediction can be calculated again and this process can be repeated again. If the accuracy parameter is not sufficient it is necessary to modify the model parameters such as inputLayerSize and stepsAheadParam. In addition, the new training is made on the expanded data set. This data set is obtained directly from PostgreSQL DB.

When the new training data set is created, the parameters of the NAR models will be modified also. In the initial process, there needs to be set up an initial size of the input layer and the critical size of the input layer. This input layer size is affecting the size of the training dataset, which can affect the performance of the analytical HW during the training and thus the algorithm will modify stepsAheadParam, if the modification of the input layer size will not improve the accuracy of the model. The algorithm uses predefined constant values for decrementing the stepsAheadParam and incrementing the parameter called inputLayerSize.

Predicted and actual true values are stored in a database so that a degree of accuracy can be calculated between these estimated and collected data. It is possible to use several statistical parameters such as root mean squared error RMSE [28], mean absolute percentage error MAPE [28], or correlation coefficient R [29]. The formulas for each statistical metric are in the equations given below.

$$\mathrm{RMSE}=\sqrt{\frac{{\sum }_{i=1}^{evalSize}{\left(x_i-y_i\right)}^2}{evalSize}} ,$$

(3)

$$\mathrm{MAPE}=\frac{100}{evalSize}{\sum }_{i=1}^{evalSize}\left|\frac{x_i-y_i}{x_i}\right| ,$$

(4)

$$\mathrm{R}=\frac{1}{evalSize-1}{\sum }_{i=1}^{evalSize}\left(\frac{x_i-\overline{x_i}}{s_x}\right)\left(\frac{y_i-\overline{y_i}}{s_y}\right) ,$$

(5)

where variable x represents the real values measured by sensors, y represents the predicted values by NAR neural network model, s_x and s_y are the sample standard deviation for the real measured values and the predicted variables. It is necessary to have a certain number of samples of the predicted variable in the DB. This quantity is defined in the evalSize parameter. To be able to evaluate the above-mentioned statistical metrics, a minimum time must elapse, which depends on two parameters. It is the time on which the model calculates the predictions, and the second parameter is the size of the parameter evalSize. How these two parameters affect the final time is shown in the figure below.

In the Figure 9 there is explained why it is unable to calculate accuracy of the NAR neural network model, right after the model is trained. Accuracy needs to be calculated on new samples which were not a part of the training dataset. To calculate any accuracy metric, it is necessary to have two arrays: one array which contains the real data samples and the second one which consists corresponding predicted values to a time which is defined by a parameter called stepsAheadParam. Final time of the model parameter adjustments can be long, mainly because of these two parameters and the other prerequisites. For the time t that needs to pass after the training and so the accuracy metrics can be calculated is defined by the formula given below:

$$t=t_{sampling}evalSize ,$$

(6)

The time difference between two consecutive samples in the DB is t_sampling. The parameter which defines the amount of the samples which are used for the accuracy calculation is evalSize.

4. Results of the Proposed Algorithm for the Automatic Parameters Set up and the Prediction Results

In this research, there was used a NAR neural network with two hidden layers. This NAR network had hyperbolic tangent activation functions in each layer. The first hidden layer was consisting of 100 neurons and the other one was consisting of 75 neurons. The size of these two hidden layers was constant. The algorithm was correcting in this test only the time delay input layer size and the steps ahead parameter. For the algorithm tests presented in this article, we decided to set up the initial parameters with these specific values given in

Table 1. Initial parameters settings for the algorithm.

Name of the Parameter	Value
stepsAheadParam	100
inputLayerSize	100
Maximum inputLayerSize	1000
Minimum stepsAheadParam	30
Minimum R-value	0.75

below.

After the first parameter correction, the correlation coefficient between the real measured data and the predicted data for 100-time steps ahead reached a value of R = 0.20548. There is a given regression plot and a preview of the prediction on specific samples in the figures below. This value was far from the desired R = 0.75 so there was necessary for the algorithm to do multiple parameter corrections.

After 7 corrections of the model parameters, the final NAR model accuracy reached a value of R = 0.76128 which is larger than the desired accuracy defined in the previous table. The input layer was increased from the initial value of 100 to 300 and the stepsAheadParam was corrected from 100 to 60. After this correction, the model was able to reach the desired accuracy.

The results of the correlation coefficient between the estimated predictions and the real data during of each parameter correction is given in the table below. The performance of the initial configuration of the model is given in the Figure 10. The performance after the algorithm ended is given in the Figure 11.

The process of the parameter corrections is given in Figure 12 and

Table 2. Process of corrections of the parameters by using the proposed algorithm.

Parameter Correction	Input Layer Size	Prediction Steps Ahead	Correlation Coeficient R
initial	100	100	0.20548
1.	200	100	0.41881
2.	300	100	0.71273
3.	400	100	0.64898
4.	300	90	0.69094
5.	300	80	0.69529
6.	300	70	0.73602
7.	300	60	0.76128

. The algorithm started with the initial values which are given in the Table 1. The blue line represents the value of the input layer size during the modifying process of the model parameters. This parameter is increased by a constant value of 100 until the reach of the 4th correction of the model. In the 4th step of the parameter correction the value of the chosen criterion of the accuracy drops, so the algorithm sets the previous value of the size of the input layer of the NAR neural network model in the next parameter correction. In the next correction the algorithm is modifying the value of the stepsAheadParam by decreasing this parameter by a constant value of 10. After decreasing this parameter by a constant value, the algorithm was able to set up the model parameters, so the final accuracy reached value of 0.761. After reaching this desired accuracy the algorithm is ended.

5. Conclusions

This research aimed to propose a way in which a predictive module could be implemented in an existing machine, which would make it possible to predict the development of specific physical quantities for a certain time in advance without the use of commercially available third-party analysis platforms. In this research, we focused mainly on models of artificial neural networks NAR that meet the above requirements.

The dataset used in this article contains accelerated data captured by the M5StickC microcontroller and the integrated IMU sensor. This device can be always used in the initial stages of the analysis of sensed vibrations without interfering with the actual structure of the industrial machine itself. The way in which this module should be deployed in a real machine was tested on a Siemens ET 200SP device using a written .NET framework console application using libraries such as S7NETplus, npgsql [30], and encog [31].

It is often necessary to choose the appropriate settings for these models to be able to predict the specified time into the future with sufficient prediction accuracy. Therefore, an algorithm is introduced in this article that automatically searches for specific parameter values so that the model meets the prediction accuracy criteria as a result. Testing of this algorithm took a place in the MATLAB environment. Firstly, the parameter was set to the specific values from Table 1. At initialization, the model achieved a prediction accuracy of R = 0.20548. With stepsAheadParam = 100, the model calculates the estimated values for one step in advance and moves the estimated value to the input layer. With the inputLayerSize = 100, after calculating the 100th prediction, the entire layer is filled with values which are containing inaccuracies. As the input layer size increases, the ratio of correct and estimated values decreases, and the resulting accuracy increases. Decreasing the parameter stepsAheadParam also affects the resulting prediction accuracy. Each change of the size of the input layer needs to have a new model and also training of this model and also a new training dataset preparation. After adjusting the parameters stepsAheadParam and the inputLayerSize to values which are specified in Table 2 the model was able to predict with the resulting correlation coefficient R = 0.76128.

Using such an algorithm, it is possible to deploy such prediction on several crucial parameters without setting and searching for specific parameter values. At the end of this algorithm, the model parameters are set to optimal values in the terms of the resulting accuracy of specific NAR neural network models.