Time series prediction using the adaptive resonance theory algorithm ART-2

The algorithm of the adaptive resonant theory ART-2 is based on the ideas of dynamic clustering and the unsupervised learning model. The classic application of the ART-2 algorithm is related to the solution of pattern recognition problems in the framework of the neural network approach. The article proposes a modification of the adaptive resonance theory ART-2 as applied to the solution of the time series (TS) prediction problem. A description of the TS forecasting algorithm based on ART-2, its properties and application features, as well as the results of a study of TS free electricity prices of the “day-ahead market” (DAM) in Russia is here. The obtained results allow us to conclude about the prospects of using ART-2 to study the structure and prediction of TS with a periodic (seasonal) component.


Introduction
The task of forecasting is a cornerstone in the field of Data Mining and in general in the field of AI, if we consider the problem of constructing AI as a problem of building bio-inspired intelligence. The latter is explained by the fact that the main function of the brain is the function of predicting the behavior of the external environment in order to obtain advantages for survival. In applied artificial intelligence systems, the problem of forecasting is usually considered as the problem of predicting the values of a time series describing the change in the price of shares or currency, meteorological indicators, water level in rivers, seismic activity, etc.
Currently, there are many different predictive models, such as the autoregressive integrated moving average (ARIMA) models, regression trees, recurrent neural networks and finally, an ensemble time series prediction approach, combining different models [1][2][3][4].
All these forecasting models somehow come down to using correlation between some set of values , = 1, … , so called window and the following value +1 . In the case of using recurrent neural networks, it is better to talk about associative connection than about correlation, but the essence does not change.
Let us try to consider the problem of forecasting time series from the point of view of bionics, i.e. problem solving in a bio-similar way.
When solving problems of prediction by the biological brain, he does not have at his disposal the entire time series in which he can learn. The time series appears dynamically in the process of activity. Of course, the time series in this case is not a sequence of values of a single parameter, but a sequence of multidimensional images described by many different types of parameters that enter the system at arbitrary points in time. But to begin with, we simplify the problem, reducing it to the usual time series, 2 and we will further consider this simplification. It is logical to assume that the brain solves the problem of forecasting as a problem of recognizing patterns (situations) and for a similar image it uses a previously predicted value for a previously encountered one. Moreover, in the process of the system functioning, new, previously unseen images appear, which must be remembered and used later, i.e. there is ongoing (incremental) learning. Such process can be viewed as a dynamic clustering process implemented in an Adaptive Resonance Theory, proposed by Grossberg in 1976 [5] and developed by him together with Carpenter [6].
Until now, the applications of the ART-2 algorithm are known only for solving problems of pattern and situation recognition.
The article solves the problem of modification (adaptation) of the algorithm in order to apply it to study the structure and predict the values of the time series.
The study of the algorithm was carried out on the real TS of the "day-ahead market" (DAM) free electricity price. The data is provided by the Siberian Branch of PJSC RusHydro. At present, the task of short-term (hourly) forecasting of the DAM prices is becoming increasingly relevant due to the reform of the energy industry and the creation of the wholesale electricity and capacity market (WECM). The task of forecasting free hourly electricity prices is new for Russia and requires the development of efficient algorithms and forecasting techniques.

Description of the ART-2 algorithm as applied to TS prediction
The task of predicting next value of the time series can be reduced to the task of clustering. In this case, the cluster is recognized by the vector-window, and this cluster corresponds to the following (predicted) value (in the future will be called as value of cluster). Lower we give a mathematical description of the algorithm.
Initial data: Let = ( 1 , 2 , … , )time series, describing the change of a certain indicator at discrete points in time t=1,2, … , N. Values of TS are normalized and brought to measurement range from 0 to 1.
In matrix form: ). (1) The first step of the algorithm: The first fragment of TS is considered 1 . Fragment 1 associated with the first cluster, the center of a cluster is declared a vector 1 = 1 , weight of the cluster: 1 = 1, and the predicted value for first cluster is assigned value following the fragment 1 : 1 = +1 .
m-th step of the algorithm: m-th fragment of TS id considered. Calculates the Euclidean distance between the fragment and the centers of the clusters that are formed in the previous steps: wherenumber of clusters. Minimum distance is: The predicted value is the value of the cluster with the minimum distance of Euclidean : ̂+ = .
If > , then a new cluster is formed corresponding to the m-th fragment of TS and cluster value: +1 = + .
If ≤ , then the center of the cluster and its weight are recalculated (adapted to the new recognized fragment) according to the formulas: where min ( )weight of cluster at previous step, min( )center of cluster at previous step. The cluster value corresponding with cluster is recalculated according to the formulas similar to (4) and (5).
Below is a description of the algorithm in the form of pseudo code: To study the proposed prediction algorithm, a program was developed in the Delphi 7.

The study of electricity prices based on the application of the proposed algorithm
The developed algorithm was used to predict the TS of the free price for electricity of the "day-ahead market". For the study, a fragment of TS of hourly prices for January 2017 was selected, a total of 744 observations. The initial TS was divided into two parts: training (544 observations) and test (last 200 observations).
The TS of prices under investigation has a non-linear structure, namely, a trend of complex shape, periodic and random components. To use the algorithm, the initial TS was represented as an expansion: where ( )trend function described by a fifth degree polynomial: where , = 0,1, … ,5parameters of trend, ( ) -function, including periodic and random components, for the model description of which the algorithm was used.
To estimate the parameters of the trend component of TS, the numerical iterative Gauss-Newton method was used. According to the results of identification, the coefficient of determination, explained by the trend function, was -0.21. Figure 1 shows the TS of electricity prices with the imposition of the trend model. Further, the model description of the component ( ) was performed using our proposed algorithm. To estimate the periodicity of TS ( ) and to determine of optimal parameters of the algorithms the periodogram was built. According to the results of the analysis, the peak values of the periodogram correspond to periods 24 and 12. In this way TS ( ) has a clearly defined period of oscillation, which describes the change in the price of DAM during the day. Therefore, a window length of 24 was set as an algorithm parameter.
The figure 2 shows the TS of electricity prices with the imposition of a TS model, built with a radius of cluster 0.4. Analysis of the graph allows us to conclude that there is a good visual correspondence between the studied TS and TS model, both on the training and on the test parts of TS. The table 1 shows the accuracy of TS forecasting of DAM prices for different values of the cluster radius for the training and test parts of TS, namely, the mean absolute error, the standard deviation of the error, the coefficient of determination. Accuracy indicators are given for the initial TS prices DAM Y(t), including trend component ( ). Analysis of the results indicates a high accuracy of the identification of TS ( ): the determination coefficient on the learning part of TS with a cluster radius of 0.5 is 0.87, on the test portion -0.66. Excluding the percentage of variation attributable to the trend component of TS (21%), the contribution of the model description of the component ( ) using the algorithm: 66% (training portion) and 45% (test portion). Accuracy of identification ( ) increases with decreasing cluster radius and is up: 69% (training portion) and 47% (test portion). The number of created clusters when specifying a cluster radius of 0.5 and 0.4 is respectively 51 and 92. A relatively large number of clusters can be explained by the structure of the component, in which, apart from clearly defined oscillations with periods of 24 (day) and 12 (half a day), there are also periodicities of other orders, although their contribution to the change is much smaller.

Results and Discussion
The proposed algorithm was also tested on artificial TS of a periodic structure. According to the results of a comprehensive study of the algorithm, we can draw the following conclusions.
1. The accuracy of prediction of TS depends on the specified parameters of the algorithm (window length, cluster radius). As the cluster radius R decreases, the number of clusters K increases and the accuracy of the algorithm improves.  6 2. As the window length L increases with the same cluster radius R, the prediction accuracy increases. The optimal window length for a TS with a clear periodic structure is equal to or a multiple of the TS period. 3. The proposed algorithm can be used to predict: seasonal TS with a clearly defined periodic structure, with a given oscillation period; TSs are characterized by repetition (within the statistical error) of certain sequences of values, and in the general case, the sequences can be repeated after an interval of time unequal to a constant. The possibility of using an algorithm for predicting TS with non-equidistant moments of measuring TS values is most important, since virtually the entire classical mathematical apparatus of TS prediction is designed for discrete TS with equal intervals of measurement of the series.
The disadvantages of the algorithm include the absence of a collapsed parametric analytical description of the model, which complicates its interpretation. In addition, a sufficiently long observation period of the investigated TS is required for learning the algorithm, the accuracy of identifying the initial values of TS is low due to the small number of clusters.

Conclusion
The article provides a mathematical description and the results of a study of the time series prediction algorithm, based on the ideas of adaptive resonance theory.
The efficiency of the algorithm is illustrated by the example of analyzing the TS of free electricity prices. As a result of the identification, a model of the investigated TS was built, which has a high degree of adequacy to the initial data. The obtained results allow us to conclude about the prospects of using the algorithm for short-term forecasting of electricity prices for future periods.
The algorithm presented in the article can also be applied in any subject areas for the description and analysis of discrete time series with a periodic structure, and the moments of measurement of values of TS may not be equidistant. The absence of a tight binding of an algorithm to time is its most important property and significantly expands the scope of use of the algorithm.