Smart Adaptive Big Data Analysis with Advanced Deep Learning

2.1 Feature extraction

Feature extraction means dimension reduction in pattern recognition and image processing. In literature, feature extraction and selection methodologies include a wide variety of topics, e.g. classification is important in [39] The use of statistical features is specialized: arithmetic mean and standard deviation are used for process data, root-mean-square, kurtosis and peak values for signals.

Generalised moments and norms extend this analysis to a wide range of features [40]. The generalised norm is defined by

where the order of the moment p ∈ R is non-zero, and N is the number of data values obtained in each sample time τ. The norm (1) calculated for variables x_j , j = 1, n, have the same dimensions as the corresponding variables. The norm ||τMjp||pcan be used as a central tendency value if all values xj>0,i.e.||τMjp||p∈R.[41]. The norm can be extended to variables including negative values [31]. Variables x_j can be process measurements, peaks of spectra, measured waveform signals and sparse data.

2.2 Signal processing

Signal processing methods transform, combine or divide the waveform data, including sound, vibration, images or sensor data, and all these may have components from several sources. Blind source separation (BSS) methods are used in separating signals to find useful signals [42]. Subset selection techniques are in literature presented in wide meanings, e.g. feature selection techniques include modelling, optimisation and classification in [43] In high dimensional systems, a subset of variables is selected without altering the original representation of the variables [43]. Feature extraction transforms high dimensional data to lower dimensions by constructing combinations of variables. Wavelet decomposition is used for finding local features or compressing the data [44] Spectrum analysis, e.g. fast Fourier analysis (FFT), represents the signal in the frequency domain [45]

Filtering and smoothing are widely used for process data, but derivation and integration can reveal interesting features from waveforms. The calculation of the time domain signal x^(α)(t), which is based on a rigorous mathematic theory [46], is performed with three steps. The fast Fourier transform (FFT) is used for the displacement signal x(t) to obtain the complex components {X_k}, k =

0, 1, 2, . . , (N − 1). The corresponding components of the derivative x^(α)(t) are calculated by

where ω = 2πf , α ∈ R is the order of derivation. Finally, the resulting sequence is transformed with the inverse Fourier transform FFT⁻¹ to produce the derivative signal. The appropriate order of derivation is α − 2 for the acceleration signals. [41]

Generalised spectral norms are calculated for waveform signals from the frequency spectrum by

where {X_j}^(α) is the sequence of complex numbers, representing different frequency components of the signal {x_j}^(α) [46] This kind of norm can be used, to provide for information about the change in signal in a certain frequency range or frequency ranges [47].

2.3 Image processing

Digital image processing aims to enhance images or to extract some useful information from them. In big data analytics, algorithms are used to detect and isolate shapes from images or video streams. Concepts and techniques are discussed in [48] The hardware and software components need to be used together to facilitate an early detection of problems, e.g. product defects and changes in process operation. The online optical monitoring based on image analysis revealed useful information from the process and can be used in forecasting the quality of biologically treated wastewater [49]

2.4 Sparse measurements

Sparse condition monitoring measurements are analysed with the methods presented above for the waveform signals, the only difference is the sparsity of values. Laboratory analyses are based on sampling and can be frequent only if automatic sampling is used. These measurements may contain spectroscopy. Uncertainty fundamentally affects the decisions that are based upon the measurement result [50]

Maintenance and operation performance can be assessed with various measures: harmonised indicators are based on cost, time, man-hours, inventory value, work orders and cover of the criticality analysis, key performance indicators (KPIs) reflect the critical success factors and the goals, the overall equipment effectiveness (OEE) includes non-financial metrics for the manufacturing success. Reliability-centered maintenance (RCM) is based on statistical analyses and statistical process control (SPC) is used in monitoring a process through the control charts. [51] These indicators need interpolation and uncertainty handling (Fig. 2).

2.5 Nonlinear scaling

Nonlinear scaling brings various measurements and features to the same scale by using monotonously increasing scaling functions x_j = f (X_j) where x_j is the variable and X _j the corresponding scaled variable. The function f() consist of two second order polynomials, one for the negative values of X_j and one for the positive values, respectively. The corresponding inverse functions X_j = f⁻¹(x_j) based on square root functions are used for scaling to the range [-2, 2], denoted as linguistification. The monotonous functions allow scaling back to the real values by using the function f(). [28]

The parameters of the functions are extracted from measurements by using generalised norms and moments. The support area is defined by the minimum and maximum values of the variable, i.e. the support area is [min (x_j), max (x_j)] for each variable j, j = 1, m. The central tendency value, c_j, divides the support area into two parts, and the core area is defined by the central tendency values of the lower and the upper part, (c_l)_j and (c_h)_j, correspondingly. This means that the core area of the variable j defined by [(c_l)_j , (c_h)_j] is within the support area. The orders, p, corresponding to the corner points are chosen by using the generalised skewness,

The standard deviation σ_j is the norm (1) with the order p = 2. [30]

The scaling functions monotonous and increasing if the ratios,

are both in the range [13,3], see [29] The coefficients of the second order polynomials are represented by

whereΔcj−=cj−(cl)j and Δcj+=(ch)j−cj.

2.6 Uncertainty

The feasible range can be defined as a type-2 trapezoidal membership function since the norm values obtained from different time periods have differences, i.e. the parameters of the scaling functions can be represented as fuzzy numbers. A strong increase in uncertainty may demonstrate a change of operating conditions. The ratios αj− and αj−are calculated by applying the interval arithmetic on (5). The constraint range [13,3]must be taken into account before calculating the coefficients (6). Then the extension principle is used for calculating the scaled values as fuzzy numbers.

2.7 Natural language

The nonlinear scaling approach provides an unified solution for natural language interpretations since all the scaled variables are in the same range [-2, 2]. The integer numbers {-2, -1, 0, 1, 2} correspond labels {very low, low, normal, high, very high} or {high negative, negative, zero, positive, high positive}, for example, and represented as fuzzy numbers, which can be modified by fuzzy modifiers, which are used as intensifying or weakening adverbs. The resulting terms,

correspond to the powers of the membership in the powering modifiers (Table 1). The vocabulary can also be chosen in a different way, e.g. highly, fairly, quite [52] Only the sequence of the labels is important. Linguistic variables can be processed with the conjunction (and), disjunction (or) and negation (not). More examples can be found in [18].

3.1 Intelligent indices

The basic form of the intelligent index is a scaled feature or measurement but more indices can be developed as the weighted sums of several scaled features. In [30], the cavitation index of a Kaplan turbine was based on a single scaled feature and several faults of the supporting rolls of a lime kiln required two features. Linguistic principal components (LPCs), which extend the linear PCA by using the nonlinear scaling, are generalisations of this. Intelligent condition and stress indices provide an unified approach to use different measurements and features in condition monitoring [30].

3.2 Temporal analysis

Temporal analysis focused on important variables provides useful information, including trends, fluctuations and anomalies, for decisions on higher level recursive adaptation. Trend analysis produces useful indirect measurements for the early detection of changes. For any variable j, a trend indexIjT(k)is calculated from the scaled values X _j with a linguistic equation

which is based on the means obtained for a short and a long time period, defined by delays n _S and n _L, respectively. The index value is in the linguistic range [−2, 2], representing the strength of both decrease and increase of the variable x_j. [35, 53]

An increase is detected if the trend index exceed a threshold IjT(k)>ϵ1+. Correspondingly, IjT(k)<−ϵ1−for a decrease. The derivative of the index IjT(k),denoted as ΔIjT(k), is used for analysing the full set the triangular episodic representations (Fig. 3). Trends are linear if the derivative is close to zero: −ϵ2−<ΔIjT(k)<−ϵ2+. Area D close to [2, 2] and area B close to [−2, −2] are dangerous situations, which introduce warnings and alarms. Areas A and C mean that an unfavourable trend is stopping.

Figure 3

Triangular episodic representations defined by the index IjT(k)and the derivative ΔIjT(k).

Severity of the situation can be evaluated by a deviation index, which is a weighted sum of X_j(k), IjT(k) and ΔIjT(k). This index has its highest absolute values, when the difference to the set point is very large and is getting still larger with a fast increasing speed [53]. This can be understood as a third dimension in Fig. 3. The trends of the parameters αj−,αj−,Δcj−andΔcj+give useful information about changes of the scaling functions. The ranges of these parameter are [13,3],[13,3],[14,34](cj−min(xj)) and [14,34](max(xj)−cj), respectively. The changes in the case structure are seen in the trends of the interaction coefficients.

The trend analysis is tuned to applications by selecting the time periods n _L and n _S. Further fine-tuning can be done by adjusting the weight factors wjT1 and wjT2used for the indices IjT(k) and ΔIjT(k) The thresholds ϵ1+=ϵ1−=ϵ2+=ϵ2−=0.5. The calculations are done with numerical values and the results are represented in natural language.

The fluctuation indicators calculate the difference of the high and the low values of the measurement as a difference of two moving generalized norms:

where the orders p _h ∈ ℜand p _l ∈ ℜ are large positive and negative, respectively. The moments are calculated from the latest K_s + 1 values, and an average of several latest values of ΔxjF(k)is used as an indicator. [54]

3.3 Interactions

The basic form of the linguistic equation (LE) model is a static mapping in the same way as fuzzy set systems and neural networks, and therefore dynamic models will include several inputs and outputs originating from a single variable [28] Adaptation of the nonlinear scaling is the key part in the data-based LE modelling (Fig. 4). All variables can be analysed in parallel with the methodology described above and assessed with domain expertise. Interactions are analysed with linear modelling methodologies from the scaled data in the chosen time period. In large-scale systems, a huge number of alternatives need to be compared, e.g. in a paper machine application, 72 variables produced almost 15 million three to five variable combinations. Correlations and causalities based on domain expertise are needed to find feasible variable groups [56].

Figure 4

Data-based modelling with linguistic equations, modified from [55].

Fuzzy set systems and fuzzy arithmetics expand the application areas in following ways [57]:

• – LE models replace linear models in TS models;

• – Fuzzy calculus is applied in models by using LE models, fuzzy inputs and/or coefficients both in the antecedent and consequent part;

• – Use fuzzy inequalities in developing fuzzy facts for the fuzzy rule-based systems.

Domain expertise is important in these modules. Neural networks can represent very complex nonlinear interactions but only highly simplified models are needed when the nonlinear scaling is successfully defined. Modelling and simulation methodologies of complex systems are discussed in more details in [33] In decomposed systems, the composite local models consisting of partially overlapping models are handled by fuzzy logic [58].

Complexity is gradually increased with decomposition and higher level structure. Case based reasoning (CBR) integrates problem solving and learning in variety of domains [59] but does not prescribe any specific technology [60]. Therefore, it is a feasible methodology for integrating the overall system. Evolutionary computation provide efficient tools for all these systems since everything is defied by parameters.

3.4 Dynamic modelling

External dynamic models provide the dynamic behaviour for the LE models developed for a defined sampling interval in the same way as in various identification approaches discussed in [10]. Dynamic LE models use the parametric model structures, ARX, ARMAX, NARX etc., but the nonlinear scaling reduces the number of input and output signals needed for the modelling of nonlinear systems. For the default LE model, all the degrees of the polynomials become very low:

for the scaled variables Y and U.

Process phases can have totally different models with different variables. Also phenomenological models can be included in the overall system but their parameters need to be adapted to the operating condition by computational intelligence.

4.1 Recursive scaling

The parameter of the scaling functions can be recursively updated by using the norms (1) with the orders defined in the tuning. The norm values are updated by including new equal sized sub-blocks in calculations since the computation of the norms can be done from the norms obtained for the equal sized sub-blocks, i.e. the norm for several samples can be obtained as the norm of the norms of the individual samples:

where K_s is the number of samples {xj}i=1N. In automation and data collection systems, the sub-blocks are normally used for arithmetic mean (p = 1).

The parameters of the scaling functions can be recursively updated with by including new samples in calculations. The number of samples can be increasing or fixed with some forgetting or weighting [31]. The orders of the norms are redefined if the operating conditions change considerably. The new orders are obtained by using the generalised skewness (4) for the data extended with the data collected from the new situation. If the changes are drastic, the calculations are based on the new data only. The decision of starting the redefinition is fuzzy and the data selection is important.

4.2 Interactions

Linear regression and parametric models are used in the recursive tuning of the interaction equations. The set of equation alternatives (Fig. 4) is useful in the recursive analysis since the set is validated with domain expertise. The LE approach uses the preference sequence: scaling, shape of scaling functions and interaction equations, which is consistent with the stages of adaptive fuzzy control: first scaling, then the shape of membership functions and finally rulebase.

The interaction models are not changed if the scaling functions change only slightly. The coefficients are obtained by using the data collected from the chosen time period if the feasible range is changed. Uncertainties can be calculated by comparing the coefficients extracted from several short periods.

Considerably revised scaling functions may require updates for the interactions as well. However, the retuning is started only if the current equations do not operate sufficiently well. The earlier chosen set of alternative equations is used first. New equations are included if new variables become important. The selected variable groups (Fig. 4) are analysed first. Considerable changes in operating conditions mean that the full data-based analysis is needed. This level forms the model basis for the case-based reasoning (CBR), see [56].

4.3 Fuzzy logic

The recursive data analysis produces parameters for the scaling functions and interactions. Uncertainties of the parameters are obtained for any time period which containing several sub-blocks, i.e. the variables are represented by fuzzy numbers. Changes in operating conditions are detected by comparing the similarities of the original and modified fuzzy numbers. The detection is based on fuzzy inequalities <, 6, =, > and > between the new fuzzy parameters and the fuzzy parameters of the case. The resulting 5X5 matrix includes the degrees of membership of these five inequalities for five parameters. The results are interpreted with the natural language interface which provides an important channel in explaining the changes to the users.

4.4 Changes of operating conditions

Changes of the scaling functions and interaction coefficients are symptoms of changes in operation. The intelligent trend analysis provides early warning about changes in variable levels, fluctuations and uncertainty. All the variables and intelligent indices are represented in the same range [-2, 2], i.e. the same analysis and linguistic interpretation can be applied in all of them. The corresponding levels and their degrees of membership can be used in the fuzzy decision making.

The full analysis is needed fairly seldom although the process changes considerably. For example, new phenomena activate with time in wearing, but the models used in prognostics can be updated by expanding the scaling functions (Fig. 5). The generalised statistical process control (GSPC) introduced in [61] could give an early warning.

Figure 5

Recursive adaptation in prognostics [62].

Layer 1 - Connections

Variable specific features are extracted and specifications defined for process measurements (Section 2.1), waveform signals (Section 2.2), images and videos (Section 2.3), spectral data (Section 2.1) and sparse measurements (Section 2.4). The generalised norms are beneficial in providing similar settings for various applications, especially for waveform signals where Edge computing is needed for local calculations, especially for the waveform signals and image data.

Layer 2 - Conversion

Features are converted to the same scale by extracting the meanings with the variable specific nonlinear scaling (Section 2.5). The output includes feature specific uncertainties (Section 2.6) and the results are presented in natural language (Section 2.7). Recursive scaling is available (Section 4.1) and the temporal analysis provides information about trends (Fig. 3) and fluctuations (Section 3.2). This layer is the key to the advanced deep learning by providing a feasible solution to divide the conversion and cyber levels. The differences of scaling functions between operating areas can be used in selecting possible cases.

Layer 3 - Cyber

Interactions are analysed for the intelligent indices (Fig. 4): indicators may combine several indices (Section 3.1) and versatile interaction models can be developed by linear methodologies (Section 3.3). Dynamic structures are included if needed (Section 3.4). In practice, case-based solutions are important: local composite models can be sufficient but the interactions might also be highly different in different operating conditions. The need for additional cases is finalised in this layer. The overall system can be managed by case-based reasoning (CBR). The recursive tuning is the key to expanding the system (Section 4.1). The resulting intelligent analysers can be single scaled indices or combinations of them. Phenomenological models are important extensions of this layer.

Layer 4 - Cognition

Service solutions are developed by combining intelligent analysers and forecasting models in Monitoring. Changes of operating conditions (Fig. 5) are detected and the situation awareness is improved in the risk analysis. The Domain expertise is essential in Decision support systems. The natural language interface defined in the conversion level (Section 2.7) is important in this layer: all important features and indices are available in scaled and linguistic forms.

Layer 5 - Configuration

Automatic solutions for control and maintenance solutions are developed by combining intelligent analysers and control (Fig. 6). The controller can include many special control actions which are activated when needed [63]. Condition monitoring is the key to the improved Condition-based maintenance when real-time measurements are processed though the layers discussed above.

The advanced deep learning uses gradually refining layers: informative features are needed in the conversion layer which forms the basis for the intelligent analysers, and finally, the service and automation solutions combine these data-driven solutions with the domain expertise. The levels of Smart adaptive systems are further refined by the recursive analysis within these layers. The adaptation to a changing environment has two sub-levels: first updating the scaling functions (Layer 2) and then interactions (Layer 3) if needed. A short-term memory is needed for incremental or on-line learning, a long-term memory for recognising context drifting.

Similar settings are realised with the set of equation alternatives (Fig. 4). Successful past solutions and the idea of reasoning by analogy are used: the generalised norms and nonlinear scaling provide compact solutions for this level. The nonlinear scaling makes similar settings more widely available also in Cyber, Cognition and Configuration layers.

The adaptation to a new or unknown application includes the full data analysis and modelling (Fig. 4). In real applications, the constraint of starting from zero knowledge is modified to building new knowledge or, at least, improving the existing one.

The learning layers and the modular application structures together with edge computing are promising for cyber-physical systems: measurement technology, intelligent analysers, control and maintenance are realized as agents which are communicating through I²oT (Fig. 6). For a waveform signal, the local calculations reduces the amount of data with a factor 10⁵ even if several features are extracted. The analysis of images and videos results a number of features. Already this and conversion layer make cloud computing possible but in technical systems, the local calculations are preferred for the cyber level as well when known compact structures are used.

The modules of the smart adaptive data analysis have been developed and tested in various applications, including monitoring [53, 61], control [31, 63], diagnostics [56], condition monitoring and maintenance [57, 64, 65], and management [51, 66]. The basis of the calculation chain was introduced in MMEA, see [6]. The local calculations and integration of systems in collaborative automation have been discussed in [8]

Many artificial intelligence approaches, especially neural computing, rely highly on unsupervised methods and simultaneous processing of massive datasets. The hierarchical deep learning ideas, which improve the solution, have been developed for heterogeneous systems, which include also unstructured data. This type of methodologies do not utilize the domain expertise and known operational information. Layers 1 and 2 are needed, especially for the more structured data. Artificial intelligence can be useful in finding possible interactions in the Cyber layer but the results must be assessed through the advanced analysis presented above.

Connections

Generalised norms operate for extracting features from process measurements, peaks of spectra, measured waveform signals and sparse data. Measured waveform signals can be transformed, combined and divided before the feature extraction. Image processing is used for detecting and isolating shapes from images and videos.

Conversion

Generalised norms and moments are used in the data-driven tuning of the monotonously increasing scaling functions. The nonlinear scaling is the key approach of the advanced deep learning since it extracts the meanings of the feature levels and opens new possibilities for temporal analysis and uncertainty estimation. The parametric definitions allow recursive analysis for all these.

Cyber

Interactions between the scaled values can be analysed with linear methodologies: versatile indicators can be constructed as the weighted sums of indices. The compact models also allow case-based systems expanding through recursive adaptation. Local composite models and dynamic structures extend solutions to intelligent analysers further. Neural deep learning can be a part of this level.

Cognition

Domain expertise is used in combining service solutions and intelligent analysers obtained from features, indices and models. Resulting systems can be used in monitoring and decision support.

Configuration

Automatic solutions for control and maintenance utilize services for varying operating conditions in large scale complex systems. Trade-off between solutions is handled with fuzzy logic.