Advances in Artificial Intelligence Methods Applications in Industrial Control Systems: Towards Cognitive Self-Optimizing Manufacturing Systems

Abstract

Industrial control systems play a central role in today’s manufacturing systems. Ongoing trends towards more flexibility and sustainability, while maintaining and improving production capacities and productivity, increase the complexity of production systems drastically. To cope with these challenges, advanced control algorithms and further developments are required. In recent years, developments in Artificial Intelligence (AI)-based methods have gained significantly attention and relevance in research and the industry for future industrial control systems. AI-based approaches are increasingly explored at various industrial control systems levels ranging from single automation devices to the real-time control of complex machines, production processes and overall factories supervision and optimization. Thereby, AI solutions are exploited with reference to different industrial control applications from sensor fusion methods to novel model predictive control techniques, from self-optimizing machines to collaborative robots, from factory adaptive automation systems to production supervisory control systems. The aim of the present perspective paper is to provide an overview of novel applications of AI methods to industrial control systems on different levels, so as to improve the production systems’ self-learning capacities, their overall performance, the related process and product quality, the optimal use of resources and the industrial systems safety, and resilience to varying boundary conditions and production requests. Finally, major open challenges and future perspectives are addressed.

1. Introduction

Currently, manufacturing companies have to deal with many challenges in order to remain competitive within the rapidly changing market dynamics and framework conditions, including socio-economic, environmental and cultural aspects.

Short delivery times and customization along with comparable production costs and high-quality parts are order-winning properties and therefore vital for business. Additionally, accelerated product developments and industrializations are important attributes to quickly cover market developments with brief time to markets. Moreover, upcoming regulations as well as growing social and environmental requirements represent additional challenges to be faced by the industry. Key enabling factors to face such challenges are production machines, whose developments are crucial to meet the cited and constantly growing demands of modern manufacturing companies.

To keep pace with these ongoing evolutions and master upcoming challenges, industrial control systems are a key factor for advanced production machines and industrial plants [1]. Closed loop and real time control solutions are already widely used and are well established on various levels of production systems. However, with increasing complexity of the manufacturing systems and control requirements, further developments of control algorithms are indispensable to address the demand for future manufacturing systems.

Promising approaches for improved industrial control systems are based on recent developments in artificial intelligence (AI). AI has obtained wide popularity in the past years and is one of the most pursued topics in research and industry. Progress in computing hardware and software has facilitated the collection of data and its exploration with AI for various use cases. Initiatives such as Industry 4.0 have further highlighted the potential of AI-based methods and have pushed them to the centre of attention in many fields [2]. A report from McKinsey & Company states that AI could contribute with a value of USD 3.6–6.6 trillion annually in the supply-chain management and manufacturing sector [3].

Especially in the area of manufacturing, AI-based developments further expand the capabilities and pave the way for more advanced industrial control systems. Current applications and trends are shown in the following section for various levels.

In addition to existing works about applications of AI or machine learning in manufacturing and Industry 4.0 [4,5,6,7], this work provides a perspective overview of the current state of the art, recent developments and future challenges for each manufacturing process and control level.

Therefore, this paper is structured following a hierarchical and functional bottom-up approach: starting from the processing and elaboration of sensor signals to the job scheduling operations of the whole production plant. Thereby, the focus lies on identifying and illustrating most promising artificial intelligence-based methods coping with current challenges at each industrial control and automation level, including the related data acquisition, process modelling, real-time control and task scheduling operations. Due to the plethora of existing algorithms, the aim is not to present a complete overview of existing approaches and methods, but to highlight the most promising ones and the potential of their future development and applications to face modern industrial control and automation challenges, by means of emerging AI-based methodologies. In particular, the key role of the integration of AI methods in industrial control solutions is to strive for future self-optimizing production systems. For the successful implementation of self-optimizing machines, all components over all hierarchies must be optimally designed and operational. Figure 1 schematically illustrates the main element of self-optimizing production systems and forms the basis for the focus of the first sections in the present review and survey paper. Starting from a controller dedicated to a set of well-defined process variables, multiple aspects must be considered to enable an accurate and reliable control that is elementary for subsequent higher-level control structures. The first step of each controller is the direct or indirect measurement of the process variable under consideration. In order to obtain the most relevant information and controller inputs, sensor signals must be processed and elaborated accordingly. Since amount, type and quality of sensor signals can vary significantly, different methods are required to process them in an appropriate manner. In particular, trends towards the utilization of multiple sensors and the consequently increasing data availability require new developments towards suitable approaches for sensor and data fusion. Related current approaches and AI-based methods are discussed in Section 2.

A further important element within optimized controllers and manufacturing applications are process models. Depending on the use case, these process models can be generated conventionally with multi-physics simulations as finite element methods or more recently, data-driven approaches enable the generation of process models based on the previously acquired data. However, not all process models are suitable for control applications. An overview of available approaches and recent developments concerning AI-based process models is given in Section 3. By processing the measurement values and querying process models, the optimizer determines the optimal controller output. Depending on the application, these processes must be performed very fast or even in real time and with as much autonomy as possible, requiring intelligent and specialized solutions. Therefore, promising AI-based methods to implement effective optimizers are presented in Section 4.

A further concatenation of multiple optimal controllers enables an advanced control strategy over all hierarchical process levels. The main elements of self-optimizing machines are their capability to autonomously adapt to changing requirements and circumstances. Therefore, the task scheduler coupled with multiple cascaded control chains allows one to redefine and modify control objectives of control chains towards higher-level optimal objectives. Thereby, an example for the cascaded control chains could be the process control of an additive manufacturing process, where the first controller guarantees a constant energy density during the deposition process and the second controller acts layerwise by modulating the process parameter for the subsequent layer to obtain an equal deposition height. In an exemplary additive manufacturing process, the two control loops could represent the regulation of two laser sources, simultaneously building various features of a more complex part. A simplified overview of a self-optimizing framework is displayed in Figure 2.

AI methods can enhance the self-optimization capacity of a cognitive industrial control system at different hierarchical and functional levels. The combination of models and real-time data, as well as of real-time control and job scheduling functionalities, allows for the development of self-optimizing industrial machines and systems with new elevated degree of autonomy. Current methods and developments of job scheduling functionalities are described in Section 5, whereas approaches and frameworks towards cognitive self-optimizing machines are outlined in Section 6.

2. Methods for Industrial Data Fusion at Sensor, Feature and Decision Level

Efficiency, repeatability, robustness, and quality of manufacturing processes are of great importance in today’s production sites. Monitoring systems enable to capture parameters and system states in real time, allowing to effectively control manufacturing processes. The quality and performance of these automation systems stand and fall with the reliability and precision of available information. As a result, robust and accurate process measurements are of crucial relevance to satisfy higher-quality requirements.

A common approach to tackle these issues is a fast developing and huge research field called sensor fusion. The scope of data fusion, also known as sensor fusion, multi-sensor data fusion, multi-sensor integration, data combination or data aggregation is to obtain improved information with higher accuracy by merging multiple information sources. It fosters the robustness, reliability and confidence of the analysed information and improves the temporal and spatial resolution [8]. Initial developments of sensor-fusion-based systems go back to the military industry, where the authentication and tracking of dynamic objects is of crucial importance. Recently, further sectors such as the automotive sector experience a strong increase in sensor fusion for autonomous vehicles to accurately map their environment.

There have been multiple attempts to define a definition and framework for sensor data fusion. One of the most popular definitions has been presented by the Joint Directors of Laboratories (JDL) [9] describing sensor fusion as a multi-level process handling the association, correlation and combination of data and information from single and multiple sources. This section focuses on the application of AI-based algorithms for industrial data fusion. For a deeper insight in further data fusion architectures, the reader is referred to [10,11,12].

To capitalize on the benefits of sensor data fusion, some challenges must be overcome for a successful implementation. There are various data properties affecting the feasibility and success of the data fusion process. The main challenges of data fusion arise from the heterogeneity of data sources and individual imperfections of each data source such as uncertainty, imprecision or granularity. An overview of typical data fusion issues is displayed in Figure 3.

Depending on the considered data type and its fusion issues, suitable fusion algorithms must be selected. Today’s available algorithms only address a subset of these aspects. Due to the multitude of applications and available algorithms, the following section does not claim to be complete, but identifies promising methodologies for applications in industrial control solutions. Furthermore, there are many ways to classify fusion algorithms. Meng et al. [13] presented the algorithms according to their abstraction level as signal level, feature level and decision level, whereas Sasiadek et al. [14] divides fusion approaches in probabilistic models, least-square techniques and intelligent fusion.

On a signal level, the most intuitive and easiest way to increase the signal-to-noise ratio of redundant sources is to apply the weighted average or the least squares method on captured data. A more sophisticated and widespread method is the Kalman Filter. The Kalman Filter considers the statistical characteristics of the measurements while still allowing for real-time processing of the data. Kalman filtering is a discrete and time invariant approach and only holds for linear models. Unfortunately, practical problems often have strong non-linear relations. To overcome these limitations, Extended Kalman Filter (EKF) [15], Unscented Kalman Filter (UKF) or the Particle Filter (PF) [16] also known as Sequential Monte Carlo (SMC) have been developed to handle non-linear relationships. A drawback of the latter method is the increasing computational effort and rising amount of required data.

Regarding the feature level, unsupervised learning algorithms such as K-means clustering or self-organizing maps can be used to classify relationships between entities and extract further information. These methods are particularly suitable for applications where patterns or geometrical relations need to be detected. Further methods to address pattern recognition problems are adaptive resonance theory (ART), ARTMAP or Fuzzy ART.

For applications with limited bandwidth, Challa et al. [17] applied support vector machines (SVM) to compress the information in sensor fusion for large amounts of samples. Further algorithms and their applications are presented by Fung et al. [18]. Depending on the use case and its requirements, the suitability of the data fusion algorithm must be evaluated individually. Thereby, the target application defines the required accuracy, computational complexity, processing power and available amount of data.

King et al. [16] introduced a classification depending on the degree of complexity. Thereby, low-complexity algorithms such as k-means allow for an enhanced battery life for wearable health monitoring systems due to their reduced computational complexity and resource requirements.

In

Table 1. Overview of data fusion algorithms at sensor, feature and decision level.

Abstraction Level	Method	Applications/References
Sensor Level	Weighted average	[19]
	Kalman Filter (EKF, UKF)	[20,21]
	Particle Filtering	[22,23]
	Dempster-Shafer method	[24,25]
Feature level	k-nearest neighbour (k-NN), k-means	[26]
	Decision Trees	[27]
	Support vector machines (SVM)	[28]
	Artificial neural networks (ANN)	[29,30,31]
	Gaussian mixture model (GMM)	[32,33]
Decision level	Bayesian inference	[34,35]
	Fuzzy logic	[36,37]

, the presented algorithms are distinguished regarding their application or abstraction level as sensor fusion, feature fusion and decision fusion. In general, there is a wide range of possible data fusion applications.

Typical examples of the military industry are the location and tracking of moving objects. Depending on the target, multiple sensor platforms are used, and various sources as acoustic signals or electromagnetic radiation can be observed [38]. More recent applications are in the health care [39] or telecommunication sectors as wireless sensor networks. The widespread machine learning algorithms such as ANN, SVM or GMM lead to an increased adoption in sensor fusion approaches and enhance the possibilities of sensor fusion.

However, the efficient handling of big amounts of data in industrial applications remains an important challenge for the future. Approaches called consensus filters try to address this issue by implementing distributed filters to create scalable algorithms [40,41]. More recent developments were published in the automotive sector, where fast developments are noticeable. Future self-driving or autonomous vehicles require reliable information about their environment and are therefore predestined for sensor fusion [42,43,44]. Wei et al. [45] fusioned LiDAR and a vision system to reduce the false positive detection of the LiDAR system utilizing neural networks to project the camera to the LiDAR space. A more detailed review in this sector is presented by Wang et al. [42] and Alatise et al. [44].

These developments will probably lead to further adoption and improvements in manufacturing industries. Besides data-related issues, the interaction and data fusion of human inputs and machine data will play an important role in the future and requires more research to be developed in the near future.

3. Empirical Data Driven Methods for Production Process Modelling

Production process modelling enables the development of stable and repeatable process models, which are of crucial relevance for manufacturing companies. Production processes can be viewed from different perspectives ranging from the overall production process chain including raw material delivery, manufacturing steps and the final stock exploitation with the final product to simulations of the microscopic material behaviour during a manufacturing step [46]. This section focuses on the modelling of manufacturing processes for control purposes. The following sections consider high-level perspectives accounting for multiple manufacturing machines and systems.

In the domain of manufacturing processes, as for example additive manufacturing, models can be classified in analytical, numerical or empirical approaches [47]. Analytical models consist of the mathematical analysis of a physical problem described by partial differential equations (PDE) and given boundary conditions. Given the assumptions made, the obtained solution accurately represents the reflected domain. However, the process to determine analytical solutions is not trivial, and for complex processes, advanced numerical methods and demanding computational efforts are necessary to solve the considered physical equations.

Numerical methods are typically finite element method (FEM), boundary element method (BEM), finite difference method (FDM) or finite volume method (FVM). The solving of PDE is an essential element for multi-physics simulations enabling the simulation of complex manufacturing processes. Offline process simulations allow to perform parameter or design engineering without performing costly experiments and are therefore widely spread in R&D departments. An important limitation of these methods is the computational burden for complex processes including multiple physical phenomena. Furthermore, the computational effort scales drastically with increasing simulation domain sizes. Besides computational drawbacks, setting up a model is not trivial and requires highly skilled employees to define model parameters. The model definition and its assumptions are crucial to obtain representative results for a predefined process window.

In certain applications, it can be more suitable to create empirical models by performing a series of experiments and analysing the observations. Thus, relations between process inputs and obtained measurements can be captured without knowing the detail about the physical phenomena. This method is in particular suitable for low-cost experiments, where numerical or analytical models would require more resources. To approximate the captured observations in terms of an exploitable model, various approaches exist. One of the most straightforward methods in statistical analysis is linear regression, where a line is fitted to capture the relation between two variables. For more complex non-linear applications, machine learning based methods as neural networks (NN) have experienced a widespread adoption in recent years [48,49,50,51,52,53]. Due to the increased availability of computational resources, the increasing development of machine learning libraries and the possibility to approximate highly non-linear relationships lead to many applications in industry, as for example machining operations and additive manufacturing [54]. The approximation capacities can be customized by varying NN hyperparameters as the number of hidden layers or the number of neurons of each layer. A further advantage of the NN-based model is the possibility to apply the model for similar processes. Therefore, only few additional data points are required to fine tune the already trained model by adapting the model to the changed conditions [55]. However, the main drawbacks of NN are the required amount of data to train the network and the reliability of the predictions. The structure of NNs makes it difficult to interpret and predict the behaviour, in particular for unseen data points.

Recent developments combine the advantages of physics-based simulations and neural networks by including the physical knowledge in the training of the neural network [56,57,58,59]. In contrast to conventional neural networks, physics-informed neural networks contain additional loss terms. Besides the prediction error, terms satisfying the partial differential equation of the simulated domain penalize solutions disagreeing with the physical equations. As a result, less data are required to train and optimize the weights of the network, and predictions are more stable. Moreover, the additional loss terms shrink the solution space and prevent overfitting of the model. For data scare applications, this method is very promising but needs further research for industrial applications. In particular, the scalability needs to be investigated since the additional computations for each iteration and the coordinates dependent on PDE are potentially limiting factors for an adoption on larger part dimensions.

With a similar objective, but without directly implementing the physics of the considered process, a recent method has been presented by Li et al. [60]. The authors presented the Fourier neural operator (FNO) to learn the underlying multiple partial differential equations (PDE) by mapping from a functional parametric dependence to the solution. The proposed architecture method appears to be significantly faster than traditional PDE solvers. To train the FNO, a sequence of timeframes of a physical phenomenon is given as input to predict the following time frames. The inputs are processed by transforming the input into the frequency space, filtering the higher frequency modes and transforming the filtered data back to the time space; furthermore, the inputs are multiplied with weights and summed up with the filtered results. Subsequently, the results are passed through an activation function, and the described layer is repeated multiple times. The advantages of the method are the mesh invariance and the capability of learning the relevant physics behind the process. Therefore, this method could be a promising approach for the modelling of manufacturing processes. First adoptions for solving the PDE for two-phase flow fields have been demonstrated by Zhang et al. [61]. However, the transfer from theoretical physical problems to empirical data is still missing.

Besides AI-based algorithms inspired by the structure of NN, probabilistic approaches as Bayesian Optimization (BO) have also been proven to be suitable for data scare applications. Due to the probabilistic nature of BO, probabilities and uncertainties of the prediction are computed. These values enable the development of data sampling methods, minimizing the uncertainty of the predictions. As a result, less experimental data are required to obtain a model of the manufacturing process. Maier et al. [62] successfully applied this method to optimize grinding parameters after a few trials for an unknown workpiece and tool combination.

Table 2. Overview of sensor fusion algorithms.

Modelling Type	Modelling Methods	Applications
Analytical	Analytical	Well-known and simpler models
Numerical	FEM, FVM	Complex processes, limited domain size and computational expensive
Empirical/Data-driven	Statistical analysis, e.g., linear regression	Well-known, low computational cost, for less complex problems
	Neural network	Mapping of nearly arbitrary relations, with sufficient experimental data
	Gaussian processes	Process parameter optimization, difficulties in handling high-dimensional problems, small amounts of available data
	Physics-Informed Neural Networks	Process modelling, small amounts of data
	Fourier Neural Operator	Mesh invariant modelling methods. Novel approach needs to be tested for real use cases

summarizes the mentioned process modelling methods.

However, a general challenge of process simulations remains the lack of generalizability of models and the increased computational burden for complex processes. Simulation domains are often tiny volumes and only represent a small element of the manufacturing step. Previously mentioned recent developments could be promising to handle both larger simulation domains and complex processes. Particularly, data-driven methods have recently shown great potential. Further research regarding the explainability and generalizability of data-driven methods must be carried out to enable an imminent transition to industry.

4. Machine Learning Applications for Production Machines Real Time Control

Real-time control of manufacturing machines is a further essential element of production machines to achieve repeatable and stable processes with high-quality parts. Applications range from accurate positioning of machine axes to adaptive control of process parameters as the laser power during laser welding. Today’s industrial control systems are mainly based on highly consolidated solutions, as for example PID controllers for axis control. The simple structure, ease of implementation and long experience in parameter tuning lead to an established technology for control applications. However, there were many further developments in the controller design and automated parameter tuning [63]. AI-based methods have been increasingly adopted in recent years to optimize control parameters tuning or to develop new types of control methods.

In general, machine learning methods can be divided into supervised, unsupervised and reinforcement learning (RL) [64]. Supervised learning trains for example an NN by reducing the mean squared error between predictions and the corresponding labelled solution. In unsupervised learning, no information about the ground truth is available. Unsupervised learning often consists in clustering or reducing data dimensionality by grouping correlated data points. In contrast to supervised and unsupervised learning, RL forms a semi-supervised learning approach, where an agent interacts with the environment and a reward is given for performed actions. The algorithms explore the environment by executing actions and receiving rewards for the actions and decisions taken. The optimal execution order of the actions, the policy, is not known and will be determined during the learning process. A good introduction into Deep RL is given by Mousavi et al. [65]. RL can be further subdivided into dynamic programming, Monte Carlo methods and temporal difference methods [66].

Regarding PID parameter tuning, RL has been successfully applied to find suitable controller parameters. Besides traditional approaches such as trial and error or Ziegler–Nichols [67], Wang et al. [68] utilized an actor critic RL approach to determine the PID parameters. The results demonstrated the algorithm’s ability to track non-linear systems while ensuring adaptability and robustness. Besides parameter tuning, RL also experiences an increased adoption within process control applications.

Table 3. Overview of common and new control algorithms.

Type	Control Method
Traditional	PID, fractional PID
Fuzzy Logic	Fuzzy controller
Reinforcement Learning	Temporal difference (TD), SARSA (State–Action–Reward–State–Action)
	Soft Actor–Critic (SAC) or Asynchronous Advantage Actor–Critic (A3C)
	Q-learning, Double Q-learning

lists the currently most adopted RL algorithms for control applications.

Nian et al. [66], Shin et al. [63] and Lee et al. [69] recently published reviews of machine learning and process control methodologies. In particular, Shin et al. [63] compared reinforcement learning with model predictive control algorithms. For complex and dynamical systems, model predictive controllers (MPC) represent an effective alternative to PID controllers. Due to their ability to explicitly consider constraints and predictively optimize the controller output, MPC solutions are already well established in the industry. Important for the applicability of MPC are sufficient slow system dynamics so as to execute the model online during every sampling step. Reinforcement learning for process control appears to be a promising method to reduce online computation times. The m ain differences lie in the modelling approach, where RL algorithms learn the model through trial and error, whereas MPC requires a model developed with first principles or process identifications. Furthermore, the online execution time of model-free RL is faster than MPC, since only a forward run of the policy network is required, whereas the MPC requires solving an optimization problem at each execution. A general comparison between deep RL and MPC for optimal control is given by Lin et al. [70] and is displayed in

Table 4. Comparison of deep reinforcement learning and model predictive control for optimal control [72].

Comparison Criteria	Reinforcement Learning	MPC
Solution form	Neural networks	Model based optimization
Online computation time	Low	High, especially for high-degree non-linear systems
Optimality-seeking capability	Near-optimal	Near-optimal with long prediction horizons
Generalization issue of machine learning	Reduced optimality	N/A
Handling of hard constraints	Under development	Yes
Handling of modelling errors, control delays, and/or disturbances	Better with larger errors	Worse if not robust MPC

. More details about the comparison of MPC and RL can be found in [70,71].

Due to the possibility of offline training, reinforcement learning is suitable for real-time control. Famous results of reinforcement learning were achieved by DeepMind beating human players in games such as Go [72]. However, these kinds of simulated environments are perfectly known and are therefore particularly suitable for the learning of reinforcement algorithms. In contrast to simulated environments, industrial processes suffer from high levels of imprecision and uncertainty (see sensor fusion; Section 2). The training requires a lot of data and interactions with the environment, which can be performed by interacting with a model. However, accurate models covering the entire manufacturing process over all parameter combinations are extremely complex, and experiments are costly. As discussed in Section 3, AI-based modelling approaches could therefore be suitable to model the process and serve as an environment for the RL algorithm. In contrast to traditional control approaches, there are no stability guarantees for RL controllers. Compared to traditional control methods, the development of RL control algorithms is still in an early stage. Nevertheless, the general learning capability of RL is promising for the future. Furthermore, research is needed to search for methods to improve stability and convergence of RL algorithms. For further applications as control approaches, safe RL will be, in the future, an important topic to enhance the reliability of RL-based controllers [73,74].

5. Real-Time Scheduling Methods for Flexible Manufacturing Systems

Trends towards customized products and shorter delivery times lead to a major challenge for manufacturing factories automation and management systems. The increase in product variants introduces significant non-production times due to system reconfiguration and set up needs at each product changeover. Besides internal production uncertainties, external factors such as a varying customer demand, supply chain ruptures or even raw material price dynamics increase the complexity level of the manufacturing configuration, planning and scheduling tasks [75]. To cope with these challenges, today’s manufacturing companies need not only high-performance and flexible processes but also robust concepts to handle foreseen variances and unforeseen disturbances with minimal impact and small additional production costs [76,77]. Approaches to raise the production flexibility can range from improvements and development of adaptive work cells with self-adaptive control functions [78,79] to optimized production planning and scheduling solutions [80].

The mapping and optimization of these increasingly complex production processes is an important field of research. Recent works underline the relevance of production scheduling and show a tendency towards the following topics in the last years [81]:

• Self-organization of resources;
• Self-regulation of the production process;
• Self-learning capacities of the overall manufacturing industrial system.

These topics highlight the importance of autonomous and self-optimizing scheduling approaches to handle the overall growing production dynamics. Frequently applied scheduling optimization objectives are the maximization of machine utilization and the minimization of the lead time to produce a product [82]. Additionally, an increased sensibility and awareness of the climatic impact lead to further requirements such as the minimization of utilized resources and the reduced emission of greenhouse gases.

Once the scheduling objectives are defined, various approaches exist to implement a job shop scheduling policy. A general classification divides them into centralized, distributed or more recent hybrid scheduling and control architectures (HCA) [83]. These hybrid structures aim to combine the advantages of both approaches, while reducing their drawbacks. State-of-the-art architectures are multi-agent systems (MAS), holonic manufacturing systems (HMS) or product-driven systems (PDS) [83]. Another classification presented by Zhang et al. [84] groups the job shop scheduling models by the problem space in the following five types: basic type job shop scheduling problem (JSP), flexible JSP, multi-resources FJSP model (MrFJSP), multi-plants-based MrFJSP (MpFJSP) and MpFJSP with smart factory.

Independently of the scheduling architecture or model, dynamic scheduling approaches are required to cover and react on production events in real time. Events can be classified in resource- or job-related events [85]. Thereby resource-related events include process-linked incidents, as for example machine breakdowns or material shortages. Examples for job-related events are job cancellation or a change in priorities. Handling these dynamic and unforeseen events, while considering constraints and variations of each production process, is a key challenge of industrial production scheduling solutions. To cope with the challenge, accurate production models are required to obtain realistic results. However, with increasing model complexity, computation times and resulting optimization times increase significantly, limiting the real-time application potential of advanced scheduling methods. To overcome these issues, new technologies and algorithms are required to model and optimize the industrial operation scheduling problem.

Promising approaches are dynamic HCA with self-adaptive mechanisms to improve the agility and reactivity [86]. Cardin et al. [75] presented an optimized reactive control architecture (ORCA) consisting of a global and local control layer. Important is the distinction between normal and disrupted mode. In the case of a local perturbation, the local optimizer triggers the switch to disrupted mode, where the local optimizer controls the entity without the input of the global control layer. As a result, the system is able to handle reactive and local disturbances, while maintaining a global optimal solution at the start of each time window. The overall architecture is shown in Figure 4, illustrating the different control levels and the disrupted mode.

However, a crucial element for effective industrial scheduling solutions is an algorithm that optimizes the objective functions in dynamic rescheduling operations. An overview of scheduling algorithms is shown in Figure 5 [84]. With increasing complexity, approximate methods are more suitable to optimize the production scheduling problem.

In recent years, AI-based methods have received increasing attention and have been more commonly applied to support the scheduling process in high-performing industrial applications. Predictive maintenance enables to predict and even anticipate production disturbances. The research in this field has increased drastically due to the exponentially increasing amount of available production data and the adoption of AI-based techniques [87,88]. Developments in this area are especially helpful for scheduling solutions to anticipate disturbances and future changes, so as to proactively adapt the scheduling solution. Within the developments under Industry 4.0, new frameworks such as the MpFJSP with smart factory are presented [84]. Thereby the actual job shop situation can be measured in real time by various sensors and identification technologies. As a result, the actual situation can be compared to the planned scheduling solution. AI-based methods such as deep learning technologies enable to extract information and learn from the production processes autonomously to provide the necessary basis for decision making and new scheduling solutions identification.

Priore et al. [89] presented an approach to predict the suitable decision rule with machine learning based methods. Thereby, ensemble methods have been successfully applied to exploit the prediction of multiple algorithms and increase the prediction accuracy. Another work using ensemble methods has been presented by Lubosch et al. [90]. Thereby, a Monte Carlo Tree Search was used to develop a flexible scheduling algorithm. To achieve a good tradeoff between exploration and exploitation, the algorithms’ hyper parameters are tuned with a Gradient Boosted Decision Tree (GBDT), avoiding a manual parameter tuning. Recently, reinforcement learning-based production scheduling systems have gained increasing attention in research [91]. Thereby a clear research trend towards more reliability-aware algorithms can be identified.

Besides further research in modelling and optimization algorithms, human aspects need to be considered in future adaptive automation solutions as well [92]. In particular, human–robot collaboration is a promising approach for higher productivity and product quality [93,94]. Worker-aware adaptive solutions enable the possibility to harmonize human well-being to obtain a synergic human-automation solution, including the related job scheduling aspects. Bettoni et al. [95] and Valente et al. [96] demonstrated an approach of adaptive huma–machine collaboration, enabling an effective reduction of the physical and mental stress of the operator. The monitoring of physiological parameters and classification with random forest algorithms allow for estimating the state of the operator and adapting the behaviour of the human-aware manufacturing systems, also considering the system evolution over time, as outlined in Bracco et al. [97].

6. Emerging Cognitive Approaches for Self-Optimizing Machines

The key idea behind self-optimizing machines (SOM) or self-optimizing production systems (SOPS) is to enable industrial production systems to produce customized high-quality parts while satisfying productivity demands [98]. Therefore, highly autonomous production machines are required to maximize the counteracting targets of efficient production and customization. Enhanced autonomy enables the system to produce complex parts, while handling and reacting to unforeseen events autonomously without any intervention of external operators [99].

Adapting the machine state automatically towards the objectives requires increased decision-making capabilities [100]. Thereby, the decision-making process can be compared to the human decision-making requirements [101]. These cognitive processes consist of the perception of the environment, its interpretation, the crosslink with existing knowledge and the subsequent decision with a coupled action. The general cognitive process is displayed in Figure 6 and illustrates the high-level control strategy.

The illustrated control structure includes the elements of each control loop. Cascading these process control loops forms a fundamental element to achieve this elevated degree of autonomy. These process control loops enable the SOM to react on changing objectives and subsequently adapt the machine state levelwise top–down. The cascaded implementation allows one to act on various influences on different impact levels, ranging from high-level product modifications to microscopic tool wear during the manufacturing process. Figure 7 visualizes a cascaded self-optimization loop for an additive manufacturing process. Thereby, the control loops are linked top–down to consider production objectives within the process control and act on each process level. Based on the Viable System model of Stafford Beer, Permin et al. [102] presented a concept of the Viable System model, including different levels of production and production management. Thereby, objectives are divided into normative production management, strategic production management and tactical production management.

A main element of the SOM and its cascaded process loops is the exploitation of multiple information sources to capture and control the machine state. Moreover, the link between all processing elements allows for the process control over multiple levels. Thereby, process models as described in Section 3 support the control process by predicting target parameters or by optimizing control outputs with model-based controllers. Thombandsen et al. [103] presented a framework of a model-based self-optimizing manufacturing machine. Within this concept, accurate observations and the understanding of their relation to the product quality are crucial prerequisites for self-optimizing machines. Based on sensor signals, machine states and operating points are identified, and control parameters are computed by a model-based optimization.

A more higher-level perspective of a SOM is presented by Möhring et al. [104] and is displayed according to the framework in Figure 8. Within this concept, the process starts with generation of the trajectory for the imported part to manufacture. During this process, the planning process considers already processed and machine properties to optimize the toolpath. Further optimizations of the settings are performed by modelling and simulation of the process and the machine. These models enable the prediction of various product target properties, as for example the surface quality or part geometry. Afterwards, the actual machining process is performed, and monitoring solutions record the real-time process behaviour and interact with real-time controllers. The actual process measurements allow one to automatically update and improve the process model. Furthermore, the inline measurements form the input of real-time controllers to perform accurate control tasks, such as axis positioning. Considering the machine characteristics and measured process deviations, real-time controllers are constantly refined. Further details and examples can be found in Möhring et al. [104].

Essential prerequisites for SOM are developments in data fusion, process modelling and real-time control mentioned in the previous sections. In particular, the implementation of suitable measurement devices is crucial for the subsequent process steps. Only if the machine state is captured properly by accurate and robust measurements can valid process models be generated and subsequent optimal actions selected. Key enabling technologies as artificial intelligence and machine learning are promising approaches to process and analyse sensor data. These techniques are necessary to build cascaded process control loops based on comprehensive process observations, accurate process models and the knowledge about quality-influencing factors. Even if there have been promising advancements in various applications, there is still a lack in industrial implementations of comprehensive SOM solutions. A general overview of AI applications in manufacturing is given by Monostori [105]. Mayr et al. [106] presented an adaptive self-learning control approach to compensate for thermal errors on five-axis machine tools. Based on data captured during the process, a thermal autoregressive with exogenous input (ARX) model is identified and updated during the process to reduce thermal errors. Dittrich et al. [107] presented a self-optimizing process planning approach for five-axis milling to automatically compensate for tool deflections. Combining a simulation model with support vector regression enabled the prediction of shape errors. The subsequent automated toolpath adaption resulted in a shape error reduction by 50%.

The overall promising benefits of SOM are becoming more and more realistic with recent progresses in computational power, sensor techniques and AI algorithms. Autonomous selection of data sampling points enables an efficient generation of process models [64]. Based on these models, integrated optimization processes pave the way for autonomous self-tuning of parameters to guarantee a constant product quality.

To enable further widespread in industrial applications, faster and data-scarce simulation and analysis methods will be needed. AI algorithms are promising techniques to generate fast models and to handle process uncertainty, while updating the model during manufacturing processes. However, these approaches suffer from low data efficiency and reliability of its predictions and are therefore still critical regarding safety requirements.

Further challenges are the process dynamics, requiring fast optimized control values to control the process. Further developments are needed to cope with the complexity and dimensionality of the optimization problems.

Moreover, more research will be needed to integrate the human interaction and transparency of the decisions made and an interactive machine user interface.

7. Major Ongoing Research Directions and Future Perspectives

The presented work introduced recent progress on different hierarchic and functional levels of industrial control solutions, discussing how AI-based methods enable further enhancements in many applications and sectors. Promising algorithms and methods were mentioned and discussed for every level following a bottom-up approach, starting from the sensor level and data fusion to more complex applications such as self-optimizing machines.

Thereby, the (self)-learning properties of AI-based algorithms are promising approaches to deal with complex processes. Compared to conventional FEM-based modelling approaches, AI-based methods come with short inference times, which are important for real-time production systems. However, future research is required to overcome the drawbacks of AI-based methods as far as the model reliability and explainability are concerned. Approaches such as neural networks are difficult to interpret for data points outside the training set. A linked issue is the required amount of data for the training process, which can be a burden for data scare applications. Furthermore, data quality in terms of uncertainty, outliers or missing data points is crucial for accurate training. The data preparation can be time-expensive and requires specialized skills. Depending on the application, AI approaches should be evaluated individually and compared to conventional methods in terms of effort and resulting costs. The mentioned factors can be currently limiting factors for a widespread exploitation of AI methods in industrial control and automation solutions, particularly in small to medium companies (SME). To assess the suitability of AI algorithms, Bettoni et al. [108] proposed a conceptual framework for SMEs. However, ongoing research and developments in this field are constantly showcasing new applications and pushing the technology towards industrial applications. These developments foster the wide spread of AI-based methods and facilitate the accessibility for companies. For data scare applications, transfer learning or physics-informed neural networks are promising methods to deal with this limitation and will potentially increase the interpretability of the models. Regarding control applications, reinforcement learning approaches are constantly gaining an interest in research, and new approaches and use cases will permit to understand and assess their suitability for industrial use. The combination of advanced monitoring, modelling, control and scheduling methods will in the future allow for the development of self-optimizing machines, leading to production machine improvements in terms of product quality, productivity and resource efficiency, and representing a crucial point for the next generation of human-centric manufacturing systems.