The rise of the machines: A state-of-the-art technical review on process modelling and machine learning within hydrogen production with carbon capture

This study aims to present a compendious yet technical scrutiny of the current trends in process modelling as well as the implementation of machine learning within combined hydrogen production and carbon capture (i.e. blue hydrogen). The paper is intended to accurately portray the role that machine learning is anticipated to play within research and development in blue hydrogen production in the forthcoming years. This covers the implementation of machine learning at both material and process development levels. The paper provides a concise overview of the current trends in blue hydrogen production, as well as an intro to machine learning and process modelling within the same context. We have reinforced our paper by first summarising a brief description of the key “ tools ” used in machine learning and process modelling, before painstakingly examining the implementation of these techniques in blue hydrogen production and the less-discovered merits and de-merits. Ultimately, the paper depicts a clear picture of the advancements in machine learning and the major role it is expected to play in accelerating research and development in blue hydrogen production on both material and process development fronts. The paper strives to shed some light on the key advantages that machine learning has to offer in blue hydrogen for future research work.


Hydrogen production: types, current and future trends
Climate change is our most pressing issue of the 21st century as outlined in the latest Intergovernmental Panel on Climate Change (IPCC) report (IPCC et al., 2023).CO 2 concentrations have now exceeded 420 ppm globally, there is a need to decarbonise globally at a rapid rate (IPCC et al., 2023).Since 2015 and the signing of the Paris agreement (UNFCCC, 2015), countries have, at least "on paper", committed to ensuring that global warming does not exceed 2 • C (with an ambition to limit this to just 1.5 • C).Within the UK since 2019 and the declaration of a climate emergency.A policy framework has been developed known as "build back greener" (DESNZ & BEIS, 2021).This framework outlines a way to ensure the meeting of Net Zero by 2050 in comparison to 1990 levels of CO 2 emissions.Part of this strategy to ensure we reach Net Zero is the development of carbon capture and storage (CCS) technologies, transitioning from oil and gas to renewable resources and an increase in hydrogen production (BEIS, 2021).Deployment of CCS technologies is critical to ensure the continued supply of low-carbon energy within developing countries (Masoudi Soltani et al., 2021).In the UK, a key part of the Net Zero strategy is to develop CCS technologies, especially within industries such as cement and steel (DESNZ & BEIS, 2021).Hydrogen has been identified as a low-carbon energy storage molecule (van Renssen, 2020), and has been identified as a potentially viable alternative to fossil fuels as a clean fuel for transport in the aviation and shipping industry (Ishaq et al., 2022).
As of 2021, 94 million tonnes of hydrogen are produced globally, and by 2030, this is expected to rise to a minimum of 105 million tonnes by 2030 (well below the 200 million tonnes required to ensure Net Zero) (IEA, 2021).Hydrogen is already used across a vast range of industries such as steel and fertiliser production (IEA, 2022).It is an energy carrier that when used in a fuel cell, produces no CO 2 emissions.Although when used as a fuel, it produces no CO 2 emissions, conventional hydrogen production methods lead to high CO 2 emissions.As of 2021, 900 million tonnes of CO 2 emissions were from hydrogen production (IEA, 2022).There are a number of processes via which hydrogen is produced, these methods are often referred to as the colours of hydrogen shown in Fig. 1.Among these methods, three routes are of interest: Grey, Blue, and Green hydrogen.Grey hydrogen is the hydrogen produced from fossil fuels; blue hydrogen is hydrogen produced from fossil fuels where the carbon is captured, utilised, and stored.Green hydrogen is hydrogen via electrolysis that is powered by renewable resources.Fig. 2 shows that currently the main and most viable methods of hydrogen production are grey and black.Both methods are associated with high CO 2 emissions.Unlike these methods, green hydrogen has no CO 2 emissions from the production of hydrogen; however, there are a number of key technologies and economic issues with the scale-up of this technology (Masoudi Soltani et al., 2021).Blue hydrogen presents another viable alternative.
In this approach, grey hydrogen production plants undergo retrofitting with carbon capture units to ensure the capture of CO 2 produced.This retrofitting allows for the utilisation of the existing infrastructure, enabling a rapid scale-up and deployment of low-carbon hydrogen production.However, the success of this approach, hinges on the effective management of the captured CO 2 , such as considering the proximity to and accessibility of carbon storage facilities (Masoudi Soltani et al., 2021).
The key challenge within low-carbon hydrogen production is the cost of low-carbon routes in comparison to conventional hydrogen production processes.Currently, the cheapest route for hydrogen production is grey hydrogen production.This is due to a variety of factors such as technology readiness level (TRL) of renewable technologies, and the inherent increased operational costs for the separation of CO 2 (Masoudi Soltani et al., 2021).Fig. 3 shows a comparison of the levelised cost of hydrogen (LCOH).It is projected that by 2030, low-carbon hydrogen production routes will be significantly cheaper due to advancements in technology (IEA, 2022).In addition to these advancements in technological development, incentives by government bodies provide support to companies in developing these technologies, these government interventions are vital to ensure that low-carbon hydrogen production is an economically viable alternative (Mazloomi and Gomes, 2012;Quarton and Samsatli, 2021;IEA, 2022).
Grey hydrogen is commonly produced via steam methane reforming (SMR) and auto-thermal reforming (ATR).These processes lead to high CO 2 emissions (over 900 million tonnes of CO 2 per annum as of 2022) (IEA, 2022).Blue hydrogen production provides a pathway to utilise existing infrastructure by upgrading grey hydrogen production plants with carbon capture technologies, such as amine scrubbing.With these technologies being available now, we have an immediate solution to reducing our CO 2 emissions.Moreover, there has been further development with this process to increase the process efficiency and intensification via Sorption-enhanced Steam Methane Reforming (SE-SMR) (HyPER, 2019;Barelli et al., 2008).
Recent studies have suggested that optimised blue hydrogen processes can be a viable long-term option to ensure that the hydrogen is of high purity and the CO 2 is captured and stored (Bauer et al., 2022;George et al., 2022).With the ongoing conflict in Ukraine, the middle east and the COVID crisis, the supply chain issue has caused further disruption to energy supplies.Hydrogen has been identified to be a viable energy storage molecule with zero-carbon energy storage (Mazloomi and Gomes, 2012;IEA, 2022).In the UK, producing low-carbon hydrogen is vital to energy security in the forthcoming years (BEIS, 2022).W. George Davies et al.

Machine learning: incorporation into hydrogen production
Machine learning (ML) has shown to be a valuable tool for engineers within the past decade.It has been utilised within chemical engineering across the previous decades; however, there has not been a widespread deployment of the technology due to a variety of reasons (e.g.lack of data, computational power, and data accessibility) (Schweidtmann et al., 2021).Recent trends in machine learning have coincided with cost-effective yet increased computational power with accessible and easy-to-use programming environments (Dobbelaere et al., 2021).
Within hydrogen production, the advancement of ML has provided an opportunity for ML to be incorporated into many aspects of hydrogen research (process modelling, optimisation and soft-sensor development).Furthermore, the advancement of novel process configurations such as SE-SMR, provides a need to screen for materials such as combined sorbent catalyst material (CSCM) to further optimise the process.There has been a recent trend to incorporate ML into the process modelling and optimisation of adsorption processes such as pressure swing adsorption (PSA), used in hydrogen production processes to produce high-purity H 2 .Conventional modelling of this highly-dynamic process is computationally slow, and ML-based models can provide efficient and robust models in comparison to conventional process modelling (Dat Vo et al., 2019;Subraveti et al., 2019;Pai et al., 2020;Vo et al., 2020;Tong et al., 2021;Yu et al., 2021).
With the new industrial wave (industry 4.0) upon us, increasing digitisation of chemical production plants such as hydrogen production has meant research areas such as ML, internet of things (IoT) and automation have become important within the realm of chemical engineering (Ghobakhloo, 2020;Örs et al., 2020;Lian et al., 2021;Sleiti et al., 2022).Blue hydrogen is becoming an ever-important necessity to enable decarbonisation at a rate to ensure Net Zero by 2050.With green hydrogen currently not at the point to be scaled up, blue hydrogen must be implemented as quickly as possible.Taking advantage of ML can accelerate the deployment of these processes, by providing both cost-effective and time-efficient routes for the development of low-carbon processes.This includes the development of cost-effective yet efficient catalysts and sorbents as well as the optimisation of operational parameters to ensure high hydrogen (and CO 2 ) purity, whilst minimising the costs.

Motivation for this paper and the paper's outlook
Recent reviews have been published on blue hydrogen production (Oni et al., 2022), as well as on machine learning within chemical engineering (Mowbray et al., 2022) with some work focused on machine learning within biohydrogen production (Kumar Sharma et al., 2022).To the best of our knowledge, there has been no in-depth technical review of machine learning within blue hydrogen production as of today.Review papers within the last couple of years have mainly focused on machine learning within carbon capture (Rahimi et al., 2021;Yan et al., 2021).With blue hydrogen being identified as a key tool to reach a low-carbon economy (Bauer et al., 2022), it is important to further develop and optimise novel low-carbon hydrogen technologies and see how machine learning can be utilised in the development and deployment of these technologies.This review paper provides a comprehensive technical review of the literature on ML within hydrogen production, outlining how ML could potentially overcome common issues within conventional techniques of process modelling as well as the incorporation of machine learning within digital twin technology.
To prepare this review, we have scrutinised 153 literature sources within the domain.Our search has been mainly focused on reviewing papers published from 2015 to 2023 which have been mostly indexed by Scopus with some coming from google scholar.Fig. 4 shows the trend in publications on ML, as well as showing the trend of ML in hydrogen production, hence underscoring its importance and potentials in this realm of research.
In regards to the methodology, a thorough search was done via Scopus and Google Scholar, where 114 research papers were selected relevant to either conventional process modelling of H 2 production or  ML within H 2 production.The papers selected were then scrutinised and summarised.This painstaking scrutinising led to 54 research papers being included in this review paper.A comparison was made between conventional process modelling and process modelling integrated with ML concerning the accuracy and speed of the optimisation.Issues including a lack of data on simulation time for conventional process modelling were addressed by selecting fewer papers that focused on the speed of simulation and comparing them with the ML-based models.
The paper follows with a section on blue hydrogen production methods, discussing in detail, the materials used for blue hydrogen production as well as novel process configurations.Then the latest research within conventional process modelling, simulation, and optimisation of blue hydrogen, processes discusses the challenges associated with conventional process modelling, and how machine learning can help to overcome some of these problems.Then, the discussion delves into the topic of machine and deep learning, exploring the important considerations involved in integrating machine learning techniques into process modelling and H 2 production.It then critically evaluates the latest research on the implementation of ML in blue hydrogen production, across both the material scale and process scale.The paper concludes by evaluating the key impacts ML has made on low-carbon hydrogen production and shedding light on future research towards combined ML and hydrogen production.

Blue hydrogen production
This section provides an in-depth review of different methods of blue hydrogen production including the methods of capturing CO 2 , as well as a brief review of the sorbents and catalysts that have been used for blue hydrogen production.In-depth reviews of blue hydrogen production can be found elsewhere (Masoudi Soltani et al., 2021;Nnabuife et al., 2022).

Overview of blue hydrogen processes
Hydrogen has conventionally been produced by SMR for decades.This method is categorised as grey hydrogen production, as it produces approximately 9 tonnes of CO 2 per tonne of H 2. The addition of carbon capture technologies can ensure the CO 2 produced is captured and stored, which converts the grey hydrogen production into blue hydrogen production.In a conventional SMR process, the (de-sulphurated) natural gas and steam are fed into the process and are compressed (1-25 bara) and heated (650 • C) before entering the reformer where reactions R1-R3 take place.Heat/energy must be supplied to the reformer due to the reaction being endothermic.The resulting product (i.e.syngas) is then fed to the water-gas shift (WGS) reactor where R2 reactions take place.Fig. 5 shows the conventional SMR process without any carbon capture.R3 highlights the overall reaction during this SMR process.
3.1.1.Steam methane reforming with carbon capture technologies SMR with carbon capture technologies is a similar process as described previously but with the addition of separation steps following the WGS reactor once a significant amount of H 2 and CO 2 is produced (Antzara et al., 2015).Conventionally, to separate the CO 2 from the hydrogen, amine scrubbing has been employed first to separate carbon dioxide from hydrogen, and then the CO 2 -lean H 2 stream is fed to a PSA column for hydrogen purification (Fig. 6).The CO 2 capture unit can be placed before or after the PSA column.This method allows for existing SMR plants to be retrofitted with carbon capture technology.In addition to amine scrubbing, adsorption processes such as PSA or temperature swing adsorption (TSA) have been identified for not only H 2 purification but also CO 2 capture (Boot-Handford et al., 2014).These processes can also be retrofitted whilst simultaneously overcoming some of the issues associated with amine scrubbing (lower energy demands for regeneration of sorbent for adsorption processes compared to their absorption counterpart) (Wang and Song, 2020).
By introducing extra separation steps within the process there will be an inherent increased CAPEX/OPEX.There has been much work on lower the costs through technological advancements such as developing cheaper catalyst material (Nkulikiyinka et al., 2022) and thermodynamic analysis of the process to ensure optimised heat integration (Antzara et al., 2015).An alternative method to reducing costs is through policy intervention by providing financial incentives to reduce CO 2 emissions.Such approaches have been adopted globally to reduce costs of low-carbon and renewable technologies (IEA, 2022).

Sorption-enhanced steam methane reforming
SE-SMR intensifies the SMR process equipped with carbon capture.In this process, the sorbent (commonly CaO) is placed in the reformer rather than being used in an individual unit, hence further process intensification.No WGS reactors is theoretically necessary.SE-SMR presents an alternative process configuration (Fig. 7), that aims to intensify the process.Integrating the sorbent in the reformer allows for two key process improvements: A. Simultaneous removal of CO 2 within the reformer as it is being produced.This shifts the equilibrium to the right (Le Chatelier's principle) and allows for more H 2 to be produced (i.e.improved conversion) (Broda et al., 2013); B. Reduction in coke deposition onto the catalyst via reducing the steam needed, and lowering the reaction temperature, due to the exothermic nature of the formation of the calcium carbonate via carbonisation of CaO (Faheem et al., 2021).
Within the reformer, R3 and R4 take place in-situ and together.With the reverse of R4 taking place in the Calciner.The addition of calcium oxide as the sorbent leads to an exothermic reaction and provides heat for further H 2 generation via WGS reactions, hence the lack of a need for an additional WGS reactor (Cherbanski and Molga, 2018).There has been some research looking into changing the sorbent due to the issues associated with the cyclability of CaO because of its deactivation after a number of cycles (Shokrollahi Yancheshmeh et al., 2016).

Auto-thermal reforming
Auto-Thermal Reforming (ATR) is like the SMR process; however, it introduces O 2 within the reformer along with the steam and methane.Here, oxygen reacts with methane according to reaction R5.It is a highly exothermic reaction and the heat generated from R5 helps to carry out the endothermic reforming reaction (R3) (Faheem et al., 2021).

Recent process development of blue hydrogen production
There has been further development of blue hydrogen production processes to further optimise for lower cost and higher efficiencies.Table 1 highlights the latest development in low-carbon hydrogen production and summarises the latest advancements in process development to further reduce the CAPEX/OPEX of conventional approaches, increase thermodynamic efficiency, as well as optimising the process to ensure high CO 2 capture rates.Some of these works have been detailed in recent literature (Ishaq et al., 2022;Nnabuife et al., 2022;Oni et al., 2022).
As shown in Table 1, the main area of process development has been the process intensification by introducing sorbents into the reformer (SE-SMR, SE-SMG, SE-CLR) for combined CO 2 capture and H 2 production.Furthermore, integrating low-carbon heat sources such as CLC and solar power, provides methods of reducing the CO 2 output.Work has also focused on integrating different feedstock (SE-SMR) due to the effect of methane as a greenhouse gas (Howarth and Jacobson, 2021).Conventional process modelling has had an important role to play in development of low-carbon H 2 production to assess process feasibility of these novel developments at low-cost.Technologies such as CLC are being integrated into the process to provide a source of low-carbon heat, whilst simultaneously allowing for a reduction in CO 2 being released as well as reducing the energy demand for conventional CO 2 capture processes such as amine scrubbing (Fan et al., 2012;Bahzad et al., 2019).Conventional process modelling involves the modelling of a process using first-principle equations such as heat and mass transfer, to determine the outputs of a process.Section 4 will provide an in-depth analysis on conventional process modelling and the recent advancements made in regards to H 2 production.
The recent development of process modelling has looked to incorporate ML within process modelling of hydrogen production.The development of surrogate models of H 2 production via ML allows for full process optimisation via the identification of key operating parameters and key performance indicators (KPIs) (Vo et al., 2022).This can also be applied to individual unit operators as well, such as a PSA unit (Subraveti et al., 2019;Yu et al., 2021).Reducing the dimensions of the model through the selection of key operating parameters, allows for increased speed of simulation.This increased speed of simulation, explicitly shown within the dynamic modelling of these processes, allows for improved process control and can provide in-sight in real time of H 2 production processes.Section 6 will delve deeper into the application of ML within the modelling of hydrogen production processes.

Development of materials for blue hydrogen
Within blue hydrogen production, research has been focused on developing catalysts, used in the reformer and water gas shift reactors.Commonly nickel-based catalysts are used in the reformer, but there are issues with sintering at high temperatures within a conventional SMR reformer (Maqbool et al., 2021).High-temperature WGS reactors conventionally use copper as the catalyst, which is expensive (Saeidi et al., 2017).More recently, there has been a focus on developing CSCM for the SE-SMR processes (Nkulikiyinka et al., 2022).Table 2 provides an overview of common materials used in blue hydrogen production.
In regards to this the development of materials within hydrogen production, modelling these molecules via molecular modelling techniques such as density functional theory (DFT) can provide insight into the interaction between the new material and CO 2 molecules or its interaction with CH 4 .In the realm of material development, ML has found applications in the creation of innovative materials, particularly in the screening of catalysts and sorbents using quantitative structureactivity relationship (QSAR) methods.This has been utilised with the development for CO 2 capture materials such as metal organic frameworks (MOFs) (Avci et al., 2018;Dureckova et al., 2019;Burns et al., 2020).In comparison to conventional material screening which involves a trial and error synthesis of compounds, the development of QSAR provides a framework in which vast amounts of compounds can be screened and ranked according to the KPIs (the activity of the material).Further elaboration on this topic will be provided in Section 6.

Conventional process modelling within blue hydrogen
Numerous low-carbon hydrogen production methods such as SE-SMR and SE-ATR are currently at a low TRL.However, some of these processes are being developed at pilot scale to better understand how to A gasification reactor is introduced in which biomass is gasified at high temperature to produce syngas and methane.This is then introduced as feedstock to the reformer to produce H 2 .The use of biomass as a precursor allows for zero emissions but there is a high energy penalty associated with the gasification process.

Li et al. (2020)
Solar-aided steam methane reforming (SA-SMR) The pre-reformer is added to the SMR process that uses solar power to heat the molten salt.Using solar-assisted heat generation provides low CO 2 emissions.scale them up for optimal heat integration, reformer sizing, and thermodynamic efficiency.This is crucial to ensure high H 2 and CO 2 purity and a high CO 2 capture rate.To achieve an optimised process for lowcarbon hydrogen production, it is essential to conduct process modelling and subsequent simulation.There has been a significant amount of research focused on process modelling, simulation and optimisation of low-carbon hydrogen production processes (Sinaei Nobandegani et al., 2016;Phuakpunk et al., 2018;Costamagna et al., 2020;Yan and Thanganadar, 2020;Dat Vo et al., 2021;Faheem et al., 2021;Maqbool et al., 2021;Shahid et al., 2021;Gunawan and Singh, 2022;Nobandegani et al., 2022;Quirino et al., 2022;Babamohammadi et al., 2023;Capa et al., 2023).Here we reviewed the latest research on the process modelling, simulation, and optimisation of blue hydrogen production processes.Specifically, the focus has highlighted the need for an alternative optimisation approach to the conventional optimisation of processes.
Conventional process modelling software such as Aspen Plus and gPROMS, model chemical processes via the modelling of physical and chemical properties of the system and the inclusion of unit operators.Significant effort within the past decade has focused on the modelling (both steady-state and dynamic) and optimisation of these blue hydrogen production methods (Costamagna et al., 2020;Yan and Thanganadar, 2020;Dat Vo et al., 2021;Gunawan and Singh, 2022;Nobandegani et al., 2022;Babamohammadi et al., 2023;Capa et al., 2023;Mostafa et al., 2023).
The development of blue hydrogen processes has included an array of modelling work.Dat Vo et al. (2021) looked at the dynamic modelling of the reformer, as a circulating fluidised bed reactor (CFB).An SE-SMR process was developed on a semi-central scale, capable of producing 48 tons of hydrogen a day.The simulation showed a reduced production cost of H 2 (12%) and a higher energy efficiency (82%) in comparison to conventional SMR (Dat Vo et al., 2021).A sensitivity analysis of main operating variables (temperature, pressure, velocity and S/C ratio) upon process performance, suggests inlet temp of the bubbling fluidised bed is the most important variable as it has considerable effects on production rate, CO 2 capture, cost and energy efficiency (Dat Vo et al., 2021).
Furthermore, it is important to compare alternative process configurations of SE-SMR to determine the optimum process configuration, that ensures for high CO 2 capture rate and high H 2 purity.Yan and Thanganadar (2020) investigated six process arrangements that were developed for blue hydrogen production: (1) SE-SMR, (2) SE-SMR + PSA, (3) SE-SMR + CLC, (4) SE-SMR + PSA + CLC, (5) SE-SMR + PSA + Oxyfuel Combustion and (6) SE-SMR + PSA + H 2 Recycle.These processes were compared in terms of five key performance indicators via a thermodynamic analysis: Cold Gas efficiency, Net efficiency, CH 4 conversion, H 2 purity, and CO 2 capture.A sensitivity analysis concluded case 4 (SE-SMR + PSA + CLC) was the optimal process configuration for high H 2 purity (100%) and a high CO 2 capture rate (100%), in which the reformer temperature was 600 • C, the pressure was 25 bara and S/C ratio was 5.These demonstrate the potential of low TRL technology such as CLC and SE-SMR to produce low-carbon H 2 (Yan and Thanganadar, 2020).
In the context of process optimisation, sensitivity analysis and other optimisation approaches can be highly informative.By utilising first principle modelling, it becomes possible to gain a thorough understanding of the process and identify the key operating parameters that need to be optimised to achieve high performance in selected key performance indicators (KPIs).However, it is important to note that conventional optimisation approaches, such as sensitivity analysis, typically utilise a one-factor-at-a-time (OFAT) approach.Although this approach can be useful in isolating individual operating parameters, it does not reveal how these parameters interact with each other, which may require further analysis.Previous research has focused on utilising the design of experiment (DoE) technique which aims to optimise via the parallel evaluation of the chosen input parameters (Gorbounov et al., 2022).While this technique is commonly used in experimental practices (Gorbounov et al., 2023), some research has explored its application in process modelling and optimisation as well (Sinaei Nobandegani et al., 2016;Phuakpunk et al., 2018;Quirino et al., 2022).
Within the literature, work has been focused on using design of experiment (DoE) (Gorbounov et al., 2022), a technique that aims to conduct optimisation via the parallel evaluation of the chosen input parameters.Commonly, the literature has demonstrated the effectiveness of this method in optimising the design of reformers for steam methane reforming, and in optimising unit operators of low-carbon hydrogen production such as SE-SMR.Using DoE for process model optimisation is a useful tool as it allows for the parallel evaluation of multiple input parameters.This has been shown in recent literature with (Babamohammadi et al., 2023) in which they developed a detailed rate-based model was implemented to simulate the reformer in SE-SMR, as well as the SE-ATR processes to investigate the effects of operational variables of the process and their impacts on KPIs including H 2 purity and CO 2 capture rate.The application of DoE in process modelling is effective in understanding the best operating conditions for maximum efficiency and how different key parameters interact with the KPIs.Recent work has explored computational approaches, such as Latin hypercube sampling (LHS), which splits the dataset into square grids and selects seemingly random data while ensuring variability within the dataset by selecting a square grid containing data from each row and column (Galeazzi et al., 2023).
Another notable trend within this multivariable approach has been the development of optimisation algorithms such as genetic algorithm (GA), which paired with process modelling can provide a detailed approach for multi-variable optimisation (Cherif et al., 2022).GA is an algorithm for optimisation that mimics biological evolution.The algorithm begins by generating a random population of solutions.At each step individual solutions from the current population are selected to act as parents to generate the next generation.Over successive populations, the algorithm tends towards an optimal solution (Bagherian et al., 2021;Cherif et al., 2022) has focused on using GA for the optimisation of a novel design for an ATR reformer.The novel configuration improved the performance of the ATR, with a reduction in the operating temperature (24.8%), and enhanced CH 4 conversion (27.2%).Further optimisation utilising a GA was employed to optimise catalyst arrangements, for maximum H 2 yield associated with the lowest local wall temperature.H 2 yield slightly increased with a decrease in maximum local wall temperature (39.3%), suggesting an improvement in heat exploitation in the novel design (Cherif et al., 2022).
Due to the application of hydrogen within fuel cells, there is a need for extremely high-purity H 2 which requires the adaptation of further purification processes.In the realm of blue hydrogen, purification processes such as PSA and TSA are highly dynamic processes carried out in columns.When utilising conventional process simulators, such as Aspen Plus, and optimising by changing relevant operating conditions, simplifications and assumptions are often made due to the computational intensity of simulating non-uniform inlet streams throughout the process.Otherwise, with conventional approaches, the modelling and optimisation will be computationally time-consuming.
In the development of digital twins for these processes, ML is a crucial component due to its ability to significantly increase processing speed.The use of ML can significantly increase processing speed and help the model to optimise these processes more accurately while considering the complex dynamics involved in the process.By training models with large datasets of process data, machine learning algorithms can identify patterns and optimise process variables in real-time to achieve optimal performance, which ultimately leads to improved efficiency, reduced costs, and increased H 2 purity.Consequently, machine learning is an essential tool in the development of digital twins for blue hydrogen production, and modelling and optimisation of the process.
When optimising adsorption-based processes associated with blue hydrogen production (i.e.TSA and/or PSA), often the cyclic nature of these processes requires a set of non-linear PDEs to be solved until it reaches a cyclic steady state.Conventional modelling and subsequent OFAT approach to optimisation (sensitivity analysis) are computationally slow.Utilising DoE and other optimisation techniques such as GA that allow for multi-objective optimisation (MOO), provides an accurate and computationally quick solution to optimisation problems.Within the literature there has been an increasing trend to utilise ML with GA to optimise processes such as H 2 purification, this will be further discussed in section 6.

Machine learning: a technical summary and application within H 2 production
This section first begins with a brief explanation on the methods and principles of machine learning (ML) and deep learning (DL) for the reader to build a practical insight into the tools that are used in process modelling, optimisation and material development within blue hydrogen production.The chapter then discusses the recent developments within ML and how they have been incorporated within H 2 production.

Machine learning: an overview
ML is a branch of artificial intelligence (AI) that involves the use of algorithms and statistical methods by computers to progressively enhance their performance and outcomes.As Arthur Samuel an AI pioneer put it "machine learning is a field of study that gives computers the ability to learn without explicitly being programmed."(MIT Sloan, 2021).ML effectively describes the process in which machines learn from problem-specific training data.ML is a methodology that enables the enhancement of non-linear systems through optimisation (Rahimi et al., 2021).It can be applied across a vast range of research areas from natural language processing (Hirschberg and Manning, 2015), image classification (Wu and Chen, 2016) and robotics (Correa-Baena et al., 2018).
According to different learning modes, ML can be categorised as supervised, unsupervised and reinforcement learning.Supervised learning is used when the dataset is labelled to understand the relationship between output variables and input variables (Gianey and Choudhary, 2018).There are two types of supervised learning algorithms: classification and regression.Classification algorithms predict discrete output variables from input variables, regression algorithms predict continuous output based on the input variables (Ketabchi et al., 2023).Examples of supervised learning algorithms include: linear regression, logistic regression, decision trees and naïve bayes (Gianey and Choudhary, 2018).
Unsupervised learning is another learning approach in ML which can be used when the dataset is unlabelled, to find patterns through the data without human intervention (Dey, 2016).Unsupervised learning can be classified into three groups: association, dimensionality reduction and association (Dey, 2016).Clustering is a technique that categorises or separates unlabelled data by identifying similarities or differences (Dey, 2016).On the other hand, association employs a rule-based approach to discover relationships between variables within each dataset (Schweidtmann et al., 2021).Dimensionality reduction reduces the dimensions of the dataset in such a way that the subsequent representation retains meaningful properties of the original data (Yan et al., 2021).Dimensionality reduction techniques such as principal component analysis (PCA), has important application within machine learning models of hydrogen production processes to enhance the modelling with respect to speed.Semi-supervised learning is a hybrid technique that combines both supervised and unsupervised learning methods.It leverages a limited set of labelled data to guide the classification and extraction of information from a larger unlabelled dataset (Schweidtmann et al., 2021).Common unsupervised algorithm includes: PCA, K-means clustering, K-mode clustering, Apriori and single value decomposition (Dey, 2016).
Reinforcement learning represents a novel approach to learning that distinguishes itself from conventional methods.Unlike traditional approaches that rely on input-output pairs.Reinforcement learning empowers an agent to learn through trial and error within an environment, leveraging feedback from its own experiences.By providing a set of permissible actions and associated constraints, reinforcement learning obviates the need for a pre-existing training dataset.Instead, it reinforces successful outputs, enabling the development of optimal recommendations for problem-solving (Horvitz and Mulligan, 2015).The agent is the object that seeks to achieve a goal within the environment.The environment provides states in which the agent operates in.The agent uses policy and reward to learn within the environment (Sutton and Barto, 1998).Policy is the method in which to map the actions of a given state.Fig. 8 shows the relationship between these meanings within reinforcement learning.The feedback given to the agent is commonly referred to as a reward.Developing an optimal policy is key to success in reinforcement learning, the agent often faces a problem known as exploration vs exploitation in which the agent must explore new states whilst maximising its reward at the same time (https://towardsdatascience.com/reinforcement-learning-101-e24b50e1d292).
ML has recently been utilised in the process modelling of chemical systems for the production of blue hydrogen (Dat Vo et al., 2019;Subraveti et al., 2019;Pai et al., 2020;Vo et al., 2020;Tong et al., 2021;Yu et al., 2021), and in the molecular screening for novel materials (such as catalysts and sorbents) (Khurana and Farooq, 2016;Avci et al., 2018;Burns et al., 2020;Mashhadimoslem et al., 2021;Nkulikiyinka et al., 2022;Cheng et al., 2023).Process modelling has historically been done within chemical engineering by a physical system represented by mathematical equations that are governed by physical laws, whereas ML-based models utilise data that could come from literature, W. George Davies et al. experimental or first-principle modelling data, and use that data to train ML-based models to simulate or predict the output of the system (CO 2 capture rate and H 2 purity) (Pistikopoulos et al., 2021).A recent development of ML has been the application within the realm of process modelling.For example, the operation of adsorption columns can be modelled via ML.This also applies to the modelling of reactors specifically applied to hydrogen production (e.g. the modelling of the reformer).In the domain of sorbent synthesis and process optimisation, a significant volume of data is accessible in literature encompassing various sorbents and processes.This data can be employed to train and construct ML-based models.
ML provides researchers with tools to model highly non-linear systems.Key factors must be considered such as, ethical considerations like the interpretability of the model when employing them in STEM-related fields (Ketabchi et al., 2023).Nevertheless, multiple ML techniques have been applied within the modelling of these processes such as random forest (RF) which balances interpretability and predictive ability well (Eduardo et al., 2022).Artificial neural networks (ANNs) have been extensively employed in the literature in low-carbon hydrogen production, due to their flexibility within the architecture.Due to this predominant use of neural networks and deep learning techniques within the literature, the next section will solely focus on neural networks and deep learning methods, outlining how they've been advanced within the last decade.

Artificial neural networks
ANNs are non-linear based mapping structures, which maps the input from the output, in between these inputs and outputs are layers of nodes (known as the hidden layer).The output node values are determined by the activation function of the nodes in the hidden layer and the interconnections between the nodes (Yan et al., 2021).Fig. 9 provides a representation of the process.The ANN structure is inspired by the principle of information processing in biological systems (Janiesch et al., 2021).
As mentioned previously, a recent trend has been the development of neural networks for modelling engineering processes.Neural networks have been widely employed since the 1940s (Schmidhuber, 2015).There was an initial disregard for neural networks due to a trend towards logical inference (Schmidhuber, 2015).Nevertheless, the combination of the development of convolutional neural networks (CNN) for image identification allowed for a return for the use of neural network.The employment of ANN exploded in the 2010s with the advent of deep learning and big data (Panerati et al., 2019a).
A neural network consists of an input layer, hidden layers, and an output layer.The data is passed through the layers to produce an output.The input layer is the data that is initially fed into the network.The hidden layers within the network do the computing and extract the key features to produce an output, which is the layer that provides a result from the input and its interaction with the neural network.Each layer can comprise multiple neurons, the basic structure of the neuron is described in Fig. 10.

How does a neural network learn?
Each neuron receives a multiplied version of inputs and random weights (which is attuned in the learning process).Within the neuron, the weights and inputs from the previous layer are linearly combined, which is then added with a static bias value (unique to each neuron layer).This is then passed to an appropriate activation function which decides the final value to be generated by the neuron (Bengio et al., 2015).This is described in equation ( 1).
where z is the neuron, X n is the data input, W n is the weight applied to each data input to the neuron, and b is the static bias value.Different activation functions are utilised based on the characteristics of input values.The activation function plays a crucial role in determining whether a neuron should be activated or remain inactive.It adds non-linearity to the system (dependant on the activation function) (Bengio et al., 2015).Once the output is generated, the loss function is next calculated (Janiesch et al., 2021).Non-linear activation functions, such as leaky ReLU, enable deep learning (Bengio et al., 2015).
The loss function measures the model's accuracy during the training process.It evaluates how well the model, models the dataset.Within ML the goal is to minimise the loss function to ensure the output generated, matches the output from the original dataset.This minimisation of the loss function is commonly done by gradient descent (Lillicrap et al., 2020).Gradient descent works by starting at a random point and determining the gradient from that point.The descent to minimise this loss function is determined by the direction and the learning rate.These data points are needed to calculate the partial derivatives, to allow it to reach the global minimum (Bengio et al., 2015).The size of the gradient will determine the size of the parameters such as the weight and bias of   each neuron.This minimisation of the loss/cost function is done during backpropagation (BP), where the weights and biases of each vector are adjusted to match the output of the actual output (https://medium.com/yottabytes/everything-you-need-to-know-about-gradient-descent-applied-to-neural-networks-d70f85e0cc14).BP plays a crucial role during the learning process as it provides a way for the ANN to converge on a global minimum after multiple iterations of the dataset.BP uses error signals that are feedback into the neurons to adjust the weights and bias, in order to minimise the error within the ANN to ensure the output is correct (Lillicrap et al., 2020).
The learning rate determines the rate at which the gradient descent occurs to reach the global minimum, this is a key hyperparameter within the process.If the learning rate is small, then the training of the model is computationally slow and too large, and the global minimum may be missed.The learning rate, and activation function, as well as the number of layers and neurons are known as neural network hyperparameters.These cannot be determined by the learning algorithm and must be set manually (Bengio et al., 2015).An epoch is a key hyperparameter within the learning process.It is when the training dataset has been passed through the neural network once.In general practice, neural networks are typically trained over multiple epochs until the loss function reaches a sufficiently low value.However, the selection of the number of epochs requires careful consideration.If too many epochs are used, the model may become overfit, resulting in a lack of generalisation and robustness in its predictions.Conversely, if too few epochs are employed, the neural network may become underfit, leading to inaccurate output values from the model (https://medium.com/@upendravijay2/what-is-epoch-and-how-to-choose-the-correctnumber-of-epoch-d170656adaaf).
Commonly, the data used to train the neural network, is split into training, testing and validation data.The training data accounts for most of the data split and is the data that is used to learn the patterns within the dataset; the validation set is used after each epoch and is used to tune the hyperparameters and confirm that the loss function is being minimised.It also ensures that the neural network is not overfitted to the training data.The test set provides unseen data for the ANN in which to determine how well the neural network model performs.Determining the split of training, validation and test data is dependent on a variety of factors.Effectively, there is no "golden rule" to split the data set (V7labs, 2021).

Advancements within machine learning
ML has been an expanding research field within the last 20 years as mentioned, with application in a vast range of fields.In this section, the advancements are discussed within the context of process engineering and STEM-based research.

Advancements within neural networks
Advancements have been made within the last few decades on developing different types of neural networks for different applications.Table 3 provides an overview of recent advancements made within neural network and a brief description of each type.
The flexibility of the neural networks, in regards to the type of algorithms used for backpropagation and learning, the networks architecture as well as a choice of loss function and activation function.This flexibility allows for ANNs to be fine-tuned for a specific purpose.Specifically, in that regard, this has led to the development of neural networks that allow for deep learning which will be discussed in the next section.

Advancements within deep learning
Deep learning (DL) is a significant branch of ML, especially within the last decade.Complex neural networks, consisting of more than three layers, are employed to analyse extensive datasets (Panerati et al., 2019b).Deep learning provides a way to automate both feature extraction and model building, meaning it can be extremely useful in noisy and unstructured datasets (Janiesch et al., 2021).They contain advanced neurons that use advanced operations.This allows for the deep-learning neural networks to be fed with raw input data to discover a representation needed for the corresponding learning task (automated feature extraction) (Janiesch et al., 2021).DL has become of increasing interest with the development of large language models such as ChatGPT (Birhane et al., 2023).
DL and ANNs are commonly referred to as "black-box".Within data science, a black-box approach is a model that produces a useful output but is not interpretable (Rudin and Radin, 2019), whereas conventional process modelling software such as Aspen Plus utilises a "white-box" approach to modelling in which models are constructed from first principles of physical equations in which mass, momentum and physical equations govern how the process operates (described mostly by partial differential equations) (Bikmukhametov and Jäschke, 2020).However, the time in which to develop these models are high, especially within large processes that have complex non-linear behaviour, such as the production of low-carbon hydrogen.Combining the two-modelling approaches allows for a hybrid model.This type of modelling approach has arisen within engineering and the natural sciences to enforce physical laws within an ML-based framework (Karniadakis et al., 2021;von Stosch et al., 2014;Bikmukhametov and Jäschke, 2020).Combining the interpretability of the first principle-based modelling and the speed of ML models allows for improved modelling and simulation of chemical processes.
A recent phenomenon within the natural sciences has been physicsinformed neural networks (PINNs) in which biases are introduced within the learning process that allow for the learning process to identify physically acceptable solutions (Karniadakis et al., 2021).Three main types of biases can be introduced, observational biases, inductive biases and learning biases.Over conventional modelling, they allow for high dimensional problems to be solved, together with the quantification of uncertainty.Within blue hydrogen production incorporating these PINNs into digital twins could be highly beneficial regarding its hybrid approach of combing data with physical models, allowing for enhanced use within chemical engineering.

Table 3
Common neural network architecture.

Feedforward neural network
The simplest architecture for a neural network information travel from the input to the output in one direction, not a deep learning technique due to a lack of automated featured extraction and no backpropagation occurring (Janiesch et al., 2021).Recurrent neural network (RNN) Designed output from any layer is then feedback as an input into previous layers (Pascanu et al., 2013;Panerati et al., 2019a).Designed for time-sequential data, the architecture of the neurons allows for storage of the data to enable sequential learning, simple RNNs have issues such as vanishing gradient (Emmert-Streib et al., 2020).Long-short term memory (LSTM) LSTM is similar to RNNs in that it allows for storage of the data; however, it overcomes the issues such as vanishing gradient (common occurrence in RNN) by introducing a "forget gate".LSTM has been heavily applied to the analysis of video data (Emmert-Streib et al., 2020).

Convolutional neural network
Neurons are organised in 3D layers and produce a convolution operation on their input vectors (Alzubaidi et al., 2021).

Modular neural network
Different networks that function independently form one another, the networks do not interact with one another and work independently in achieving an output ( Vaswani et al., 2017).

Autoencoders
A type of neural network, commonly used in image recognition.It aims to learn data encodings in an unsupervised manner (Bengio et al., 2015).
W. George Davies et al.

Ensemble learning
A recent development within ML research has been ensemble learning, which involves combining multiple ML algorithms and combining the predictions produced by each model.Research has shown it to be more accurate in comparison with a single ML algorithm (Zhang et al., 2022b;Mohammed and Kora, 2023).Methods such as RF is a prominent example of an ensemble learning method which combines multiple decision trees to generate a prediction.There are many ways to ensemble multiple ML algorithm such as bagging, boosting and stacking (Alam et al., 2020).A recent development within ensemble learning is its incorporation within deep learning to overcome issues such as high variance within deep neural networks (Alam et al., 2020).
Ensemble learning has been used within the modelling of hydrogen production, with it being employed usually to compare against a neural network (Nkulikiyinka et al., 2020;Zhao et al., 2021).RF has been employed as a soft-sensor for predicting the concentrations of gases within the reformer in the SE-SMR process and has shown a slightly better performance in the predictive results in comparison to using an ANN (see section 6) (Nkulikiyinka et al., 2020).It has also been employed to predict the output of the gasification of biomass for hydrogen production, comparing with four different ML methods with RF outperforming them all (Zhao et al., 2021).The improved predictive performance of ensemble learning highlights the benefits of integrating ensemble learning further into H 2 production.

How and why has machine learning been integrated into hydrogen production?
In regards to ML integration within process modelling of chemical processes and more specifically H 2 production.ML integration within H 2 production has often involved the collection of data either through the literature, industry or first-principle simulation.A PCA is often run on the dataset rather than using a "raw dataset", used to reduce the dimensions whilst maintaining accuracy by selecting the key features.These features are then used as the input for the ML algorithm (commonly ANNs), these models are then trained, tested and validated as per the previous section.Often the ML model is developed it is often optimised using a GA, to provide optimal performance of the model and identify operating parameters in which the optimal output occurs.This method has been commonly used within the development of "surrogate" models for unit operators such as the PSA unit (Subraveti et al., 2019;Yu et al., 2021).A vast array of literature has applied this method to either optimise specific unit operators (Dat Vo et al., 2019;Subraveti et al., 2019;Pai et al., 2020;Vo et al., 2020;Tong et al., 2021;Yu et al., 2021) or entire hydrogen production processes (Alsaffar et al., 2020;Nkulikiyinka et al., 2020;Oh et al., 2022;Vo et al., 2022;Wang et al., 2022;Gul et al., 2023).
As mentioned previously ML has been utilised in H 2 production within the molecular, material and process scale of modelling.The utilisation of ML and specifically in modelling low-carbon hydrogen production holds significant value.In regards to ANNs specifically the flexible structure allows for easy customisation to suit a wide range of applications, including supervised, unsupervised, and reinforcementbased learning.Moreover, neural networks offer computational efficiency compared to previous models, making them a viable choice.They exhibit high adaptability and can correlate diverse inputs and outputs, which proves advantageous for tackling multi-objective optimisation problems, such as simultaneous hydrogen production and CO 2 capture.This flexible structure combined with automated feature extraction provides a computationally quick model, that if trained correctly, can produce highly accurate outputs.However, careful consideration is required to ensure neural networks are trained correctly, such as ensuring an R 2 value across the training, testing and validation dataset.
Furthermore, an understanding of how neural networks learn is key to ensure accurate and robust models.ANNs/ML integration within process modelling occurs when white box (first principle) modelling is insufficient in regards to developing analytical solutions (the time taken to solve the PDEs).Often in complex processes such as H 2 production but specifically for units such as the PSA column for H 2 purification.A numerical method increases the efficiency to provide solutions.An indepth comparison will be discussed in section 6.A thorough understanding of ANNs in relation to the structure (architecture) and the choice of learning algorithm are important to ensure a correct model.

Machine learning in blue hydrogen production: a critical Outlook
In this section, ML models for both grey and blue hydrogen have been examined, acknowledging the similarities in their processes.Additionally, we have explored and discussed the implementation of ML in these hydrogen production processes, ranging from material screening and development to comprehensive plant optimisation.By doing so, we have shed light on the existing applications of ML and identified areas where its utilisation can be expanded in blue hydrogen production.Table 4 provides a descriptive overview of the research that will be discussed in detail in the sections below.Furthermore, a detailed comparative assessment has been done on the current literature with regards to process modelling.Conventional process modelling has been used as a benchmark to assess ML within blue hydrogen production to highlight the advantages of utilising ML and potential pitfalls.

Machine learning in material development for hydrogen production
Within the realm of material development, ML has been extensively implemented both in the molecular modelling of catalysts and materials for CO 2 capture with DFT calculations and molecular dynamics, as well as QSAR (Fiedler et al., 2022) -a method of identifying materials for a select function based on its structure.Within low-carbon hydrogen production, there has been a significant focus on the development of catalysts for the hydrogen evolution reaction.This is due to the expense of the platinum catalyst used.Some work has focused on implementing ML within DFT calculations (Ugwu et al., 2022;Bokinala et al., 2023;Wu et al., 2023) to discover novel catalysts for the hydrogen evolution reaction.Conventional molecular modelling such as DFT requires the solving of the Hamiltonian over time and still requires first-principles calculations to solve the model in order to determine the electronic structure which can then be used in materials screening (Lee, 2015).
With regards to material development, ML has been extensively applied within the pharmaceutical industry and catalyst design (Wang et al., 2019;Mehta et al., 2021).It has recently been applied to amine-based carbon capture and blue hydrogen production (Yang et al., 2023), and more recently to adsorbents that have the optimum properties for CO 2 capture (Yan et al., 2021).Within blue hydrogen specifically, there has been a focus on the development of novel materials such as CSCM with the development of an ML model for the selection of CSCM that have optimal properties for application within SE-SMR (Nkulikiyinka et al., 2022).Work has also looked at catalyst development within the WGS reactor (Kim et al., 2022) which is often expensive in comparison to the nickel used within the reformer.Some discrepancies have been reported in the literature which affect the screening for catalysts for the WGS reactor (Kim et al., 2022).Future work must correctly characterise catalyst formulation in order for the future enhancement of the WGS process with respect to catalysts.
Materials have also been screened for processes such as CLC (Yan and Mattisson, 2020).This highlights the application of ML to material development in blue hydrogen production.It can increase the speed of the workflow for the discovery and synthesis of materials.However, consideration must be given in the use of neural networks as interpretability is of vital importance to ensure an understanding of how and why the structure affects the performance of the material within a process.A recent development has been the integration of the scales of modelling from the material scale to the process scale.ML provides a framework in   The R 2 value of 0.99 highlights the advantage of using ANN to predict the performance of an adsorbent quickly and effectively.Khurana and Farooq (2016) Developed a two-step screening process for adsorbents.
Step two was a meta-model developed for the VSA process ensuring that the adsorbent implemented had minimum energy and maximum productivity for the process.
Results indicated the ANN model was highly accurate and several adsorbents were identified that showed a superior performance in comparison to 13X zeolite.which sorbents can be screened and then placed in a PSA column to further assess its performance, this has been done for sorbents for CO 2 capture with recent work comparing with the industry standard Zeolite 13X (Burns et al., 2020).Furthermore, the interest in metal organic framework (MOFs) for CO 2 capture and H 2 purification and molecular modelling as well as synthesis of these MOFs has allowed for a large database to be developed (Avci et al., 2018;Dureckova et al., 2019).This database and utilisation of ML has provided a framework for screening and subsequent process scale modelling of these sorbents at high-speed, increasing the speed at which these sorbents can be employed within industry (Saenz Cavazos et al., 2023).
Further work can be done on developing databases of materials derived from waste or biological sources.There has been some work in which a neural network was used to select materials/molecules for carbon capture from a database of biological waste sources (Mashhadimoslem et al., 2021).Furthermore, work needs to be done to develop larger databases, as well as further work to determine sources of waste material that can be used as a precursor material for the synthesis of sorbents for CO 2 capture to reduce costs and enhance performance in these processes.
This QSAR screening for materials using ML, specifically the use of PCA and neural networks, can enhance and optimise the synthesis of this material by utilising PCA.There is the advantage of identifying the key structural parameters that affect the properties.This can then be fed into a neural network to determine the optimum materials that have high performance for the desired purpose, by reducing the dimensions of the model it allows for increased speed whilst maintaining accuracy.Future work must look at increased interpretability too within ML-assisted QSAR.The implementation of ML provides a robust framework (if trained correctly), that is able to screen for materials at high speed.

Machine learning to enhance process modelling for hydrogen production
ML has been effectively used with blue hydrogen production for the optimisation of whole processes, optimisation of individual process units and soft sensors of different process units within the process for the prediction of outputs such as H 2 purity and CH 4 conversion.Highlighted within this section shows a trend of integrating ML within process modelling for soft-sensor development as well as the use of ANN and GA to optimise both individual process units and whole processes.Furthermore, we look at the recent interest in hybrid modelling and why that allows for improved performance whilst maintaining accuracy and interpretability.The following section provides an in-depth discussion of the ML role has played in the past five years and how it will continue to play a role within H 2 production process modelling.The neural network model showed a good performance with the error between the ML performance and the Experimental data at 0.206%.
W. George Davies et al.

Comparison of conventional process modelling with ML-based process modelling
A significant portion of the literature has looked at the optimisation of blue hydrogen production processes using ML via either the optimisation of whole processes, or the modelling of individual process unit.Specifically, process units like PSA which are highly dynamic processes and therefore, development of detailed and dynamic process models and subsequent simulation of these PSA columns are time-consuming (Li et al., 2018;Zhang et al., 2022a).There has been a trend towards using ML for the modelling and optimisation of carbon capture processes (Subraveti et al., 2019;Yu et al., 2021).As shown by Subraveti et al. (2019) optimising the PSA column using a neural network and optimisation algorithm provides a quicker alternative than the conventional method of modelling.Some work has also included optimisation of the integrated CO 2 and H 2 production process such as the incorporation of sorbent into the reformer for both SMR and ATR (Krzywanski et al., 2018;Nkulikiyinka et al., 2020;Vo et al., 2022;Gul et al., 2023).Table 5 shows the simulation time and accuracy of ML-surrogate models for both process units of H 2 production and entire processes.
Table 5 highlights the increased speeds at which these ML models obtain optimised results when running as a simulation, the increased computational time for first-principle dynamic simulations would be insufficient within an industrial setting whereas developing a surrogate model for either the individual process units or the whole process provides increased speed whilst at high accuracy.As shown in the table when integrating ANNs into process modelling, in order to optimise this surrogate model a GA is used.Utilising GA for the optimisation of ANNs provides a robust framework in which GA can be used to provide optimum KPIs.GA can explore search spaces efficiently to find a global optimum rather than local minima or maxima.
Table 6 showcases the conventional process modelling simulation time of both process units and whole processes of hydrogen production, for each case the time taken for the simulation to run is significantly slower with respect to its ML counterparts.This white-box approach to modelling although provides interpretability is significantly slower, due to the process in which it runs the simulation, modelling the dynamic performance of a PSA unit conventionally requires the solving of multiple non-linear PDEs until it reaches a cyclic steady state.Utilising ML can increase the speed at which PSA and H 2 production processes can be simulated.
Although utilising ML is computationally quick, the accuracy of the model is of great importance.As identified by Rebello et al. (2022) the structure of the neural network is important in relation to the function of the model.A PSA unit for the Fischer-Tropsch was modelled using multiple types of ANNs (FNN, RNN, DNN), these ANNs were both used for simulation and prediction purposes.A comparison was made based on the performance of the ANN for each purpose.It was shown that the ANN architecture significantly affects the performance of the model depending on what function the ANN model is being used for (simulation or prediction).
These two predictor structures are known as nonlinear autoregressive networks with exogenous inputs (NARX) and nonlinear output error (NOE) for nonlinear systems (such as neural networks) (Rebello et al., 2022).NARX depends on past measurements of input and output, in which it assumes errors associated with measurements from input (Rebello et al., 2022).NOE does not depend on past measures of output to make forecasts and assumes errors associated with output.For long-term simulations, NARX is not as good due to the cumulative effect of the error.Within the literature, understanding what neural network can then be used for a specific function to identify which type of neural network, can be applied to the problem at hand.

Soft-sensor development
Soft-sensors' ability to provide real-time information dependent on process conditions provides an asset to continuous processes, as it provides an insight into hard-to-determine information that conventional sensors are not able to do.Nkulikiyinka et al. (2020) has shown the benefits of incorporating soft-sensors into blue hydrogen production by comparing RF and ANNs for soft sensors for the reformer and calciner in the SE-SMR process.It allows for improved process control by providing insight into the process that would be hard to determine otherwise.Further work must ensure these sensors are well trained, tested and validated so that once implemented into real-world applications the error is minimised.One new method of doing this is utilising Monte-Carlo techniques within the training process.This approach was applied to a PSA unit in the Fischer-Tropsch process by Costa et al. (2022).Monte-Carlo training employs a stochastic approach to training the neural network in which a Monte-Carlo simulation is run and trains a new model with each sample.This process allows for the identification of the uncertainty in the predictive results of the soft sensor.The same principles can be applied for H 2 purification in blue hydrogen production in order to identify the confidence within the prediction the soft sensor makes.This soft-sensor development is of importance within biomass gasification for hydrogen production.There has been increasing research within the area (Ayodele et al., 2021;Sezer and Özveren, 2021;Kumar Sharma et al., 2022).Due to the variation within the biomass, uncertainty can arise from the gas concentration produced leading to uncertainty in the H 2 produced from different biomass samples.ML allows for the quantification of hydrogen production from biomass and can lead to highly accurate predictive models for H 2 yield and purity (Ayodele et al., 2021;Sezer and Özveren, 2021;Kumar Sharma et al., 2022) A key issue within the increased use of neural networks and application within chemical engineering especially with the development of soft sensors, is the interpretability of these models as these models are mostly trained on historic literature data or pilot plants (Schweidtmann et al., 2021), this allows for models to have a large dataset and a model that can accurately model a process based on real-world data.Having a fully developed understanding from a first-principle viewpoint is important to understanding the operational factors that drive the process to ensure a stream of highly pure H 2 is being produced, whilst simultaneously the CO 2 capture rate remains high.Conventional modelling using software such as gPROMS or Aspen Plus utilises a white-box modelling approach in which the outputs of the process are based on first-principles equations (mass and heat transfer) as well as thermodynamic and kinetic data.A hybrid approach allows for an interpretable model that can either run a simulation or predict outputs of a process at high speed, allowing for informed decision-making to improve the output, which in regard to blue hydrogen production is H 2 purity, CO 2 capture rate and CH 4 conversion.

Hybrid modelling
Hybrid modelling allows for greater interpretability of the ML-based model.Literature has discussed this hybrid approach to modelling as a key path forward within process simulation (Zander et al., 1999;Pistikopoulos et al., 2021).A recent development within ML within the natural sciences has been PINNs with a focus on the molecular scale of modelling, combining PDEs within the neural network for the modelling of hydrogen processes could provide significant benefits such as uncertainty quantification, increased speed and robustness of models whilst simultaneously providing greater interpretability to strictly ML-based models.Another method which has been more frequently employed within the literature is a combined approach which uses both first-principle modelling as well as a surrogate model developed via ML to produce an output in regards to the KPIs (Hong et al., 2022;Vo et al., 2022).By utilising this approach, it can enhance interpretability without a significant cost to the increased speed of black box models.
Hybrid modelling within blue hydrogen has been increasingly used, especially in regard to integrating the surrogate model with the dynamic model for economic optimisation (Vo et al., 2022).The hybrid approach can be well applied within the industry for H 2 production plants providing insight in-real time into the effects of the operating parameters on the KPIs.Specifically, if those KPIs relate to the economic output of the plant (such as the LCOH).Recent works have demonstrated the superior performance of a hybrid approach to optimise economic output (Vo et al., 2022).In regards to blue hydrogen production, this is highly important, since the LCOH of blue hydrogen production is of greater expense, so ensuring these production plants are optimised in regards to economic activity can provide a competitive edge to these processes.
The combination of white-box modelling (first principle) with blackbox modelling allows for interpretability with increased speed, allowing for interpretable results in-real time.Therefore, the hybrid approach provides engineers working within the H 2 production plant, with quick, interpretable results.These results allow for informed decisions to be made to ensure that the productivity of the plant can be increased in real-time.

Machine learning for whole plant optimisation and digital twins
Whole plant optimisation is key to ensure profits are high due to the early TRL for many blue hydrogen technologies (SE-SMR), currently, it is not at the scale for industry use, whereas grey hydrogen is.ML has been used to optimise a whole SMR plant based in Spain, in which a modular neural network to optimise the plant to increase H 2 purity and the profitability of the plant (Pardo et al., 2020).There has been little work using plant data due to the early TRL of these technologies, therefore a lack of a large dataset to develop a neural network model.
As mentioned earlier, digital twins have become an increasing area of interest within chemical engineering due to increased digitisation and advancements in technology within automation, ML and the IoT.Within the last decades, there has been a rise in the use and development of digital twins within chemical and power plants.Implementing these digital twins within hydrogen production plants can allow for improved process safety (a concern for hydrogen), process optimisation, and enhanced process control for flexible operation in real-time (Shin et al., 2019;Hwangbo and Sin, 2020).This allows for enhanced performance of the chemical plant by increasing product output whilst minimising costs.Furthermore, this increasing digitisation provides a framework in which the hydrogen supply chain can be optimised from production to storage and utilisation in real-time, allowing for optimised scheduling and logistics within the hydrogen supply chain.
Furthermore, a key area in which RL can be utilised within process plant is to enhance the cyber security of chemical plants such as H 2 production (Santorsola et al., 2022).The transition to industry 4.0 provides opportunity to enhance process control and improve OPEX via increasing automation and the IoT.However, increasing automation may lead to increasing susceptibility to cyber-attacks.Ensuring these cyber-attacks are minimal is key to our energy security (BEIS, 2022).Ensuring hydrogen production plants are safe is key to ensuring a safe transition, whilst reinforcing energy security.

Machine learning in blue hydrogen production: what's next?
Machine learning has been utilised to great effect within H 2 production.Table 7 below provides advantages and disadvantages to utilising ML within H 2 across all scales of modelling.Furthermore, the table outlines recommendations of where future research should be directed to further integrate ML within hydrogen production.
ML is able to provide increased speed of modelling whilst maintaining accuracy.The scales of modelling in which has been applied to has ranged from the molecular scale to the process scale and whole plant scale.With regards to PSA/TSA modelling lots of work has focused on the optimisation of these processes using ML (specifically ANNs) as well as the development of soft-sensors for this process.As highlighted by Rebello et al. (2022) careful consideration is required in determining the structure of the neural network and the purpose for which the neural network is being used (simulation or prediction).Although this was applied to the Fischer-Tropsch process, the same principles still apply to the H 2 purification processes.Similarly, in regards to soft-sensor development, recent work focused on the Fischer-Tropsch PSA unit in regards to the performance of the soft-sensor.Costa et al. (2022) showcased the use of a stochastic training technique to enhance the performance, to be implemented into hydrogen production for not only the PSA unit but also for the reformer, WGS and the whole processes.The integration of RF for soft sensors shown by Nkulikiyinka et al. (2020) highlights the improved process performance of ensemble learning techniques.Future work should incorporate ensemble learning to develop soft sensors for H 2 production.Specifically, the employment of a deep-ensemble learning approach to soft sensors of processing units in blue hydrogen production.
RL is also a key area in which chemical engineering research has focused on Shin et al. (2019), Hwangbo and Sin (2020) and Khan and Lapkin (2022).Future research should focus on the use of RL in cyber-security.Ensuring H 2 production is safe is critical for energy security.With the transition into industry 4.0, there will be increasing digitisation of chemical processes, this digitisation will have an increased risk of cyber-attacks, future research should focus on how RL can be incorporated in regard to process control and cyber security.Another area in which RL can be applied is process synthesis of novel processes.Some work has been done already with Khan and Lapkin (2022) regarding ethylene production in which reinforcement learning was used to develop six process configurations for ethylene production to optimise the process.This application allows for further development of processes that can be further optimised to reduce costs whilst maintaining similar or higher H 2 purity and CO 2 capture rates.
ML has a key role to play within process modelling and material development not only within H 2 production but across chemical engineering.Transitioning towards a low-carbon future is essential to reducing the effects of climate change.This transition involves the deployment of infrastructure and industrial processes to ensure this transition happens.ML provides a framework in which to develop these processes (such as blue hydrogen production) at increased speed and ensure that they can make an immediate impact when deployed to ensure the transition is both quick and effective.

Conclusion and future prospective
Blue hydrogen is a necessity to ensure Net Zero is reached by 2050.The recent development in blue hydrogen production has been applied within three main areas: 1. Discovery and development of novel material and molecules to serve as catalysts for the reforming and WGS reactions and sorbents involved in the CO 2 capture process; 2. Optimisation of process performance and predictions of outputs via soft sensors; 3. Dynamic modelling of PSA unit for H 2 purification to improve speed of modelling, whilst maintaining accuracy.
The utilisation of ML provides increased speeds of optimisation of processes as well as increased speeds of discovery of new materials.This is becoming more critical with climate change being an ever-present threat and our time to reduce the impacts of climate change diminishing.Developing solutions that are effective at an efficient rate is critical.ML impact within blue hydrogen has already shown that it can reinforce and accelerate optimisation on both material and process levels.There is still further improvement in how ML can be implemented within blue hydrogen production.Future work should focus on a hybrid modelling approach to modelling these blue hydrogen production processes as well as the integration of the scales of modelling to further enhance speed whilst maintaining interpretability.With low-carbon hydrogen being an increasing necessity to ensure net-zero to prevent and minimise the effects of climate change.Incorporating ML to enhance the speed of developing blue hydrogen production materials and processes is a necessity to ensure a low-carbon future.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper

Table 7
Advantages and disadvantages to incorporating machine learning within hydrogen production.

Advantages
Disadvantages Future Research Direction

Material Scale
• Increased speed of material development • Interpretability of the ANN-based QSAR framework is insufficient.
• Integration of material screening with process performance of selected material • Impact of these materials on the WGS reactor Process Scale • Increased speed of modelling in comparison to conventional modelling approaches.• The development of soft sensors allows for real-time process optimisation.• GA combined with ANNs provides an excellent robust framework for the optimisation of PSA units.
• Interpretability utilising just ML reduces the interpretability of the process.

Fig. 1 .
Fig. 1.A brief description of the colour spectrum of hydrogen.

Fig. 4 .
Fig. 4. Trends in the publication of research topics in this review.The graph on the left highlights the trend in the publication within machine learning.The graph on the right highlights' trends in the number of publications on machine learning with hydrogen production.

Fig. 8 .
Fig. 8. Representation of the relationship between the agent and the environment within reinforcement learning adapted from Sutton and Barto (1998).
W. George Davies et al.

W
. George Davies et al.

Table 1
Novel methods for low-carbon hydrogen production.

Table 2
Material for blue hydrogen production.the SMR.Cheaper than using noble metals.Catalyst support via metal oxides (Ni/ MgAl 2 O 4 ) has been introduced to reduce poisoning and increase conversion efficiency. is used in the SE-SMR process due to the low cost of CaO, and adequate CO 2 capture rate.A key issue is a drop in CO 2 sorption capacity over multiple cycles due to sintering of CaO.CaO owing to its unique layered structure and high surface area.However, the sorption capacity is much lower in comparison to other sorbents.It has a high CO 2 selectivity and can capture CO 2 efficiently within a large range of temperatures (450-650 • C).It also has a lower regeneration reaction temperature (700 • C).However, it has a low hydrogen production rate due to a combined effect of low sorption capacity and slow desorption rate.

Table 4
Research overview of machine learning within hydrogen production.

Table 5
Accuracy and simulation time of ML process modelling of blue hydrogen production units and processes.

Table 6
Time of simulation for conventional modelling of blue hydrogen production processes and units.
• Different neural network structures although provide increased flexibility, can lead to different model performances.Careful consideration of neural network architecture must be considered dependent on the function of your model.• Stochastic modelling/training for soft-sensors model development • Ensemble learning to enhance the predictive capabilities of models • Hybrid modelling, further developing a hybrid approach to ensure high speed whilst maintaining interpretability • Reinforcement learning, utilising RL for H 2 production plant safety and developing novel processes for low-carbon hydrogen production.