A computer vision approach to improve maintenance automation for thermal power plants lubrication systems

1. Introduction

Thermal power plants (TPPs) represent an important asset of the current energy infrastructure with particular reference to developing countries (Fausing et al., 2020). According to the statistical bulletin on National Economic and Social Development in 2020 released by the National Bureau of Statistics of China, the country's electricity generation capacity in 2020 was 7,779.06 billion kWh, of which, the thermal power generation capacity was 5,330.25 billion kWh, accounting for 68.52% of the national electricity generation capacity (National Bureau of Statistics of China, 2021). In India, the major portion of total captive power comes from TPPs from coal, oil, natural gas, etc., of which coal-fired units contribute 41% of global electricity (Chandrasekharan et al., 2014). TPPs are industrial plants that use coal, oil and natural gas as fuel to produce electricity. In modern TPPs the core equipment are boilers, steam turbines and generators. The basic requirements of TPPs operations are to ensure safety, economy and power quality (Raza and Hameed, 2022). From an operational perspective, modern TPPs consist in very complex industrial integrated systems, characterized by extensive pipework (Bao et al., 2022) requiring continuous improvement in all aspects of their operations, including maintenance, in order to ensure competitive operational performance (Singh and Gurtu, 2022). Therefore, accurate and timely detection of anomalies is essential for the safe and economic operation of complex TPP environments (He et al., 2022).

In TPPs, the lubrication system supplies lubricating oil for auxiliary equipment to reduce the friction in the rotating mechanical parts of the equipment. Most common lubricants are oil and grease, in reducing equipment wear at the same time also play the role of sealing, cooling, clean, anticorrosion (Sarkar, 2017). The basic purpose of a lubricant is to reduce friction and wear between two surfaces moving relative to one another. In most cases, a lubricant also dissipates heat, prevents rust corrosion, acts as a seal to outside contaminants and flushes contaminants away from bearing surfaces (Nowak et al., 2019). However, the equipment utilized for oil transportation for a long time is exposed to various degradation mechanisms such as corrosion, cracking, fatigue, erosion, etc. These problems are likely to lead to the occurrence of oil spills (Rachman and Ratnayake, 2019). Lubrication system failures during the production process will lead to insufficient lubricant supply, with consequent accelerated equipment wear, oil leakage, overheating and ultimately affect the boiler operation (Sarkar, 2017). Leakage of lubricating oil is an important source of pollution. These oils biodegrade relatively poorly in soil and water and degradation products and wear debris may adversely affect agricultural areas, forests, harbors and parks (Nowak et al., 2019). The chemical composition of lubricating oil has a potential chronic toxicity, with negative impact on animals and humans (Mangas et al., 2014). Additional safety concerns involve lubricating oil low ignition point, that, along with the operating temperature of the equipment could result in accidental combustion (Tao, 2016). Such considerations highlight a strong need of pipeline maintenance operations, to ensure the safety of the pipeline, its related equipment and the overall plant.

In this context, an effective maintenance plan is the essential approach to lubricating system failure risk prevention (Yang, 2004). Maintenance is classified into three types: improvement maintenance (IM), preventive maintenance (PM) and corrective maintenance (CM) (Divya et al., 2022). IM aims at optimizing maintenance in the design stage, while PM seeks to predict failure to ensure the normal operation of equipment, and CM involves timely maintenance operations after failure. In engineering practice, IM and PM cannot completely eliminate all failures therefore CM is still widely used in actual production environments (Wang et al., 2014). As a maintenance method to find the cause of the failure from the fault phenomenon, CM plays a very effective role in the maintenance and improvement after the failure. As in TPPs, image recognition technology be used as a CM tool to monitor liquid leakage pipelines and provide timely corrections and optimizations (Bao et al., 2022).

Due to the large number of pipes in TPP with close spatial arrangement and highly variable lighting conditions, computer vision is usually used as the main enabling technology for maintenance automation. Literature review show interesting contributions in the use of ultraviolet (UV) fluorescence for monitoring purposes. Simeone, Woolley et al. utilized UV fluorescence to monitor clean-in-place industrial cleaning processes for a variety of food fouling (Simeone et al., 2018; Woolley et al., 2018). A. A. Ionin et al. used ultraviolet filaments to detect thin oil films on water surface and investigated the possibility of using ultraviolet laser pulses as a spectrometric method for detecting oil and oil surface films (Ionin et al., 2016). Sharikova et al. developed an ultraviolet laser-induced fluorescence system for real-time monitoring of mineral content in tap water (Sharikova and Killinger, 2008). Kavanagh et al. utilized fluorescence to detect oil-bearing wastewater produced by bitumen extraction (Kavanagh et al., 2009). Contributions on oil detection on water surfaces using UV-based techniques can be found in (Chase et al., 2010; Sun et al., 2011; Son et al., 2011) for various types of process waters. As regards power equipment components, Zhang Jing et al. designed and developed a portable oil leak detection device for high-voltage power equipment based on ultraviolet fluorescence effect (Liu et al., 2019). Xu-Xu Li et al. studied the composition and fluorescence characteristics of oil in industrial equipment and proposed a method to detect oil leakage by using the fluorescence characteristics (Li et al., 2022). Dong Baoguo proposed a color-characterization-based oil detection method for oil leakage in transformer equipment, by analyzing the Hue-Saturation color histogram from digital images (2013). Yan Haijun et al. proposed a method for detecting oil leakage in pipes by determining the extent of the oil leakage by judging the approximation between an image taken in real time and a UV image taken previously without an oil leakage (Yan et al., 2019). Zhou Ju proposed to use UV intensity control to obtain real-time oil leakage information in the monitored target area, and this method was applied to monitor oil leakage on the target surface (Ju, 2019).

An extensive survey at Huaneng thermal power plant facilities, Shantou, China, aimed at characterizing the lubrication system pipework for maintenance automation purposes.

Such survey shows a large number of equipment and pipework to be inspected have a complex and close space arrangement, making image acquisition particularly challenging; The lighting conditions in the factory workshop is highly variable in terms of light color and intensity, resulting in a hard image acquisition process. Oil leakage from lubricating systems appear as irregularly shaped semi-transparent effusions on various types of surfaces resulting technically difficult to identify for most image-based technologies. At present, most of the lubricating oil leakage inspection is manually carried out by operators, resulting in a low efficiency, time-consuming procedure with possible safety concerns. Despite an evident potential in terms of accuracy an safety, automatic maintenance techniques are not fully deployed in such complex environment (Lu et al., 2020).

The literature review on oil leakage detection techniques and the survey carried out in a real plant environment, highlighted a number of research and technological gaps for automatic maintenance systems development and implementation. In this respect, the method of detecting oil droplets in water (Ionin et al., 2016; Sharikova and Killinger, 2008) is not suitable for the detection of oil leakage in power transformers, due to the following limitations: (1) The detection accuracy will decrease with the decrease of circulating water purity; (2) Need a lot of manual detection; (3) Serious waste of circulating water; (4) Complex equipment installation and a high number of detection points; (5) High system cost (Chen, 2010). A general limitation of any machine vision-based oil leakage detection system consists in the complex environment to be inspected. Variable lighting conditions and the distributed presence of pieces of equipment represent a technological challenge for digital imaging-based inspection (Lu et al., 2017). In addition, the UV imaging can be badly affected by the reflective characteristics of auxiliary items, such as plastic tags, rubber tubes, plastic straps and other plastic products which can produce image noise under a UV light source. Overall, due to the complexity of the actual environment (see Figure 1) and the limitations of the state-of-the-art methods, the existing oil leakage methods cannot be directly used in TPPs environments for maintenance automation purposes.

To overcome the above-mentioned issues, this paper proposes a novel real-time inspection system using an automated guided vehicle (AGV), a dual light source illumination and a tailored image processing algorithm. The AGV moves along a pre-defined inspection path to acquire digital images in correspondence of a number of preset inspection points. The dual light source allows for a reduction of image background noise while providing a proper illumination intensity throughout the inspection process. The image processing algorithm ensures a suitable inspection accuracy. Such inspection system aims at boosting the automation of maintenance operations in terms of equipment inspection with reference to lubricating oil leakage detection in TPPs. The application can include the possibility of a continuous and real-time automated inspection, allowing for a prompt maintenance of the production environment.

The remainder of this paper provides a definition of the inspection system requirements in section 2 in terms of expected performance, AGV hardware, lighting system and image acquisition. Section 3 provides for a detailed description of the inspection system modules. Section 4 reports an experimental verification of the proposed system with an industrial case study within Huaneng TTP to verify the inspection capabilities. The experimental results are presented and discussed in section 5 demonstrating the effectiveness of the proposed inspection system for oil leakage detection purposes. Concluding remarks and research outlook are reported in section 6.

2. Inspection system requirements

As regards the detection sensitivity, the system should be able to detect oil leakages as small as 100 cm². At the same time, the whole system equipment meets the following communication performance requirements: image data transmission should be guaranteed within the whole factory facility resilient to interference and obstacles (e.g. factory walls). The system data storage capacity should allow saving and handling at least 18 months of data. Finally, concerning the industrial suitability, the detection accuracy should be higher than 90%.

3. Inspection system architecture

The inspection system proposed in this paper has a modular structure as illustrated in the flow-chart reported in Figure 7. The modules are namely the image acquisition module, the image processing module and the operation module. A detailed description of the various modules is reported in the following subsections.

4. Case study

To demonstrate the effectiveness of the proposed inspection system, a case study is reported considering a real power plant facility, the Huaneng plant located in Shantou, China. In this context, two oil stations, namely Oil Station A and Oil Station B were chosen for the experimental activities. The oil station consists of one oil tank, two oil filters, thermal energy storage tank, instrument panel, oil cooler and auxiliary pipework. The base under the oil station is considered as a part of the equipment to be inspected, as shown in Figure 12. The main function of the oil station is to enable the flow of the lubricating oil through other machine components.

The lubricating oil utilized in the plant is a Total Energies No. 32 engine oil, which is highly transparent with a light-yellow tint in visible light, resulting particularly difficult to detect by the human eye considering the ambient light conditions.

The experimental setup includes an OPT-DPA6024-2 UV light source with a wavelength range of 300–450 nm. LED stripe light source (P-BL2-147-28-B) with a wavelength range of 465–475 nm. The light source controller is equipped with a 256-level brightness control, which can be adjusted manually or via software.

The image acquisition module utilizes a DS-2ZMN3008 © CMOS camera with 3 million pixels resolution (2048 × 1,536), horizontal field of view angle 57.2–3.5° (wide-angle-telescope), 12× digital zoom, aperture F1.6-F4.4, horizontal view 58.9–2.11° (wide-angle - viewing distance), close-up range 10–1500 mm (wide-angle - viewing distance). The image acquisition system is mounted on an AGV supplied by Huaneng Power Plant. The AGV size is 738 ×592 × 864 mm, and the weight is about 80 kg. The AGV has a four-wheel drive, with a top speed of 1.5 m/s, using radar navigation with 8 h battery autonomy.

A series of experimental tests has been designed and realized to validate the detection capabilities of the inspection system in terms image processing effectiveness and detection accuracy.

The testing area for the experimental validation of the methodology proposed in this paper is represented in Figure 13. 13 inspection points have been identified within the testing area. In this context, the AGV moves along a predefined inspection path covering the 13 inspection points distributed over the two oil stations, as illustrated in Figure 13. Each inspection point corresponds to a specific AGV positioning location which is designed to provide for a clear view of the inspection point itself, minimizing cluttering and shading.

Figure 14 illustrates the experimental procedure. A number of artificially simulated oil leakages were randomly placed along the 13 inspection points throughout the testing area (see Figure 13).

It is important to note that an inspection point could contain more than one inspection targets, as illustrated in Figure 15(a)-(c), possibly requiring the acquisition of more images to fully cover the whole inspection point. Specifically, a total of 23 inspection targets are considered in this paper, distributed over the 13 inspection points, as shown in Figure 13.

Experimental samples of oil leakage have been applied in correspondence of each inspection target manually, dropping a fixed amount of oil (30g) from a height of 10 cm. In this way each inspection target is characterized by an artificial oil leakage with a surface of roughly 100 cm2. Such value is to be considered as the lower threshold for detection purposes, as required by the industrial partner.

The AGV moves along the inspection path and reaches an inspection point. Here the system performs the image acquisition and processing algorithm described in section 3. As mentioned above, the number of inspection targets at each inspection point is not necessarily only 1, so the AGV needs to position the camera to cover each inspection target, then it moves to the next inspection point.

In the experimental programme performed for this research work, each inspection target has been tested 20 times by changing the location of the artificial oil leakage within the same inspection target area. In this way a total number of 460 tests were carried out.

The inspection task in these experimental tests consists in a classification problem in correspondence of each inspection target. If the system correctly detects the oil leakage the test result is recorded as correct classification. Alternatively, if the system fails to detect the oil leakage, the test result is recorded as misclassification.

5. Results and discussion

Out of 460 experimental tests, only 4 cases of misclassification were found, yielding to an overall accuracy rate of 99.13% and an error rate of 0.87%. Table 1 reports the experimental tests results for each inspection area, inspection points and inspection targets in terms of misclassified instances.

It is important to stress how the experimental fine-tuning of both lighting system and image processing parameters allowed to successfully eliminate all the false positives instances of encountered during the whole experimental campaign.

The overall classification performance appears to be highly suitable for industrial applications. As concerns the misclassifications, the following considerations can be made. For the inspection targets 2a and 13a, the main reason for the misclassified instances was the reflection of some extraneous light source on the inspection point, due to random circumstances, altering the provided illumination and over-exposing the digital images. The oil leakage in the misclassified instance at inspection target 6c was only partially included within the inspection target field of view resulting poorly illuminated by the light system therefore producing less fluorescence. A similar situation can be defined for the misclassified instance at inspection target 12c. Here, the artificial oil leakage was applied in an area for which the field of view was partially occluded by dense pipework, resulting in the presence of an obstacle between the camera and the oil leakage to be detected. Such issue suggests, for the next research stage, an improved design of the camera setup and inspection path definition, including the possibility of field of view remote adjustment.

6. Conclusions

In view of the environmental complexity of the area to be inspected, a novel automatic inspection system endowed with an AGV, a double light source and a specific image processing algorithm, was developed in this paper to aid the maintenance automation by performing oil leakage inspection on TPP equipment.

The experimental results show that this inspection system is highly stable, accurate and shows good applicability. From an image processing perspective, the proposed system effectively identifies the oil leakages and significantly reduces the negative effects of background noise, such as reflections from transparent rubber hoses and plastic equipment casings and plastic belts, typical of TPP equipment environment. Furthermore, the inspection system takes full account of the diversity and complexity of scenes within power plants and provides a real-time determination of oil leakage with significant advantages in terms of identification and maintenance support.

In this respect, the manual maintenance scenario requires 6–7 operators to likely spend more than 4 h a day in the workshop to manually inspect and record oil leakages in various parts of the equipment without interruption. With the correct configuration and implementation of the proposed inspection system, a single AGV can be utilized to inspect the whole factory for 12 rounds every day, guaranteeing better detection performances compared to the human, saving time, resources and reducing the related safety risks. During manual inspection operations, the detection accuracy usually reaches no more than 70–80%, while the accuracy of this inspection system can reach up to 99.13%. As concerns the operator safety, the manual inspection procedures required operators to hold the UV flashlight to each point being in this way exposed to long-term irreversible damages to human skin such as inflammation, cancer and other diseases. Near ultraviolet light with a wavelength of 290–400 nm can cause damage to the human eyes, which is one of the pathogenic factors of senile cataract. Therefore, the automated maintenance inspection system has a clear potential to improve the operators' working conditions.

Further research is required to improve the system capabilities specifically: (1) adaptability to ambient light intensity in the inspection area. In this respect, both hardware and software should be able to perform an adaptive calibration to dynamically compensate the illumination conditions. (2) At present, the automatic maintenance mainly consists in the automation of inspection procedures, while the maintenance operations themselves are still carried out manually based on a visual diagnosis carried out by the operators. The next research steps are aimed at providing an automatic oil leakage diagnosis by performing a root cause analysis and potentially endow the AGV with a mechanical arm to carry out the daily maintenance of wiping, cleaning, lubricating and adjusting the equipment of the TPP, so as to maintain and protect the performance and technical status of the equipment.