DETECTION OF THREE-DIMENSIONAL BELT DEVIATION AND LONGITUDINAL TEARING DEFECTS BASED ON BINOCULAR LINE LASER TECHNOLOGY

.


INTRODUCTION
Belt is the main equipment for underground transportation in coal mines and a major tool for underground coal transportation.Due to long-term exposure of the belt during use, it is susceptible to various factors such as dust, coal powder, etc.If not cleaned in a timely manner, they can lead to severe belt wear, and in severe cases, they can cause belt breakage and cause safety accidents [1][2].In addition, due to wear and tear after long-term use, the belt is prone to deviation.Belt deviation can cause unstable coal flow, leading to belt breakage accidents.At present, manual testing is mainly used, which is not only inefficient and labor-intensive, but also poses safety hazards.Therefore, automatic detection has become a development trend in detecting belt deviation and longitudinal tearing defects [3].Because the belt is an irregular object, it is easily affected by noise after collecting images, resulting in poor image quality and inability to obtain accurate results.In response to these issues, scholars have proposed various methods for image preprocessing, based on filtering, threshold segmentation, and other methods to eliminate the impact of noise.They use methods such as edge detection and corner detection to extract surface features of belts, and use machine learning to classify images [4][5].However, none of these methods can achieve good results.This study focuses on the belt conveyor and uses a combination of binocular laser technology and three-dimensional point cloud data to diagnose and analyze the tearing fault of the conveyor belt.A binocular camera with built-in line laser is used to irradiate the laser onto the belt surface for shooting and transmission to a computer for processing and analysis, thereby achieving automatic detection of belt deviation and longitudinal tearing defects.
The article conducts research through four parts.Firstly, the current research status of binocular laser technology and detection of belt deviation and longitudinal tearing defects is discussed.Secondly, it is the design of a belt deviation and longitudinal tearing monitoring system based on a binocular camera with built-in line laser.Next, performance DIAGNOSTYKA, Vol. 25, No. 4 (2024) Zhang Z, Yang C, Huang C, Gao H, Bao Y: Detection of three-dimensional belt deviation and longitudinal… 2 verification is conducted on the designed system.Finally, there is the conclusion.

RELATED WORKS
In recent years, visual inspection has become a non-contact non-destructive testing technology.It has developed rapidly and has been widely applied in various fields, and is becoming increasingly mature.To improve the measurement accuracy of rotating object's shape, Ye et al. developed a laser vision measurement system consisting of two cameras and a rotation system by converting the rotation axis parameters of binocular vision to reconstruct the three-dimensional shape of this object.After verification, the system could reconstruct the three-dimensional shape of an object at any position based on the rotation axis parameters [6].Wu et al. proposed a dual camera measurement method using line structured light to address the missing point cloud data caused by field of view occlusion in 3D measurement.They improved the accuracy of three-dimensional measurement by using binocular polar constraints to calibrate the physical parameters of the light plane.After verification, this method could effectively reduce the loss of measurement information during the measurement process [7].Wu et al. proposed a twostage defect recognition method using structured light images.They first extracted optical stripes from hue, saturation, range of values, and grayscale space, and used hole filling methods to ensure the integrity of the stripes.Then they calculated the depth information of all the light strips and ultimately formed a depth map for defect localization and measurement.The results verified the feasibility and accuracy of this method [8].Wang et al. developed a dynamic binocular stereo vision system suitable for a large field of view.By using an improved twopoint method, the initial value of camera was quickly estimated, and the pitch and yaw angles of the camera were accurately estimated using the output of the biaxial high-precision platform, achieving online measurement of the rotated three-dimensional spatial coordinates [9].Li et al. proposed a threedimensional measurement method for indoor wave fields in the spatiotemporal domain based on a combination of binocular stereo vision and digital image processing to address the shortcomings of current indoor wave field measurement methods.They used suspended particles to characterize the water air interface layer, densely reconstructed the spatiotemporal domain of indoor waves, and sliced it in space.In the end, they obtained the threedimensional distribution of wave spatiotemporal domains in different directions, thereby achieving efficient and accurate measurement of indoor wave spatiotemporal domains [10].

At present, longitudinal tear detection of conveyor belts has been a key research topic in
China.There are various detection methods for longitudinal tearing of conveyor belts, and the detection device has also developed from singularity to intelligence.Wang et al. proposed a sound-based detecting method to solve the low real-time, low reliability, and so on.They first preprocessed the sound signal, and then extracted the signal features.Finally, they established a neural network for classifying and recognizing longitudinal tearing sound signals of belt conveyors based on their acoustic signal characteristics [11].Yang et al. proposed a method for early warning of longitudinal tearing of conveyor belts based on infrared spectroscopy analysis.They monitored the changes in the infrared radiation field and used the frequency domain characteristic coefficient T to determine whether there was a risk of longitudinal tearing.These results confirmed that this coefficient could identify longitudinal tearing of conveyor belts and provide early warning [12].Zhang et al. proposed a new deep learning-based conveyor belt offset monitoring method to quickly and timely detect the deviation status.They improved the data of YOLOv5, improved their recognition ability for straight lines rather than borders, and achieved belt offset monitoring.These experiments confirmed that this method achieved a good compromise between accuracy and speed, with an accuracy of 90% [13].Liu et al. proposed a deviation detection method for belt conveyors based on detection robots and deep learning to address the issue of conveyor belts being prone to deviation.They captured images by the detection robot and used the Hough transform algorithm to detect the edges of the conveyor belt.Finally, they corrected the detected edge point coordinates and estimated the deviation using the corrected coordinates.These experiments confirmed that this method had high detection accuracy and robustness [14].Sun et al. established an evaluation system for the deviation status of curved conveyor belts that could detect, predict, correct, and warn based on the ARIMA-LSTM joint prediction model.They first constructed a conveyor belt deviation test system, and then established a mechanical model for curved conveyor belt deviation and solved it.Finally, they established an ARIMA-LSTM joint prediction model by using the series parallel weighing method.These experiments confirmed that the evaluation system had high feasibility [15].
In summary, researchers have proposed many methods to detect or avoid the occurrence of belt deviation and longitudinal tearing defects, and have also achieved certain results.However, due to the randomness of the occurrence of deviation tearing, further research is needed for fault detection and identification of longitudinal tearing of conveyor belts.Therefore, a reliable and real-time conveyor DIAGNOSTYKA, Vol. 25, No. 4 (2024) Zhang Z, Yang C, Huang C, Gao H, Bao Y: Detection of three-dimensional belt deviation and longitudinal… 3 belt longitudinal tear detection system is developed through image processing algorithms and binocular laser technology.

DESIGN OF A BELT DEVIATION AND LONGITUDINAL TEAR MONITORING SYSTEM BASED ON A BINOCULAR CAMERA WITH BUILT-IN LINE LASER
The overall architecture of the detection system is elaborated based on image processing algorithms and binocular line laser technology.Then, the overall structure is designed according to the actual detection requirements, and the collected point cloud data are preprocessed to obtain denoised and simplified point cloud data.On this basis, clustering, threshold segmentation and other algorithms are used to extract the deviation tearing features of the collected point cloud.

Overall design of the belt deviation and longitudinal tearing monitoring system
The application of belt conveyors in coal mines is becoming increasingly widespread, and the safety warning and protection of belts are particularly important.Belt deviation and longitudinal tearing are common faults that occur during the operation of belt conveyors, which can cause them to malfunction and seriously affect normal production efficiency and progress [16].Moreover, they can also cause damage to the main parts of the equipment, leading to material scattering on the belt, causing abnormal wear and tear between the belt and the rollers, thereby reducing the service life of belt [17].The belt deviation and longitudinal tearing monitoring system designed this time consists of two parts, namely, acquisition module and the back-end processor.The data acquisition is mainly responsible for collecting image data of the conveyor belt, including: binocular camera with built-in line laser, internal acquisition sensor, light source, lamp, PC, display, controller, and device.The back-end processor includes a data processing module, upper computer monitoring system, and cloud server system, and reserves a remote expert system and a mobile application access interface.

Fig. 1. Installation diagram of binocular camera with built-in line laser
The working environment of conveyor belt belongs to a dusty and humid environment.
Therefore, during data acquisition, the camera lens is easily covered by dust and is also susceptible to interference from factors such as water mist.These make it impossible to obtain normally or the image quality obtained is very poor.If not handled properly, they can cause problems such as missing point cloud data and excessive noise [18].So the study identifies that the conveyor belts' transfer points, machine heads, and other parts are prone to longitudinal tearing.And a binocular camera with built-in line laser is installed below them marking position and tilted 45 degrees to the bottom of the conveyor belt.Figure 1 shows the installation method.The research adds a stabilizing base at the bottom and stabilizes it with the ground through expansion screws, so that the whole mounting frame and the camera are not in contact with the belt conveyor, so as to avoid the vibration generated during the operation of the belt conveyor affecting the camera measurement data.Through the bottom endpoint's cloud data of conveyor belt, point cloud features are extracted and analyzed.And recognition of belt deviation and longitudinal tearing is achieved based on crack defect features and belt folding defect features.Finally, based on statistical multi frame continuous damage information, early warning recognition is achieved.Then, the clockwise angle  between the vector     from other point   to point   in set (  ) and the coordinate axis  was calculated, and the angle set  ′ was obtained by subtracting the adjacent angles in pairs.Equation (4) represents the specific calculation.
) In equation ( 4),  ′ is the angle between two vectors.The elements in the set  ′ will be sorted in descending order to find the maximum angle ′  between them.If ′  exceeds the critical value (usually π/2), it is considered an edge point.After extracting all the edge points, the edge points of the head and tail are removed, leaving only the set of edge points on both sides of the belt.The deviation angle is calculated by the angle between the initial belt edge and the camera center point, and the angle between the real-time belt edge point and the camera center point.Based on the calculated deviation angle, it is determined whether the set threshold has been exceeded.If so, it indicates that the belt has deviated, and then an alarm linkage processing is executed.If not, it indicates that the belt is normal.

Optimization and 3D reconstruction of 3D point cloud data processing
The designed system mainly consists of two parts: data analysis and multi frame information fusion.However, the original point cloud data has a large scale, high density, contains a large number of duplicate points, and is affected by various factors such as environment and hardware, resulting in a large amount of noise [19].Therefore, it still needs to be optimized and processed.Figure 3 shows the data processing of this detection system.In terms of data analysis, the main focus is on processing the single frame laser point cloud data of the belt, to extract the running and tearing features of the single frame point cloud data of the belt, as well as the position, width, and depth information of the point cloud.The point cloud data optimization strategy adopted is to first remove most noise points from the point cloud data, and then remove noise points that are closer to normal point clouds [20].Firstly,  −   is constructed to establish the topological relationship of the point cloud.Each data point in the point cloud is searched within the domain, with a neighborhood value of .Then, the Euclidean distance between the point and the center of the neighborhood is calculated in .The average distance of adjacent values of  at each point is calculated, represented by , and the standard deviation of adjacent points of each  is calculated using equation (5).
On the basis of equation ( 5), noise is removed.The method adopted by the research institute is threshold segmentation, and the threshold is set to .When  ≥ , the filtering point is determined to be a noise point, and then denoised.On the contrary, if  is below the critical value, it indicates normal and should remain unchanged.Equation ( 6) represents the threshold. =  ±  (6) The study selects a stereo simplification algorithm based on average distance.The point cloud is placed inside a cube, and the compression of the point cloud is achieved by pre-selecting the center point of the cube and satisfying threshold filtering conditions based on this distance.The specific methods are to take  = (  ,   ,   ) as the center point, point cloud data as , and then calculate the distance  from  point to any point in this cube, expressed as equation (7).
The distance  obtained above is added, and then the average value  is calculated.If  < , the threshold condition is not met, and the point cloud data is deleted.Otherwise, the point cloud is retained in equation (8).
Taking a point cloud on a line as an example, a preliminary screening is conducted first.Assuming that the quantity of point cloud data on each line is usually  0 , the quantity of point cloud data on each line scanned in real-time is calculated in order, which is  1 ,  2 ,  3 , …  −1 ,   .The threshold is set to , and the point cloud data on each subsequent line is subtracted from  0 , which is Δ.However, due to negative numbers, it is not convenient to compare.So when solving, the absolute value of Δ must be taken in equation (9).|Δ| =  0 −   ( = 1, 2, 3, … ) (9) Assuming that the y-axis point cloud coordinates obtained on a single laser line are  1 ,  2 ,  3 , …  −1 ,   , the difference Δ between the absolute values of adjacent coordinate points is calculated in equation (10).
The threshold is set to   .When Δ ≥   , the th point   and the ( − 1)-th point  −1 on the  -th laser line are considered as a pair of defect points, and the (  ,   ,   ) and ( −1 ,  −1 ,  −1 ) of this defect point are saved in matrix   , represented by equation (11).
In equation (11),   represents a defect point detected on the laser line.The above operation is repeated, and all defect points on a laser line that meet the threshold were stored in matrix   in equation (12).(12) To meet the real-time requirements of belt conveyor deviation and longitudinal tear detection, an image reconstruction method based on deviation and longitudinal tear features is adopted.After obtaining each frame of linear point cloud, it will be processed in real-time.Firstly, a larger container  is constructed to store the defect point cluster , while  is used to store adjacent points obtained during the detection.By setting the speed  and the shooting time  1 returned by the camera, the xcoordinates of all point clouds obtained on the  -th laser line can be determined in equation (13).
=   (13) In binocular camera with built-in line laser detection, the time on the first laser line collected by the camera is taken as  1 .The camera transmits realtime information in the y and z directions of all  points on the  -th laser line.Meanwhile, the xcoordinate on this laser line displays  ,1 =  ,2 = ⋯  , =  , .The point on the  -th laser beam is represented by   [1]~  [𝑛].In this way, the threedimensional coordinate ( , ,  , ,  , ) of any point   [𝑗] on the  -th laser line can be obtained.
For the y coordinate of   [2]~  [𝑛], the difference  , between   [] and the adjacent previous point   [𝑗 − 1] on the laser line  is calculated using equation (10).Comparing  , with   , if  , >   , then   [𝑗] and   [ − 1] are considered as defect point  _ .All the defect points on the  -th laser ray discovered in the above steps are calculated with two-dimensional Euclidean distances one by one from the last element of all small containers  in the current .Equation ( 14) is the calculation method.Figure 5 shows the processing method.
The distance between two lasers is determined as   , and the linear distance ，  =   −  −1 is set.To gather adjacent breakpoints with the same longitudinal tearing edge in a cluster,   > DIAGNOSTYKA, Vol. 25, No. 4 (2024) Zhang Z, Yang C, Huang C, Gao H, Bao Y: Detection of three-dimensional belt deviation and longitudinal… 6 Δ/(  <   ) needs to be met.Otherwise, it will cause the defect points on both sides of the longitudinal tearing crack to be in the same cluster container, resulting in the inability of the subsequent algorithm in equation (15). = / = Δ (15) If the distance between the last element in [] and the current processing line   is less than   , i.e. (  − [].().) <   , then a smaller container [] needs to be maintained.If the distance between the last element in [] and the current processing line   is greater than   , i.e. (  − [].().) >   , it will be removed.Figure 6 shows the processing process.

Fig. 5. Container deletion diagram
To verify the size of all containers in the current , a set length   , frame rate , and cluster size discrimination threshold   related to conveyor belt  in the x-direction are introduced.If the detected defect length is greater than   , a deviation longitudinal tear defect will occur in equation (16).

SYSTEM PERFORMANCE VERIFICATION AND RESULT ANALYSIS
To verify the feasibility and effectiveness of the proposed defect detection system, real-time performance analysis of image algorithms was conducted through pre positioning accuracy experiments, measurement error comparison under visibility changes, and other methods.Then, the realtime performance and accuracy of system detection were calculated for performance verification analysis.

Real-time analysis of image algorithms
A total of 40 sets of pre-positioning accuracy experiments were conducted, using the prepositioning algorithm to obtain the absolute value of the rectangular midpoint coordinates minus the actual measured center coordinates as the error.Figure 6 showed the experimental results.The minimum Absolute Error (AE) on the x-axis was 0.05mm, the maximum AE was 3.44 mm, and the average AE was 1.12 mm.The minimum AE on the y-axis was 0.08 mm, the maximum AE was 2.47 mm, and the average AE was 0.71 mm.The minimum AE on the z-axis was 0.26 mm, the maximum AE was 13.49 mm, and the average AE was 4.26 mm.The error of the pre positioning algorithm on the x and y axes was very small, while the error on the z-axis was the largest.In terms of the pre positioning algorithm, the accuracy had reached the practical requirements.Due to the harsh working environment of the camera, suspended particles such as sand and dust in the atmosphere could reduce visibility, thereby affecting the imaging effect of binocular vision.Therefore, mean square error (MSE) and peak signal-to-noise ratio (PSNR) are used to characterize the visibility of the camera.The smaller MSE is, the closer the image is to the original image, and the larger PSNR is, the better the image quality is, the higher the visibility is.As can be seen from FIG. 8, when the amount of smoke continues to increase, the image quality of the binocular camera continues to decline, and the measurement error of the calibration object gradually increases.When there is no smoke, the measurement error of binocular camera is about 3.16cm.With the increase of smoke amount, MSE gradually increased, PSNR gradually decreased, and image quality gradually deteriorated, resulting in a gradual increase in the measurement error, and the measurement error of the last group reached 6.51cm.Therefore, in practical application, the physical dust removal system can be set up.By setting the calibration object at a fixed position, the binocular camera can continuously observe the distance of the calibration object, and the physical dust removal system can be controlled through closed-loop feedback to keep the measurement error of the calibration object within the failure threshold, so as to prevent the binocular vision measurement method from failing due to low visibility.

Detection performance testing
To test the detection performance of the system, the study analysed it from two aspects: real-time performance and accuracy.The specific detection method was to verify real-time performance by calculating the average processing time.The DIAGNOSTYKA, Vol. 25, No. 4 (2024) Zhang Z, Yang C, Huang C, Gao H, Bao Y: Detection of three-dimensional belt deviation and longitudinal… 8 accuracy was verified by calculating the ratio of the correct number of detections to the total number of detections.For the convenience of statistical analysis, n1 represented the number of times a tear had been detected in the event of tearing.n2 represented the number of times a tear had been detected without tearing.n3 represented the number of times a tear hadn't been detected in the event of tearing.n4 represented the number of times a tear hadn't been detected without tearing.Classifying from 600 images, the values of n1, n2, n3, and n4 were statistically obtained.Table 1 showed the test results of primary and secondary inspections.The probability of correct recognition by the system was above 80%, and the effect of secondary detection was better than that of primary detection.The accuracies of the first level detection and the second level detection were calculated and compared in Table 2.The highest accuracy rate of the first level detection was 92.5%, but the average value was only 89.87%, which was not ideal for detection efficiency.The average accuracy of the two level detection was 97.37%, which was 7.5% higher than the first level detection.Table 3 showed the comparison results of time consumption between primary and secondary testing.The average processing time of the first level detection was 19.45ms.The average processing time of the two-level detection was 23.73ms, which was only 4.28ms longer than the first level detection, meeting the real-time requirements of the detection system.Therefore, the study adopted two-stage detection to improve anti-interference ability and reduce false alarm rate, enabling the system to detect 42.1 frames of images in real-time and accurately within one second.It provided a reliable and effective guarantee for the safety production of coal enterprises, greatly reducing the incidence of accidents.Taking into account vibration shocks, changes in lighting conditions, and the reason why the sensor works for a long time, the image may be accompanied by a small amount of noise.Therefore, the algorithm was used to add salt and pepper noise to the image at a 10% interval, and the foreign body was used to simulate the belt deviation and longitudinal tear defect.The test results after adding 30% salt and pepper noise were shown in Figure 10.When the noise is greater than or equal to 40%, foreign bodies cannot be detected.

CONCLUSION
The current belt conveyor deviation and longitudinal tear detection technology has problems such as low efficiency, complex detection systems, large land occupation, and high costs.In this experiment, the overall structure was designed according to the actual detection requirements, and the collected preprocessed point cloud data were extracted using clustering, threshold segmentation, and other algorithms for deviation tearing features.These results confirmed that the error of the prepositioning algorithm on the x and y axes was very small, while the error on the z axis was the largest.In the absence of smoke, the measurement error of a binocular camera was around 3.03 cm.The system only took an average of 20 to 30 milliseconds to process an image.The average accuracy of the first level detection was only 89.87%, while the average accuracy of the second level detection was 97.37%, which was 7.5% higher than the first level detection.The average processing time of the first level detection was 19.45 ms.The average processing time of the two level detection was 23.73 ms, which was 4.28 ms longer than the first level detection and met the real-time requirements.The designed cloud-

Fig. 2 .
Fig. 2. Crack detection plane expansion Figure 2 is a plan view of crack detection, where AD and BC represent the widths of two interfaces.The length of crack is equal to the distance from the starting boundary point to the ending boundary point, which is the product of the distance between columns ℎ and the number of point cloud columns .The coordinates of the starting and ending points of the crack detection area are recorded, providing a basis for future multi frame information fusion processing.The actual distance length in equation (1) must be calculated based on the current conveyor belt speed data. =  ×   (1) In equation (1),  represents the actual distance length. represents the calculated length of point cloud data. represents the conveyor belt speed. represents the time of a single frame of data. represents the number of point cloud columns in a single frame of point cloud data.After calculating the degree of longitudinal tearing of the belt, it is also necessary to calculate the degree of belt deviation.This method performs edge

Figure 7
Fig. 6.Error diagram of predetermined positionFigure7was the output diagram of adjacent point intervals in a single frame point cloud.Figure7(a) was the output image of the neighbourhood spacing of a single frame point cloud obtained through complete surface scanning.The horizontal axis represented the coordinate value of the sample point on the y-axis, while the vertical axis represented the difference in the y-coordinate of adjacent points.The distribution of sampling points along the y-direction was basically consistent at the same working height from the output image laser line.Figure7 (b)was the output of the single frame point cloud neighbourhood spacing obtained by scanning the surface with cracks and defects.When there was a defect, a pair of adjacent sample points would appear, and the difference in their y-coordinate values was much greater than the difference in general neighbouring points.Therefore, the points with a large y-coordinate spacing were considered fault breakpoints.
Figure7was the output diagram of adjacent point intervals in a single frame point cloud.Figure7(a) was the output image of the neighbourhood spacing of a single frame point cloud obtained through complete surface scanning.The horizontal axis represented the coordinate value of the sample point on the y-axis, while the vertical axis represented the difference in the y-coordinate of adjacent points.The distribution of sampling points along the y-direction was basically consistent at the same working height from the output image laser line.Figure7(b) was the output of the single frame point cloud neighbourhood spacing obtained by scanning the surface with cracks and defects.When there was a defect, a pair of adjacent sample points would appear, and the difference in their y-coordinate values was much greater than the difference in general neighbouring points.Therefore, the points with a large y-coordinate spacing were considered fault breakpoints.

Figure 7 (Fig. 7 .
Figure7was the output diagram of adjacent point intervals in a single frame point cloud.Figure7(a) was the output image of the neighbourhood spacing of a single frame point cloud obtained through complete surface scanning.The horizontal axis represented the coordinate value of the sample point on the y-axis, while the vertical axis represented the difference in the y-coordinate of adjacent points.The distribution of sampling points along the y-direction was basically consistent at the same working height from the output image laser line.Figure7(b) was the output of the single frame point cloud neighbourhood spacing obtained by scanning the surface with cracks and defects.When there was a defect, a pair of adjacent sample points would appear, and the difference in their y-coordinate values was much greater than the difference in general neighbouring points.Therefore, the points with a large y-coordinate spacing were considered fault breakpoints.

Table 2 .
Comparison results of accuracy of first-level detection and two-level detection

Table 3 .
Compares the time consuming results of first-level detection and two-level detection