Intelligent Detection Method for Froth Flotation Based on YOLOv5

YOLOv5 based flotation foam intelligent detection method is based on machine vision technology, and uses YOLOv5 based deep learning framework to tackle the problem of data mining of flotation froth. It provides real-time video of the foam, and at the same time extracts real-time status data such as the size and gradation ratio, flow direction, flow rate, and life cycle of the flotation foam for the flotation staff. This provides effective and accurate data support for standardizing and optimizing flotation control. The method is equipped with automatic detection and control reference functions of abnormal working conditions. When the flotation working condition changes, it will automatically prompt the process operators and output the control reference value of the flotation tank liquid level and air intake based on historical data to assist in adjusting the control strategy, thereby improving the response speed of abnormal working conditions. The system helps to stabilize the production and operation rate, improve the stability of flotation process control, and provide strong data support for fine flotation process control in concentrators of mining companies.


I. INTRODUCTION
Among the various operations concerned of mining companies production, flotation is an important link which directly related to the quality of concentrate products. The improvement in basic data is particularly important to the level of flotation process control. The current flotation operation is mainly based on the foam state on the liquid level to determine the flotation working conditions and the changes in the floating slurry. It is difficult for the operating process to form a corresponding standard since it is impossible to quantify the basis and the control accuracy is highly dependent on manual experience. Therefore, the appropriate method to obtain relevant basic data and ensure its accuracy, real-time, and consistency is the essential of improving the control of the flotation process [1]. In the froth flotation process, experienced workers adjust the flotation operation according to The associate editor coordinating the review of this manuscript and approving it for publication was Junchi Yan . the visual results of the flotation froth which indicates the low level of automation. In actual production, most of the workers directly judge the flotation conditions and make adjustments based on their own experiences, such as the concentration of the slurry and the number of flotation reagents [2]. The flotation process has a low level of automation and large judgment errors. Therefore, it is of great urgency to improve the flotation performance for saving manpower and material resources and achieving automatic production of working conditions.
In recent years, due to the development of artificial intelligence and deep learning technology, some encouraging progress have been made in the analysis and research of flotation foam images at home and abroad. Y Fu of Curtin University and C Aldrich of Stellenbosch University used three pre-trained neural network structures, namely AlexNet [49], VGG16 [50], and ResNet [51], to estimate foam levels from industrial image data [3]. Scholars from Hatam Ambia University in Tehran, Iran have developed a convolutional neural network (CNN), which can be used to perform the analysis under different process conditions (airflow, foaming agent dosage, pulp solids, foam depth, and collector dosage). The foam images collected by the coal flotation column are classified [4]. The gray-level co-occurrence matrix and Haralick featured set derived from the local binary model considered by Chris Aldrich and Xiu Liu of Curtin University are realized through the mature method of multivariate image analysis, combined with the traditional multivariate statistical process monitoring method, Reliable monitoring of industrial platinum group metal flotation plants [5].
The Key Laboratory of Coal Processing & Efficient Utilization, the Ministry of Education, School of Chemical Engineering and Technology, China University of Mining & Technology in Xuzhou, China tested the image characteristics of flotation foam on industrial data sets and used support vector regression to predict ash content to obtain certain industrial applications [6]. Scholars from the Department of Information Science and Engineering of Northeastern University used the transfer learning method to design a soft measurement method for iron ore tailings grade classification. The experimental results show that the proposed hybrid deep neural network is effective in the field of iron ore foam flotation [7].
It can be seen that there are still many researches based on flotation foam images, but most of them are concentrated in the laboratory and the research and application of coal flotation technology are more than a few. Therefore, in order to effectively solve the resource recovery and comprehensive utilization of non-ferrous metal mines in the process of copper flotation in my country, and effectively solve the outstanding problems of tailings loss, unstable indicators, serious foam leakage, energy consumption, and large dosage of chemicals in the copper flotation process. In this paper, combined with the actual characteristics of mines, a series of researches have been carried out on the characteristics of the foam and the identification of abnormal working conditions in the copper flotation process by using the intelligent detection method of flotation foam based on YOLOv5. Research content and successful application in the actual production process. At the same time, the research of flotation foam images is a long-term continuous work, and it will also bring more benefits to the actual flotation production process with the development of artificial intelligence, deep learning, and other technologies to bring more benefits to the actual flotation production process.
The preliminary content and the short summary of this paper will be illustrated below: 1) In this paper, the core image recognition engine built by the object detection algorithm model of YOLOv5 + SORT is applied to the flotation foam analysis technology for the first time. Through the in-depth study of bubbles in the flotation cell, YOLOv5 model is recommended to identify bubbles based on the large difference in the number and size of foam and the existence of a large number of small bubbles. By testing the system using YOLOv5s, the processing speed is the fastest on the basis of the accuracy meets the requirements. The multi-target tracking algorithm in this paper combines SORT algorithm with Kalman Filter and Hungarian algorithm, and the speed can reach 260Hz, which is 20 times faster than the ordinary algorithm.
2) The three classifiers designed in this system, which are trained by ResNet deep learning model, are applied to the recognition of flotation foam analysis system. The abnormal flotation conditions are identified by counting the proportion of shadow area in the flotation cell and using the object classification algorithm model based on ResNet.
3) This paper adopts the end edge architecture to realize edge computing and greatly reduce the pressure of data transmission. The image data is converted into the moving speed, size, state, color and other indicators of foam for transmission. Compared with information transmission based on images and videos, the resources occupied are greatly reduced, and some abnormal situations can be controlled locally and timely. 4) Using multi-modal flotation foam image recognition technology, deep learning and big data technology, online and real-time measurement and intelligent calculation of key parameters and indicators such as foam color, size, flow rate, state and texture are realized. At the same time, the data-driven model of process indicators and control indicators is established based on the analysis results of foam through multi-modal technology.

II. RELATED WORK
The research content of this paper is the flotation foam detection system based on image processing. Image processing is widely used in artificial intelligence, communication technology, physics and other research fields, but it is relatively less used in mineral processing industry.

A. FORTH FLOTATION IMAGE ANALYSIS
In terms of the current research status, scholars have completed a lot of work on the color and characteristics of flotation foam. Vallebuona described the bubble size distribution in flotation machines on an industrial and laboratory scale [17]. Murphy described the movement trajectory of the flotation foam by using the Laplace method [18]. Moolman from South Africa earlier introduced the image processing algorithm into the flotation research, and has done a lot of research [19]. At the same time, his color feature processing method of fast Fourier transform is also an important link in the development of foam image analysis [20].
Hatonen proposed to calculate the mean, standard deviation, skewness and peak value of RGB components in RGB color space. This method is simple, and the color space is related to hardware devices [21]. Combined with 3D fractal and color analysis, Bonifazi proposed a method to predict the flotation index of complex sulfide minerals [22], [23]. For the image processing of phosphorus oxide flotation foam, Lin proposed and implemented a bubble size estimation method using improved texture spectrum binary image [24]. Bartolacci reasonably collects the texture characteristics of foam through the composite application of gray level co-occurrence matrix and wavelet analysis, then establishes an empirical model of concentrate grade by the least square method, and finally carries out feedback control on the flotation process [25]. Liu Jinping extracted the texture amplitude spectrum and phase spectrum of foam image in different scales and directions by using the characteristics of Gabor transform, and identified the production status of flotation industry. This achievement has also been used in actual industrial production [26].
To sum up, image analysis has many research achievements in flotation, and has always been a research hotspot. However, the application results in actual production are relatively few. The system developed in this paper has had a positive impact on production efficiency after being used in the field. Details will be described in V. A.

B. OBJECT DETECTION ALGORITHM
Geoffrey Hinton put forward the concept of deep learning with milestone significance for computer vision in 2006 [27]. Since then, in-depth learning has entered the vision of scholars and received extensive attention. Depth learning has been developing continuously in recent years. Among them, convolutional neural network [28] (CNN), a frequently used deep learning model, was introduced by R. Girshick and other researchers took the lead in applying it to object detection tasks [31], which can greatly improve the accuracy of detection. Since then, object detection has developed at an unprecedented speed.
There are two main categories of object detection algorithms based on depth learning: 1. Detection and recognition algorithm based on regression framework. After passing the input image through a convolutional neural network, this kind of algorithm can directly obtain the predicted bounding box and the category probability of the contained objects, which belongs to one stage object detection algorithm. The detection speed has been greatly improved.
The main representatives are YOLO series and SSDs. YOLO [32] (You Only Look Once) is the first one stage detector based on deep learning Joseph et al. proposed in 2015. Compared with the previous ''proposal detection + verification'' method, the difference of this network is that the original image is divided into multiple different regions, and the probability prediction of the bounding box and category of each region is carried out at the same time. This end-to-end detection method achieves a detection speed of 45 frames per second, with high real-time performance, and can be used for video detection. YOLOv2 [33], YOLOv3 [34], YOLOv4 [35], and YOLOv5 [36] They are all improved versions of YOLO.
2. The two-stage object detection algorithm based on the Region Proposal strategy, as the name implies, is not a onestep detection process, but a coarse to fine detection.
The main representative is RCNN series. R-CNN [31] (Regions with CNN features) was established by R Girshick team proposed that candidate boxes should be extracted first during detection, and then convolutional neural networks should be used for feature extraction. This method is not used in this article and will not be repeated here.
As a popular object detection algorithm in recent years, they are widely used in various scenarios, such as scene text detection [42], [43], [44], remote sensing images [41], [46], medical images [48], etc. In this paper, we combine object detection algorithm with SORT algorithm, and the detection speed is greatly improved compared with the ordinary algorithm, thanks to YOLOv5. This article will describe this in detail in the third section.

III. DESCRIPTION OF OBJECT DETECTION PROBLEM IN FLOTATION FOAM IMAGE
The flotation process applied by the foam image analysis system is the process flow of copper and sulfur selection. The overflow slurry produced by the cyclone will be separated by the slurry distribution tank (differentiating box) after being removed by the slime removal screen and then enter the first and second series of flotation mixing tanks. The overflow ore slurry enters the flotation system after mixing and adjusting the slurry in the agitation tank. It is subjected to a rougher copper separation by the KYF320 flotation machine in the No. 1 tank, and then a second copper rougher separation by the KYF320 flotation machine in the No. 2 tank. The produced rougher concentrates are merged into the KYF70 flotation machine in the No. 2 tank for a selection. The primarily selected concentrate enters the KYF70 flotation machine for secondary selection in the No. 1 tank and produces flotation copper concentrate and copper selected tailings. Copper beneficiation tailings return to copper beneficiation I and copper rougher II in sequence. Copper rougher separation II tailings enter the second slot KYF320 flotation machine for scavenger, and enter the second slot KYF320 flotation machine for scavenger and scavenger concentrate return. Sweep the tailings into the sulfur beneficiation operation. The specific process flow chart is shown in Figure 1. This time, according to on-site requirements and process analysis, the more representative No. 2 flotation machine for the second rougher separation and the second flotation machine for the second selection was selected for image collection and analysis For the concentrator, the flotation operation is a key production link which is directly related to the quality of the concentrate product, and its process control level is closely related to the economic benefits. The current operation of flotation positions is mainly based on the judgment of the froth state of the liquid surface of the flotation tank to confirm the changes of the flotation conditions and the floating pulpSince there is no quantitative operation basis, the control accuracy is highly dependent on manual experience, and it is difficult to form a corresponding specification for the operation process. How to improve the flotation process control level, standardize the flotation process control process, and reduce the control gap caused by insufficient personnel experience has become the focus of lean flotation operation control. Therefore, it is necessary to use deep learning algorithms to overcome the problems of flotation foam object detection and extraction. Through analysis of real-time status data such as flotation bubbles size classification ratio, flow direction, flow rate, life cycle, and abnormal flotation conditions, it can effectively solve the copper flotation process tailings loss, index instability, serious foam leakage, energy consumption, and dosage of chemicals in the copper flotation process [8], [9].

IV. FLOTATION FOAM OBJECT DETECTION METHOD BASED ON YOLOV5
A. RELATED THEORETICAL BASIS 1) SORT TARGET TRACKING ALGORITHM It plays an important role in the detection performance of the multi-target tracking algorithm. By changing the detector, the algorithm efficiency can be improved by 18.9%. The SORT [10] algorithm selected in this paper is only combined with the Kalman Filter and the Hungarian algorithm, and its calculation speed can reach 260Hz, which is 20 times faster than the 2016 SOTA algorithm.
The target model is the representation and motion model used to propagate the target identity to the next frame. The SORT algorithm uses a linear constant velocity model independent of other objects and camera motion to approximate the inter-frame displacement of each object. The state of each target is modeled as: Among them, u and ν represent the horizontal and vertical pixel positions of the target center, and s and r represent the ratio (area) and aspect ratio of the target bounding box, respectively. Note that the aspect ratio is considered constant. After the object is detected in association, the target state is updated with the detected bounding box, and the velocity component is optimized through the Kalman filter framework. If there is no detection related to the target, the linear velocity model is used to simply predict its state without correction.

2) CANNY ALGORITHM
The Canny edge detection algorithm [11], [12] is a multilevel edge detection algorithm developed by John F. Canny in 1986. As of August 2014, the paper published by Canny has been cited more than 19,000 times. Canny edge detection must first perform Gaussian denoising on the image. The commonly used method of differential operation is to use the template operator to map the center of the template to each pixel position of the image. Then perform mathematical operations on the center pixel and its surrounding pixels according to the formula corresponding to the template and calculate the value of the corresponding pixel of the image.
In the experiment, the Laplacian operator, Soble operator, and Roberts operator are selected as template matrix. The Laplacian operator is a second-order differential operator, and its accuracy is relatively high, but it is too sensitive to noise, and the effect is poor in the presence of noise. The Robert operator is also very poor when the illumination is uneven, and it is more sensitive to noise. Here takes a simpler template as an example to make the following introduction: 1. Calculate the gradient values in the x and y directions to obtain the gradient magnitude and gradient direction of the grayscale: Gx = (hd(x)(y + 1) − hd(x)(y) + hd(x + 1)(y + 1) − hd(x + 1)(y))/2; (2) Gy = (hd(x)(y) − hd(x + 1)(y) + hd(x)(y − 1) − hd(x + 1)(y + 1))/2; (3) G(x)(y) = (int)Math.sqrt(Gy * Gy + Gx * Gx); (4) angle(x)(y) = Math.atan2(Gy, Gx); 2. Selection of high and low thresholds. Generally, the relationship between the high threshold Th and the low threshold Tl of the Canny operator is Tl = 0.4 * Th. The high threshold value is selected according to the purpose of binarization. Th usually selected as follows: the gradient amplitude matrix is counted in the gradient value, all the gradients are accumulated and summed, and the amplitude value of q% (q% between 0.75-0.85) is taken as the high threshold.
3. Non-maximum suppression. This is the key to edge detection. It takes the extreme value of the gradient amplitude value in the area as the edge point and scans the entire gradient amplitude map. If the point of (x, y) is greater than the amplitude of dTmp1 and dTmp2, then (x, y) is regarded as the preselected edge point, and the starting value is set to 255. The amplitude value of the dTmp1 point can be expressed in the form of G(g1) + (1-cot(sigma)) * (G(g2)-G(g1)). Similarly, the gradient amplitude value of the dTmp2 point can be obtained. G obtains a preselected edge point 4. If the gradient amplitude is greater than or equal to Th, it is regarded as the edge point and set to 255; the value below Tl is directly set to 0 and regarded as non-edge point; the one between Tl and Th is set to 125, which is regarded as the point to be detected. In this way, a preliminary edge map point is obtained.
5. Edge connection. Scan the image obtained in the previous section, and detect the 8 domain points around 255. If there are 125 points as edge points, set them to 255, and then use these newly set 255 points to find the points to be detected in the 8 domains. If so, set it to 255 until no new edge points are generated.

B. THE FLOTATION FOAM OBJECT DETECTION METHOD BASED ON YOLOv5
This paper uses the object detection algorithm model of YOLOv5 + Sort to build the core image recognition engine. Through the training of 1000+ sample data sets, the recognition and positioning of the liquid surface foam can be realized. The correct identification of abnormal conditions of flotation is achieved by counting the proportion of the shadow area in the flotation cell and using the object classification algorithm model based on ResNet. The specific implementation process is shown in figure 2.

1) SORT TARGET TRACKING ALGORITHM
The foam feature extraction part first goes through the decoding, frame extraction, and format conversion of the original camera video stream. Then send the converted image frame to the deep learning model for recognition. Finally, the detection result is marked in the video stream. The specific implementation process is shown in figure 3.

a: YOLOv5 MODEL
Due to the number of bubbles in the flotation tank is quite different. This system uses YOLOv5 model to identify the bubbles. YOLOv5 contains four models: YOLOv5s, YOLOv5x, YOLOv5m, and YOLOv5l. Among them, YOLOv5s [13], [14] is the model with the smallest model and the fastest processing speed. Due to the speed requirements in industrial inspection, YOLOv5s was chosen as the model for bubble object detection. Since YOLOv5s is the fastest in processing speed, and the size of YOLOv5s is only more than 10 MB, the size of YOLOv5s for single-precision calculations is only more than 7 MB. Figure 4 below is the network structure diagram of YOLOv5s designed for this time.

b: SORT MULTI-TARGET TRACKING ALGORITHM
First, perform object detection, and then calculate the intersection ratio between target frames with the Hungarian algorithm. If the condition is not met, delete it, and establish a new tracking direction for the current bubble. If the condition is met, update the Kalman filter, and at the same time, the Kalman filter Predict the parameters of the next target frame and perform the next cycle. The problem of multi-target tracking can be regarded as a problem of data association. The purpose is to correlate the detection results between different frames in a video sequence and assign a unique ID to each target. To solve the problem of target association, a variety of methods are generally used to model the appearance and movement characteristics of the target.
The workflow of Sort multi-target tracking algorithm: 1. A given video sequence 2. Run the object detector to obtain the bounding box of the object 3. For the detected objects, perform feature calculation and motion prediction 4. Similarity calculation, that is, calculating the probability that two objects belong to the same target 5. Target association, assign an ID to each object Establish a motion model for the target, which is used to calculate the state of the target in the next frame of the image. The paper approximates the target motion between adjacent frames as a constant velocity motion, independent of the motion of other targets and the camera. The state of each target is described by a 7 -dimensional vector: T where u and v are the horizontal and vertical pixel coordinates of the target center, s represents the size of the target, and r represents the ratio of the bounding box of the target. It should be noted that the ratio r of the bounding box of the target is considered to be constant. When a Detection is successfully associated with the Target, the bounding box of the Detection is used to update the target's status. Target's speed information can be calculated by the Kalman filter. By introducing Kalman Filter and Hungarian algorithms, real-time and excellent tracking performance can be achieved. This method does not consider the appearance characteristics of the target, and only uses the position and size of the bounding box in motion estimation and data association. The problem of short-term and long-term occlusion in the tracking process is ignored, because solving this problem will bring greater complexity to the model, which often limits the real-time operation of the tracker, and at the same time, according to the actual situation of the process, there is almost no occlusion in the foam problem.
If a Target has no Detection associated with it, only use the linear constant velocity model to predict the state of the Target. The transfer equation of the Kalman filter [15] and measurement equation are as follows: Among them, the state transition matrix and the measurement matrix are respectively: result of the previous frame, and the predicted result needs to be correlated with the detection result of the current frame. In the figure below, the blue point on the left represents the predicted value, the green point on the right represents the detection result, and the line between the two points represents a possible match. The connection here is weighted, which is obtained by calculating the IOU distance [16]. The IOU distance reflects the degree of overlap of the two bounding boxes. The target association problem can be solved by the Hungarian algorithm.

2) RECOGNITION OF ABNORMAL CONDITIONS
The abnormal working condition is judged according to the real-time status data of flotation foam combined with historical trend data. The abnormal working conditions including slurry overflow, underfloor stuck, nonfoaming, and pipeline blockage are within the alarm range. To improve the recognition accuracy of abnormal working conditions, after many field tests, it is determined that the proportion of the shadow area in the launder and the three classifiers trained by the Resnet deep learning model are used for the recognition of abnormal working conditions.
First of all, due to the high detection accuracy of the shadow area detection method while identifying ''abnormal slurry overflow'' and the low accuracy of identifying ''abnormal underfloor stuck'', the method is only suitable to identify ''abnormal slurry overflow''.
Secondly, the Resnet deep learning method has a high accuracy rate in the recognition of ''abnormal underfloor stuck'', but the detection accuracy of ''abnormal slurry overflow'' is low, so this method is used to identify ''abnormal underfloor stuck''.
Finally, according to the comprehensive analysis results of the above two algorithms, the abnormal working conditions of the flotation cell are inferred. After field practice, this method can effectively improve the system's detection accuracy. The specific coordination detection logic of the two is shown in figure 7.

V. FOAM OBJECT DETECTION AND SYSTEM VERIFICATION
This system adopts the ''end-side cloud'' architecture in the guide for the construction of smart mines in the non-ferrous metal industry. The system runs on the ''NewPre3102'' AI edge controller, so the computing power limitations of edge devices need to be considered. The overall system adopts a streamlined architecture to minimize unnecessary intermediate components and reduce the consumption of computing resources for front-end equipment. The overall system is based on the ''core flotation video detection program'', with the GPU-based YOLO object detection model engine as the core, to achieve rapid detection and result output of each acquired video frame. The overall structure of the system is shown in Figure 8.
According to the actual process requirements of the company's copper ore dressing plant, the method divides the system functions into three main aspects. One is to obtain real-time data of flotation foam characteristics through computer vision and image processing to provide data support for the fine flotation process control of the concentrator. The second is to display the processed real-time froth image, the extracted flotation froth status data, and its change trend graph at the front end, and combine with the flotation froth historical status data to output the control reference value of the floatation tank level and air intake volume. Provide a more intuitive operation basis for the operators of flotation positions. The third is to combine historical data and post-operation experience to establish a connection between the characteristics of the foam shape and the flotation operating conditions, to realize automatic alarms for changes in the flotation operating conditions.

A. FLOTATION FROTH FEATURE EXTRACTION
Using deep learning image processing algorithm and Pytorch + YOLOv5 model, through training on 500+ labeled sample data sets, the recognition, positioning, and labeling of liquid surface foam can be realized. Quantify the fluidity and stability of the flotation foam, visually reflect the state and quality of the flotation foam in the form of data, and output the following foam characteristic parameters:

1) BUBBLES SIZE
The deep learning-object detection algorithm detects and recognizes the large bubbles in the flotation froth image, and calculates the area of the identified bubbles. Then according to the artificially preset bubbles size classification, the flotation bubbles size classification ratio is calculated to reflect the stability of the flotation working condition.

2) BUBBLE LIFE CYCLE
The duration of the identified foam from formation to extinction is recorded and the average value of the foam life cycle within the field of view is counted to reflect the stability of the flotation foam, through the identification and tracking of the detected foam.

3) LIQUID SURFACE VELOCITY AND FLOW DIRECTION
Through the tracking of the identified foam movement trajectory, the flow velocity and flow direction of the flotation tank liquid surface are statistically integrated to reflect the operating rate.  For the calculation method of foam flow rate, it is necessary to first calculate the single foam speed, and then obtain the foam moving speed by calculating the average value of the single foam speed per unit time. The specific calculation method is as follows.
a) Calculation method of single foam speed: Establish a coordinate system, take the upper left corner of the picture as the far point (0, 0), and after YOLOv5 can get the target frame of the foam (x 1 , y 1 , w, h) to find the center point coordinates x 1 , y 1 are the center store coordinates, using the sort algorithm Calculate the position (x 2 , y 2 ) of the bubble in the next frame of picture, and calculate the pixel distance L of the same bubble. The formula is: where i represents the i th bubble, Li represents the distance of the i th bubble, Xi1 represents the x coordinate of the starting center point of the ith bubble, Xi2 represents the x coordinate of the end center point of the ith bubble, and yi1 represents the ith bubble The y-coordinate of the starting center point, Yi2 represents the y-coordinate of the ending center point of the i-th bubble. Through the distance formula of a single foam, the speed of a single foam can be obtained, and the formula is as follows: H i = L i / T where T is the time interval, Hi is the speed of the i-th picture The final single velocity formula is: Multiple foam velocity calculation methods: The average velocity of multiple bubbles in the same time period is as follows: c) Actual average flow velocity Calculate the actual moving speed according to the abovementioned multiple foam moving speeds, The pixel distance can be converted to the actual distance by the following formula: S 1 = S/N According to the actual image situation, a pixel can be regarded as a dot, that is, a single pixel area S 1 It can be expressed as: S 1 = πr 2 Therefore, the diameter of a single foam can be expressed as: The actual average flow velocity of multiple bubbles in unit time can be expressed as: According to the actual situation, 5 video frames are randomly selected from random videos. First, a manual review is performed to determine the number of large bubbles on the page. Then the 5 images are sent to the Yolo object detection engine to obtain the annotated images. Manually checking the accuracy of the engine labeling (whether the labeling frame completely defines the bubble, whether the frame is too large, too small, or deviating) will be the next, and the number of bubbles is identified by the engine. Finally, the labeling accuracy and the recognition recall rate are judged. The test results are as follows: As shown in Figures 9, and 10 above, the engine can accurately produce the rectangular outline of the foam. In the test, there was no labeling that was too large, too small, or deviated. The labeling accuracy can reach 99%, and the recall rate can reach over 95%. After analysis, it is found that the main reason for the remaining 5% not being detected is that the judgment of the bubbles size target when manually labeling the sample is different from each other, which causes some foams to be relatively small and not detected, but the accuracy can meet the requirements of the flow rate and flow direction.

B. DETECTION OF ABNORMAL CONDITIONS OF FLOTATION
According to the real-time status data of flotation foam combined with historical trend data, it is judged whether the current flotation production operation conditions are abnormal, to alarm the abnormal operating conditions of the flotation production operation. Achieve alarm output for abnormal working conditions such as slurry overflow, underfloor stuck, non-bubble, and pipeline blockage. Use the detection program to detect the on-site collected video, and record the measured value of the left and right grid liquid surface velocity and flow direction for each frame, and finally analyze it. Check whether the flow velocity and flow direction values monitored by the system are consistent with actual observations. At the same time, when the liquid level is abnormal, whether the flow rate and flow direction are also abnormal.
According to the analysis of the data saved in the test, it is determined that the system can measure the flow direction and velocity information of the liquid surface well, and the data is consistent with the actual observation. At the same time, when abnormal conditions occur, the flow rate and flow direction are also abnormal. The analysis is shown in the figure below: It can be seen from the flow velocity analysis graph (the blue line is the flow velocity of the left grid, and the gray line is the flow velocity of the right grid): Under normal circumstances, the flow velocity of the left and right grids are not much different, and the discounted coincidence degree is high; In the case of pipe blockage (red box), the flow rate on the left side is higher than that on the right side; and in the case of slurry overflow(green box), it can be observed that the flow rate drop down to a low level. In the flow diagram, it can also be seen that the flow direction value detected by the system is consistent with the actual situation. Under normal circumstances, the flow direction is mainly concentrated between 200-250 degrees (within the range defined by the red line), that is, it flows to the lower left. In the case of ore blockage, the flow direction is often concentrated between 150-200 degrees, that is to say, it is more inclined to the upper left. This is consistent with actual observations. In the case of slurry overflow, there is an obvious chaotic pattern in the flow direction, and the direction of the entire page continues to rotate between 0 and 360 degrees. The actual observation situation is also the same, that is, the page flows slowly and continuously rotates without obvious directionality.

C. FLOTATION OPTIMIZATION CONTROL REFERENCE
The front-end interface receives the real-time video stream and real-time status data of flotation foam after object detection and processing. The image processing results of flotation foam, flotation liquid surface foam classification statistics ratio, flow velocity and direction of flotation liquid surface and foam life cycle statistics are displayed on the front-end page and updated according to a certain refresh frequency. it is equipped with a corresponding trend graph and historical data query interface, so that the post operator can intuitively control the change of the flotation working condition and trace back the cause of the tendency of the flotation working condition. Finally, based on the analysis of the historical state data of the flotation foam, the control reference value of the current flotation tank liquid level and air intake volume is output to provide a control reference for the post operator.

D. OPERATION EFFECT
After the system is put into operation, the overall commissioning rate should reach more than 95%, the false alarm rate of abnormal operating conditions should not exceed 3%, and the false negate rate of abnormal operating conditions should not exceed 1%. It can correctly output the real-time status data of the flotation bubbles size classification ratio, flow rate, flow direction, color, and life cycle, and display its trend change graph. According to the state of the foam, the reference value of the flotation tank liquid level and the air intake volume are identified and output, which provides an intuitive operation basis for the post personnel. The screenshot of the site commissioning is shown in figure 14.

E. ECONOMIC AND SOCIAL BENEFIT INDICATORS
After the system is put into operation, the ore dressing efficiency can be significantly improved. In order to improve labor efficiency, it is estimated to save 8000 yuan/month * 12 months = 96000 yuan. It is estimated that the metal recovery rate will be improved by 0.2-0.5%. The annual copper output of a single series is 25000 tons * 0.5% * 70000/ton = 3.5 million-8.75 million. It is estimated to save about 3% by adding reagents. The comprehensive economic benefits are expected to increase by about 4-5 million annually. The system plays a great role in promoting industrial transformation and upgrading. It has obvious effects on improving the working conditions of flotation operators and protecting workers from occupational diseases.

VI. CONCLUSION
This article mainly discusses the research on the intelligent detection method of flotation foam based on YOLOv5 and its application in the flotation system of a large domestic mine. The system is based on the identification and positioning of foam in the video frame image of flotation liquid level, and calculates the flow rate, flow direction, color, life cycle of the entire liquid level, as well as the status information of foam identification and abnormal condition prediction information by continuously tracking the foam status of the liquid level. The system is equipped with the automatic detection and control reference function for abnormal working conditions, which automatically prompts the process operator to adjust the control strategy in time when the flotation working conditions change, so as to improve the response speed for abnormal working conditions. the system also can combine historical data and real-time data for analysis, output the control reference value of the flotation tank liquid level and air intake, and assist the control of post personnel. The system helps to stabilize the production operation rate, improve the stability of the flotation process control, and provide strong data support for the fine flotation process control of the beneficiation plant of mining companies. The use of this technology in the flotation process can significantly improve the flotation process level and reduce the incidence of abnormal working conditions. It has important research and application value for operational intelligence and displays information on the flotation process. In the future work research, we will further use the oriented object detection, which ranging from regression-based approaches [38], [39], [40] to classificationbased models [45], [47], and the more recent distributionbased parameterization methods [29], [30], [37] for finegrained object detection whereby the orientation of the object is estimated. These algorithms can locate the foam more accurately, mainly considering that the prediction results of these algorithms can better reflect the real bubble shape.