Abstract
With the development of the world economy and the acceleration of the urbanization process, the automobile has brought great convenience to people’s life and production activities and has become an essential means of transportation. Intelligent vehicles have the significance of reducing traffic accidents and improving transportation capacity and broad market prospects and can lead the development of the automotive industry in the future. Therefore, they have been widely concerned. In the existing intelligent vehicle system, lidar has become the leading role due to its excellent speed and accuracy and is an indispensable part of the realization of high-precision positioning. However, to some extent, the price is the main factor that hinders its marketization. Compared with the lidar sensor, the vision sensor has the advantages of fast sampling rate, light weight, low energy consumption, and low price; so, many domestic and foreign research institutions have listed it as the focus of research. However, the current visual-based intelligent vehicle environment perception technology is still prone to be affected by factors such as illumination, climate, and road type, resulting in the lack of accuracy and real-time performance of the algorithm. In this paper, the environment perception of intelligent vehicles is taken as the research object, and the problems existing in the existing road recognition and obstacle detection algorithms are deeply studied. Firstly, due to the complexity of texture feature extraction and voting calculation process of existing detection methods, and the influence of local strong texture feature interference inconsistent with road direction, a road image vanishing point detection algorithm based on combined 4-direction Gabor filter and particle filter technology was proposed. Then, aiming at the problem that the existing road image segmentation methods based on vanishing point constraint are too dependent on the edge features of road, which leads to oversegmentation easily, a method is proposed to improve the segmentation accuracy of road image by integrating road texture, road surface, and nonroad surface color features. Finally, the application of 3D reconstruction of road scene and obstacle detection technology based on binocular vision and visual navigation algorithm in intelligent vehicle trajectory tracking control is studied. Results show that the visual navigation algorithm can guide the vehicle routes along the road without a barrier, and compared with Wang Ren and two kinds of algorithm, the results show that this control algorithm effectively solves the traditional sliding mode control that is chattering phenomenon, overcomes the model matching, and does not match the interference problems, if used in the intelligent vehicle systems, it can reduce the thermal loss of electronic components and wear of actuator parts and improve the tracking accuracy.
1. Introduction
With the development of the world economy and the acceleration of urbanization, cars have become an indispensable means of transportation in people’s life and production activities. However, with the increase of car ownership, traffic accidents occur frequently, causing huge losses to people’s personal and property safety, making driving a boring and dangerous activity [1]. At present, many domestic and foreign enterprises and scientific research institutions have begun to use scientific and technological means to improve the safety of vehicles, such as fatigue warning and safe driving assistance system. Although the current level of vehicle intelligence has been very high, people are still not satisfied with the status quo, hope to develop a fully automated driving function of the vehicle, and make travel more safe and comfortable [2]. Intelligent vehicle is a comprehensive system integrating artificial intelligence, automatic control, signal processing and other interdisciplinary knowledge of computer, automation and electronics, considering the integration of vehicles and roads, and coordinating planning. It is a typical high-tech complex, and its goal is to be able to drive completely autonomously. The research on intelligent vehicles has the significance of reducing traffic accidents and improving traffic transportation capacity and broad market prospects, which can lead the future development of the automobile industry. Therefore, it has received great attention [3, 4]. The autonomy of intelligent vehicle is mainly reflected in the ability to automatically use sensors to complete the perception and understanding of the surrounding environment, plan the collision free path, and drive according to the predetermined route, so that the vehicle has full adaptive ability. In the intelligent vehicle system, the ability of environmental perception and understanding is the basis and premise. After the intelligent vehicle completes the perception and understanding of the surrounding environment, the environmental information can be provided to the control system to complete the tasks of obstacle avoidance and road tracking. Smart vehicles can sense their surroundings through sensors such as lidar, sonar, and cameras. In the existing intelligent vehicle system, lidar has become an indispensable part of high precision positioning due to its excellent speed and accuracy, but it has the problem of high cost. More than 90 percent of the information humans get when driving a vehicle comes from the visual system, which allows them to establish a relationship with their surroundings.
Environment perception is an important part of autonomous driving of intelligent vehicles. Although the research at home and abroad has achieved fruitful results, the research of visual environment perception technology is not mature enough. In this regard, this paper conducts in-depth research on the existing problems in the road recognition and obstacle detection algorithms of intelligent vehicles based on vision, in order to improve their accuracy, real-time performance, anti-interference ability, and adaptability to various road types. Data fusion based on visual sensors can not only obtain the road image ahead but also obtain rich three-dimensional road information in the road detection of intelligent driving vehicles [5]. The fusion of multisensor information can effectively improve the cognitive level of intelligent driving vehicles on the road environment. Therefore, information interaction and data fusion of various sensors in the environment perception system are essential, and data fusion based on visual sensors has always been a popular research direction for intelligent driving vehicles to achieve efficient and reliable environment perception. Multisensor information fusion is an information processing technology that contains multiple or multiple sensors at different positions. The information is processed by standard criteria to explain the environment-specific characterization. The method of multisensor information fusion can achieve better results under the condition of more diverse driving environment and has a qualitative improvement of the robustness of the system. Multisensor data fusion system and all of the single sensor signal processing or lower levels of multisensor signal processing way, compared to single sensor signal processing or lower levels of multisensor signal processing, are a low level of brain information processing imitation, unlike multisensory data fusion system, they cannot use the multisensory resource effectively. More than 90 percent of the information humans get when driving a vehicle comes from the visual system, which allows them to establish a relationship with their surroundings [6].
It is found from the research at home and abroad that the fast and practical video image vehicle detection and tracking technology is the key to the further development of intelligent transportation. According to different data sources, vehicle detection and tracking methods based on machine vision can be divided into monocular vision, stereo vision, color image, and fusion of vision and other sensors. At present, vehicle detection technology based on monocular vision and gray image has been widely promoted and applied in the field of intelligent transportation due to its advantages such as reasonable price of sensor hardware, strong real-time performance, and mature technology. Vehicle detection system based on monocular vision is usually composed of detection module and tracking module [7]. The vehicle detection module uses digital image processing algorithm and machine vision algorithm to detect the vehicle target in the image to be detected and obtain the location information of the vehicle target in the image. In the process of acquiring vehicle target, detection windows of different scales are usually used to scan the original image so as to determine whether it is a vehicle target at every possible position of the image. Therefore, the time complexity of the detection algorithm is generally high. The tracking module obtains the tracking target and initializes the tracking information according to the vehicle target obtained by the detection module and then restricts the search area to a certain range by using spatial constraints. Therefore, the introduction of tracking module can solve the problem of poor real-time performance of the detection module to a large extent. Because it is outdoor scene, rain, snow, and windy trees caused by shaking or bad weather will produce all kinds of noise in the picture; in addition to light conditions, change can also lead to different images, and the influence of shadow, and part of the mobile vehicle obscured that has made the extraction of target is becoming more difficult. At present, the target extraction mainly includes difference method, optical flow method, and energy function based method. The commonly used moving vehicle extraction methods include threshold processing algorithm and boundary tracking and region marking algorithm. The commonly used global threshold method based on image element gray level includes automatic threshold method of category variance, automatic threshold method of optimal entropy, automatic threshold method of moment invariance, and automatic threshold method of minimal error. These methods have their own advantages, but they are not well qualified for the task of rapid vehicle segmentation in traffic scenes. Low reliability and accuracy of vehicle tracking is as follows: reliability and accuracy are two important indexes in tracking [8–10]. In the long distance, when the target area is small and the mobility is not strong, the filtering tracking method is usually used to improve the tracking accuracy. In short range, when the target has a certain area and the jitter between frames is large, the matching tracking method or the window centroid tracking method is generally adopted to keep the tracking stability and accuracy. In practical application, intelligent vehicle tracking based on visual sensor information fusion still faces severe challenges in road recognition and obstacle detection.
2. Vehicle Intelligent Tracking Algorithm and Model
For the transformation of rich road environment, driving routes, and other information, visual sensors can be very effective acquisition. Generally, the information in the process of vehicle driving, such as the road surrounding environment information, vehicle information in front, and traffic signal information, can be processed as visual information. A vehicle detection and tracking system usually includes three stages: moving area detection, target classification (vehicle identification) and vehicle tracking [11]. Firstly, sequence images are captured by the camera. Then, motion region is extracted from sequential images. The extracted moving area is sent to the vehicle detection module to identify the vehicle according to certain judgment criteria; in the tracking module, the track sequence of each vehicle is established to achieve tracking. One of the most important factors for vehicle detection in front is the way of image acquisition, which focuses on the selection of visual sensors. With the continuous optimization of algorithm performance, the field of target tracking has also made great progress. Although single target tracking can achieve satisfactory results, due to the complexity of multitarget tracking task, there are still some problems such as target occlusion, target color similarity, background complexity, and rerecognition of disappearing target [12]. In addition, in the process of driving, unmanned vehicles not only need to perceive the surrounding environment through sensor data but also need to combine high-precision map to plan the driving path in real time and make control decisions; so, the real-time requirements are high. Therefore, while ensuring the stability of the algorithm, how to reduce the computation of the algorithm and build a lightweight multivehicle tracking algorithm is of great significance to the environment awareness technology of unmanned vehicles.
In vehicle tracing aspect, the existing tracking algorithm in dealing with a simple background situations when the movement of the vehicle tracking problem has good effect, but because of the complexity of the target motion and target characteristic of timeliness, shade, rotation, and scale changes when tracking the target and the background interference, it is difficult to adapt to changes in the environment, the tracking effect is very poor, and it is difficult to obtain a more robust tracking results [13]. Then, the data fusion module can fuse the information of multiple vision sensors and output it to the image display module. There are mainly the following methods for image collection: (1)USB camera: light signals can be converted into digital images by photosensitive sensors, which can then be processed. USB camera is simple and clever, but it is difficult to meet the high demand for real-time engineering(2)Analog camera and image acquisition card: the image obtained by the analog camera is transferred to the digital image acquisition card for signal conversion and processing. However, its low resolution cannot meet the requirements of high quality(3)Stereo camera: multiple driving assistance functions with improved safety and comfort can be realized by using stereo cameras [14]. The stereo camera provides an extensible platform that combines the proven capabilities of a monochrome camera with the advantages of stereo technology for 3D environment detection. With a powerful lens system, the camera detects a horizontal line-of-sight range of 45 degrees and provides a 3D measurement range of more than 50 meters. Now, for stereo cameras, research is still in its infancy, and the costs are high(4)Digital camera: the digital camera can capture the image and convert it into a continuous picture. Digital camera is generally integrated with a charge coupled device (CCD) sensor with high resolution and a digital signal processor (DSP) with fast operation, which can instantly digitally transform the acquired image and transfer it to the computer for timely processing. Its real-time performance is strong and processing for digital signals; so, the electromagnetic interference is small. Nowadays, digital cameras are used for vehicle detection. Intelligent vehicle tracking model algorithm framework is shown in Figure 1
The principle of camera internal parameter calibration is equivalent to placing the projection screen in front of the pinhole and realizing the transformation from the objective world to the digital image through a set of known world coordinates and camera pixel coordinates [15]. Let the object be the rigid point, and the transformation between its world coordinates , , and is where is the orthogonal identity matrix, is the three-dimensional translation vector, and is the matrix.
Any pointin space is connected to the cameraandthe focal point of the image plane and the projection position. Using the geometric relationship of central keyhole imaging, the following proportional relationship can be obtained:
The homogeneous coordinates and matrix can be expressed as
The formula for transforming image plane coordinates into pixel coordinates is where and are the coordinates of the origin of the coordinate system , and the physical dimensions of each pixel in the axial and axial directions are and .
The formula for converting world coordinates to pixel coordinates is where , , is the internal parameter, is the external parameter, and is the projection parameter.
In each simulation, the actual trajectory and reference trajectory of the vehicle are sampled at 50 Hz frequency [16]. Since the dimensions of transverse position and yaw angle are inconsistent, the following formula is used to normalize the two tracking quantities:
Then, the average tracking error of the whole track can be calculated by taking the two norm of the error of each sampling point and denoted as
The performance of the proposed algorithm should be measured from two aspects: tracking accuracy and real-time performance [17]. Therefore, the following performance indicators can be defined: where is the standardized tracking error, is the calculation time of the standardized algorithm, and and are, respectively, the weights of the tracking error and algorithm calculation time, which mainly depend on whether we want the algorithm to provide higher accuracy or faster response in practice. Note that the smaller the calculated performance indicator is, the better the performance of the selected threshold is [18].
When the sensor detects that the distance from the obstacle in front is less than the safe distance, the obstacle avoidance decision will be triggered. The safe distance can be calculated by the following formula: where is the longitudinal speed of the vehicle, is the speed of the obstacle vehicle, is the maximum deceleration of the vehicle, is the maximum deceleration of the obstacle vehicle, is the detection and response time, and is the minimum distance between two vehicles [19]. The vehicle acceleration threshold is also considered in the setting of safe distance.
A monocular camera is mounted on the front end of the smart vehicle. The general idea is to have as many objects of interest in the field of view as possible and as few background areas as possible. The intelligent vehicle runs at a certain speed and hopes that the road in the front view will be as long as possible, so as to have enough time to judge the road situation ahead, which requires that the height of the camera installation should not be too low, and the depression angle should not be too large [20]. However, at the same time, if the vision is too far, there will be many interferences (such as sky, trees, and ditches) in the distance of the road and on both sides of the road, making it difficult to extract navigation information. On the other hand, the near field of view of the intelligent vehicle should not be too far away; otherwise, it will not be able to deal with the situation that appears temporarily in front of the intelligent vehicle, which requires that the depression angle of the camera installed should not be too small. Therefore, it is necessary to carry out the preprocessing operation of the original image, which can be divided into three steps: gray histogram and its correction, edge enhancement, and image segmentation [21].
At present, the research on target tracking algorithm has been quite thorough, but the tracking effect is often not ideal if only using the existing target tracking algorithm to track the vehicle without considering the characteristics of the vehicle target [22]. Due to the vehicle target in the process of driving will encounter such conditions as weather changes, video acquisition equipment jitter, vehicle target mutual occlusion, and angle of view changes frequently, resulting in the collected information noise distortion greatly, resulting in a sharp increase in the difficulty of tracking the vehicle target; therefore, when designing the tracking algorithm, the characteristics of vehicle targets should be taken into account, and the algorithm structure should be constructed accordingly. At present, researchers have carried out a large number of studies on vehicle trajectory tracking, but there are still several difficulties in the following aspects: (1)Tracking accuracy and stability of trajectory tracking: in low speed and small curvature conditions, the controller can generally ensure good tracking accuracy and stability, but in medium-high speed and road curvature and other harsh conditions, the system reliability is poor, and intelligent vehicles will appear unstable or out of control phenomenon [23](2)Real-time control algorithm: vehicle is a system with high real-time requirements, especially in high-speed conditions, and only high real-time control algorithm can make the vehicle have a higher safety [24]. However, the general automatic data acquisition system (ADAS) driving assistance function is based on the vehicle controller, and the computing platform is often embedded single chip microcomputer. Therefore, how to ensure the real-time computing is the control algorithm design needs to consider(3)Obstacle avoidance in complex environment: in a higher level of intelligent driving, the actual vehicle trajectory tracking problem should not only be limited to fixed roads but also make accurate and safe obstacle avoidance for surrounding vehicles, so as to achieve higher intelligence
3. Information Fusion Method
Multisensor information fusion is a common basic function in human and other logic systems. Data fusion methods are divided into prefusion and postfusion. The prefusion is to fuse the original signals received by various sensors together through a fusion method and then extract the target location and speed information. After fusion, each sensor processes its own original data and extracts the target location and speed information and then fuses the characteristic data extracted by different sensors through the fusion system [25–27]. Data fusion can also be divided into series fusion, parallel fusion, and hybrid fusion according to the way of information interaction. In series fusion, each level of sensor needs to obtain the output information from the previous level of sensor, and all the perceptual information will be successively transmitted to the last sensor and integrated. In parallel fusion, each sensor directly sends its own output results to the sensor fusion center for data synthesis. The information does not pass through other sensors in the transmission process, and the output information of sensors does not affect each other. Hybrid fusion is the combination of series fusion structure and parallel fusion structure.
In fact, multisensor information fusion is a universal and common basic function in nature. Since people’s different sensory organs have different functions and sensory characteristics, they can feel the simultaneous situation in different places. This process is fast, difficult to measure, and also adaptive. Make full use of multiple sensor multisensory information fusion of different information, and redundancy and complementary information on space and time are combined according to certain standards; in order to obtain the observation environment fusion reasoning that is the core of the multisensor information fusion system, the problem is how to time-varying dynamic characteristics for a complex environment and target [28]. On the premise of difficult to obtain prior knowledge, the environment model of target aircraft with good robustness and adaptive ability is established. How to effectively control the recursive estimation and how to construct the model of uncertainty are the central problems to be further solved when seeking the general design method and developing the actual system.
Information fusion in general integration is to build a decision system with certain intelligence based on the information obtained from various information sources in the distributed computer system [29]. In information fusion when analyzing and processing the information in the comprehensive integration, the following tasks shall be completed in sequence: (1)Information collection: according to the scope of the field in which the problem is analyzed, the relevant information items are widely collected and extracted from the distributed network database, and the format is converted(2)Information identification: identify the extracted information, remove the false and retain the true, and determine the credibility(3)Correlation processing: it requires quantitative analysis of the correlation of multiple information source data and divides the data into different sets according to certain discrimination principles, and the data in each set are associated with the same source(4)Fusion processing: deciding the choice of information obtained from information sources; verify and modify each information item with reference to other information sources; verify and analyze, supplement and synthesize, coordinate and modify, and estimate the information of different information sources; analyze and synthesize the information provided in real time; generate comprehensive information through analysis and judgment(5)Establish work information database: generate work information database for the use of analysis models and experts in various fields and establish links between work information base and information source
In order to determine whether the vehicle passes through the detection area, and then establish the corresponding tracking target, vehicle speed, vehicle track, and other vehicle information should be obtained, through which the video vehicle can be efficiently detected and tracked [30]. However, because of the shadow of the video vehicle, the influence of complex background, and the factors such as the camera shake, to video vehicle detection accuracy, algorithm and real-time tracking the amount of calculation in the process of some interference and influence, therefore, the genetic algorithm (GA) must fully study how to eliminate the impact of these unfavorable factors.
In essence, trajectory tracking of intelligent vehicles means that when the controlled vehicle receives the reference path output from the path planning layer, it can track safely, in real time and smoothly with the smallest tracking error [31]. Vehicle is composed of multiple parts of highly coupling complex system, and its ground movement process is very complicated; so, it will encounter in the process of the actual path tracking of nonlinear, multiobjective constraint. It is necessary to minimize the difference between the output of the vehicle in the predicted time domain and the 10000-square meters data trace of the reference rail. The cost function is established:
Considering the influence of target vehicle’s speed and penalty function on obstacle avoidance, the following obstacle avoidance function is selected:
The objective of the upper trajectory planning controller of the layered control system is to reduce the deviation between the vehicle and the reference trajectory as much as possible while avoiding obstacles smoothly. The functional model of multisource information fusion is shown in Figure 2. The penalty function is used to represent the obstacle avoidance process, and the specific form of the upper model predictive controller is as follows:
4. Analysis and Discussion of Calculation Results
The traditional Camshift is a tracking algorithm using color information [32]. It adopts the “peak value” tracking idea and finds the most similar area through continuous iteration. It has strong practical and real-time performance. However, Camshift only adopts color feature, which is not enough to distinguish the jamming target and background with similar color to the target. In this paper, the color and length between perpendicular (LBP) texture characteristics are fused to improve the tracking accuracy and stability to a certain extent. However, in different scenarios, the accuracy and stability of tracking cannot be well solved only by improving the characteristics of moving vehicles. Therefore, the accuracy and stability of vehicle tracking in different scenes can be ensured by using the probability information of vehicle space movement on the basis of color and texture features. Since the movement track of the tracking target is from far to near and the speed of movement is fast, infinitely variable transmission (IVT) is greatly affected. In the second half of the video, the repetition rate of Online AdaBoost (OAB) decreases, and there are illumination changes. The method in this paper has affine invariance and has little influence on the environment. Expressway vehicle detection results are shown in Figure 3.
Experimental results show that the visual navigation algorithm presented in this paper can correctly guide vehicles along the road and successfully avoid obstacles on the road. In addition, while three kinds of control algorithm can effectively overcome the interference of model matching and does not match, the vehicle lateral displacement and angular error can be asymptotically stable convergence to the steady state and ensure that the vehicle can accurately track the given reference trajectory. If applied to intelligent vehicle system in practice, it can reduce the heat loss of electronic components and wear of executive parts and improve the tracking accuracy and other functions (wear of brush and other parts will directly lead to the output of moving parts that is inconsistent with the expected value, resulting in tracking accuracy reduction, and even system collapse). Proved through the experiment in this paper, the visual navigation algorithm can guide the vehicle routes along the road without a barrier, and the proposed control algorithm can effectively solve the problem of traditional chattering of the sliding mode control algorithm, make the control input signal smooth, and at the same time, effectively overcome the model matching and does not match the interference problems, and ensure that vehicles were able to properly track the reference trajectory visual navigation. Freeway video algorithm tracking results are shown in Figure 4.
Real-time schematic diagram of high-speed vehicle running is shown in Figure 5. As we can see, for obstacle avoidance decision-making figure, at the beginning of the obstacle avoidance decision-making set to 1, obstacle avoidance control below for the sensor measures the distance of the car and, as can be seen from the diagram, the relative distance of vehicles has been reduced, between 12 s to 15 s of some mutations, and this is because the accused of lane changing vehicle is conducted to avoid obstacles. The detection range of the sensor changes, so that the sensor cannot detect obstacles in front when the vehicle heading angle is large; so, the output of the sensor has some mutations. After 14.5 s, the output of the sensor remains 0, indicating that the controlled vehicle has completed obstacle avoidance and overtaking of the vehicle with obstacles. In addition, under the condition of low relative speed, the controlled vehicle made obstacle avoidance decision when it was 23.36 m away from the vehicle in front in order to obtain better driving efficiency. The average normalized Euclidean distance error of the proposed algorithm is reduced by 0.0102, 0.0127, and 0.0235, and the computational efficiency is improved by 64, 79, and 273 times, respectively. The tracking effect of the proposed algorithm is more stable and has higher robustness.
Based on the linear road model with vanishing point constraint, road segmentation is transformed into a Bayesian posterior probability density estimation problem. The texture feature of road is described by the proportion of direction consistency. The similarity between image pixels and “road surface” pixels is calculated using nonlinear transformation function and self-supervision strategy, which is used as the probabilistic prototype for measuring visual features of road surface and nonroad surface. Finally, according to the Bayesian principle, the above three visual features are fused together to segment the road surface by maximizing the Bayesian posterior probability density estimation. Compared with Rasmussen and Kong, the most representative methods in the field of unsupervised or semisupervised road segmentation, the experimental results show that the accuracy of the proposed algorithm is improved by 3.73% and 7.48%, respectively. The average position error of the proposed method is only 4.30 pixels, which is far less than the tracking error of the IVT tracking algorithm and OAB tracking method, and the overlap rate is 89.83%, which is also much higher than that of the IVT tracking algorithm and OAB tracking algorithm, and has good robustness. Track of a moving vehicle is shown in Figure 6.
Experimental results proved that the visual navigation algorithms can guide the vehicle routes along the road without a barrier; the traditional sliding mode control is chattering phenomenon, overcomes the model matching, and does not match the interference problems, if used in the intelligent vehicle systems, it can reduce the thermal loss of electronic components and wear of actuator parts and improve the tracking accuracy. This article system in the resolution of 6001200 average on the gray image detection time is 0.8 s, and the original parts of deformation model resolution on this average detection time are more than 8 s; you can see in this paper that the vehicle detection scheme not only on the real-time deformation model is superior to the parts and can still maintain a relatively good accuracy. The results show that the MOTP of the proposed DBMoT algorithm reaches 97.61% under different frames, which is 2.89% higher than the other three algorithms on average. The tracking accuracy of DBMoT is higher, and the target drift and error matching problems are solved at the same time. Vehicle tracking results are shown in Figure 7.
The whole sample space includes 572 image samples containing 3000 vehicles and negative samples of various scenes without vehicles. The vehicles are marked by hand. Positive samples are rear images of various models, including sedans, SUVs, and minivans. The angle of view also includes three angles, namely, front rear, left rear, and right rear. Negative samples include freeway scene and street scene. Lane detection results under different road conditions are shown in Figure 8, and the intelligent car first stops in place, starts to accelerate to about 18 s, then keeps at about 60 km/h, and then begins to decelerate. The relative distance with the vehicle in front keeps increasing at first, which is why the white vehicle in front accelerates from stopping to 30 km/h, while the vehicle stops at the same place. When the distance reaches a certain value, the vehicle accelerates and catches up with the vehicle. Therefore, the distance starts to decrease, and the obstacle avoidance decision is made when the distance is detected at about 26 s. The synchronous trajectory tracking and obstacle avoidance control algorithm based on model predictive control starts to plan the trajectory and solve the control quantity. At about 30 s, the distance becomes 0, because at this time, the vehicle in front is beyond the detection range of millimeter wave radar. In addition, Gaussian difference graph weakens the photosensitivity of the image, which makes the algorithm in this paper have good robustness to the change of illumination. The multivirtual trigger area multitarget tracking is shown in Figure 9.
One is to track the vehicle in front of the vehicle in the same lane, which is simple. The other is the vehicle lane change and then redetects and tracks the vehicle in the lane. The algorithm in this paper focuses on the construction of multiscale space, and Gaussian difference maps are all single-channel grayscale maps with simple data structure. Through observation, it can be seen that at the speed of 72 km/h, the designed model predictive controllers (MPC) have appeared obvious tracking error, and at the point of 55 m to 75 m and 100 m to 120 m large curvature, the lateral position tracking error once reached 1 m, in the structured road, and is about to pull out of the current lane. Test results under different weather and road conditions are shown in Figure 10. As can be seen from the yaw velocity diagram, the amplitude of yaw velocity is close to 30°/s at high speed, and the frequency is very high, which affects the stability of the car body on the one hand and greatly affects the ride experience on the other hand. However, under the rolling optimization of model predictive control, the controller can finally converge to the reference trajectory.
(a) 2D results
(b) 3D results
Results of tracking vehicles in a specific lane are shown in Figure 11. The number of FP and FN, especially FN, decreased significantly after adding a scale. The number of FP decreased from 265 to 148, and the number of FN decreased from 325 to 179. The accuracy of the original model is 94.61%, and the recall rate is 89.56%. After increasing the scale, the accuracy of the model test was 97.05% and the recall rate was 94.16%. Although the accuracy of the model is not improved much, which is because the residual network has a good ability to extract image features, the recall rate of the model is greatly improved after scale optimization, and the model is also increased from 0.8796 to 0.9366 after scaling. Therefore, the target vehicle is in a stable state in the whole obstacle avoidance process and meets all constraints of safe driving, indicating that the layered controller has good real-time performance, which enables the target vehicle to achieve smooth obstacle avoidance, adapt to the change of speed, and has high accuracy and reliability.
The vehicle detection and tracking system implemented in this paper can detect, track, and count multilane vehicles simultaneously. In this paper, multiple traffic passing videos were used to test, and the detection and tracking effects were summarized. The test data mainly included detection rate, tracking rate, detection accuracy, and tracking accuracy. Detection accuracy and tracking accuracy are subjective. If the detection coverage of the vehicle target is less than 60%, we believe that a detection error has occurred. If there is an obvious mid-term target loss or only less than 60% of the vehicle body and more than 150% of the vehicle body surrounding area in the tracking process, it is judged as a tracking error. In the test of this paper, a total of 542 vehicles from three videos were counted, and the detection rate reached more than 92%. The follow-up rate was 87%. The detection accuracy and tracking accuracy are lower than the detection rate and tracking rate, respectively, which are 86% and 84%, mainly because the detection is incomplete after the trigger, and the tracking is lost during the process. The rendering of the improved single Gaussian background modeling algorithm is shown in Figure 12.
5. Conclusion
With the rapid development of economy, the number of cars has exploded, bringing convenience to people as well as many traffic problems. It has brought good news to solve traffic system problems. How to recognize and track a specific target on a complex road and in a large number of vehicles is crucial in the development of intelligent transportation in the future. How to adaptively carry out real-time and accurate vehicle tracking for various actual scenes has always been a difficulty in academic research. The main research achievements of this paper include the following: (1)Based on the visual sensor information fusion model, train network to detect vehicles in the picture. Aiming at the poor detection effect of tiny network, two lightweight models, DarkNet-19 and ResNet-18, were used to replace original feature extraction network, respectively, and the detection accuracy of ResNet-18 for feature extraction increased from 79.57% to 87.96%. Aiming at the problem that the optimized network is still poor in detecting small objects, this paper proposes to increase the prediction scale to solve this problem. The detection accuracy of the vehicle detection network trained by secondary optimization is 14.09%(2)Finally, the whole scheme is deployed on the video big data processing platform combining Hadoop with Storm, which has fast running speed and high data processing ability, to ensure the real-time performance of vehicle tracking. Different from the traditional image feature information tracking scheme, this scheme uses deep learning to achieve. In order to verify the feasibility and portability of the scheme, the vehicle tracking scheme is transplanted into the development board(3)Proposed a multivehicle vision tracking algorithm based on neural network. Different from other deep learning networks, neural network can better learn advanced deep hierarchical features by minimizing reconstruction errors and double-layer networks. Through learning strategy, the proposed multivehicle visual tracking algorithm can effectively solve the problem of error matching. For the problem of target drift, the paper introduces time information to calculate the dynamic duration to update the appearance model in real time(4)Establish corresponding observation model based on fusion features. The vehicle features were extracted by perceptual features and principal component analysis to build a fusion feature model, which was integrated into the vehicle tracking framework based on particle filter. The proposed algorithm improves the real-time performance of multitarget tracking. The results show that robustness of the proposed algorithm is improved, and it can provide continuous and reliable positioning information for intelligent vehicles. It has the characteristics of low cost and can basically solve the lighting, occlusion, loss, and congestion in target tracking
In this paper, the laser radar and cameras and data fusion of lane detection, the model has good performance in structured road test, but in practical applications, the traffic condition is complicated, and the accuracy and real-time detection model of the road is a big challenge; so, how to automatically drive vehicle in complex environments accurately extracts the lane region. At the same time, to ensure the real-time lane detection, we need to solve the problem. Therefore, there are still many problems in this subject that need further discussion, mainly in the following aspects: (1)Laser radar in direct detection of lane edge and auxiliary lane detection area in the parallel data fusion process: all is the use of grid method and the DBSCAN algorithm of point target detection area and road edge detection; therefore, depth study of point cloud data available for training and the road edge points existing in point cloud data is extracted. More lidar scanning layers can be used to obtain road edge data, and then the lane edge can be fitted by spline curve or Bessel curve fitting method, so as to realize the detection of curves and other complex roads, and improve the accuracy of lidar auxiliary detection of curves and other complex road areas in parallel data fusion(2)In the process of tandem data fusion lane detection, this paper uses the traditional method Hough transform to detect lane edges. Although the accuracy of lane detection in complex background environment is improved by dividing precise regions of interest, it is difficult to adapt to the complex and changeable lane environment. Therefore, the positive and negative samples in the image information can be automatically obtained by the region of interest, and then the machine learning SVM is trained to realize the real-time detection of different lane areas
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research was funded partially by the General Project of Science and Technology Program of Beijing Municipal Education Commission, grant number KM201911417011.