Research on Vehicle Pose Detection Method Based on a Roadside Unit

Vehicle pose detection plays a vital role in modern automotive technology, which can improve driving safety, enhance vehicle stability and provide important support for the development of autonomous driving technology. The current pose estimation methods have the problems of accumulation errors, large algorithm computing power, and expensive cost, so they cannot be widely used in intelligent connected vehicles. This paper proposes a vehicle pose detection method based on an RSU (Roadside Unit). First, the on-board GPS performs the positioning of the target vehicle and transmits the positioning information to the RSU through the UDP (User Data Protocol). Next, the RSU transmits a forward command to the OBU (On-board Unit) through the UDP. The OBU sends the command to the ECU (Electronic Control Unit) to control the vehicle forward. Then, the RSU detects and tracks the vehicle. The RSU takes pictures of two images before and after the movement and obtains the coordinates of the four angle points and the center point by image processing. The vehicle heading direction is determined by the moving direction of the center point of the front and rear two images. Finally, the RSU captures the vehicle images in real time, performs the process of tracking, rectangular fitting and pose calculation to obtain the pose information and transmits the information to the OBU to complete the whole process of vehicle pose detection and information transmission. Experiments show that the method can realize accurate and efficient detection of vehicle pose, meet the real-time requirements of vehicle pose detection, and can be widely used in intelligent vehicles.


Introduction
As a key vehicle perception technology, vehicle pose detection can accurately identify and understand the movement state of the vehicle, so that the driver or the intelligent driving vehicle can make more accurate and safe driving decisions.In the field of intelligent driving, vehicle pose detection can also provide key data support for automatic driving systems, and improve the perception of road conditions and obstacles, so as to better cope with the complex traffic environment.Therefore, the research on vehicle pose detection is necessary and meaningful.The camera has the advantages of easy installation, strong adaptability to the environment and low cost, so it has an important application value in vehicle pose detection.At the same time, with the rapid development of V2X (Vehicle to Everything) communication technology, RSU (Roadside Unit) plays an increasingly important role in intelligent transportation systems.Through its positional advantages, extensive coverage, and efficient data transmission capabilities, RSU provides strong Sensors 2024, 24, 4725 2 of 23 support for realizing intelligent driving and intelligent transportation systems.The vehicle pose detection method based on the RSU can provide reliable pose information in real time due to the simplicity of its solution method and the high camera frame rate.With the advantages of low environmental requirements, strong adaptability, and low cost, this method can compensate for the disadvantages of easily losing GPS signals, the high cost of radar, and sensor drift, making it widely applicable.
This paper proposes a vehicle pose detection method based on the RSU, which can obtain vehicle pose information in real time and with high precision, laying a foundation for the vehicle in stability control, driving safety, and path planning.The remaining structure of the paper is as follows.The second section discusses the current vehicle pose detection methods.The third section details the rectangular fitting method, the vehicle pose calculation method, and the vehicle heading direction detection method.The fourth section shows the training results of target detection and the experiment results of this method in the real vehicle scenarios.In the fifth section, the proposed method is summarized and the subsequent research is also presented.

Related Works
Accurate vehicle pose information is the key component of autonomous driving systems.Scholars, both domestically and internationally, have conducted extensive research on vehicle pose detection methods.At present, the mainstream vehicle pose detection methods are the camera-based methods, the lidar-based methods, and the multi-sensor combination algorithms.
Because of the ability of cameras to capture rich image information and the increasing maturity of deep learning algorithms, camera-based vehicle pose detection methods are widely used.The camera-based methods can automatically extract and learn abstract features from the data, which have strong adaptability to the vehicle pose changes under different angles and light conditions.Xu et al. [1] integrated deep learning with enhanced digital maps to accurately estimate vehicle heading angles.Huang et al. [2] presented a lightweight deep learning framework for effective prediction performance in unobscured vehicle scenarios.Gupta et al. [3] introduced an efficient vehicle pose estimation architecture based on multi-scale deep learning for precise estimation with manageable network complexity.Zhao et al. [4] proposed a deep learning-based FPN method for monocular camera-based vehicle pose estimation to improve speed and accuracy.Lyu et al. [5] used the vehicle 3D priors to estimate the 6D pose and improved the detection accuracy and speed.The camera-based methods rely on large amounts of labeled data and have limited generalization ability in specific scenarios, while having a high demand for computational resources.
The lidar-based methods can provide a high precision distance measurement and can accurately capture the details of the vehicle surrounding environment, thus enabling 3D positioning and attitude detection of the vehicle.Wang et al. [6] introduced a lidarbased method for estimating the pose and velocity of the ego vehicle and surrounding moving obstacles simultaneously.Zhao et al. [7] presented a real-time tracking algorithm using L-Shape fitting, enhancing performance and accuracy.Yang et al. [8] proposed a vehicle pose estimation method based on edge distance for boundary rectangle-based pose estimation.Gu et al. [9] developed a registration algorithm with road information constraints, demonstrating accuracy and effectiveness in practical use.The lidar-based vehicle pose detection methods provide superior robustness in complex environments by exploiting 3D geometric information.However, the lidar-based methods may face limitations in some applications due to their uneven density, high processing complexity, and the dependence on precise registration and localization.
The multi-sensor combination algorithms have a wide range of applications, especially playing a key role in environmental perception as well as in navigation and positioning.By integrating data, more comprehensive, accurate, and real-time information acquisition from different sensors is enabled.Wu et al. [10] added a GPS heading angle to the measurement vector, and established its error model, which effectively improved the accuracy of the GPS heading angle.Balazadegan Sarvrood et al. [11] proposed a method that combined visual odometry, light detection and ranging odometry, and a simplified inertial measurement unit.This method achieved an accurate estimation of pose information in urban areas with surrounding buildings.Xu et al. [12] proposed a new multi-information fusion vehicle pose estimation method to achieve precise vehicle pose estimation even during GNSS interruptions.Yang et al. [13] proposed a method which used 4D radar and a camera to robustly estimate vehicle pose to use the complementary characteristics of each sensor.The method showed excellent performance and enhanced robustness.An et al. [14] fused inertial and multi-antenna GNSS measurements to provide higher precision for positioning and attitude estimates.To address the unique demands of safety-critical applications in challenging observation conditions, Li et al. [15] introduced an innovative tightly integrated RTK (Real-Time Kinematic)/INS (Inertial Navigation System) algorithm to ensure the continuous delivery of precise and dependable positioning outcomes.Bersani et al. [16] presented an innovative integrated algorithm that provided an accurate estimation of the vehicle pose for the panning algorithm in multiple challenging scenarios.To identify the pose accurately, CNS (Celestial Navigation System) is commonly combined into the INS/GNSS integration, leading to an INS/GNSS/CNS integrated navigation system [17][18][19][20][21].The multi-sensor combination algorithms have the advantages of improving accuracy, enhancing robustness and adapting to multiple scenarios, but also face challenges of complexity, cost, and energy consumption.
Based on the above research, the current camera-based vehicle pose detection methods have limited generalization ability in specific scenes and have a high demand for computing resources; the lidar-based methods have a high cost, processing complexity, and severe dependence on accurate registration and positioning; and the multi-sensor combination algorithms have high complexity and are not widely used.In this paper, RSU is used to realize vehicle pose detection, UDP communication is used to realize information transmission between OBU and RSU, and the whole detection process is completed by controlling vehicle forward, vehicle heading direction detection, target detection and tracking, vehicle rectangle fitting, and pose calculation.This method solves the problems of high pose detection power, high cost, and weak real-time performance.With the development of Internet of Vehicles technology, this method has great practical application significance.

Method
The vehicle pose detection method based on the RSU can detect the vehicle pose without any prior information and meet the real-time requirements in the actual process of vehicle driving.Figure 1 shows the specific flow of the vehicle pose detection method based on the RSU.
The multi-sensor combination algorithms have a wide range of applications, especially playing a key role in environmental perception as well as in navigation and positioning.By integrating data, more comprehensive, accurate, and real-time information acquisition from different sensors is enabled.Wu et al. [10] added a GPS heading angle to the measurement vector, and established its error model, which effectively improved the accuracy of the GPS heading angle.Balazadegan Sarvrood et al. [11] proposed a method that combined visual odometry, light detection and ranging odometry, and a simplified inertial measurement unit.This method achieved an accurate estimation of pose information in urban areas with surrounding buildings.Xu et al. [12] proposed a new multiinformation fusion vehicle pose estimation method to achieve precise vehicle pose estimation even during GNSS interruptions.Yang et al. [13] proposed a method which used 4D radar and a camera to robustly estimate vehicle pose to use the complementary characteristics of each sensor.The method showed excellent performance and enhanced robustness.An et al. [14] fused inertial and multi-antenna GNSS measurements to provide higher precision for positioning and attitude estimates.To address the unique demands of safetycritical applications in challenging observation conditions, Li et al. [15] introduced an innovative tightly integrated RTK (Real-Time Kinematic)/INS (Inertial Navigation System) algorithm to ensure the continuous delivery of precise and dependable positioning outcomes.Bersani et al. [16] presented an innovative integrated algorithm that provided an accurate estimation of the vehicle pose for the panning algorithm in multiple challenging scenarios.To identify the pose accurately, CNS (Celestial Navigation System) is commonly combined into the INS/GNSS integration, leading to an INS/GNSS/CNS integrated navigation system [17][18][19][20][21].The multi-sensor combination algorithms have the advantages of improving accuracy, enhancing robustness and adapting to multiple scenarios, but also face challenges of complexity, cost, and energy consumption.
Based on the above research, the current camera-based vehicle pose detection methods have limited generalization ability in specific scenes and have a high demand for computing resources; the lidar-based methods have a high cost, processing complexity, and severe dependence on accurate registration and positioning; and the multi-sensor combination algorithms have high complexity and are not widely used.In this paper, RSU is used to realize vehicle pose detection, UDP communication is used to realize information transmission between OBU and RSU, and the whole detection process is completed by controlling vehicle forward, vehicle heading direction detection, target detection and tracking, vehicle rectangle fitting, and pose calculation.This method solves the problems of high pose detection power, high cost, and weak real-time performance.With the development of Internet of Vehicles technology, this method has great practical application significance.

Method
The vehicle pose detection method based on the RSU can detect the vehicle pose without any prior information and meet the real-time requirements in the actual process of vehicle driving.Figure 1 shows the specific flow of the vehicle pose detection method based on the RSU.(1) In order to accurately identify the target vehicle in complex road conditions, the initial position information of the target vehicle is obtained by using the on-board GPS, and the information is transmitted to the RSU through UDP in the OBU.
(2) After receiving the position information of the on-board GPS, the RSU camera takes pictures of the vehicle on the road and performs the target detection.Since the GPS position information of the vehicle has been obtained, the target vehicle can be extracted from the target detection.(3) Preliminary pose calculation is performed on the initial image P 0 of the target vehicle to extract the center point coordinate, preparing for determining the vehicle heading direction.(4) The RSU sends forward command to the target vehicle through the UDP.After the OBU of the target vehicle receives the command, it sends the command to the ECU to control the vehicle forward.(5) The RSU tracks the target vehicle in real time, and performs a preliminary pose calculation for the real-time image P t to obtain the center point coordinate.(6) The vehicle heading direction is judged according to the moving direction of the center point coordinate of the initial image P 0 and the center point coordinate of the real-time image P i after the vehicle advances.(7) The small movement distance between the two front and rear images may not be enough to determine the vehicle heading direction due to the high frame rate of the RSU camera.If the vehicle heading direction cannot be judged, return (5).otherwise, continue to track the vehicle in real time.(8) The RSU sends the pose information of the vehicle to the OBU in real time to complete the whole process of pose detection.

Data Interaction Based on the UDP
In this paper, data sharing between the RSU and OBU is realized through the UDP.The UDP has low resource consumption and fast processing speed, and provides connectionless datagram services to terminal devices.To enable real-time information transmission between RSU and OBU for transmitting small data volumes such as vehicle position and heading angle information, this paper employs UDP for data transmission.Firstly, the OBU transmits its own vehicle position information to the RSU through UDP for the RSU to locate the target vehicle for vehicle pose detection.Secondly, after the RSU locates the target vehicle, it uses UDP to send a forward command to the target vehicle.While sending the command, the RSU captures real-time images of the target vehicle to determine the vehicle heading direction.Finally, during the vehicle pose detection phase, the RSU continuously detects the vehicle in real time, calculates the pose, and then transmits the pose information to the OBU.
Before transferring data, the two communicating parties using the UDP do not need to establish a connection, so there is no delay in the connection establishment.With the development of network technology, especially in short-distance transmission areas, UDP data transmit well in real time relative to TCP. Figure 2 shows the format of the UDP message.The UDP message is divided into two parts: UDP message header and UDP data area.The header consists of the source port, the destination port, the message length, and the checksum.When the source host sends the data to the target host, it only needs to send the packet to the IP address and port number of the target host without any preconnection.If the target host is ready to receive the data, it will receive and process the packet.The application layer protocol will guarantee the reliability of communication.At the same time, because UDP only provides a low level of error control, in the process of data transmission, the transmission delay and the system running memory is small, and the data transmission speed is high, which can be suitable for data transmission in scenarios with high real-time requirements.Due to the lack of existing UDP communication programs in MATLAB (MathWorks, Natick, MA, USA), the programs are written on the computers of RSU end and OBU end to receive and transmit data.Figure 3 shows the architecture of the communication port.The vehicle pose detection algorithm is provided on the RSU terminal computer, and the OBU terminal computer provides the target vehicle ID, time, and GPS positioning information.All the information is transmitted through the communication port built by the UDP.

Vehicle Rectangular Fitting
In the top view, the shape of the vehicle is basically rectangular, so the external rectangle of its outline can accurately reflect the pose of the vehicle.Seeing the vehicle as a rectangle can simplify the process of pose estimation and reduce the calculation complexity.The heading angle and position of the vehicle are estimated more accurately by the boundary and position information of the rectangle.Moreover, the rectangular model is suitable for different types of vehicles and can provide stable fitting results at a variety of perspectives and distances.Figure 4 shows the flow chart of the target vehicle rectangular fitting.Figure 5 shows drawing effect of the target vehicle extraction.Figure 6 shows rectangle outline extraction of the target vehicle.Figure 7 shows rectangular fitting of the target vehicle.Due to the lack of existing UDP communication programs in MATLAB (MathWorks, Natick, MA, USA), the programs are written on the computers of RSU end and OBU end to receive and transmit data.Figure 3 shows the architecture of the communication port.The vehicle pose detection algorithm is provided on the RSU terminal computer, and the OBU terminal computer provides the target vehicle ID, time, and GPS positioning information.All the information is transmitted through the communication port built by the UDP.Due to the lack of existing UDP communication programs in MATLAB (MathWorks, Natick, MA, USA), the programs are written on the computers of RSU end and OBU end to receive and transmit data.Figure 3 shows the architecture of the communication port.The vehicle pose detection algorithm is provided on the RSU terminal computer, and the OBU terminal computer provides the target vehicle ID, time, and GPS positioning information.All the information is transmitted through the communication port built by the UDP.

Vehicle Rectangular Fitting
In the top view, the shape of the vehicle is basically rectangular, so the external rectangle of its outline can accurately reflect the pose of the vehicle.Seeing the vehicle as a rectangle can simplify the process of pose estimation and reduce the calculation complexity.The heading angle and position of the vehicle are estimated more accurately by the boundary and position information of the rectangle.Moreover, the rectangular model is suitable for different types of vehicles and can provide stable fitting results at a variety of perspectives and distances.Figure 4 shows the flow chart of the target vehicle rectangular fitting.Figure 5 shows drawing effect of the target vehicle extraction.Figure 6 shows rectangle outline extraction of the target vehicle.Figure 7 shows rectangular fitting of the target vehicle.

Vehicle Rectangular Fitting
In the top view, the shape of the vehicle is basically rectangular, so the external rectangle of its outline can accurately reflect the pose of the vehicle.Seeing the vehicle as a rectangle can simplify the process of pose estimation and reduce the calculation complexity.The heading angle and position of the vehicle are estimated more accurately by the boundary and position information of the rectangle.Moreover, the rectangular model is suitable for different types of vehicles and can provide stable fitting results at a variety of perspectives and distances.Figure 4 shows the flow chart of the target vehicle rectangular fitting.Figure 5 shows drawing effect of the target vehicle extraction.Figure 6 shows rectangle outline extraction of the target vehicle.Figure 7 shows rectangular fitting of the target vehicle.(1) The initial image P0 is used for canny edge detection [22][23][24].Firstly, Gaussian blur is performed to reduce the effect of noise in the images on edge detection.Secondly, the gradient of the image is calculated using the Sobel operator.Thirdly, the image is processed by non-maximum suppression and refined in the gradient direction to make the edge more refined and accurate.Then, double threshold detection is performed to divide the edge pixels of the image into three categories: strong edge, weak edge, and non-edge.By setting two thresholds, the pixels in the gradient image are divided into high-threshold and low-threshold pixels, where the high-threshold pixels are strong edges and the low-threshold pixels are weak edges.Finally, the weak edge points are connected to connect them with the strong edge points to form the complete edges.(2) The image after edge detection is processed by closed operation.The image is corroded to eliminate small holes and small connection areas in the image, making the target more compact.After the corrosion operation, the image is expanded to increase the target in the image and fill the holes in the target to make the target more complete.
(3) Fully fill in the image.The image after closed operation is completely filled, the convex packets and areas of the image are calculated, the area of the convex packets is calculated again, the target with the largest area is retained, the centroid is extracted for further target analysis and processing, the other targets are deleted, and finally, the filled image Pi of the vehicle is obtained.

Vehicle Pose Calculation Model
The roadside equipment end mainly detects the environment within its visual range through the camera loaded on the road experiment unit.The installation height and the pose angle of the camera can be set in advance.Figure 8 is the RSU camera model, which shows the position relationship between the RSU camera and the road plane.Xc, Yc, and Zc are the axes of the camera coordinate system.Xw, Yw, and Zw are the axes of the world coordinate system.The installation height of the RSU camera is h.(1) The initial image P 0 is used for canny edge detection [22][23][24].Firstly, Gaussian blur is performed to reduce the effect of noise in the images on edge detection.Secondly, the gradient of the image is calculated using the Sobel operator.Thirdly, the image is processed by non-maximum suppression and refined in the gradient direction to make the edge more refined and accurate.Then, double threshold detection is performed to divide the edge pixels of the image into three categories: strong edge, weak edge, and non-edge.By setting two thresholds, the pixels in the gradient image are divided into high-threshold and low-threshold pixels, where the high-threshold pixels are strong edges and the low-threshold pixels are weak edges.Finally, the weak edge points are connected to connect them with the strong edge points to form the complete edges.(2) The image after edge detection is processed by closed operation.The image is corroded to eliminate small holes and small connection areas in the image, making the target more compact.After the corrosion operation, the image is expanded to increase the target in the image and fill the holes in the target to make the target more complete.(3) Fully fill in the image.The image after closed operation is completely filled, the convex packets and areas of the image are calculated, the area of the convex packets is calculated again, the target with the largest area is retained, the centroid is extracted for further target analysis and processing, the other targets are deleted, and finally, the filled image P i of the vehicle is obtained.

Vehicle Pose Calculation Model
The roadside equipment end mainly detects the environment within its visual range through the camera loaded on the road experiment unit.The installation height and the pose angle of the camera can be set in advance.Figure 8 is the RSU camera model, which shows the position relationship between the RSU camera and the road plane.X c , Y c , and Z c are the axes of the camera coordinate system.X w , Y w , and Z w are the axes of the world coordinate system.The installation height of the RSU camera is h.The vehicle heading angle refers to the angle between the center speed of the vehicle and the horizontal axis under the world coordinate system.At low speeds, the movement of the vehicle in the vertical direction is usually negligible, so the center velocity direction of the vehicle is consistent with the longitudinal central axis of the vehicle.After obtaining the vehicle rectangular fitting box, the coordinates of the four corner points in the image coordinate system can be obtained: A1 (xc1, yc1), B1 (xc2, yc2), C1 (xc3, yc3), and D1 (xc4, yc4).Converting the coordinates of the four corner points into the world coordinate system, A (xw1, yw1), B (xw2, yw2), C (xw3, yw3), and D (xw4, yw4) are obtained.Figure 9 shows the position of the four angle points of the vehicle in the world coordinate system.Through the coordinates of the four angle points, the angle between the central axis L and the horizontal coordinate axis, which is the calculated heading angle θ and the vehicle center point O, taken as the positioning point, can be obtained.
( ) 4 The vehicle heading angle refers to the angle between the center speed of the vehicle and the horizontal axis under the world coordinate system.At low speeds, the movement of the vehicle in the vertical direction is usually negligible, so the center velocity direction of the vehicle is consistent with the longitudinal central axis of the vehicle.After obtaining the vehicle rectangular fitting box, the coordinates of the four corner points in the image coordinate system can be obtained: A 1 (x c1 , y c1 ), B 1 (x c2 , y c2 ), C 1 (x c3 , y c3 ), and D 1 (x c4 , y c4 ).Converting the coordinates of the four corner points into the world coordinate system, A (x w1 , y w1 ), B (x w2 , y w2 ), C (x w3 , y w3 ), and D (x w4 , y w4 ) are obtained.Figure 9 shows the position of the four angle points of the vehicle in the world coordinate system.Through the coordinates of the four angle points, the angle between the central axis L and the horizontal coordinate axis, which is the calculated heading angle θ and the vehicle center point O, taken as the positioning point, can be obtained.tan θ = y w4 + y w3 − y w2 − y w1 x w4 + x w3 − x w2 − x w1 (1) (x w , y w ) =  In practice, there is a difference between the calculated heading angle and the actual heading angle, so it is stipulated that when the vehicle heading orientation is consistent with the front half axis of the Xw in the world coordinate system, the heading angle is 0°, and the clockwise heading angle is positive.Then, the relationship between calculated  In practice, there is a difference between the calculated heading angle and the actual heading angle, so it is stipulated that when the vehicle heading orientation is consistent with the front half axis of the X w in the world coordinate system, the heading angle is 0 • , and the clockwise heading angle is positive.Then, the relationship between calculated heading angle θ and actual heading angle φ is shown in Figure 10.In practice, there is a difference between the calculated heading angle and the actual heading angle, so it is stipulated that when the vehicle heading orientation is consistent with the front half axis of the Xw in the world coordinate system, the heading angle is 0°, and the clockwise heading angle is positive.Then, the relationship between calculated heading angle θ and actual heading angle φ is shown in Figure 10.

Vehicle Heading Direction Detection Model
The judgment of the vehicle heading direction is an important process of the vehicle heading angle detection.In this paper, the forward command is sent to the vehicle through communication to control the vehicle forward and the vehicle heading direction is judged by the moving direction of the vehicle center point.When the vehicle takes the rectangular fitting, the movement direction of the center point of the vehicle may not be on the same straight line, that is, there is an angle.In this case, the movement direction of the center point of the vehicle cannot be simply used as the vehicle heading direction, and the vehicle heading direction needs to be accurately judged to calculate the pose.Figure 11 shows the specific steps.Figure 12 shows the schematic diagram of the vehicle heading direction judgment.
(1) Calculate the initial image pose.Obtain the initial image P 0 , conduct the preliminary pose calculation for it, and obtain the center point coordinate C 0 . ( (4) Calculate the minimum clip angle β.Calculate the angles a 1 , a 2 , a 3 , a 4 between the displacement vector v 0 and the side midpoint vectors v 1 , v 2 , v 3 , v 4 .
(5) The minimum vector clip angle β is obtained: (6) Determine the vehicle heading direction.The vector direction corresponding to the minimum clip angle β is the vehicle heading direction.

Vehicle Heading Direction Detection Model
The judgment of the vehicle heading direction is an important process of the vehicle heading angle detection.In this paper, the forward command is sent to the vehicle through communication to control the vehicle forward and the vehicle heading direction is judged by the moving direction of the vehicle center point.When the vehicle takes the rectangular fitting, the movement direction of the center point of the vehicle may not be on the same straight line, that is, there is an angle.In this case, the movement direction of the center point of the vehicle cannot be simply used as the vehicle heading direction, and the vehicle heading direction needs to be accurately judged to calculate the pose.Figure 11 shows the specific steps.Figure 12 shows the schematic diagram of the vehicle heading direction judgment.
(5) The minimum vector clip angle β is obtained: ( ) min , , , a a a a (6) Determine the vehicle heading direction.The vector direction corresponding to the minimum clip angle β is the vehicle heading direction.During the training process, the model was optimized using the stochastic gradient descent optimizer.The initial learning rate is 0.01, the MiniBatchSize is set to 20, the MaxEpochs is set to 300, and the input resolution of the image is 1530 × 2720 [25][26][27].Figure 14 shows the training loss for each iteration.During the training process, the model was optimized using the stochastic gradient descent optimizer.The initial learning rate is 0.01, the MiniBatchSize is set to 20, the MaxEpochs is set to 300, and the input resolution of the image is 1530 × 2720 [25][26][27].Figure 14 shows the training loss for each iteration.

Training Accuracy
In order to evaluate the training effect more accurately, the evaluation indexes such as detection accuracy, true accuracy, and overall accuracy were used to evaluate the model performance.The calculation method is shown in Equations ( 7)-( 9):

Training Accuracy
In order to evaluate the training effect more accurately, the evaluation indexes such as detection accuracy, true accuracy, and overall accuracy were used to evaluate the model performance.The calculation method is shown in Equations ( 7)-( 9): TP is the number of samples detected as vehicles and actually vehicles.FP is the number of samples detected as vehicles but not actually vehicles.TN is the number of samples detected as not vehicles but actually vehicles and FN is the number of samples detected as not vehicles and actually not vehicles.According to the detection results, the confusion matrix [28][29][30][31][32] was obtained as shown in Table 1.It can be seen from Table 2 that the effect of the training for vehicles detection is remarkable, which can meet the needs of target vehicle identification for pose detection.

Pose Detection Experiment of the Real Vehicle Scenarios
The experiment was operated as follows.The UAV and the terminal computer form the RSU equipment, in which the UAV camera is the RSU camera, and the terminal computer realizes the image and data processing and communicates with the vehicle OBU.To enable the UAV to simulate the RSU as much as possible, the experiment was conducted in a windless environment.The UAV flies to a certain height and then hovers.The UAV camera should be kept parallel to the ground, namely, the pitch angle is 0 • .When the vehicle enters the RSU perception area, the OBU sends the on-board GPS position information to the RSU through UDP.After the RSU accepts the information, the vehicle is obtained and the initial image obtained by the RSU is sent to the RSU through Raspberry PI for processing.The RSU computer first conducts the target detection.By matching the result of the target detection and the GPS information, the detection target is considered as the actual target vehicle.The preliminary pose calculation of the target vehicle was performed to obtain the coordinate of the vehicle center point.At this point, the RSU sends a forward command to the vehicle, and the OBU receives the command and forwards it to the ECU to control the vehicle to move forward.The RSU tracks the vehicle and completes the preliminary pose calculation of the vehicle in real time to obtain the coordinate of the vehicle center point.By comparing the initial center point coordinate of the vehicle with the real-time center point coordinate after moving forward, the vehicle heading direction can be determined.Subsequently, the RSU continues to track the vehicle, detect its pose, obtain the center coordinates of the vehicle and the heading angle information, and transmit the information to the OBU, completing the entire process of vehicle pose detection.To verify the effectiveness of the vehicle pose detection method based on the RSU under different driving modes, the experiments were conducted in both straight-line and turning driving states.Table 3 shows UAV parameters, Figure 15 shows experimental vehicle equipment, Figure 16 shows RSU equipment and Figure 17 shows the relative position of the UAV and the camera.different driving modes, the experiments were conducted in both straight-line and turning driving states.Table 3 shows UAV parameters, Figure 15 shows experimental vehicle equipment, Figure 16 shows RSU equipment and Figure 17 shows the relative position of the UAV and the camera.The vehicle heading direction detection experiments were conducted.Figure 18 shows the system equipment (the red represents device name and installation location, the blue represents communication mode, and the black represents data transmission content).The experiment scenarios are shown in Figure 19 (the red arrow indicates the driving direction of the vehicle).Figure 19a shows the actual experiment route of the straight-line scenario (where the green arrow indicates the heading direction and the red arrow is the moving direction of the center point); Figure 19b shows the map identification route of the straight-line scenario; Figure 19c shows the actual experiment route of the turning scenario; Figure 19d shows the map identification route of the turning scenario.The experiment results are shown in Figure 20 (the green arrow indicates the heading direction and the red arrow is the moving direction of the center point).Figure 20a-c show the detection effect of the head with no deviation between the moving direction of the center point and the heading direction; Figure 20d-i are the detection effect of the heading direction with different deviations.As can be seen from the figures, the heading direction detection method has a significant effect.Figures 21-23 shows the results of target vehicle detection, target vehicle tracking and target vehicle pose calculation, respectively.Figure 24 shows the communication results.The vehicle heading direction detection experiments were conducted.Figure 18 shows the system equipment (the red represents device name and installation location, the blue represents communication mode, and the black represents data transmission content).The experiment scenarios are shown in Figure 19 (the red arrow indicates the driving direction of the vehicle).Figure 19a shows the actual experiment route of the straightline scenario (where the green arrow indicates the heading direction and the red arrow is the moving direction of the center point); Figure 19b shows the map identification route of the straight-line scenario; Figure 19c shows the actual experiment route of the turning scenario; Figure 19d shows the map identification route of the turning scenario.The experiment results are shown in Figure 20 (the green arrow indicates the heading direction and the red arrow is the moving direction of the center point).Figure 20a-c show the    (g) (h) (i)

Detection Speed Analysis
The average time used by the vehicle pose detection method based on the RSU and the GPS method was compared to obtain the pose detection data, and the detection speed of the two methods was compared.The detection time is shown in Table 4.

RSU GPS
Detection time/s 0.063 0.235 Table 4 shows that the vehicle pose detection method based on the RSU takes less time and detects faster when obtaining the pose detection data than the GPS-based method.This is because this method obtains many image data, and the pose solution process is simple, while data obtained by GPS have a low frequency, and there is a delay.Therefore, the detection speed of the vehicle pose detection method based on the RSU method is fast.

Detection Accuracy Analysis
The trajectory and heading angle plots obtained from the vehicle pose detection method based on the RSU and the GPS-based method are shown in the figure below.Figures 25 and 26 shows the results of the straight-line scenario and the turning scenario, respectively.Due to the different data acquisition frequencies of this method, GPS, and RTK (which has a lower frequency), interpolation was performed on the RTK data to facilitate error analysis.As the data trend obtained in this process from RTK is relatively regular, linear interpolation is used to interpolate the data according to the frequencies of RSU and GPS to obtain the interpolated results.A comparison is then made between the interpolated results and the detection results from RSU and GPS, generating error plots.A detailed analysis of the errors is presented in the table below.The columns in the table represent max for maximum error, min for minimum error, mean for average error, std for standard deviation, and rmse for root mean square error.All values are in meters.From the Table 5, it can be seen that whether in the straight-line scenario or the turning scenario, the performance of the vehicle pose detection method based on the RSU is overall better than the GPS-based method.This is because the vehicle pose detection method based on the RSU has strong adaptability to the environment and a faster detection speed, enabling it to capture the changes in the vehicle pose during driving in a timely manner.On the other hand, GPS has positioning delays, making it unable to reflect realtime changes in the vehicle pose.Additionally, during turning maneuvers where the environment changes significantly, GPS accuracy is affected by environmental factors, leading to decreased precision.The method presented in this paper exhibits some jitter phenomenon, which is attributed to the unstable process of rectangular fitting, particularly in the fitting of the rearview mirror.The instability occurs during the rectangular fitting process; where both rearview mirrors may be fitted, only one rearview mirror may be fitted, or no rearview mirror may be fitted, thereby affecting the precise detection of trajectory points.Furthermore, during the target detection process, parts of the environment that do not belong to the vehicle are also detected, leading to the extraction of other feature points during pose detection and impacting the final detection results.In summary, the vehicle

Conclusions
This paper presents a pose detection method based on the RSU method.In the process of vehicle pose calculation, the paper analyzes the pose information of vehicles with different heading orientations to avoid pose misjudgments caused by vehicle orientation.Comparison experiments were conducted outdoors between the GPS-based pose detection method and the method in this paper.The experimental results of the position and heading angle were compared using mean, max, min, std, and rmse as evaluation metrics.The results indicate that, due to the simplicity and stability of the detection method in this paper, it has a high accuracy.The detection speed of the GPS-based pose detection method was compared with this method, and the processing speed of this method is faster due to the high frame rate of the RSU camera.
The method presented in this paper has the advantages of high accuracy and fast processing speed, and is suitable for various vehicle scenarios such as parking lots, areas with dense and high buildings, and cargo yards.This paper is innovative in the pose detection field, using the method based on the RSU to realize the accurate detection of the vehicle pose.However, the security of the data transmission process in this paper is still insufficient.The subsequent research will focus on the security direction of the data transmission process, and ensure the security of the data transmission process by strengthening the encryption algorithm, adopting two-factor authentication, and establishing the secure data channels to confront the increasingly severe cyber security threats.The problems of increased transmission time brought by these measures will also be considered.At the same time, regarding the increased communication delay that may arise from a single RSU serving multiple OBUs, multi-channel and time allocation, priority and quality of service management, and channel management and resource allocation will be considered to improve the performance of the method proposed in the paper.

Figure 4 .
Figure 4. Flow chart of the target vehicle rectangular fitting.

( 4 )
Rotate Pi.Rotate Pi at different angles (from 1° to 90°) to find the minimum external rectangle.(5) The bounding box information is calculated for each rotated image, including the upper left coordinate (x, y), width (w), and height (h) of the bounding box.(6) Calculate the minimum area angle ω.The area of each rotated image is calculated based on the width and height of the bounding box to find the rotation angle that makes the area of the external rectangle minimum.(7) The image Pi is rotated according to the optimal angle found to obtain the final rotated image Pw. (8) The coordinates of the four corner points (xn, yn) (n = 1, 2, 3, 4) after rotation are calculated to complete the fitting process of the image minimum external rectangle.(9) The coordinates of the four corner points (xrn, yrn) (n = 1, 2, 3, 4) of the external rectangle in the initial image P0 are calculated based on the rotation angle ω, completing the final process of rectangular fitting.(a) Initial image

Figure 4 .Figure 5 .Figure 6 .
Figure 4. Flow chart of the target vehicle rectangular fitting.the filled image Pi of the vehicle is obtained.(4) Rotate Pi.Rotate Pi at different angles (from 1° to 90°) to find the minimum external rectangle.(5) The bounding box information is calculated for each rotated image, including the upper left coordinate (x, y), width (w), and height (h) of the bounding box.(6) Calculate the minimum area angle ω.The area of each rotated image is calculated based on the width and height of the bounding box to find the rotation angle that makes the area of the external rectangle minimum.(7) The image Pi is rotated according to the optimal angle found to obtain the final rotated image Pw. (8) The coordinates of the four corner points (xn, yn) (n = 1, 2, 3, 4) after rotation are calculated to complete the fitting process of the image minimum external rectangle.(9) The coordinates of the four corner points (xrn, yrn) (n = 1, 2, 3, 4) of the external rectangle in the initial image P0 are calculated based on the rotation angle ω, completing the final process of rectangular fitting.

Figure 5 .Figure 5 .
Figure 5. Drawing effect of the target vehicle extraction (the left, middle, and right images show the image processing effects in different directions).

Figure 6 .
Figure 6.Rectangle outline extraction of the target vehicle.Figure 6. Rectangle outline extraction of the target vehicle.

Figure 6 .
Figure 6.Rectangle outline extraction of the target vehicle.Figure 6. Rectangle outline extraction of the target vehicle.

Figure 6 .
Figure 6.Rectangle outline extraction of the target vehicle.

Figure 7 .
Figure 7. Rectangular fitting of the target vehicle (the left, middle, and right images show the final effects in different directions).

Figure 7 .
Figure 7. Rectangular fitting of the target vehicle (the left, middle, and right images show the final effects in different directions).

( 4 )
Rotate P i .Rotate P i at different angles (from 1 • to 90 • ) to find the minimum external rectangle.(5) The bounding box information is calculated for each rotated image, including the upper left coordinate (x, y), width (w), and height (h) of the bounding box.(6) Calculate the minimum area angle ω.The area of each rotated image is calculated based on the width and height of the bounding box to find the rotation angle that makes the area of the external rectangle minimum.(7) The image P i is rotated according to the optimal angle found to obtain the final rotated image P w .(8) The coordinates of the four corner points (x n , y n ) (n = 1, 2, 3, 4) after rotation are calculated to complete the fitting process of the image minimum external rectangle.(9) The coordinates of the four corner points (x rn , y rn ) (n = 1, 2, 3, 4) of the external rectangle in the initial image P 0 are calculated based on the rotation angle ω, completing the final process of rectangular fitting.

Figure 8 .
Figure 8. Schematic diagram of the road camera (RSU camera) model.

Figure 8 .
Figure 8. Schematic diagram of the road camera (RSU camera) model.

Figure 9 .
Figure 9. Schematic diagram of the pose calculation.

Figure 9 .
Figure 9. Schematic diagram of the pose calculation.

Figure 9 .
Figure 9. Schematic diagram of the pose calculation.

Figure 10 . 10 .
Figure 10.The schematic diagram of the conversion relationship between actual heading angle and calculated heading angle.(a) The schematic diagram when heading angle is 90°; (b) the schematic Figure 10.The schematic diagram of the conversion relationship between actual heading angle and calculated heading angle.(a) The schematic diagram when heading angle is 90 • ; (b) the schematic diagram when heading angle is 270 • ; (c,d) the schematic diagram of heading angle calculation in other cases.

Sensors 2024 ,
24, x FOR PEER REVIEW 10 of 23 diagram when heading angle is 270°; (c,d) the schematic diagram of heading angle calculation in other cases.

Figure 11 .( 1 )
Figure 11.Flow chart of the vehicle heading direction judgment.(1) Calculate the initial image pose.Obtain the initial image P0, conduct the preliminary pose calculation for it, and obtain the center point coordinate C0. (2) Calculate the real-time image pose.Conduct the preliminary pose calculation for the real-time image Pt to obtain the center point coordinate Ct, and the four side midpoint coordinates C1, C2, C3, C4. (3) Calculate the displacement vector v0 and the side midpoint vector vn.The center point displacement vector 0 0 t v C C = − .Calculate the side midpoint vectors v1, v2, v3, v4 from the center point of the vehicle Ct obtained through the real-time bounding box to the midpoints of the four sides of the rectangle C1, C2, C3, C4.

Figure 11 .Figure 12 .
Figure 11.Flow chart of the vehicle heading direction judgment.

4. 1 .
Object Detection 4.1.1.Dataset TrainingThe vehicle pose detection method proposed in this paper calculates the vehicle pose by vehicle rectangular fitting.In order to identify the target vehicles in the traffic scenes,

Figure 12 .
Figure 12.Schematic diagram of the vehicle heading direction judgment.(a), (b) and (c) represent the judgment of the heading direction in different situations, respectively.

4 . 23 Figure 13 .
Figure 13.Part of the dataset images.During the training process, the model was optimized using the stochastic gradient descent optimizer.The initial learning rate is 0.01, the MiniBatchSize is set to 20, the MaxEpochs is set to 300, and the input resolution of the image is 1530 × 2720[25][26][27].Figure14shows the training loss for each iteration.

Figure 13 .
Figure 13.Part of the dataset images.

Figure 13 .
Figure 13.Part of the dataset images.

Figure 14 .
Figure 14.Training loss for each iteration.

Figure 14 .
Figure 14.Training loss for each iteration.

Figure 17 .
Figure 17.The relative position of the UAV and the camera.

Figure 17 .
Figure 17.The relative position of the UAV and the camera.

Figure 19 .Figure 5 .Figure 20 .Figure 25 .
Figure 19.Experiment scenarios.(a) shows the actual experiment route of the straight-line scenario; (b) shows the map identification route of the straight-line scenario; (c) shows the actual experiment route of the turning scenario and (d) shows the map identification route of the turning scenario.

Figure 20 .
Figure 20.The vehicle heading direction detection.a-c show the detection effect of the head with no deviation between the moving direction of the center point and the heading direction; d-i are the detection effect of the heading direction with different deviations.

Figure 20 .
Figure 20.The vehicle heading direction detection.(a-c) show the detection effect of the head with no deviation between the moving direction of the center point and the heading direction; (d-i) are the detection effect of the heading direction with different deviations.

Figure 20 .Figure 21 .
Figure 20.The vehicle heading direction detection.a-c show the detection effect of the head with no deviation between the moving direction of the center point and the heading direction; d-i are the detection effect of the heading direction with different deviations.

Figure 22 .Figure 22 .Figure 23 .
Figure 22.Target vehicle tracking.(a-c) represent the target vehicle tracking results at different times, respectively.

Figure 23 .
Figure 23.Target vehicle pose calculation.(a-c) represent the target vehicle pose calculation results at different times, respectively.

Figure 23 .Figure 24 .
Figure 23.Target vehicle pose calculation.(a), (b) and (c) represent the target vehicle pose calculation results at different times, respectively.

Sensors 2024 ,Figure 20 .Figure 25 .Figure 25 .Figure 25 .
Figure 25.Results of the straight-line scenario.(a) The absolute trajectory comparison result of the straight-line scenario; (b) the absolute trajectory error comparison result of the straight-line scenario; (c) the heading angle comparison result of the straight-line scenario; (d) the heading angle error comparison result of the straight-line scenario.

Figure 25 .Figure 26 .
Figure 25.Results of the straight-line scenario.(a) The absolute trajectory comparison result of the straight-line scenario; (b) the absolute trajectory error comparison result of the straight-line scenario; (c) the heading angle comparison result of the straight-line scenario; (d) the heading angle error comparison result of the straight-line scenario.
Rotate P i to P w Calculate (x n , y n ) ) Calculate the real-time image pose.Conduct the preliminary pose calculation for the real-time image P t to obtain the center point coordinate C t , and the four side midpoint coordinates C 1 , C 2 , C 3 , C 4 .(3) Calculate the displacement vector v 0 and the side midpoint vector v n .The center point displacement vector v 0 = C t − C 0 .Calculate the side midpoint vectors v 1 , v 2 , v 3 , v 4 from the center point of the vehicle C t obtained through the real-time bounding box to the midpoints of the four sides of the rectangle C 1 , C 2 , C 3 , C 4 .

Table 1 .
The confusion matrix of the detection results.

Table 3 .
Table of UAV parameters.

Table 3 .
Table of UAV parameters.

Table 4 .
Comparison of the average detection time.