A Centralized Processing Framework for Foliage-Penetration Human Tracking in Multistatic Radar

A complete centralized processing framework is proposed for human tracking using multistatic radar in the foliage-penetration environment. The configuration of the multistatic radar system is described. Primary attention is devoted to time of arrival (TOA) estimation and target localization. An improved approach that takes the geometrical center as the TOA estimation of the human target is given. The minimum mean square error paring (MMSEP) approach is introduced for multi-target localization in the multistatic radar system. An improved MMSEP algorithm is proposed using the maximum velocity limitation and the global nearest neighbor criterion, efficiently decreasing the computational cost of MMSEP. The experimental results verify the effectiveness of the centralized processing framework.


Introduction
Human target detection and tracking have great potential in military, safety, security and entertainment applications [1].A number of technologies have been used for human detections, including computer vision [2], lidar [3], radio frequency identification devices (RFID) [4] and radar [1], [5][6][7].Computer vision and lidar performance is degraded in dusty, foggy and non-line-of-sight environments.RFID is a cooperative sensor and can't be used in noncooperative conditions.Radar is a privileged tool to detect humans in poor visibility conditions (e.g., at night, fog, smoke, and in through-wall and foliage-penetration applications).
Multistatic radar with a low frequency wideband transmitting signal has fine localization precision, a large covered area, and the ability to penetrate the foliage [5], [8].It is highly suitable for foliage-penetration surveillance.A multistatic radar system usually consists of one transmitting antenna and several spatially distributed receiving antennas.Due to cost and size limitations, antennas of such a system generally have a large beam angle and can only provide limited angle-of-arrival information.By the high range resolution of wideband signals, time-of-arrival (TOA) is commonly used for target localization [6], [9].
Multi-target tracking in multistatic radar is a complex task that requires clutter suppression, target detection, TOA estimation, target localization and target tracking.Human detection is full of challenges due to the low reflectivity of the human body and the severe clutter of the foliage-penetration environment.An appropriate clutter suppression algorithm should be used to remove most clutter and allow the response of the moving human to be detected.Target detection and TOA estimation provide the range measurements for target localization.They should insure relatively low miss and false detection rates, and accurate TOA estimation.Target localization is a key challenge in multistatic radar system.Especially in the multiple-target environment, it is necessary to identify, among all available bistatic range measurements of all receiving channels, which combinations of measurements correspond to true targets.Wrong measurement-to-measurement association leads to the well-known ghost issue [10], [11].For human target tracking, a decentralized framework has been proposed in [12].It uses the human slow-time feature for measurement association.The relatively expensive computation cost and the non-real-time tracking limit the application of the decentralized framework [9], [12].
If the distance between receiving antennas is small enough, the measurement association method proposed in [6] can be applied.However, this configuration has limited localization precision.In this paper, we consider some more general approaches for measurement association in multistatic radar system.The S-D assignment algorithm is an option, but the assignment problem is NP hard for S ≥ 3 (S is the number of receiving channels) [13].An efficient method proposed in [14] defines a list of potential targets based on a certain metric (a potential target corresponds to a combination of the measurements from each receiving channel).It avoids resolving the optimum assignment problem and identifies the maximum likelihood targets by re-peatedly selecting the highest potential target on the list.This method needs to compute the metric for all the possible combinations and has heavy computational cost.We will offer an improved algorithm in Sec. 3. Target tracking provides the results of the moving trajectories of humans.It can be implemented by the Kalman filter-based multitarget tracking (MTT) system described in [15], [16].
The paper is organized as follows.First, the multistatic radar system is described in Sec. 2.Then, a centralized framework for foliage-penetration human tracking is illustrated in detail in Sec. 3. In Sec. 4, the performance of the proposed approach is evaluated using the experimental data.Finally, some conclusions are drawn in Sec. 5.

Multistatic Radar System
The diagram of our multistatic radar system is presented in Fig. 1.
The multistatic radar consists of one transmitting antenna and three receiving antennas.Antennas are connected to the transmitter and receivers by cables with lengths limited to 6 m.The transmitting signal is a linear frequency modulation continuous wave with bandwidth 150 MHz at ultra-high frequency band.The radar has a range resolution of 1 m and is able to penetrate bushes to detect concealed moving targets.The receiving signal is mixed with the transmitting signal to reduce the sampling rate in the receiver.Detailed parameters are provided in Tab. 1.The sampling data of each receiving channel are organized as a 2-D matrix.The column of the matrix represents the sampling of the received signal in one pulse repetition interval (fast time), and consecutive pulses are stored in successive columns (slow time).By performing fast Fourier transform to the columns of each matrix, the range profiles can be obtained.To improve the signal to noise ratio, we divide the slow time dimension into multiple coherent processing intervals (CPIs) and add the range profiles in each CPI.The data after accumulating processing are y s (t, τ), where t represents the propagation time, τ represents the slow time whose interval is a CPI, and the superscript s = 1,2,3 represents the data of channel s.

Centralized Tracking Framework
As decentralized tracking is not capable of real-time tracking [9], we design a centralized tracking framework for real-time human tracking in multistatic radar.The centralized tracking framework is presented in Fig. 2.
Clutter suppression, target detection and TOA estimation are performed in each receiving channel.The bistatic range measurements, which can be derived from the TOA values, are fused to estimate the target positions by a multitarget localization algorithm.Finally, MTT is performed to obtain target trajectories.Each step is described in detail in the following.

Clutter Suppression
Generally, the received signal strength of the human body is weaker than the bush clutter, the antenna coupling and other ambient static clutter.Due to the signal attenuation of bushes and leaves, the human echoes become much weaker.The clutter suppression phase is essential to remove most clutter and allow the response of moving humans to be detected.There are several background subtraction approaches for clutter suppression, such as accumula- tive average, moving average, and exponential average [17].We chose exponential average background subtraction in our application for its robust performance and low complexity.Its implementation can be found in [17], [18].The data after exponential average background subtraction processing are y c s (t, τ), where the subscript c represents the output of the clutter suppression phase.

Target Detection
The purpose of detection is to decide whether targets are present in the examined radar signals.Constant false alarm rate (CFAR) detectors provide good and robust performance for human detection in wide band radar [7].Based on the Neyman-Person criterion, they have the maximum probability of detection for a given false alarm rate [19].Among them, cell averaging CFAR (CA-CFAR) is the most common detector for moving target detection [9], [20].However, in multi-target scenarios, nearby targets will increase the threshold leading to high misdetection rate.We apply ordered statistics CFAR (OS-CFAR) because it has better detection performance in the multi-target environment [7], [21].
The output of the OS-CFAR detector is given as follows: where T is the adaptive threshold provided by OS-CFAR, the subscript d represents the output of target detection phase.After OS-CFAR, one range profile corresponds to a vector with binary values.

TOA Estimation
A human target is a typical extended target.With the range resolution of 1 m, the fast time response of a normalsized person covers about 4 m in our multistatic radar system.To improve the accuracy of TOA estimation, we set the bistatic range sampling interval to 0.2 m.Therefore, one person is usually represented by several TOA values.To simplify the target localization, each detected person should correspond to only one TOA.A method that takes the leading edge as the TOA estimation of the human target has been proposed in [6].Due to the specific motion characteristics of people, leading edge may indicate the arms and legs, which move ahead of torso.Additionally, the range resolution of 1 m is not fine enough to use the leading edge TOA estimation method.An improved approach is to select the geometrical center (the center of the extended TOA values of the human) as the TOA estimation.If the length of the filter does not match the human extension or if the distance of the targets in the range profile is close, the average filter method may generate unwanted TOA estimations [9].Here, we propose a robust geometrical center TOA estimation method.
The geometrical center TOA estimation method consists of three steps.The first two steps are the same as the leading edge TOA estimation method in [6].As the parameters of our system are different from [6], we give whole steps as follows: 1. Accumulation of detector output with interval length corresponding to the extended size of the target.
where W p represents the extended size of the target, and the subscript e represents the TOA estimation phase.As the bistatic range sampling interval is set to 0.2 m, the extended size 4 m of a normal person corresponds to the interval length W p = 20.

Generation
where T lim determines the minimal number of reflections that confirm a target.T lim is set to 8 in our system.After the above two steps, the majority of the false alarms and misdetections can be mitigated.

3.
Substitution of extended TOA values by single TOA estimation.
In this step, each continuous TOA interval is substituted by, at most, two TOA estimations.The pseudocode procedure of geometrical center TOA estimation is given in Tab. 2.
The variable N t is the total number of samples of ( , ) s e y t   in the range profile.TOA s m (τ) represents the geometrical center TOA estimation of the mth target at slow time τ in channel s.The operator x gives the integer nearest to x.If two targets are close enough in range, their extended TOA values may overlap and cause continuous TOA interval.The variable tNum considers the above case, and if tNum ≥ 2, it is predicted that there are two targets in the continuous TOA interval.When the loop ends, the number of targets and their corresponding TOA estimations can be estimated.The performance of the TOA estimation method will be verified by the TOA estimation results and human tracking results in Sec. 4.
By multiplying the TOA estimations with the electromagnetic wave propagation velocity c, bistatic range measurements are given: where m s is the estimated target number in channel s.Target localization can be performed by the Taylor series algorithm [9], [22] with the selected bistatic range measurement corresponding to the target in each channel.Then, the question is raised of which combinations of measurements in the receiving channels correspond to true targets.
Pseudocode procedure of geometrical center TOA estimation.

Target Localization
For multi-target localization in a multistatic radar system, one has to identify, among all available bistatic range measurements of all receiving channels, which combinations of measurements correspond to true targets.An efficient method has been proposed for range-to-range association in multi-target multi-sensor environment [14].We introduce this method for our measurement association problem and propose an improved algorithm.
Assume that the numbers of output range measurements in each channel are m 1 , m 2, and m 3 , respectively.We will consider three measurements from three receiving channels, making the total number of possible combinations (potential targets) in the initial list m 1  m 2  m 3 .One can calculate a presumed position of the target using the selected combination by a certain localization algorithm.Then, a metric can be calculated that is based on all the range measurements upon the selected combination.Here, we chose mean square error (MSE) as the metric.For instance, a potential target at (x, y) can be estimated upon a combination {r 1 i1 , r 2 i2 , r 3 i3 } (r s i s represents the i s th measurement in receiving channel s).The MSE metric is defined as follows: the estimated bistatic range, (x t , y t ) is the position of the transmitting antenna, and (x s , y s ) (s = 1,2,3,) is the position of the receiving antenna.
First, we can set a maximum permitted MSE and delete the combinations with MSEs larger than the permitted MSE.Then, the list of potential targets is ordered by the MSE metric, with the minimum MSE at the top and the maximum MSE at the bottom.This approach assumes that the potential target highest on the list has the maximum likelihood of being the true target.Select the potential target highest on the list and eliminate all other lower ordered potential targets that are based on any of the measurements upon which the selected target was based.The list can thus be pared down.The operation is repeated for the next highest potential target remaining on the list until the number of selected targets reaches the preset target number or the list is empty.As the method selects the target with minimum MSE and pares down the list in each process loop, we call it minimum MSE paring (MMSEP) algorithm.The MMSEP algorithm can be summarized by the following steps: 1. List all the combinations, estimate the position of potential targets, and calculate the MSE of each combination.Delete the combinations with MSEs larger than the permitted MSE.

2.
Order the list of potential targets by their MSE, with the minimum MSEs at the top and the maximum MSEs at the bottom.

3.
Select the potential target highest on the list and eliminate all other lower ordered potential targets based on any of the measurements upon which the selected target was based.Add the selected potential target to a list of true targets.

4.
Repeat step 3 until the number of selected targets reaches the preset target number or the list is empty.
The main computational cost lays in the first step, which must compute the locations and the MSEs of all possible combinations.In this paper, we propose the following method to reduce the computational cost in the original algorithm.
Assuming that the Cartesian location at slow time k -1 is: where m k -1 is the estimated number of the targets at slow time k -1.Then, the bistatic range of target j in each receiving channel can be calculated: The bistatic range measurements in each receiving channel at slow time k are: where m k s is the number of measurements in receiving channel s at slow time k.
We define the following inequality: where V max is the maximum moving velocity of the human (normally V max < 10 m/s), r is a preset range error, and T is the time interval between slow time k -1 and k (i.e., CPI).As the moving distance in T is limited by 2V max T (2V max is the maximum bistatic velocity [23]), the measurements satisfying inequality ( 9) have the highest likelihood to be associated with target j.The position of j at slow time k can be calculated based on the associated measurements.Delete the associated measurements from R k s and repeat association for other targets in P k -1 until j = m k -1 or until at least one set among R k s , s = 1, 2, 3 is empty.After above processing, if all sets R k s , s = 1, 2, 3 are not empty, then perform MMSEP algorithm upon remaining measurements.As the global nearest neighbor (GNN) criterion is used in inequality (9), we call the proposed procedure as GNN-MMSEP algorithm.The GNN-MMSEP algorithm is summarized as follows: 1. Perform the MMSEP algorithm for target localization at slow time k = 1.The positions of the targets are stored in set P 1 ，k = k + 1.

If all sets among R k s
, s = 1, 2, 3 are not empty, perform MMSEP algorithm for target localization upon remaining measurements and add the positions to P k .k = k + 1.Return to step 2.
The GNN-MMSEP algorithm performs MMSEP for target localization at the first slow time interval.At successive slow time intervals, it uses GNN criterion for target association to decrease the computational cost of target localization for all possible combinations in MMSEP.The 5th step maintains the targets that are not associated with the targets at the previous slow time interval to be located.
There are two factors that can affect the target localization performance of the GNN-MMSEP algorithm.

1.
If there are intensive clutter measurements in the receiving channel, inequality (9) may associate clutter measurements with the targets.As the TOA estimation method above can reject clutter, the influence of clutter can be omitted.Additionally, wrong associations will generate false position at current slow time interval and the false position has less possibility to be associated with the targets at successive slow time intervals.The true target position will be relocated in step 5.The algorithm itself is robust to sparse clutter environment.

2.
If targets are close in range, it also may cause wrong associations.Our algorithm is also robust in this case, as the position calculated by wrong combinations at current slow time interval is less likely to be associated with the targets at successive slow time intervals.The true target position will be relocated in step 5.
The experimental results in Sec. 4 demonstrate the performance of the GNN-MMSEP algorithm.

Target Tracking
The locations of the targets calculated by the range measurements normally have certain random error.Generally, target tracking provides more accurate and smoothing trajectories.For MTT, all the measurement-to-track association, track filtering, track initialization and track maintenance steps should be considered.We chose a Kalman filter based MTT framework, as it has been demonstrated robust and efficient for human tracking [15].Detailed description of the implementation of MTT system can be found in [15], [16].

Experimental Scenario
The multistatic radar is localized behind a thick and impenetrable bush.Two people move on the other side of the bush.The bush is approximately 2 m thick, 1.8 m high and 9 m wide.The positions of antennas are set as follows: TX (1,0), RX1 (-4,2), RX2 (-1,1) and RX3 (4,2), where TX/RX indicates the transmitting antenna and receiving antenna, respectively.The antennas were set to insure that they are not in line-of-sight of the moving human.The experimental scenario is illustrated in Fig. 3.The data used in this paper were recorded in a cloudy day with gentle breeze in September.The relative humidity of the air was about 60%.The bush had bushy leaves.

Results
The slow time-range image of the receiving channel one is presented in Fig. 4. Figure 4(a) is the image before clutter suppression.The human trajectories are submerged in the clutter and difficult to be detected.By exponential average background subtraction clutter suppression method, most clutter is removed and human trajectories are clearly revealed in Fig. 4(b).Note that the performance of clutter suppression in the foliage-penetration environment relies heavily on the weather.Further study for clutter suppression will be taken under different weather.The OS-CFAR detection and the TOA estimation procedure are performed on the range profiles after clutter suppression.Figure 5(a) is the result of the leading edge method proposed in [6].It takes the leading edge of the OS-CFAR outputs as the TOA estimation of the targets, which may produce errors as described in Sec.3.3, and it may miss the measurements of the further target when targets are close to each other in the range direction.We have also performed the GNN-MMSEP algorithm upon the measurements of the geometrical center TOA estimation for target localization.Figure 6(c) shows the localization and tracking results of GNN-MMSEP.When targets cross in the range profile, this algorithm may associate wrong measurements, as described in Sec.3.5.Wrong measurement combinations will lead to wrong localizations.Therefore, the discontinuities of the trajectories in Fig. 6(c) are slightly worse than Fig. 6(b).However, this small gap can be tolerated in human tracking applications.Although comparable localization error was presented in Fig. 6(b) and Fig. 6(c), GNN-MMSEP approach indeed demonstrated a lower computational cost in comparison with the original MMSEP approach.In our experimental scenario, the total processing time of 120 slow time intervals by MMSEP is 1.56 s.However, the GNN-MMSEP algorithm took only 0.064 s (CPU: Intel i5-3470, MATLAB version: R2012b).By using the GNN-MMSEP algorithm, we improve the computational efficiency of target localization by more than 20 times.
To evaluate the performance of the tracking framework, a mean tracking error metric is defined as: where (x(τ), y(τ)) is the person's ground truth at slow time τ (we assume that the human are walking at a constant velocity), and (x ̅ (τ), y ̅ (τ)) is the person's predicted position provided by the tracking results.Table 3 shows the tracking precision of different TOA estimation and target localization methods.The tracking framework using leading edge TOA estimation has the worst performance.The mean tracking error is more than 1 m.By geometrical center TOA estimation, the tracking frameworks with the MMSEP and GNN-MMSEP location algorithms nearly have the same tracking precision.

Conclusion
In this paper, we have proposed a complete centralized processing framework for foliage-penetration human tracking in multistatic radar.A geometrical center method is proposed for the more accurate TOA estimation of human targets.The MMSEP algorithm is introduced for more general human targets localization in a multistatic radar system.The proposed GNN-MMSEP method has shown better performance in terms of computational efficiency in comparison with the original MMSEP algorithm.The experimental results show the validity of the proposed centralized tracking framework for foliage-penetration human tracking.The introduced centralized tracking framework also has great potential for other applications, such as through-wall target tracking, air vehicle tracking, and ship tracking in a harbor.

. 4 .
are three bistatic range measurements in three receiving channels at slow time k that satisfy the inequality (9), calculate the position upon the measurements, add the position to set P k , and delete the measurements in R k s Perform step 5 if j = m k -1 or at least one set among R k s

Fig. 6 .
Fig. 6.Tracking results.(a) MMSEP localization with leading edge TOA estimation.(b) MMSEP localization with geometrical center TOA estimation.(c) GNN-MMSEP localization with geometrical center TOA estimation (red lines are the ground truth, green circles are the localization results, and blue lines are the tracking results).
Figures 6(a) and 6(b) are the tracking results of MMSEP localization using the leading edge and geometrical center TOA estimation methods.The red lines represent the ground truth of the moving people (Person A walks from P1 to P2; Person B walks from P3 to P4), the green circles are the localization results of the MMSEP algorithm, and the blue lines are the tracking results of the MTT system.The discontinuities of the trajectories in Fig.5are due to the losses of range measurements in the case of target trajectory crossing.As the leading edge TOA estimation loses more measurements, the discontinuities are more serious than in the geometrical center method.The tracking error of the geometrical center TOA estimation are smaller shown in Fig.6(b), which further demonstrates the superiority of the geometrical center method.
of continuous TOA intervals.
Tab. 3. Mean tracking error between tracking results and the ground truth.