Leveraging neural network uncertainty in adaptive unscented Kalman Filter for spacecraft pose estimation

This paper introduces an adaptive Convolutional Neural Network (CNN)-based Unscented Kalman Filter for the pose estimation of uncooperative spacecraft. The validation is carried out at Stanford’s robotic Testbed for Rendezvous and Optical Navigation on the Satellite Hardware-In-the-loop Rendezvous Trajectories (SHIRT) dataset, which simulates vision-based rendezvous trajectories of a ser-vicer spacecraft to PRISMA’s Tango spacecraft. The proposed navigation system is stress-tested on synthetic as well as realistic lab imagery by simulating space-like illumination conditions on-ground. The validation is performed at diﬀerent levels of the navigation system by ﬁrst training and testing the adopted CNN on SPEED+, Stanford’s spacecraft pose estimation dataset with speciﬁc emphasis on domain shift between a synthetic domain and an Hardware-In-the-Loop domain. A novel data augmentation scheme based on light randomization is proposed to improve the CNN robustness under adverse viewing conditions, reaching centimeter-level and 10 degree-level pose errors in 80% of the SPEED+ lab images. Next, the entire navigation system is tested on the SHIRT dataset. Results indicate that the inclusion of a new scheme to adaptively scale the heatmaps-based measurement error covariance based on ﬁlter innovations improves ﬁlter robustness by returning centimeter-level position errors and moderate attitude accuracies, suggesting that a proper representation of the measurements uncertainty combined with an adaptive measurement error covariance is key in improving the navigation robustness.


Introduction
Nowadays, more than 30,000 monitored pieces of debris are orbiting the Earth, including non-functional spacecraft, abandoned launch vehicle stages, and fragmentation debris 1 .Altogether, this debris pose a threat to commercial satellites as well as human spaceflight, undermining safety and operations in orbit.At the same time, autonomous refueling and repairing is regarded as the most viable solution to extend the lifetime of active satellites in orbit.In this context, advancements in the field of autonomous navigation were made in the past years to enable key technologies for future In-Orbit Servicing (IOS) and Active Debris Removal (ADR) missions (Tatsch et al., 2006;Wieser et al., 2015).For such scenarios, the estimation of the pose (relative position and attitude) of an uncooperative target object by an active servicer spacecraft represents a critical navigation task.Compared to cooperative closeproximity missions, the pose estimation problem is complicated by the fact that the target object does not aid the relative navigation and may not be functional.Hence, optical sensors on the servicer spacecraft are well suited to pose estimation because they do not rely on any sensing hardware on the target, such as Global Navigation Satellite Systems (GNSS) sensors and antennas.In this context, pose estimation systems based solely on a monocular camera are recently becoming an attractive alternative to systems based on active sensors or stereo cameras, due to their reduced mass, power consumption and system complexity (Sharma et al., 2018;Pasqualetto Cassinis et al., 2019).However, a significant effort is still required to comply with most of the demanding requirements for a robust and accurate monocular-based relative navigation system.Notably, the aforementioned navigation system cannot rely on known visual markers, as they are typically not available on an uncooperative target.Since the extraction of visual features is an essential step in the pose estimation process, advanced Image Processing (IP) techniques are required to extract keypoints (or interest points), corners, and edges on the target body, such as the Speeded Up Robust Features (SURF) Bay et al. (2008), the Scale Invariant Feature Transform (SIFT) (Lowe, 2004), and the Oriented FAST, Rotated BRIEF (ORB) Rublee et al. (2011).In modelbased methods, the detected features are then matched with pre-defined features on an offline wireframe 3D model of the target to solve for the pose.This is usually achieved by solving the Perspective-n-Points (PnP) problem Sharma and D'Amico (2015) with standard pose estimation solvers such as the Efficient Perspective-n-Point (EPnP) Lepetit and Moreno-Noguer (2009), the Efficient Procrustes Perspective-n-Point (EPPnP) Ferraz et al. (2014b), or the multi-dimensional Newton Raphson Method (NRM) Ostrowsky (1966), additionally using the RANdom SAmple Consensus (RANSAC) method to improve the robustness against outliers.Essentially, a reliable detection of key features is critical to guarantee safe operations around an uncooperative target, e.g.under adverse illumination conditions.
Unfortunately, standard IP algorithms usually lack feature detection robustness when applied to space images D' Amico et al. (2014), undermining the overall navigation system and, in turn, the close-proximity operations around the uncooperative target.From a pose initialization stand-point, the extraction of target features can in fact be jeopardized by external factors, such as adverse illumination conditions, low Signal-to-Noise ratio (SNR) and Earth in the background, as well as by target-specific factors, such as the presence of complex textures and features on the target body.Moreover, most of the IP methods are based on the image gradient, detecting textured-rich features or highly visible parts of the target silhouette Pasqualetto Cassinis et al. (2019).As such, the detected features are image-specific and can vary in number and typology depending on the image histogram.This means that most of these techniques cannot accommodate an offline feature selection step, resulting in a computationally expensive image-to-model correspondence process to ensure that each detected 2D feature is matched with its 3D counterpart on the available wireframe model of the target object.
In recent years, Convolutional Neural Networks (CNNs) are emerging as a valid alternative to more traditional pose estimation systems, with two main CNNbased architectures currently standing out.Initially, endto-end architectures in which a single CNN replaced the entire pose estimation pipeline were adopted Sharma et al. (2018), Sharma and D'Amico (2020), Shi et al. (2018), Sonawani et al. (2020).However, since the pose accuracies of these systems proved to be lower than the accuracies returned by standard PnP solvers, especially in the estimation of the relative attitude Sharma et al. (2018), keypoints-based architectures stood out as the preferred option.Specifically, degree-level average orientation errors were achieved by keypoints-based methods as opposed to average orientation errors one order of magnitude larger achieved by end-to-end methods.These averages were computed across test images of the Tango spacecraft from the PRISMA mission D' Amico et al. (2013) as part of the Spacecraft Pose Estimation Dataset (SPEED) challenge Kisantal et al. (2020), Huo et al. (2020) (Sharma et al., 2019).In keypoints-based CNN systems, a CNN is used only at a feature detection level to replace standard IP algorithms, and the output features are fed to a PnP solver together with their body coordinates, which are made available through the wireframe 3D model of the target body.Due to the fact that the trainable features can be selected offline prior to the training, the matching of the extracted feature points with the features of the wireframe model can be performed without the need of a large search space for the image-model correspondences, which usually characterizes most of the edges/ corners-based methods (D'Amico et al., 2014).However, due to a lack of availability of representative space images, these CNN systems often need to be trained with synthetic renderings of the available target model.As a result, their feature detection robustness on more realistic images is usually unknown and difficult to predict.Therefore, the synthetic datasets used to train the CNNs tend to fail in representing the textures of the target as well as the external illuminations, resulting in inaccurate detections and low pose estimation accuracies when tested on images from different sources, such as spaceborne images (Kisantal et al., 2020;Pasqualetto Cassinis et al., 2022).In this context, three desirable aspects stand out.First of all, novel techniques shall be investigated to improve the performance of CNN-based pose estimation systems trained using synthetic images on actual space imagery.This aspect is referred to as the domain shift problem (Tobin et al., 2017;Ben-David et al., 2007;Sugiyama and Muller, 2005).Second, the interface between the CNN and the navigation filter shall be explored to assess the performance of a CNN-based navigation filter on representative relative trajectories.Third, a proper validation framework shall be sought to test the robustness of the proposed CNNbased system against representative images of the target spacecraft, generated in a laboratory environment which simulates space-like conditions.Notably, a calibration framework shall be established which returns an accurate reference for the pose between the monocular camera and the target mockup for each generated image (pose labels), in order to be able to quantify the CNN performance at both keypoints detection and pose estimation levels.
The domain shift problem in CNNs has been extensively investigated in recent years to leverage the domain gap from synthetic training to real test imagery, either via data augmentation (Tobin et al., 2017;Jackson et al., 2018;Geirhos et al., 2019) or via domain adaptation (Donahue et al., 2017(Donahue et al., , 2016)).In relation to the latter, domain adaptation techniques were leveraged by the winners of the recent Spacecraft Pose Estimation Challenge (SPEC) to tackle the domain gap challenges in the Next Generation Satellite Pose Estimation Dataset (SPEED+) (Park et al., 2023).Although domain adaptation techniques are often effective and can produce impressive results by adapting the CNN on a specific target domain post training, they require the target domain images and synthetic training images simultaneously to perform adaptation, and hence they are not domain-agnostic.On the other hand, data augmentation techniques consist of introducing variations in the synthetic training domain without any a priori knowledge of the target domain.In essence, the idea is to extend the standard data augmentation effects, such as random cropping, zooming, rotation, flipping etc. with texture and complex illumination variations.By doing that, Tobin et al. (2017) showed that a CNN can generalize from synthetic environments to new domains by using an unrealistic but diverse set of random textures.Following this line of reasoning, Jackson et al. (2018) and Geirhos et al. (2019) further discovered that by randomizing textures during training, CNNs can learn the shape of objects rather than textures, improving their robustness to domain shift.By extending the above methods from terrestrial applications to space scenarios, some of the authors (Pasqualetto Cassinis et al., 2022;Park et al., 2021a;Park et al., 2019) already highlighted that augmenting the CNN training with texture randomization can lead to a considerable improvement in the overall pose estimation performance under large domain gaps in the target's texture.However, preliminary results showed that the CNN performance decreases under highly adverse illumination conditions, suggesting that additional augmentation techniques are required to tackle the domain shift from an illumination standpoint (Pasqualetto Cassinis et al., 2022).
Besides domain shift, additional challenges arise in relation to the applicability of CNNs for relative navigation in space.Notably, both pose initialization and pose tracking are not well suited to produce pose estimates at high frequencies, especially due to the computationally expensive IP in combination with pose estimation.Furthermore, solving for the pose solely from CNN detections can only provide a prediction from actual sensor data without accounting for any modeling of the external environment.Therefore, it is not guaranteed that the estimation can reliably deal with unwanted components in the measurements, i.e. image noise or adverse illumination conditions.Finally, quantities such as the translational and rotational velocities of the target spacecraft with respect to the servicer spacecraft can hardly be estimated together with the pose.Therefore, filtering techniques are usually used in combination with the camera measurements and the actual pose estimate in order to return full-state (pose and relative velocities) solutions at high frequency (Sharma and D'Amico, 2017).Many navigation filters for close-proximity operations were investigated in recent years in the context of pose estimation.The reader is referred to Pasqualetto Cassinis et al. ( 2019) for a comprehensive overview.However, limited focus has been given to the interface between a CNN-based system and a navigation filter.Earlier works by some of the authors (Pasqualetto Cassinis et al., 2021) suggested that a tightly-coupled approach should be more suited than a loosely-coupled one, if a CNN is used to extract features from the monocular images.As opposed to a loosely-coupled approach in which the detected features are transformed into an estimated pose prior to the filter, a tightly-coupled approach directly feeds the detected target features into the filter.However, limited research was provided on how to model the features detection uncertainty into a representative measurement error covariance.Other authors proposed a method based on projection vector, in which the covariance is associated to the image gradient magnitude and direction at each feature location (Cui et al., 2019), or a method in which covariance information is derived for each feature based on feature's visibility and robustness against illumination changes (Harvard et al., 2020).However, in all these methods the derivation of features covariance matrices is a lengthy process which generally cannot be directly related to the actual detection uncertainty.This is because it may not be possible to associate the gradient information to the image processing algorithm.Moreover, this procedure could not be easily applied if CNNs are used in the feature detection step, due to the difficulty to associate statistical meaning to the IP tasks performed within the network.In this context, a novel method was recently presented by some of the authors in Pasqualetto Cassinis et al. ( 2021), in which the output of the CNNs is directly exploited to return relevant statistical information about the detection step.This method consists in creating a statistical distribution from the CNN heatmaps, and to derive a representative covariance matrix from each feature based on the heatmap's dispersion.However, despite the promising preliminary finding this heatmaps-based method has not been extensively tested on realistic space imagery.Specifically, it is still unclear whether the magnitude of such a covariance can effectively represent the feature detection uncertainty.
In the context of on-ground validation, several laboratory testbeds exist to generate Hardware-In-theLoop (HIL) images of a target spacecraft's mockup with a monocular camera (Wilde et al., 2019), e.g. the robotic Testbed for Rendezvous and Optical Navigation (TRON) at Stanford University (Kisantal et al., 2020), the GNC Rendezvous, Approach and Landing Simulator (GRALS) at the European Space Research and Technology Centre (ESTEC) (Zwick et al., 2018), the European Proximity Operations Simulator (EPOS) at the German Aerospace Agency (DLR) (Kru ¨ger and Theil, 2010), and the Platform-art facility at GMV (Dubanchet et al., 2020).The TRON testbed was the first of its kind to provide the generation of large-scale pose estimation datasets of a target mockup satellite under multiple camera-target geometries and illumination conditions.The resulting Spacecraft Pose Estimation Dataset (SPEED) and its extension SPEED+ guarantee a reliable validation of CNN-based pose estimation systems on static images with a considerable domain gap from their synthetic counterpart.However, the validation of a CNN-based navigation system comes with additional challenges that stem from the fact that it is hard to recreate representative relative trajectories and realistic illumination conditions on-ground.Despite the recent advancements in the on-ground generation of realistic space imagery which led to the creation of the HIL images of the SPEED+ dataset (Park et al., 2021a) (Park et al., 2021), the recreation of rendezvous trajectories on-ground remains a demanding task that has not been fully explored in most of the mentioned testbeds.The recreation of realistic rendezvous trajectories on-ground has been showcased by some of the authors at ESTEC's GRALS facility (Pasqualetto Cassinis et al., 2022).In their setup, a v-bar trajectory around a mockup of the Envisat spacecraft was recreated by controlling both the target rotation and the translation of the monocular camera through two robotic arms, thus extending the testbed capability from the generation of static datasets to actual relative trajectories.However, only a static sun lamp was used to illuminate the target, leading to simplified illumination conditions.Furthermore, the calibration of the overall system was complicated by an inaccurate calibration of the target mockup, leading to lower accuracies in the reference pose labels compared to the TRON setup.In a recent effort to solve the above issues, the capabilities of the TRON facility were extended towards the generation of the Satellite Hardware-In-the-loop Rendezvous Trajectories (SHIRT) dataset (Park and D'Amico, 2022), a dataset of synthetic and realistic imagery of two close-proximity trajectories around the Tango spacecraft with high-fidelity calibration and more realistic illumination conditions of a typical LEO orbit (Park and D'Amico, 2022).
Building on the previous findings of some of the authors (Pasqualetto Cassinis et al., 2022;Park et al., 2021a;Pasqualetto Cassinis et al., 2021), this paper aims at solving the above-mentioned challenges in a sequential fashion.First, a data augmentation pipeline centered on light augmentation is introduced to solve the domain shift problem.An existing technique, proposed by Sakkos et al. (2019) for terrestrial applications, is exploited to generalize the illumination conditions during the CNN training.The performance of the CNN-based pose estimation system is then evaluated on realistic imagery of the SPEED+ dataset at a pose estimation level.Next, the CNN system is combined with an Unscented Kalman Filter (UKF) to address the performance of the proposed system at a navigation filter level.To this end, the SHIRT dataset is used in the evaluation in order to compare the filter performance in synthetic scenarios with the performance on realistic lab imagery.To cope with the challenges in the representation of the measurements uncertainty, an adaptive scheme is proposed in which the measurement error covariance is estimated online by scaling the heatmaps-based representation based on the filter innovations.
In summary, the main contributions of this work are: 1. To propose a novel data augmentation scheme based on light randomization in order to improve the CNN performance on realistic space imagery 2. To propose a new covariance adaptation method based on CNN heatmaps to capture measurements uncertainty within the navigation filter 3. To validate a tightly-coupled, CNN-based UKF on realistic trajectories generated on-ground using the TRON facility.
The paper is organized as follows.Section 2 introduces the proposed navigation framework.The TRON testbed and the image acquisition procedure are described in Section 3. In Section 4, the CNN training, validation and testing phases are detailed, with special focus to the data augmentation pipeline.Section 5 presents the preliminary pose estimation results on SPEED+ test datasets.Next, the proposed navigation filter is described in Section 6 together with the novel adaptive scheme for the measurement error covariance.Sections 7,2,3,4,5,6,7,8 describe the simulation environment and show the navigation results on the two recreated close-proximity trajectories.Finally, Section 9 provides the main conclusions and recommendations.

System framework
From a high-level perspective, a model-based monocular pose estimation system receives as input a 2D image and matches it with an existing wireframe 3D model of the target spacecraft to estimate the target pose with respect to the servicer camera.As illustrated in Fig. 1a, the pose estimation problem consists in determining the position of the target's centre of mass t C and its attitude with respect to the camera frame C, represented by the inverse of the rotation matrix R C B .The Perspective-n-Points (PnP) equations, relate the unknown pose with a feature point p in the image plane via the relative position r C of the feature with respect to the camera frame.Here, r B is the point location in the 3D model, expressed in the body-frame coordinate system B, whereas f x and f y denote the focal lengths of the camera and C x ; C y À ) is the principal point of the image.From these equations, it can already be seen that an important aspect of estimating the pose resides in the capability of the IP system to extract features p from a 2D image of the target spacecraft, which in turn need to be matched with pre-selected features r B in the wireframe 3D model.Notably, a wireframe model of the target needs to be made available prior to the estimation.Notice also that the problem is not well defined for n < 3 feature points, and can have up to four positive solutions for n ¼ 3 (Fischer and Bolles, 1981).Generally, more features are required in presence of large noise or symmetric objects.
The validation pipeline of the proposed navigation system is shown in Fig. 2 and consists of the following main stages: 1. Calibration Procedure and Image Acquisition: The adopted monocular camera is calibrated with respect to a scaled 1:2 mockup model of the Tango spacecraft via a dedicated procedure, in order to associate accurate reference labels of the pose between the camera and the mockup.Once the setup is calibrated, laboratory images of the target mockup are generated by the camera and associated to their corresponding pose labels.Trajectory design and execution is carried out in order to create a representative laboratory database (SPEED+) as well as to perform rendezvous trajectories around the mockup (SHIRT).The calibration procedure and the trajectory generation pipeline are taken from  additionally estimate the relative translational and rotational velocities.4. Post-Processing and Validation: The results of the proposed CNN-based system on the HIL images are validated against the reference pose labels, derived from the calibration setup.This is performed at both pose estimation and navigation levels.

Relative navigation
This work considers a servicer spacecraft flying relative to a target spacecraft, with the relative motion being described in a Local Vertical Local Horizontal (LVLH) reference frame co-moving with the servicer (Fig. 1b).Furthermore, it is assumed that the servicer is equipped with a single monocular camera.The relative attitude of the target with respect to the servicer can then be defined as the rotation of the target body-fixed frame B with respect to the servicer camera frame C, where these frames are fixed to each spacecraft's body.The vector from the origin of the camera frame to the origin of the target frame defines their relative position.Together, these two quantities characterize the pose.This information can then be transformed from the camera frame to the servicer's center of mass by accounting for the pose of the camera with respect to the LVLH frame.Beside the Cartesian representation of the relative motion between the servicer and target spacecraft, the relative state can also be parametrized as a function of the absolute orbital elements of the two spacecraft.This paper uses the Relative Orbital Elements (ROE) state introduced by D'Amico (2010), which are defined in terms of the classical Keplerian orbital elements as: where the subscripts t and s indicate the target and servicer spacecraft respectively, and s i and c w represent the sine and cosine of the argument of perigee x and inclination i, respectively.Notice that this set of ROEs is nonsingular for non equatorial orbits of arbitrary eccentricity, but is singular for equatorial orbits.

TRON testbed
The TRON facility at SLAB, visualized in Fig. 3, includes a control room and an 8 Â 3 Â 3 m simulation room which consists of various components and machineries to 1) simulate the vision-based rendezvous trajectory of a servicer spacecraft with a camera to a target spacecraft, and 2) emulate the high-fidelity spaceborne illumination conditions to maximize the realism of the images captured by the camera.TRON comprises two 6 degrees-of-freedom KUKA robot arms and a set of Vicon motion track cameras to reconfigure an arbitrary pose between a camera and a target mockup model, as well as multiple Earth albedo light boxes and a sun lamp to simulate highfidelity spaceborne illumination conditions (Park et al., 2021b).The calibration of the facility is performed via a dedicated multi-source Robot/World Hand/Eye (RWHE) calibration procedure which fuses readings from KUKA and Vicon to accurately estimate the pose of the adopted monocular camera with respect to the target mockup.Millimeter-level position accuracy and millidegree-level orientation accuracy were obtained on a subset of closerange images of the Tango mockup.The reader is referred to Park et al. (2021b) for a detailed description.

SPEED+ dataset
The SPEED+ dataset was created in the TRON facility to be used in the ongoing international Satellite Pose Estimation Challenge, with the main objective to evaluate and compare the robustness of machine learning models trained on synthetic images (Park et al., 2021a).The dataset is built upon the existing SPEED dataset (Kisantal et al., 2020) by increasing the number of synthetic and HIL images, whilst extending the illumination conditions simulated in the facility and improving the accuracy of the pose labels.SPEED+ consists of synthetic images generated in OpenGL and realistic images generated in TRON with light boxes and sun lamp (Fig. 4).Fig. 5 illustrates sample SPEED+ images for the synthetic, lightbox, and sunlamp domains.Overall, the test images of SPEED+ extend the full orientation space and distance up to 10 m with realistic re-creation of Earth albedo and direct sunlight present in spaceborne imagery.

SHIRT dataset
A detailed overview of the generation of the SHIRT dataset can be found in Ref. Park and D'Amico (2022).For the HIL images, the first step is to design a rendezvous trajectory which resembles a typical close-proximity scenario.Next, the motion of the two KUKA robots is commanded in order to recreate the desired trajectory, taking into account the scale of the target mockup and the constraints of the facility.In order to simulate representative illumination conditions, the true location of the Sun is used to intermittently switch the light boxes on/off and capture the correct inclination of the Sun with respect to the mockup.Fig. 6 shows a comparison of synthetic and HIL images for the same poses, where the synthetic images were generated for the same illumination and trajectory inputs.As can be seen, the TRON illumination manages to capture the correct inclination of the Sun used in the synthetic renderings.Besides, a clear domain gap is present between synthetic and HIL images, ensuring that the validation of the proposed CNN-based system can be carried out in challenging domain shifts.Specifically, by comparing the first (a-b) and second (c-d) rows of Fig. 6, it can be seen that domain gaps can result in either a difference in the illuminated side of the target (a-b) or in a different contrast between the illuminated and shadowed parts of the target (c-d).

Convolutional neural network
The main reason for an emerging interest in CNNs for feature extraction lies in the capability of the convolutional layers to extract high-level features of objects with improved robustness against image noise and illumination conditions as compared to standard IP algorithms (Pasqualetto Cassinis et al., 2021).As shown in Fig. 7, the first essential step of keypoints-based CNN systems is represented by an Object Detection Network (ODN), e.g.Faster R- CNN Ren et al. (2017), R-FCN Ren et al. (2016) or MobileNet Howard et al. (2017), placed before the main CNN.The ODN regresses the coordinates of a bounding box around the target object, in order to crop a Region Of Interest (ROI) and to increase robustness to scale, variation, and background textures.The cropped ROI is then fed into a Keypoint Detection Network, which convolves with the input image and outputs a set of feature maps.These so-called heatmaps are detected around preselected features on the target object, such as corners or interest points.The 2D pixel coordinates of the heatmap's peak intensity characterize the predicted feature location, with the intensity and the shape indicating the confidence of locating the corresponding keypoint at this position (Pavlakos et al., 2017).Notably, the selection of the 1.Gaussian, shot, impulse and speckle noise.2. Gaussian, defocus, motion and zoom blurs.3. Spatter, color jitter and random erase.
Finally, light augmentation is introduced to generalize the illumination conditions during training.The proposed approach uses the data augmentation method introduced    by Sakkos et al. (2019) for terrestrial applications and extends it to the SPEED+ dataset.Fig. 8 illustrates the light augmentation pipeline for a sample image I of the SPEED+ dataset.First, a ROI is extracted around the target spacecraft and used to create an illumination mask M at random locations around the target.The mask consists of an illumination circle with centre where H ; W are the height and width of the extracted ROI and k 2 1 5 ; 1 2 À Á .Since uniformly modifying all pixels within the illumination circle generate unrealistic results, the Euclidean Distance Transform (EDT) is further applied to the mask in order to model light attenuation (Sakkos et al., 2019).For each pixel in the circle, the EDT assigns a number that is the distance between that pixel and the nearest nonzero pixel.
Once the mask is created, the original image is augmented to create three distinct effects: where I Aug is the augmented image, M is a mask of the same size as the input image I, and z 1 ; z 2 are random integers.Fig. 10 illustrates the three filters applied to sample image I. Notably, several illumination effects can be introduced in the datasets by interchanging these filters whilst varying the disc diameter and location.In the current implementation, 50% of the synthetic images are augmented with light augmentation, of which 50% have a dark filter, 25% a bright filter, and 25% local and global filters.

Train, validation and test
Table 1 lists the number of images used in the Train, Validation, and Test datasets together with the data augmentation breakdown.The ideal synthetic images of the SPEED+ dataset are already split into 80:20 train/validation sets.The augmentation pipeline described in Section 4.1 is used to extend both sets.During training, the validation dataset is used beside the training dataset to compute the validation losses and avoid overfitting.The Adam optimizer (Kingma and Ba, 2015) is used with a cosine decaying learning rate with initial value of 10 À3 and decaying factor of 0.1.Finally, the CNN is tested on the lightbox and sunlamp sets.In this way, the performance of a CNN trained solely on synthetic imagery can be assessed on the realistic imagery simulated in TRON.

Pose estimation
Following the promising pose estimation results achieved in ADR scenarios in recent studies (Kisantal et al., 2020;Chen et al., 2019;Sharma and D'Amico, 2017;Sharma and D'Amico, 2015), the Efficient Perspective-n-Points (EPnP) method followed by Gauss-Newton refinement (Lepetit and Moreno-Noguer, 2009) is selected to estimate the pose from a set of detected features.This method solves the PnP problem in Eqs.(1, 2) in closed-form with the EPnP algorithm, and uses the estimated pose as an initial guess for an iterative pose refinement.Notably, the CNN-based extraction of pre-defined features allows a direct match with the wireframe model, without the need of a large search-space for the 2D/3D correspondences which characterizes the more standard feature extractors described in Section 1.

Error metric
To evaluate the performance of the pose estimation system, two separate error metrics are generally adopted, in accordance with Kisantal et al.Kisantal et al. (2020).Firstly, the translational error E T between the estimated relative position tC and the ground truth t C is computed as Secondly, the attitude error E R is measured in terms of the Euler axis-angle error between the estimated quaternion q and the ground truth q, where denotes the quaternion multiplication and the subscripts s; v refer to the scalar an vectorial part of the error quaternion b.Furthermore, a combined score is created by combining both position and attitude errors, Note that when evaluated on SPEED+ lightbox and sunlamp samples, the modified SPEED score E Ã pose (Park et al., 2021a) is used to zero out the errors smaller than the thresholds based on the TRON calibration, i.e., for individual sample,

Pose estimation results
The pose estimation results of the CNN-based system in Fig. 7 are presented for the SPEED+ test dataset, in order to evaluate the capability of the CNN to bridge the HIL imagery.Additionally, the proposed pose estimation system is evaluated on the PRISMA25 dataset, which consists of 25 flight images of the Tango spacecraft from the PRISMA mission (D'Amico et al., 2013).Despite the limited number of images in PRISMA25, the comparative study between SPEED+ and actual flight images allows an assessment of the scalability of the proposed data augmentation method to different target domains, whilst providing insight to the applicability of HIL images as a surrogate of flight images for validation.

SPEED+
Figs. 11, 12 show the pose estimation results in terms of the Cumulative Distribution Function (CDF) across the test datasets.Referring to Section 4.1, the results are reported for an augmentation-free CNN training, a training with the noise pipeline, and a training with both noise and light augmentation.This is done in order to assess the impact of each augmentation on the CNN performance.As can be seen, the introduction of light augmentation into the CNN training greatly improves the overall pose estimation performance on both the lightbox and the sunlamp subsets.Specifically, pose errors E T < 0:1 t C k k; E R < 10 and E T < 0:1 t C k k; E R < 13 are achieved in 80% of the lightbox and sunlamp subsets, respectively (Figs. 11,12).Fig. 13 illustrates highly accurate pose estimation results on a representative subset of HIL images with lightbox illuminations, Earth in the background, and sunlamp illuminations.Conversely, Fig. 14 illustrates two representative scenarios characterized by large pose errors.As can be seen, near-eclipse illumination conditions and highly adverse sunlamp reflections can still jeopardize the CNN performance, despite the adopted data augmentation pipeline.Notably, similar effects can be observed in some of the highly accurate pose estimates (Fig. 13), suggesting that the CNN performance could be affected by small visual artifacts not visible by the human-eye.

PRISMA25
Fig. 15 shows the CDF pose estimation results for the PRISMA25 dataset in terms of the total SPEED score in Eqn. 9.The results are compared with the lightbox and sunlamp scenarios in order to assess the scalability of the proposed light augmentation on flight images.Notably,  60% of the images are characterized by highly accurate poses with mean errors E T ¼ 0:5 m, E R ¼ 2:5 , proving the effectiveness of the proposed augmentation pipeline.Notice also that the CDF of PRISMA25 is bounded by the lightbox and sunlamp CDFs for most of the scores, suggesting that the TRON illuminations represent realistic best/worst case bounds for illuminations in actual space imagery.
Besides, Table 2 lists the mean results associated to the CDF in Fig. 15, whilst comparing the proposed light augmentation method with another augmentation pipeline which exploits texture randomization (Jackson et al., 2018).As can be seen, light augmentation provides a considerable improvement over texture randomization.

Navigation filter
In earlier works (Pasqualetto Cassinis et al., 2021;Sharma and D'Amico, 2017), the so-called Multiplicative Extended Kalman Filter (MEKF) was used to cope with the challenges in the attitude parametrization, which either suffers from the singularity (e.g.Euler angles) or normalization (e.g.quaternion).The latter can happen because a quaternion representation may cause the associated filter error covariance to be singular, since the quaternion set consists of four parameters to describe the 3DOF attitude.The MEKF, introduced for the first time by Lefferts et al. (1982), aims at solving the above issue by using two different parametrizations of the relative attitude.A three element error parametrization, expressed in terms of quaternions, is propagated and corrected inside the filter to return an estimate of the attitude error.At each estimation step, this error estimate is used to update a reference quaternion and is reset to zero for the next iteration.Notably, the reset step prevents the attitude error parametrization from reaching singularities, which generally occur for large angles.However, despite its promising results on ideal rendezvous scenarios with a high measurement frequency and synthetic images as measurements, the MEKF could considerably decrease its performance under low measurement frequencies and realistic space imagery, which is the case for the scenarios investigated in this paper.
In the proposed navigation system, a UKF Julier and Uhlmann ( 2004) is used instead, in order to capture the system nonlinearity via the unscented transform, which uses a finite set of deterministic samples instead of linearizing the nonlinearity itself.Specifically, the Unscented Quaternion Estimator (USQUE) introduced by Crassidis and Markley ( 2012) is adopted.The USQUE has proven more robust under low measurement acquisition frequency when compared to a more standard EKF/MEKF.Furthermore, the expected error is generally lower than the EKF, and the unscented transform avoids the derivation of Jacobian matrices (Crassidis and Markley, 2012).Similar to the MEKF, a three-component attitude-error vector is used to represent the quaternion error vector.
In a standard EKF, the state vector for pose estimation based on ROE is a 13-dimension vector composed of the relative ROE as well as the relative quaternion and rotational velocity, where a s represents the semi-major axis of the servicer spacecraft, q C B ¼ q 0 q v ½ is the quaternion set that represents the relative attitude, and x C B=C is the angular velocity of the target with respect to the camera, expressed in the camera frame.In the USQUE, the modified state vector propagated inside the filter becomes a 12-dimension vector, where dp is four times the Modified Rodrigues Parameters (MRP) r,  A high-level description of the proposed navigation pipeline is provided in Fig. 16.Each subsystem is described in the following sections.

Measurement error covariance computation
Compared to the methods discussed in Section 1 (Cui et al., 2019;Harvard et al., 2020;Ferraz et al., 2014a), the proposed method derives a measurement error covariance matrix associated to each feature directly from the heatmaps detected by the CNN, rather than from the computation of the image gradient around each feature.In this method, the ith non-zero pixel around the heatmap's peak is extracted to derive a covariance matrix C i , Here, w i is a normalized weight based on the gray intensity I i at each pixel location, n is the number of pixels in each feature's heatmap, x; y are the coordinates of each heatmap's pixel, and p ¼ p x ; p y À Á is the heatmap's peak location.Note that an assumption is made that the spread around the peak can represent the feature covariance, although the spread of the heat around the peak would give the expected value of the square of the map instead.Fig. 17 shows the overall flow to obtain the covariance matrix from the CNN heatmaps.The reader is referred to Pasqualetto Cassinis et al. ( 2021) for an exhaustive description of each step of the method.

Prediction
The first step of the filter is to generate sigma points v i ½ from the current state vector xk by using the standard formulation of the UKF Julier and Uhlmann ( 2004): where N ¼ 12 is the dimensionality of the system and k is a scaling factor tuned offline.The sigma points are then propagated through the dynamic equations.For the translational motion, the ROE state is propagated assuming an unperturbed Keplerian orbit, Fig. 16.High-level description of the UKF flow together with the measurements processing and adaptive covariance.For the attitude, each sigma sample dp i ½ is transformed to the error-quaternion dq i ½ , which is used to generate the quaternion multiplicatively via q where denotes the quaternion multiplication.The dynamics update of the quaternion representation of relative orientation assumes perturbation-free motion, where Dt is the time-step between filter calls and x is the normalized angular velocity, representing the direction of the target rotation axis relative to the servicer.Afterwards, q The mean of dp À k provided by the unscented transform is used to update the nominal state quaternion.
The relative angular velocity x C B=C is propagated via the following equation (Capuano et al., 2020): where J is the target's inertia matrix, s are the external torques, and E denotes the Earth-Centered-Inertial (ECI) frame.Note that in the current implementation no control torques are modeled for neither the target nor the servicer, i.e. s B ¼ s C ¼ 0. After propagation, the sigma points are used to derive a mean state estimate and its associated error covariance, where the weights w i ½ are a function of k; N and Q k is the process noise covariance.

Correction
The measurement update follows the projections described in Eqs.(1, 2), in which the sigma points of the relative position t C are derived from the ROE state given the knowledge of the servicer's orbital elements state and its attitude with respect to the ECI frame.Once the expected measurements Z i ½ given each sigma point are computed, the mean expected measurement and innovation covariance can be computed similar to the mean and covariance of the filter state, where R k represents the measurement error covariance matrix.In the proposed system, R k is a time-varying block diagonal matrix constructed with the heatmaps-derived covariances C i in Eq. ( 13), where the scaling coefficient c k is described in Section 6.4.Notice that C i can differ for each feature in a given frame as well as vary over time.Preliminary navigation results (Pasqualetto Cassinis et al., 2021;Pasqualetto Cassinis et al., 2020) already showed that such heatmaps-derived covariance matrix can capture the statistical distribution of the measured features and improve the measurement update step of the navigation filter.At this stage, outliers are removed from the mean measurements by computing the Mahalanobis Distance M i between the ith feature and its corresponding filter innovation term D z k , where D z k ¼ z À h x k ð Þ and h is the nonlinear transformation in Eqs.(1, 2).The Mahalanobis distance is a measure of the distance between a point and a distribution.In this case, the point is the keypoint detected by the CNN in the image and the distribution is the reprojected feature and its associated covariance.The threshold M t to select outliers is determined by where p m is the desired probability that a measurement would result in M i P M t , given that the correspondence between the detected and reprojected features is correct (Thrun et al., 2005).In other words, if M i P M t it is highly unlikely that the ith keypoint correlates to its reprojection as predicted by the filter, and the feature can be rejected.In this way, filter robustness can be improved during low visibility periods of the target spacecraft in which wrong CNN detections may occur.Notably, this feature rejection scheme reflects an important advantage of incorporating a navigation filter compared to relying solely on the CNN detection and PnP solver solution.
Finally, the corrected state estimate xk is obtained from the propagated state xk , the innovation D z k , and the Kalman Gain K k , where K k is a function of the state error and innovation covariance, and K k D z k represents the state correction.After correction, the new attitude error dp is reset to zero at each iteration.

Covariance adaptation
The heatmaps-based covariance described in Section 6.1 already proved to accurately represent the measurements uncertainty of the CNN detections in synthetic images with ideal illumination conditions (Pasqualetto Cassinis et al., 2021).However, a reliable representativeness of the heatmaps cannot always be guaranteed in realistic space images under challenging illumination conditions.Despite capturing the shape of the distribution, the magnitude of the heatmpas-derived covariance could indeed fail at representing the actual detection uncertainty of the CNN.This could lead the navigation filter to trust inaccurate features and ultimately diverge.
Referring to Fig. 16, the above challenges are addressed by adaptively estimating the measurement error covariance through a new technique which leverages the heatmapsbased covariance together with existing covariance matching techniques.This approach is based on the adaptive state noise compensation method for estimating the process noise covariance introduced in Stacey and D'Amico (2021).
By replacing the theoretical covariance of the innovations, also known as pre-fit residuals, in a Kalman filter with an empirical estimate, the measurement error covariance of the ith pair of pixel measurements at time step k can be estimated as Myers (1974) Ri Here, the innovation D z i;k is the difference between the true and expected ith pair of pixel measurement at time step k, taking into account all measurements through time step k À 1.The matrix S k ¼ S k À R k is computed from the sigma points passed through the nonlinear measurement models in Eq. ( 22) (Thrun et al., 2005).The portion of S k corresponding to the ith pair of pixel measurements is S i;k 2 R 2Â2 .Eq. ( 27) assumes a filter at steady state and that the pixel measurement errors are zero-mean.Typically, Eq. ( 27) is averaged over some finite length sliding window of filter output.Such covariance matching techniques, also referred to as innovation based estimation, are not guaranteed to converge to the true estimate of the error covariance.However, they are widely used and have been shown to work well in practice (Myers and Tapley, 1976;Sullivan and D'Amico, 2017;Karlgaard, 2010;Fraser and Ulrich, 2021;Mohamed and Schwarz, 1999).
Assuming that the pixel measurement errors of different features are not correlated with each other and that the associated heatmaps are representative of the shape of their covariance yields the model Here, c k is introduced to account for the uncertainty in the magnitude of the covariance.Using all the pixel measurements at a single filter call, a pseudo-coefficient c Ã k can be estimated through a weighted least squares fit between the diagonal elements of the right hand side of Eq. ( 28) and the corresponding elements of Eq. ( 27).The resulting solution for c Ã k is The vector b k is the concatenation of all the main diagonal elements of the covariance matching estimates Ri;k 2 R 2Â2 for all i at time step k.The vector A k contains the corresponding diagonal elements of each C i;k .The weighting matrix W k is chosen as the theoretical covariance of b k such that covariance matching estimates with less uncertainty have a greater influence on the solution of c Ã k .The covariance between any two elements of The scalar b R a i;k is either the first or second element of the main diagonal of Ri;k as denoted by a. Eq. ( 33) is just an element of S k squared multiplied by two.Thus, where the factor of 2 from Eq. ( 33) has been dropped because it does not change the result of Eq. (30).The Hadamard power, , denotes an element-wise power.Eq. (34) assumes that b k is ordered in the same way as the pixel measurements are ordered in the measurement vector provided to the filter.For computational efficiency, the weighting matrix is approximated as diagonal by setting all offdiagonal elements to zero.As a result, the inverse of the weighting matrix required in Eq. ( 30) can be computed element-wise along the main diagonal.
The variance of a weighted least squares estimate (Gill and Montenbruck, 2012) is The final estimate of c k is obtained by combining the c Ã estimates over a sliding window of length N through where Upper and lower inequality constraints are easily added to Eq. ( 37) by setting the estimated c k equal to any constraint that is violated.Note that some lower bound greater than zero should always be included to guarantee that the measurement error covariance is positive definite.Any filter calls where no pixel measurements are provided to the filter are excluded from the sliding window.The weighting matrix in Eq. ( 40) is assumed diagonal because the innovations are not correlated in time for a filter at steady state (Mehra, 1972;Kailath, 1968).After each measurement update, c Ã k ; P c Ã k , and c k are computed through Eqs. ( 30), (36), and ( 38) respectively.The resulting c k is used in the following measurement update to compute the measurement error covariance of each pixel measurement through Eq. ( 28).

Simulations
The investigated rendezvous scenarios of the SHIRT dataset involve the Mango and Tango spacecraft from the PRISMA mission (D'Amico et al., 2013).The simulations take place in Low Earth Orbit (LEO) with initial ROEs listed in Table 3. ROE 1 is purely an along-track separation describing a standard v-bar hold point, whereas ROE 2 describes a mid-range hold point (Sharma and D'Amico, 2017), meaning that the relative trajectory slightly drifts away from the v-bar.The reference truth motion of each spacecraft is numerically propagated with 1 s interval using rigorous force models including GRACE's GGM05S geopotential of order and degree 120 (Ries et al., 2016), NRLMSISE-00 atmospheric model Picone et al. (2002), analytic lunisolar third-body gravity (Gill and Montenbruck, 2012), and solar radiation pressure (SRP) including a conical Earth shadow model.Moreover, the spacecraft attitude is perturbed via analytical gravitygradient (Wertz, 1978), air drag, and SRP effects.The satellite parameters for these forces match the modeled parameters of the Mango and Tango spacecraft specified in D'Amico (2010).The servicer's angular velocity is initialized to always align the camera boresight with the along-track direction, whereas the target's initial angular velocity about its principal axes is set to x 0 ¼ 1 0 0 ½ T =s ð Þ for ROE1 and x 0 ¼ 0 0:4 À 0:6 ½ T =s ð Þ for ROE2.The images are captured every 5 s.However, a measurement acquisition time of 30 s is used in this work, in order to represent a severely limited on-board processing power.Fig. 18 shows a montage of the HIL images for both ROE trajectories.

Results
In this section, the navigation results are presented for the ROE 1 and ROE 2 trajectories.Similar to the pose estimation error metrics introduced in Section 5.1, the norms of the estimated translational and rotational velocities are compared to their ground truth values to return the estimation error.Table 4 reports the obtained navigation results for both synthetic and HIL scenarios in terms of mean error at steady state.The mean values after two orbits are computed across a time interval of 600 s at steady state.

Synthetic scenarios
Figs. 19, 20 show the navigation performance on both relative trajectories when synthetic images are used as measurements.Results are reported for the pose estimation error along with the standard deviation derived from the state covariance matrix.In these scenarios, centimeterlevel position error and degree-level attitude error are achieved at steady state after two relative orbits.Also, notice that the state covariance increases at approximately half of each relative orbit before quickly decreasing.In these periods, some of the CNN detections become inaccurate due to the challenging near-eclipse conditions in the target's visibility.However, thanks to the adaptive heatmaps-based measurement error covariance the filter can capture the detection uncertainty and reflect it in a larger state covariance.This is a desirable behaviour as it prevents the filter from diverging due to an inaccurate representation of the measurements uncertainty.

HIL scenarios
As mentioned in Section 6.4, the magnitude of the heatmaps-based covariance is not guaranteed to reflect the actual measurements uncertainty in realistic imagery, especially under highly challenging illumination conditions.As such, the state covariance could be updated with an inaccurate measurement error covariance (Eq.( 22)), leading to the divergence of the filter immediately after initialization.This was observed for both ROE trajectories when a simple heatmaps-based covariance was used.However, the achieved performance of the filter can benefit from the proposed covariance adaptation scheme.After the estimated scaling factor is introduced (Eq.( 28)), the filter shows an increase in robustness towards inaccurate measurements and does not diverge.Figs.21, 22 show the nav-igation performance on both relative trajectories when the HIL images are used.Thanks to the covariance adaptation, centimeter-level position error is achieved after two relative orbits, which compares well with the results in the synthetic scenario.However, a larger uncertainty in the attitude estimate can be seen in both trajectories, with steady-state mean errors E T % 10 À 15 .A degraded performance in the relative attitude due to challenging HIL images was already observed by some of the authors in a similar    4 indicate that the proposed system can estimate the relative rotational velocity with relatively small errors.Note also that, due to a non-ideal tuning of the process noise covariance Q, the navigation error exceeds the 3-r bounds in some portions of the orbits.The same behaviour is observed when varying the initial conditions for both the state and the covariance.For a more optimal tuning of the navigation filter, which goes beyond the scope of this paper, the reader is referred to Park and D'Amico (2022), in which the process noise covariance is adaptively estimated based on the filter innovation.
Overall, the gained filter robustness highlights the benefits of adaptively correcting the measurement error covariance.Notably, the proposed covariance adaptation scheme was applied to a more standard, gradient-based covariance method (Cui et al., 2019) which does not exploit the CNN heatmaps, in order to compare its performance with the heatmaps-based method.Preliminary results showed a much worse navigation performance with mean attitude errors E R > 100 after one relative orbit, suggesting that the image gradient does not correlate well with the CNN uncertainties.This indicates that not only the adaptive covariance is essential for filter robustness, but also that

Conclusions
This paper introduces the validation of an adaptive Convolutional Neural Network (CNN)-based navigation filter for the pose estimation of an uncooperative spacecraft by an active servicer spacecraft equipped with a monocular camera.First, the performance of the proposed system is evaluated at a pose estimation level by testing the adopted CNN on the realistic space imagery of the SPEED+ dataset, captured from Stanford's Testbed for Rendezvous and Optical Navigation (TRON).By adopting a novel data augmentation pipeline centered on light augmentation, the system is proven capable of bridging the domain gap between the synthetic training images and the HIL test images, showcasing the benefits of generalizing the illumination conditions during training.Specifically, pose errors E T < 0:1 t C k k; E R < 10 and E T < 0:1 t C k k; E R < 13 are achieved in 80% of the lightbox and sunlamp subsets, respectively.Furthermore, by assessing the pose estimation performance on both Hardware-In-the-Loop (HIL) images and actual space imagery from the PRISMA mission, the data augmentation method is shown to be robust against changes in the target domain.
At a navigation filter level, the system is validated by assessing the performance of the proposed CNN-based Unscented Kalman Filter (UKF) on both synthetic and HIL images of two representative rendezvous trajectories around the target spacecraft.Results on the synthetic scenarios indicate that the system can accurately estimate the relative state with centimeter-and sub-degree-level pose errors, thanks to the heatmaps-based representation of the CNN detection uncertainty.In the HIL scenarios, the inclusion of an adaptation scheme for the measurement error covariance returns centimeter-level position errors and moderate attitude accuracies (E R % 10 À 15 ) at steady-state, preventing the filter from diverging during periods of low measurement accuracy.Remarkably, results suggest that a proper representation of the measurements uncertainty combined with an adaptive measurement error covariance is key in improving the navigation robustness.
Recommendations for future work are identified at different levels of the proposed navigation system.First of all, other data augmentation techniques shall be investigated together with other CNN architectures, with the aim to further improve the CNN detection accuracy prior to the navigation filter.As an example, the impact of combining texture and light augmentation on the CNN performance could be explored.At a filter level, the modeling of the process noise covariance should be improved in order to better capture the uncertainty in the state estimate.Finally, the applicability of the proposed CNN-based system shall be demonstrated on a representative space processor in order to carry out a real-time performance analysis.In this context, the interface of the proposed system with guidance and control should also be investigated in closed-loop tests.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Fig. 1 .
Fig. 1.Schematic of the pose estimation problem using a monocular image (a) and representation of the relative motion framework (b)(Curtis, 2005).

Fig. 2 .
Fig. 2. Illustration of the proposed on-ground validation of the CNN-based pose estimation system.

Fig. 7 .
Fig. 7. Proposed CNN architecture and interface with the PnP solver.

Fig. 12 .
Fig. 12. Cumulative Distribution Function of Position (a) and Attitude (b) errors over the sunlamp subset of SPEED+.

Fig. 17 .
Fig. 17.Derivation of covariance matrices from CNN heatmaps.Accurate, moderate and coarse detections are represented.The displayed ellipses are derived from the computed covariances by assuming a confidence interval 1r ¼ 0:68.

Fig. 18 .
Fig. 18.First five HIL images for ROE 1 (a) and ROE 2 (b) trajectories.The images were generated every 30 s to represent limited-on board processing power.

Fig. 19 .
Fig.19.Pose Estimation Error in the ROE 1 synthetic scenario.The ROE state is converted to position (left) and the quaternion state is converted to 3-2-1 Euler angles (right).The errors are shown along with the 3r error bound.Note that the error in the Z direction is characterized and by an initial offset in the position and a sudden jump in attitude.The position offset is a result of a relatively large error in the initial a priori knowledge of the camera boresight direction.For the attitude, this could instead be a consequence of filter contamination with wrong measurements, which is not captured by the measurement covariance matrix.

Fig. 20 .
Fig. 20.Pose Estimation Error in the ROE 2 synthetic scenario.The ROE state is converted to position (left) and quaternion state to 3-2-1 Euler angles (right).The errors are shown along with the 3r error bound.

Table 1
Description of Train, Validation and Test datasets together with data augmentation breakdown.
Fig. 11.Cumulative Distribution Function (CDF) of Position (a) and Attitude (b) errors over the lightbox subset of SPEED+.

Table 3
Initial mean servicer orbital elements and reference relative trajectories parametrized in ROE spacePark and D'Amico (2022).

Table 4
Mean navigation results after two orbits for ROE 1 and ROE 2 trajectories, computed across a time interval of 600 s at steady state.Results are reported for both synthetic and HIL scenarios.