Markerless head motion tracking and event-by-event correction in brain PET

Subjects were enrolled in institutional review board–approved studies that were also approved by the Yale Radiation Safety Committee. Subjects were also enrolled in a separate institutional review board–approved protocol for evaluation of the camera system. All subjects gave written informed consent.

All studies were performed on a Siemens Biograph mCT (Jakoby et al 2011) using both the Vicra and UMT systems. The UMT was mounted on the gantry of the scanner, and time synchronization was applied between UMT and mCT (see Supplement). In a 15 min phantom study, three Na-22 point sources (80 kBq each) and the Vicra tool were attached to a mannequin head (figures 1(a), (b)). Thirteen manual translation and rotation step motions were performed every minute, and the phantom was held static between movements. CT was used for attenuation correction (AC). The phantom was stationary during the CT acquisition and the first minute of the PET scan.

A 15 min human study mimicking the phantom study was performed with the same point sources and without tracer injection. A volunteer was instructed to perform step head motions of translation and/or rotation every minute. The volunteer was instructed to remain static for the first minute and between step motions. No CT was acquired.

As part of ongoing studies, twelve 120 min PET dynamic human studies were performed using ¹⁸F-FPEB (Wong et al 2013, Mecca et al 2021) (153 ± 24 MBq injection, metabotropic glutamate 5 receptors, N = 8) and ¹¹C-LSN3172176 (500 ± 209 MBq, muscarinic M1 receptor, N = 4) (Naganawa et al 2021). CT acquired prior to the PET was used for AC. Each participant underwent separate T1-weighted MRI.

UMT utilized a stereovision camera with infrared structured light to capture the subject's real-time 3D facial surface. The system consisted of an illumination class I laser (940 nm) and two infrared cameras which collected reflected optical signal. These data were transferred to a dedicated processor to calculate point clouds (a list of spatial coordinates) at up to 30 Hz. There were ∼20 000 points per cloud, with spatial accuracy <0.2 mm as validated by a high-resolution motion stage. The point clouds were down-sampled by averaging point location within a grid of 3 mm voxels (Steinbrucker et al 2014).

To perform motion estimation, each point cloud (figure 1(c)) was registered to a reference point cloud collected at the beginning of each scan, using a rigid-body ICP registration algorithm (Besl and Mckay 1992). The average calculation time for a 120 min human protocol was approximately 30 min. This algorithm found a 6 degree-of-freedom solution to minimize the sum of the squared distance between points in each point cloud pair (Chen and Medioni 1992). A bounding box was manually selected and applied on the first frame, and the reference cloud was segmented within the bounding box to include the upper face/cheek area and exclude all non-face areas. During motion tracking, the ICP algorithm was only applied within the bounding box. If the rotation or translation output from the ICP registration exceeded empirically set thresholds (3 milliradian rotation or 0.5 mm translation in any direction), the current moving frame was considered to contain significant motion. Subsequently, the bounding box was repositioned based on the transformation matrix and applied to subsequent frames. The final transformation matrix was converted to the PET coordinate system via a pre-calculated calibration matrix (see Supplement). No filtering was applied to the motion data.

The UMT framework was compared with the Vicra, which was used as the reference. The Polaris Vicra tracking system (Northern Digital Inc., Waterloo, Canada) used infrared illuminators and stereo cameras to sense 3D positions of reflective spheres, which were mounted to the subject's head with a 'tool'. Data were collected at 30 Hz.

To measure head motion between CT and PET, a second Vicra reflection tool ('bed tool') was attached to the patient bed and was tracked along with the tool on the patient's head ('head tool'). We assumed that the patient's head was motionless relative to the bed during the CT acquisition, so the spatial relationship of the head tool relative to the bed tool was constant, i.e. a fixed transformation matrix. With the bed tool, Vicra can track the motion from CT pose to PET pose to eliminate attenuation mismatch. For this evaluation, the transformation matrix between CT and PET poses was applied in the proposed UMT framework, since UMT did not measure the motion between PET and CT (see Supplement). In the future, we will develop CT automatic segmentation and registration algorithms for UMT.

Image reconstruction was performed using the MOLAR (Motion-compensation OSEM list-mode algorithm for resolution-recovery reconstruction) platform (Jin et al 2013a). Motion information at 30 Hz provided by Vicra or UMT was converted to a text file with one line for each measurement, including time tag and the 12 values in the rigid transformation matrix. Then, EBE MC reconstructions were performed by reassigning the endpoints of each line-of-response (LOR) according to the motion information. Both methods used the same reconstruction pipeline and frame timing. OSEM reconstruction (3 iterations × 21 subsets) with spatially invariant point-spread-function (PSF) of 4 mm full-width-half-maximum (FWHM) was used (Jin et al 2013a). Time-of-flight information was included in the reconstruction and no post-smoothing filter was applied. An isotropic voxel size of 0.5 mm was used for the point source reconstructions and 2 mm was used for the ¹⁸F-FPEB and ¹¹C-LSN3172176 studies. Dynamic PET data were reconstructed into 33 frames: 6 × 30 s, 3 ×1 min, 2 ×2 min, 22 ×5 min. For the phantom and human point source studies, the six parameters of rigid motion measured by the Vicra and UMT systems were compared. A full 15 min frame and fifteen 1 min frames were reconstructed. To assess within-image blurring, the regions around each point source were fitted to a three-dimensional (X: lateral, Y: anterior-posterior, and Z: axial), Gaussian model with 8 parameters (peak height, background, X, Y, and Z center locations, and X, Y, and Z FWHMs). Measurements made in the reconstruction of the 1st minute, i.e. no motion, were used as the reference. Center shift of the point sources was measured by the Euclidian distance (mm) of the estimated centers between target and reference reconstructions. For the ¹⁸F-FPEB and ¹¹C-LSN3172176 human studies, region of interest (ROI) analysis was applied (see Supplement). For each ROI, paired sample t-tests (one-tailed) were applied to the absolute value of the percent differences with respect to the Vicra data, to test statistically if the UMT error was smaller than that of the no motion correction data.

Two motion correction metrics were evaluated. The first metric was based solely on motion data. Due to head motion and facial expression changes, e.g. mouth breathing, ICP registration may be inaccurate. To quantify the UMT MC quality, we proposed a metric called registration quality (RQ), which quantified registration accuracy as the fraction of points in the point cloud that were accurately registered:

$$\begin{eqnarray}&&\mathrm{RQ}=\frac{N(m^{\prime} )}{N(m)}\end{eqnarray} \tag{ 1 }$$

$$\begin{eqnarray}&&{m}^{{\prime} }\in \left\{{m}_{i}\right|\varphi \left({m}_{i},{r}_{i}\right)\leqslant {d}^{2}\},\end{eqnarray} \tag{ 2 }$$

where N is the number of points (∼5000 points per cloud), $m$ represents the registered moving frame, $r$ represents the reference frame, and $i$ indexes the points in the point cloud. $\varphi$ is a squared Euclidean distance operator quantifying the closeness of ${m}_{i}$ and ${r}_{i}.$ The distance threshold, d, was empirically set at 2.4 mm (80% of the UMT voxel size). To quantify overall registration quality, the time duration with RQ below an empirical threshold (0.97) was tabulated for each study.

The second metric was based on PET raw data. We have previously used COD as a data-driven method to detect motion (Lu et al 2019, Ren et al 2017). The central coordinates of the LOR of all events were averaged over 1 s intervals to generate COD traces in X, Y, and Z. A follow-on to COD was motion-corrected COD (MCCOD), which was an objective quality control approach to assess rigid motion information (Sun et al 2022a); see Supplement for details. MCCOD displacements indicated motion estimation errors.

In the human study, we generated MCCOD traces based on both Vicra and UMT data to qualitatively evaluate UMT motion tracking performance. In addition, we assessed the effectiveness of the RQ metric by comparing 'spikes' in this trace to large visible MCCOD displacements.

For the phantom point source study, the mean and standard deviation (SD) of rotation and translation differences tracked by UMT and Vicra were 0.39 ± 0.71° and −0.60 ± 2.18 mm, respectively. Qualitative results show that UMT and Vicra motion tracking are in an overall very good agreement (figure 2(a)). Small discrepancies in rotation (0.65 ± 0.10°) and translation (1.08 ± 0.28 mm) in X at 4–5 min as well as rotation (0.59 ± 0.06°) in Y at 8–9 min were observed. Short period disagreements ('spikes') between Vicra and UMT occurred at the times of step motions; these may be caused either by the Vicra tool wobbling or by poor performance of the UMT ICP registration. The SD of rotation and translation during the static time periods between step motions for UMT were 0.04° and 0.05 mm, and for Vicra were 0.07° and 0.39 mm, i.e. there were less high-frequency fluctuations between step motions for the UMT than the Vicra; this difference in variability is visible in figure 2(a).

Zoom In Zoom Out Reset image size

Figure 3(a) shows reconstructed axial, sagittal, and coronal images of the point sources in the phantom study; these slices intersect the highest pixel value for each source for each method. For the 0–1 min reference phase (figure 3(a), top row, no motion), isotropic resolution was observed in the X–Y (axial) planes for all three points while poorer resolution is observed in the Z direction, perhaps caused by the spatially-variant resolution of the mCT (Jakoby et al 2011). For phases with motion (figure 3(a), rows 2 and 3, 15 min reconstructions), UMT yielded similar peak intensity as the reference image, while Vicra yielded lower peak values consistent with less-accurate motion compensation. For comparison, no motion corrected (NMC) images are shown as maximum intensity projections in comparison to those for UMT and Vicra (see supplemental figure 2).

Zoom In Zoom Out Reset image size

Table 1(a) shows the analysis of the full (15 min) and frame-based (individual 1 min periods) reconstructions of the point sources for phantom study. Numerically, for all the points and in all directions, UMT showed smaller FWHM values than Vicra (table 1(a)) in both the full analysis (0.35 ± 0.27 mm) and the frame-based analysis (0.09 ± 0.08 mm). UMT also yielded smaller center shifts than the Vicra (differences of 0.12 ± 0.10 mm in full analysis and 0.57 ± 0.02 mm in frame-based analysis). Due to the nature of the mannequin, i.e. no facial expressions, this study represented the best possible UMT performance.

For the human study with point sources, figure 2(b) shows the translation and rotation tracked by UMT and Vicra. For X axis rotation and translation, good agreement between UMT and Vicra was found, with mean ± SD differences of −0.18 ± 0.61° and 0.05 ± 0.93 mm, respectively. For the other motion parameters (Y axis rotation and translation, Z axis rotation and translation), larger differences were found: 0.34 ± 0.36°, −0.07 ± 1.31 mm, −0.46 ± 0.72° and 1.09 ± 1.33 mm, respectively. Like the mannequin study, UMT yielded lower noise in translation tracking than the Vicra. Figure 3(b) illustrates the full 15 min reconstruction of the point sources in the human study. Visually, UMT outperformed Vicra for Point 2, exhibiting a higher peak intensity. For the other sources, both UMT and Vicra displayed lower peak intensities than the reference, accompanied by elongated tails in the axial and coronal views.

Table 1(b) shows analysis results of the human point source study. In the full analysis, at Point 2 (∼5 cm above the left ear of the volunteer), UMT provided better spatial resolution than the Vicra and yielded similar peak intensity as the reference image. For Point 1 (close to top of head) and Point 3 (∼5 cm above the right ear), UMT yielded comparable spatial resolution as the Vicra in X and Z directions, but resolution degradation in Y was observed for Points 1 and 3 for UMT. In addition, UMT yielded substantially larger center shift (by 0.62 ± 0.13 mm) in the full analysis. In the frame-based analysis, UMT had smaller FWHM values than Vicra for all points (0.15 ± 0.08 mm), similar to the mannequin study. As for center shift, Vicra outperformed UMT for 2 points; the average difference was 0.26 ± 0.45 mm for all points. Compared with the mannequin study, UMT had poorer performance of center shift in the human study, perhaps due to its sensitivity to facial expressions such as mouth/nose movement, hands touching face, etc.

For the human studies, we compared the real-time motion tracking information from UMT and Vicra in one typical ¹¹C-LSN3172176 subject with moderate motion (figure 4). Over a 2 h scan, the mean and SD of rotation and translation differences tracked by UMT and Vicra were 0.84 ± 0.62° and 0.15 ± 0.72 mm, respectively. The translation values in all directions agreed, but there were some minor drifts in Y rotations, with mean and SD differences between UMT and Vicra of 0.51 ± 0.35°. These discrepancies may be caused by factors such as nonrigid motion (which can affect UMT) and tool slippage and wobbling (which affects the Vicra). As seen in figure 4, similar to the point source studies, UMT yields lower noise than the Vicra; this is most clearly visualized in the translation results. Quantitatively, we calculated the translation motion SD within 1 min intervals and averaged this value over the 120 min scan across all 12 subjects. The result showed that Vicra average SD was 0.49 mm while UMT was 0.27 mm.

Zoom In Zoom Out Reset image size

Figure 5 shows reconstructed images for the two tracers with no motion correction (NMC) or motion information from UMT or Vicra. In figure 5(a), compared with NMC, UMT and Vicra both improved image resolution, e.g. at frontal cortical regions (arrows). This area usually suffers more rotation-induced motion since it is far from the head's center of rotation. In figure 5(a), minor visual differences were observed between Vicra and UMT methods. In figure 5(b), UMT and Vicra yield improved lateral cortical structures (arrow) while NMC was blurred. These examples show that UMT can achieve similar motion tracking performance to Vicra in human studies.

Zoom In Zoom Out Reset image size

Table 2 shows the SUV analysis over 2 h scans for large ROIs for NMC and UMT, with percentage differences reported with respect to Vicra. For ¹⁸F-FPEB, the average difference between UMT and Vicra was very small (mean ± SD 0.05% ± 0.73%), and the inter-subject SD of UMT-Vicra difference was much less than NMC (5.16% ± 3.11%). For ¹¹C-LSN3172176, the average difference between UMT and Vicra was a bit larger (0.84% ± 1.15%), and average SD across subjects between UMT and Vicra was also much less than NMC (3.46% ± 2.82%). For both tracers, grey matter (GM) has higher uptake than white matter, NMC underestimated GM and overestimated white matter while UMT values had a maximum mean difference of 1.18% compared to Vicra. Paired sample t-test showed that NMC absolute bias was significantly greater than UMT (p < 0.05) in 12 / 14 ROIs (p < 0.1 in the remaining 2 ROIs). Figure 6 shows the SUV analysis for 74 small GM ROIs. In figure 6(a), the mean difference between UMT and Vicra was 0.19% while NMC showed large negative biases. Variability across subjects of the UMT results in both tracers (error bars) was small (mean ± SD 1.11% ± 0.40%). In figure 6(b), for the whole period, the corresponding SD of the % differences across ROIs was also low.

Zoom In Zoom Out Reset image size

Figure 7 shows the UMT RQ metric and COD/MCCOD results of one ¹¹C-LSN3172176 (figure 7(a)) and one ¹⁸F-FPEB subject (figure 7(b)); the COD/MCCOD data are shown for the one direction with most prominent motion (there was minimal motion in other directions). In figure 7(a), the proportion of time with RQ below 0.97 was 0.03%. The most pronounced spike in the RQ curve was at ∼45 min corresponding to a clear motion, as seen in the COD graph in the Y direction. Other spikes in RQ did not show corresponding movement in the COD graph, perhaps due to non-rigid movements (see Discussion). From 45 to 80 min, there were three head motions from COD-Y, and UMT visually outperformed Vicra in correcting these motions with smoother MCCOD results. Overall, Vicra (blue) and UMT (orange) MCCOD tracers were generally consistent, with increasing fluctuation with time expected due to the ¹¹C half-life. In terms of ROI accuracy, the motion correction quality for UMT and Vicra were comparable with average small GM ROI percent difference of 0.18% ± 1.11% in this subject.

Zoom In Zoom Out Reset image size

Figure 7(b) shows the UMT RQ and Z direction COD/MCCOD results of one ¹⁸F-FPEB subject who showed more pronounced movements. From 15–30 and 80–120 min, the RQ curve showed numerous spikes (the proportion of time with RQ below 0.97 was 10.65%) and COD-Z showed some matching displacements during these periods. Other spikes in RQ were caused by facial movement, e.g. mouth breathing (see Discussion) and these were reflected in the UMT motion correction quality from MCCOD. In this case, Vicra outperformed UMT, and the average small GM ROI difference between UMT and Vicra reconstructions was larger (−1.16% ± 3.61%).