Detection of individual brain tau deposition in Alzheimer's disease based on latent feature-enhanced generative adversarial network

OBJECTIVE
The conventional methods for interpreting tau PET imaging in Alzheimer's disease (AD), including visual assessment and semi-quantitative analysis of fixed hallmark regions, are insensitive to detect individual small lesions because of the spatiotemporal neuropathology's heterogeneity. In this study, we proposed a latent feature-enhanced generative adversarial network model for the automatic extraction of individual brain tau deposition regions.


METHODS
The latent feature-enhanced generative adversarial network we propose can learn the distribution characteristics of tau PET images of cognitively normal individuals and output the abnormal distribution regions of patients. This model was trained and validated using 1131 tau PET images from multiple centres (with distinct races, i.e., Caucasian and Mongoloid) with different tau PET ligands. The overall quality of synthetic imaging was evaluated using structural similarity (SSIM), peak signal to noise ratio (PSNR), and mean square error (MSE). The model was compared to the fixed templates method for diagnosing and predicting AD.


RESULTS
The reconstructed images archived good quality, with SSIM = 0.967±0.008, PSNR = 31.377±3.633, and MSE = 0.0011±0.0007 in the independent test set. The model showed higher classification accuracy (AUC = 0.843, 95% CI = 0.796-0.890) and stronger correlation with clinical scales (r = 0.508, P < 0.0001). The model also achieved superior predictive performance in the survival analysis of cognitive decline, with a higher hazard ratio: 3.662, P < 0.001.


INTERPRETATION
The LFGAN4Tau model presents a promising new approach for more accurate detection of individualized tau deposition. Its robustness across tracers and races makes it a potentially reliable diagnostic tool for AD in practice.


Introduction
Alzheimer's disease (AD) is a common neurodegenerative disease characterized by a progressive decline in multiple cognitive domains.Pathological changes appear decades before the onset of the disease, and the prodromal stage may exhibit highly heterogeneous clinical manifestations.Positron emission tomography (PET) imaging is a crucial part of this biomarker detection, as it allows for the identification of intraneuronal aggregates of hyperphosphorylated, misfolded tau protein (neurofibrillary tangles, NFTs).Tau PET offers extensive visualization and quantification that is unmatched at the histopathological level and the tau PET can support from the detection of Braak stages to the heterogeneity of tau pathology found in the literature (Ossenkoppele et al., 2016, Schöll et al., 2016, Sintini et al., 2019), and tau PET imaging is now widely used in the diagnosis of AD (Hall et al., 2017, Schwarz et al., 2018).
The common approaches for interpreting tau PET imaging mainly consist of visual assessment (Lin et al., 2023, Seibyl et al., 2023, Sonni et al., 2020) and semi-quantitative analysis in the regions of interest (ROIs) related to the disease (Jack et al., 2020, Leuzy et al., 2021, Lu et al., 2023a，Mathoux et al., 2024, Provost et al., 2021).Specifically, the visual assessment method, although it can incorporate individual differences, is subjective in some cases of atypical tau distributions and ambiguity.On the other hand, the semi-quantitative analysis method relies on manual segmentation and fixed templates (Provost et al., 2021), and with accumulated evidence of the various spatiotemporal tau patterns and trajectories, a simple region-specific quantification of tau PET can be considered to be less effective (Hong et al., 2022).Moreover, both visual assessment and manual segmentation methods require significant time and effort, and the fixed templates approach is insensitive to the detection of small lesions (Aksman et al., 2021, Vogel et al., 2021, Young et al., 2018).To address the above issues, researchers have proposed deep learning models, such as U-net (Ronneberger et al., 2015), DeepMedic (Kamnitsas et al., 2016), variational autoencoder (VAE) (Chen and Konukoglu, 2018) and generative adversarial network (GAN) (Baur et al., 2021) to overcome the limitations of conventional methods.The primary advantage of deep learning is that it can reduce manual work, such as image preprocessing, segmentation of regions of interest, feature extraction and fusion.However, the focus of deep learning models is to learn the distribution of labelled data, which might overlook individual differences and various tau deposition patterns, Additionally, healthy subjects may show elevated signals due to off-target binding (Zhang et al., 2021a), rendering deep learning-based individual deposition detection models for tau PET imaging unavailable at present.Then we pay attention to the unsupervised anomaly detection residual strategy method that has been applied to MRI images (Chen and Konukoglu, 2018).This method does not need data label training; and can adapt to diverse types of anomalies.Compared with traditional deep learning training strategies, it also has better generalization ability.
Therefore, we propose a data-driven unsupervised model aimed at overcoming issues related to individual heterogeneity and indistinct discrimination of normal tau signals, thereby enabling precise identification of abnormal deposition regions within individual images.Specifically, we enhance the latent layer distribution constraint of the conventional GAN model, introducing a method termed LFGAN4Tau, a Latent Feature-Enhanced Generative Adversarial Network method.Through unsupervised training on a substantial dataset of cognitively normal individuals' tau PET images, our model is capable of mapping patient images onto cognitively normal reference images and accurately locating non-normal deposited areas by comparison with the original images.Moreover, we aspire to demonstrate the potential clinical diagnostic value of our proposed model through clinical diagnosis, prediction, and its correlation with clinical assessments.

Participants
This retrospective study was approved by the Institutional Review Committee of the Alzheimer's Disease Neuroimaging Initiative (ADNI) and Huashan Hospital (Ethics approval code for Huashan Hospital: KY2018-363, KY2019-551, KY2020-043, KY2021-555), and written informed consent was obtained from each participant or legal guardian.A total of 1086 participants, including subjects from ADNI and Huashan Hospital, were included in this study.Of note, the majority of participants in ADNI were Caucasian and all subjects from Huashan Hospital were Mongoloid.Cohort 1 consisted of 483 cognitively normal (CN) individuals, 204 patients with early mild cognitive impairment (EMCI), 194 patients with late mild cognitive impairment (LMCI), and 107 patients with AD from ADNI.Detailed information regarding participant enrollment in ADNI can be found at https://adni.loni.usc.edu.All participants in the ADNI received baseline 18 F-flortaucipir tau PET scans, T1-weighted structural Magnetic resonance imaging (MRI) scans, and related clinical information, including Apolipoprotein E (ApoE) status, scores of Mini-Mental State Examination (MMSE), Clinical Dementia Rating-Sum of Boxes (CDR-SB), Montreal Cognitive Assessment (MOCA), Alzheimer's Disease Assessment-11 (ADAS11), ADAS13, and ADASQ4.436 subjects in Cohort 1 (44.1 %) underwent β-Amyloid PET scans using 18 F-florbetapir.The whole-brain β-Amyloid Standardized uptake value ratio (SUVR) values provided by ADNI were used to classify β-Amyloid abnormal (A+) and β-Amyloid normal (A− ) status (threshold: 1.18) (Chen et al., 2015, Zhang et al., 2022).Additionally, 110 patients and 27 CNs in Cohort 1 had longitudinal tau PET scans with a mean follow-up of 20.8 months, and 25 of them met the criteria for progressive cognitive deterioration (Chen et al., 2021) (defined as (a) diagnosis of dementia or (b) MMSE ≤ 24 at the last visit or (c) decline of MMSE value≥4 between the first visit and the last visit within 2 years).The flowchart illustrating the inclusion and exclusion of participants in Cohort 1 is provided in (Fig. S1).
Additionally, Cohort 2 was enrolled as an independent test dataset, comprising 29 CN individuals and 69 patients with AD from Huashan Hospital (Shanghai, China).All participants received baseline 18 F-florzolotau (also known as 18 F-PM-PBB3, 18 F-APN-1607) tau PET scans (Li et al., 2021, Zhang et al., 2022), T1-weighted structural MRI scans, and MMSE scores.The diagnosis of AD patients in Huashan was based on the 2011 NIA-AA criteria (Frisoni et al., 2011), and additionally, CDR-SB and MOCA assessments were performed.
For the subsequent training and testing of the anomaly detection model, the CNs in Cohort 1 were split into the train group and the validation group, by the ratio of 7:3 and stratified sampling according to both sex and age.AD patients in Cohort 1 were defined the as AD-ADNI group, CNs and AD patients in Cohort 2 were defined as the CN-Huashan and AD-Huashan groups, respectively.In addition, we trained additional models (T− ) on tau PET negative subjects in the CN-ADNI (see the Model Training section of the supplementary material for details).

Imaging acquisition and preprocessing
For the ADNI dataset, the raw 18 F-flortaucipir PET images and T1weighted structural MRI were downloaded from the official ADNI website (https://adni.loni.usc.edu/) and subsequently preprocessed uniformly.For the Huashan dataset, 18 F-florzolotau PET imaging and T1-weighted structural MRI were obtained as previously reported (Li et al., 2021).Subjects underwent anatomical MRI using a 3.0-T horizontal magnet (Discovery MR750; GE Medical Systems, Milwaukee, WI), followed by 18 F-florzolotau PET imaging.High-resolution T1-weighted images were acquired with the following parameters: TE of 3.2 milliseconds, TR of 8.2 milliseconds, TI of 450 milliseconds, flip angle of 12 • , acquisition matrix of 256 × 256 × 152, and voxel size of 1 × 1 × 1 mm.Huashan Hospital utilized a Siemens mCT Flow PET/CT scanner (Siemens, Erlangen, Germany) in 3-D mode to acquire PET images.To correct for attenuation, a low-dose CT transmission scan was conducted.Participants received 370 MBq of 18 F-florzolotau intravenously and rested for 90 minutes before the 20-minute imaging acquisition (90-110 minutes).The images were reconstructed using a 3-D ordered-subset expectation maximization algorithm (4 iterations; 24 subsets; Gaussian filter, 2 mm; zoom, 3) and resulted in a matrix size of 256 × 256 × 148 and a voxel size of 2 × 2 × 3 mm 3 .
All PET images were pre-processed using the Statistical Parametric Mapping toolbox in MATLAB 2016b.Firstly, each PET image was coregistered with the corresponding T1 image.Then, the transformation parameters standardized in T1-weighted MRI image standardization were used to convert the PET image to Montreal Neurological Institute (MNI) space.To increase the signal-to-noise ratios, the images were smoothed with an 8 mm full width at half maximum Gaussian kernel and resampled to a voxel size of 2 × 2 × 2 mm 3 .

Network architecture of LFGAN4Tau
The LFGAN4Tau model is based on a combination of GAN and VAE, as described in previous studies (Baur et al., 2021, Chen and Konukoglu, 2018, Shi et al., 2023).The input of the model is a tau PET image slice, as a result, the output is a residual map of the tau deposition region obtained by the residual strategy.The overall structure of the model consists of two encoders, a decoder, and a discriminator (Fig. 1 A).
The training objective of the LFGAN4Tau model is aligned with the min-max equation of GAN, which is presented in (1) (Goodfellow et al., 2020, Islam and Zhang, 2020, Kang et al., 2018): where D is Discriminator, G is Generator, z denotes a latent variable, p (z) is a prior distribution, in this case, a Gaussian distribution, and q(z) is the encoder's distribution function.More details can be found in the supplementary material.
L balance = min(L 1 ) + μmin(L 2 ) + γmin(L 3 ) (2) To solve the problem of individual heterogeneity and nonpathological tau PET tracer signals discrimination in tau PET images, we effectively regulated the potential constraint terms by combining the encoding of normal distributions in the model with a special multiple loss function (Fig. 1 B), which has been utilized in similar studies before (Shi et al., 2023, Shi et al., 2022).This regularization preserved the individual characteristics of the subjects and ultimately facilitated the detection of individual heterogeneity and pathological tau deposition.The primary loss function is L 1, as stated in (2).P (penalty) denotes the intermediate distribution of the two distributions, and λ is set to 10 by default.Furthermore, the pixel difference between the generated and original images is represented by L 2. Additionally, the consistency of L 3 in potential vectors serves as an auxiliary loss function to the GAN

Overall quality assessment of reconstructed images
To assess the overall quality of the model involved comparing reconstructed images to original images in the Train, Validation, and Test groups, three index indicators were used: the structural similarity (SSIM) (Wang et al., 2004), peak signal-to-noise ratio (PSNR), and mean square error (MSE) (Shi et al., 2023).Furthermore, to demonstrate the superiority of the multiple loss functions, we conducted ablation experiments in the same Train, Validation and Test groups by training the model without L 1, L 2, or L 3 loss functions, respectively.Furthermore, we also calculated the SUVR of individual detected tau deposited area, in other words, using the voxels defined by the residual images to calculate an SUVR (defined as residual SUVR).

Visualization of the outcome by LFGAN4Tau
We utilized both visualization and quantitative analysis methods to examine the direct output of the model.To aid in visualization, we selected several representative subjects at different stages of disease progression, displaying their tau PET images, clinical information, as well as the generated and residual images produced by the model.Subsequently, a conservative threshold of 0.01 was applied to binarize the residual images (more details of the results for different thresholds are available in the Supplementary Material), and the residual SUVR values were calculated and compared with fixed templates SUVR values for different clinical groups.Traditional SUVR values were calculated using ROIs comprising: 1) whole brain; 2) meta-ROI; 3) Braak 1&2; 4) Braak 3&4; and 5) Braak 5&6.Box scatter plots were employed to demonstrate the differences between clinical groups and A+/A− subgroups in CN-ADNI-Val, EMCI, LMCI, AD-ADNI, CN-Huashan and AD-Huashan groups.

Classification experiments
To assess the classification performance of the residual SUVR, the comparison was made between residual SUVR and fixed templates SUVR values by conducting Receiver Operating Characteristic (ROC) analysis and computing Hedges' g effect sizes in the Validation, EMCI, LMCI, AD-ADNI, and AD-Huashan groups.Specifically, we compared the SUVR values derived from the template for conventional tau deposition regions (including the 5 brain regions mentioned in the previous section) with the residual SUVR value obtained from LFGAN4Tau.We calculated the mean and 95 % confidence interval of the ROC curve and used Hedges' g effect sizes to quantify the magnitude of the difference between the two groups.

Statistical analysis
We conducted non-parametric rank sum tests to compare the demographic and clinical characteristics of subjects in the CN and disease groups.Additionally, we utilized one-way ANOVA tests to compare the residual SUVR values generated from the LFGAN4Tau model segmentation of tau-precipitated brain regions with fixed templates SUVR values between each clinical disease group and corrected the differences between groups using Tukey's multiple comparisons tests.To evaluate the impact of differentiation on disease conversion prediction, we employed the log-rank test to assess the survival difference between the high-risk and low-risk groups.We performed all statistical analyses using SPSS 25.0 and considered the level of significance to be P < 0.05.
We conducted correlation analyses to examine the association between residual SUVR and fixed templates SUVR and clinical scores of disease severity.The analysis was carried out using Spearman's correlation coefficient in the CN-ADNI-Val, EMCI, LMCI, and AD-ADNI groups in Cohort 1.The clinical indicators used in the analysis were MMSE, MOCA-B, CDR-SB, ADAS11, ADAS13, and ADASQ4.
To analyze the ability of the model to predict the risk of cognitive decline in patients with CN or MCI, we constructed a Cox proportional hazards model using longitudinal data from ADNI.We separated the data into low-risk and high-risk groups based on the upper quartile of output risk value as the threshold and used the Kaplan-Meier (KM) method for survival analysis of different groups with an endpoint event defined as progressive cognitive deterioration.

Subject characteristics
The CN group in Cohort 1 had significant differences in gender, MMSE, MOCA-B, CDR-SB, ADAS, and APOE4 status compared to the other three groups (P < 0.0001).Furthermore, there were significant age differences and β-Amyloid status between the CN group and LMCI and AD groups (P < 0.0001).Lastly, the CN and AD groups exhibited a significant difference in education years (P = 0.0001) (Table 1).The demographics and characteristics of the A+/A− subgroup are shown in Table S1.

Overall quality assessment of reconstructed images
Images from the CN group were used to validate the overall quality of the model reconstruction.The objective indicators in our model perform better on both the ADNI Train and Validation groups than other models, and also on the data from the external validation group, Huashan hospital with different tau tracer ( 18 F-florzolotau), SSIM can reach 0.967 ± 0.009, PSNR reaches 31.377± 3.633, and MSE is 0.0011 ± 0.0007 (Table 2).In addition, ablation experiments (Table 2) and comparative model results (Table S2) show that our proposed model with multiple loss functions has the best performance.Detailed results of the LFGAN4Tau(T− ) model are shown in the Model Training section of the supplementary material.

Visualization of the outcome by LFGAN4Tau
We present examples of images generated by the LFGAN4Tau model (Fig. 2; Fig. S6), and the LFGAN4Tau(T− ) model (Fig. S7), with all participants of comparable ages.On an individual level, the residual image for each subject outlined distinct brain regions.In general, the residual brain region exhibited a more diffuse pattern as the disease severity progressed.Moreover, In Fig. 2, our residual SUVRs are more discriminatory across diseases, which suggests that these individual regions from our model provided a better characterization of tau uptake values, which may be valuable for clinical applications.

Survival analysis
Based on the definition of endpoint events, the rate of progressive cognitive deterioration was 18.25 %, detailed longitudinal data are shown in Table S5.The LFGAN4Tau model exhibited a higher predictive power (hazard ratio = 3.662, 95 % CI: 1.815-7.385)compared to the other SUVRs.Furthermore, the Braak SUVRs were non-significant in the COX regression analysis (P > 0.05) (Fig. 5).

Discussion
In this study, we proposed a LFGAN4Tau method for individual tau deposition, which aimed to facilitate clinicians in achieving better diagnosis and prediction for AD.
The LFGAN4Tau model utilizes adversarial generation techniques to map a patient's tau PET distribution onto that of a CN individual.The underlying mechanism involves the collaborative learning of the model's generator and discriminator.This process yields reconstructed images, which can be roughly considered as the patient's tau PET image before the onset of cognitive impairment.The discrepancies between these reconstructed images and the original images can be leveraged to derive regions of tau deposition.To solve the problem of individual heterogeneity and non-pathological tau deposition discrimination in tau PET images, we added a multiple loss function.This multifarious loss function serves a dual purpose: first, to reinforce the constraints on latent features, thereby steering the model toward generating more accurate and meaningful outputs; and second, to equip the model with the adaptability needed to accommodate the inherent diversity in image characteristics.
The residual images obtained from the proposed method were distributed in AD-related brain regions (Fig. 2 C-F), which generally    J. Jiang et al.
followed the widely accepted pathological model of tau trajectory (Hong et al., 2022, Leuzy et al., 2019).However, abnormalities in tau accumulation reflected by the elevated signals were more prominent and easier to identify than the original images due to the clearance of interfering signals from off-target/non-specific binding (Whittington andGunn, 2019, Whittington andGunn, 2021).In most participants, the areas of abnormal tau deposition captured by residual images were consistent with a traditional limbic-predominant phenotype with a Braak-like spatial progression (Fig. 2).Residual images also indicated that there is a non-negligible number of subjects exhibiting heterogeneity in tau patterning, consistent with previous findings (Lu et al., 2023b, Vogel et al., 2021, Young et al., 2022), such as frontal and parietal-dominant (Fig. S6 A), asymmetric pattern (Fig. S6 B-C), and our residual images show "atypical" tau trajectories.Furthermore, since tau accumulation is heterogeneous not only in different regions but also within the same region, the ability to describe the precise extent of tau deposition and provide weighted intensity calculations displayed by residual images could help to avoid underestimation caused by mean calculations.Taken together, while the fixed templates SUVR values may not accurately reflect the individualized tau deposition, the proposed residual SUVRs could facilitate precise disease tracking regardless of variation in tau pathology.
We initially assessed the clinical utility of this model in tau PET imaging for diagnosis, and our method demonstrated improved performance compared to the fixed template method in classifying different entities and binary amyloid statuses (A+/A− ) in most cases, especially in the mid-disease phase.We then evaluated its efficiency on disease prognosis, and the proposed method consistently outperformed the fixed template method.The observed superior predictive performance in survival analysis for cognitive decline, with a higher hazard ratio (3.662, P < 0.001) indicates the model's promising prognostic value in AD.These improvements re-emphasized the importance of accounting for individual heterogeneity when interpreting tau PET imaging in practice (Leuzy et al., 2019, Zhang et al., 2021b).Of note, although our LFGAN4Tau model was solely trained in the 18 F-flortaucipir PET imaging from ADNI, it was independently tested not only on the same scans but also on a different tau PET imaging ( 18 F-florzolotau) in another race.The solid performance in both tests to some extent confirmed the satisfactory generalizability of the current approach.
Beyond its diagnostic role, tau PET imaging holds promise as a biomarker of disease severity in AD (Chen et al., 2021).Accurate quantification of tau deposits requires consideration of both extent and intensity.Although the fixed template method takes into account the most commonly involved regions in AD, averaging the signal over a uniform range would partially reduce the accuracy of this marker in reflecting the extent of the disease, which would further affect the precise assessment of disease progression and treatment efficacy at the individual level.As expected, the new tau biomarker residual SUVR obtained from our model showed better classification and prediction performances, and stronger associations in disease groups with clinical scales than SUVR values in meta-ROIs (both anatomical and pathological ROIs) in AD.Although its superiority remains to be tested in future clinical trials, the results now show the feasibility of using deep learning models to detect individual small lesions and identify new tau PET biomarkers in AD.
Overall, our LFGAN4Tau method showed promising results and provided a new perspective for tau PET image detection and quantification.Nonetheless, this work has some limitations.Firstly, our modified model used registered and smoothed SUVR images, which could have reduced the detailed information of the tau PET scan.Although raw images without registration and smoothing could be more appropriate for deep learning model training, it would require higher computing power and more training data.We fully recognise that the computational efficiency aspect has not been sufficiently considered in the current study.We have not been able to fully discuss the computational efficiency and scalability of the models, especially in terms of their actual deployment in clinical practice.We do consider deploying the model on better GPUs or large-scale computing platforms in the future to improve computational efficiency and accelerate the model run speed.Secondly, the model is currently more sensitive to classifying the disease in the middle and later stages, which may be because the tau pathology is not obvious enough in the early stages, and the image voxels are less distributed, which may directly result in the poorer correlation of res SUVR with ADAS13 in the CN group (Table 4).In the future, we will consider how to optimise the model with fewer voxels.In the survival analysis, more neurodegenerative markers will be compared in the future to make the study more complete.Finally, although we used data from two centres in our study, we are also aware that this may not be sufficient to cover all cases, and that the smaller number of CNs at the Huashan centre has an age difference from the ADNI data.We plan to further expand our data sources and collect data from more centres for validation in future studies to more fully assess the robustness and generalisation ability of the model.As a deep learning model, LFGAN4Tau still has some black box attributes.Although we have demonstrated the interpretability of the results using multiple methods, they cannot be used as the only evidence for diagnosis, and can only be used as an auxiliary diagnostic tool currently.Future work will focus on enhancing the interpretability and generalization performance of the model and exploring its potential applications in other medical image analysis tasks.Note: Correlations results are expressed as: r value (P value).Bold indicates the maximum value of the r value.

Declaration of competing interest
There are no conflicts of interest including any financial, personal, or other relationships with people or organizations for any of the authors related to the work described in the manuscript.

Fig. 2 .
Fig. 2. Examples of generated images and residual images.

Fig. 3 .
Fig. 3. Residual SUVR and fixed templates SUVR values for different grouping situations.(A) Data distribution with clinical subgroups.(B) Data distribution for A+/ − subgroups.(C) Data distribution under the Huashan group.

Table 1
Demographics and characteristics The data were given as mean standard ± SD or number (%).ADNI: Alzheimer Disease Neuroimaging Initiative, CN: cognitively normal, EMCI: early mild cognitive impairment, LMCI: late mild cognitive impairment, AD: Alzheimer's disease, MMSE: Mini-Mental State Examination, MOCA-B: Basic Version of Montreal Cognitive Assessment, CDR-SB: Clinical Dementia Rating-Sum of Boxes, ADAS: Alzheimer's Disease Assessment.

Table 2
The results of objective indicators.The SSIM, PSNR, and MSE of each group are presented as mean ± standard.Bold indicates the maximum value in the Test set.The Train and Validation sets are from ADNI, Test set is from Huashan.SSIM: structural similarity (higher values indicate better image similarity), PSNR: peak signal to noise ratio (higher values indicate better image quality), MSE: mean square error (lower values indicate better image quality), LFGAN4Tau: a Latent Feature-Enhanced Generative Adversarial Network method proposed in this paper, L1+L2, L1+L3, L2+L3 represent LFGAN4Tau models without L3, L2, L1 loss functions respectively.

Table 3
Results of ROC Analysis in clinical subgroups.

Table 4
Correlation between ADAS13 clinical scores and different SUVR values of tau images (in the CN group, and disease group respectively).