Perfusion parameter map generation from TOF-MRA in stroke using generative adversarial networks

Purpose: To generate perfusion parameter maps from Time-of-flight magnetic resonance angiography (TOF-MRA) images using artificial intelligence to provide an alternative to traditional perfusion imaging techniques. Materials and methods: This retrospective study included a total of 272 patients with cerebrovascular diseases; 200 with acute stroke (from 2010 to 2018), and 72 with steno-occlusive disease (from 2011 to 2014). For each patient the TOF MRA image and the corresponding Dynamic susceptibility contrast magnetic resonance imaging (DSC-MRI) were retrieved from the datasets. The authors propose an adapted generative adversarial network (GAN) architecture, 3D pix2pix GAN, that generates common perfusion maps (CBF, CBV, MTT, TTP, Tmax) from TOF-MRA images. The performance was evaluated by the structural similarity index measure (SSIM). For a subset of 20 patients from the acute stroke dataset, the Dice coefficient was calculated to measure the overlap between the generated and real hypoperfused lesions with a time-to-maximum (Tmax) > 6 s. Results: The GAN model exhibited high visual overlap and performance for all perfusion maps in both datasets: acute stroke (mean SSIM 0.88 – 0.92, mean PSNR 28.48 – 30.89, mean MAE 0.02 – 0.04 and mean NRMSE 0.14 – 0.37) and steno-occlusive disease patients (mean SSIM 0.83 – 0.98, mean PSNR 23.62 – 38.21, mean MAE 0.01 – 0.05 and mean NRMSE 0.03 – 0.15). For the overlap analysis for lesions with Tmax > 6 s, the median Dice coefficient was 0.49. Conclusion: Our AI model can successfully generate perfusion parameter maps from TOF-MRA images, paving the way for a non-invasive alternative for assessing cerebral hemodynamics in cerebrovascular disease patients. This method could impact the stratification of patients with cerebrovascular diseases. Our results warrant more extensive refinement and validation of the method.


Introduction
Acute ischemic stroke (AIS) is a leading cause of death and disability and requires rapid decision-making to optimize patient outcomes.h Traditional patient stratification strategies for recanalization via mechanical intervention often include perfusion imaging.Perfusion imaging, available both for computed tomography (CT) and magnetic resonance imaging (MRI), provides essential information about the severity and extension of ischemia in addition to vessel information (Copen et al., 2011;Wintermark et al., 2013).In MRI, dynamic susceptibility contrast (DSC) is the most common perfusion-weighted imaging (PWI) technique that requires the application of a gadolinium-based contrast agent (Jahng et al., 2014).
The use of gadolinium-based contrast agents, however, has raised concerns owing to potential health risks.Gadolinium deposition in organs has been reported, particularly in patients with impaired renal function or repeated exposure to gadolinium (Kanda et al., 2016).Additionally, PWI faces technical challenges limiting its effectiveness.One major difficulty is the selection of an appropriate arterial input function (AIF), which can significantly impact the accuracy of deconvolved perfusion parameter maps (Calamante, 2013;Potreck et al., 2022;Thijs et al., 2004).Additionally, perfusion parameter maps generated by different vendors vary considerably, making it difficult to establish standardized guidelines for interpretation (Welker et al., 2015).Consequently, there is an urgent need for alternative techniques for patient stratification without health risks and unreliable post-processing that maintain the required diagnostic information.
Recently, generative adversarial networks (GANs), have shown promise in medical imaging, especially for image-to-image translations (Yi et al., 2019;Shokraei Fard et al., 2022).The first pix2pix GAN, introduced by Isola et al. in 2017, demonstrated the capability of GANs to handle various image-to-image translation tasks by learning a mapping from input to output images (Isola et al., 2017).This approach has already been applied across various different modality combinations like the synthesis of structural MRI images from PET (Choi and Lee, 2018), to generate MRA from T1-weighted and T2-weighted MRI images (Olut et al., 2018) or MR images were translated into synthetic computed tomography (sCT) images (Maspero et al., 2018 Sep 10).
GANs consist of two neural networks, the generator and the discriminator, that compete with each other to enable the generation of high-quality images (Goodfellow et al., 2014).In the context of AIS, GANs can be employed to generate perfusion parameter maps from perfusion source images without the need for AIF selection and deconvolution, potentially simplifying and improving the diagnostic process (Kossen et al., 2022).
Image-to-image translation using GANs can be categorised into paired and unpaired methods.Paired image translation, as seen in pix2pix, requires a dataset of input-output image pairs for training, making it suitable for applications where such data is available.Unpaired image translation, exemplified by CycleGAN, does not require paired images, allowing the model to learn mappings between different domains using unpaired datasets (Zhu et al., 2017).This flexibility makes unpaired methods particularly useful in medical imaging, where obtaining perfectly aligned image pairs can be challenging.However, paired methods tend to produce more accurate results, provided sufficient training data is available (Yi et al., 2017) This work takes this idea a significant step further.We hypothesized that native Time-of-flight (TOF) images contain sufficient hemodynamic information to generate perfusion parameter maps.TOF imaging provides a correlate of cerebral hemodynamics via valuable information about vessel configuration, collateral circulation, and the presence of steno-occlusions.Native TOF imaging is free of health risks, as it does not require the administration of any contrast agent.
In this proof-of-concept study, we thus propose a GAN to generate perfusion parameter maps directly from native TOF imaging data, bypassing DSC perfusion imaging and associated limitations.We generate five perfusion parameter maps, i.e. time-to-maximum (Tmax), cerebral blood flow (CBF), cerebral blood volume (CBV), mean transit time (MTT) and time-to-peak (TTP).Our models are developed on two patient cohorts including acute stroke and chronic cerebrovascular disease patients.

Materials and methods
The study protocol for this retrospective analysis of our prospectively established stroke database was approved by the ethics committee of Heidelberg University and the ethical review committee of Charité, with patient informed consent waived for the former and written informed consent obtained from all patients for the latter.

Data and pre-processing
In total, 272 patients were included in this study from two different centers.200 patients suffered from acute stroke and were enrolled in a study at Heidelberg University Hospital.72 patients with stenoocclusive disease were included from the PEGASUS study (Mutke et al., 2014).The data was collected between 2011 and 2014 for the PEGASUS dataset at the Charité University and between 2010 and 2018 for the Heidelberg dataset.DSC MRI and TOF MRA scans were acquired for all patients.An intravenous gadolinium-based contrast agent was administered for perfusion imaging.DSC data were post-processed with Olea Sphere® (Olea Medical®, La Ciotat, France) and automatic motion correction was applied for the Heidelberg dataset.For the PEGASUS dataset, DSC data were post-processed with the PGui software (Version 1.0, provided for research purposes by the Center for functional neuroimaging, Aarhus University, Denmark) and motion correction was not available.For both datasets, raw DSC images were used to calculate perfusion maps of TTP from the tissue response curve.Maps of CBF, CBV, MTT, and Tmax were created by deconvolution of a regional concentration time curve with an AIF.For the Heidelberg dataset, the AIF was detected automatically and then visually inspected by a neuroradiology expert.For the PEGASUS dataset, the AIF was detected by a junior rater and visually inspected by a senior rater.For details about the studies, imaging protocol and data processing steps for both datasets please refer to the Appendix A.1 and A.2.
For both datasets skullstripping was applied using SynthStrip (Hoopes et al., 2022).Non-uniformity correction was performed on the TOF-MRA images and the perfusion parameter maps were co-registered using the VINCI software package using mutual information as cost function (https://vinci.sf.mpg.de/,20).All images were resized to a dimension of 256×256×128 and normalized between -1 and +1.We included all available data from both datasets.Both data sets were split into 70 % training (acute stroke data: 140, PEGASUS: 51), 20 % validation (acute stroke data: 40, PEGASUS: 13) and 10 % test (acute stroke data: 20, PEGASUS: 8) respectively.

Network architecture
The proposed network architecture is a 3D adaptation of the pix2pix GAN (Isola et al., 2017) consisting of two networks: the generator G and the discriminator D. The generator's task was to synthesize a 3D perfusion parameter map based on the corresponding TOF-MRA while the discriminator tried to distinguish between the generated data volume and the ground truth, along with the real TOF-MRA volume (Fig. 1).The implementation details of the GAN can be found in Appendix B.

Training
For each dataset, five GANs were trained corresponding to the five perfusion parameter maps (Tmax, CBF, CBV, MTT, TTP).Since the PEGASUS dataset was smaller than the acute dataset, the network parameters of the respective perfusion map from the acute data were used for pre-initialization of the network trained on PEGASUS data.
All models were trained for 200 epochs at which all models had reached convergence.Due to computational restrictions, the batch size was set to 1. Hyperparameters were tuned based on validation set performance, as outlined in Appendix C. All models were trained on a TESLA V100 GPU (NVIDIA Corporation, Santa Clara, CA, USA).The code for the 3D GAN was implemented in PyTorch (Paszke et al., 2019) and is openly available under https://github.com/claim-berlin/TOF-to-Perfusion-GAN.

Performance evaluation
After visual inspection, four metrics were calculated to measure the similarity of the generated perfusion parameter maps and the parameter maps derived from the DSC image.The structural similarity index measure (SSIM) combines differences in luminance, contrast and structure.Peak signal-to-noise ratio (PSNR) computes the ratio between the maximal possible signal power and the noise power that is entailed in the data.In both metrics, higher values correspond to increased similarity between the images.The last two metrics mean absolute error (MAE) or L1 norm and normalized root mean squared error (NRMSE) are error metrics.Thus, more similar images lead to lower metrics.Additional details about the metrics are included in Appendix D.
To quantify a clinically relevant alignment between the real and generated perfusion maps, an additional segmentation based analysis was performed measuring the overlap of the hypoperfused areas.A junior rater (OUA) with 4 years of experience in stroke imaging evaluated pairs of perfusion maps (Tmax) for the internal acute stroke test set of 20 acute stroke patients in a pseudoanonymized and randomized manner, amounting to 40 maps in total (20 real and 20 generated).The segmentation process consisted of two steps: thresholding above a Tmax value of 6 as a preliminary segmentation step, followed by the manual removal of artifacts outside of the hypoperfused area.The Dice coefficient was calculated for quantitative assessment of overlap between original and generated Tmax images.

Results
Visual inspection showed great similarities between the generated and the DSC-derived perfusion parameter maps for acute stroke patients (see Figs. 2 and 3).Particularly, in Fig. 2, the hypoperfused areas, visualized by generated Tmax, have great overlap with the ground truth.The quantitative analysis using image-based similarity metrics showed mean performances for all perfusion parameter maps with SSIM>0.87,PSNR>28, MAE<0.04 and NRMSE<0.4 (see Table 1).
The additional segmentation-based analysis demonstrated a moderate degree of overlap between the real and generated perfusion maps (see Fig. 4).One patient was excluded from this analysis due to the absence of a discernible lesion in the real perfusion map.For the remaining 19 patients of the internal acute stroke test set, we report a median Dice value of 0.490 and a mean Dice value of 0.477, with the interquartile range spanning from 0.386 (25th percentile) to 0.656 (75th percentile).Notably, in two patients, the generated hypoperfused area appeared on the contralateral side relative to the real hypoperfused area resulting in no overlap between real and generated lesions.
For the PEGASUS dataset including patients with chronic stenoocclusive cerebrovascular disease, the generated perfusion maps also show high similarities to the ground truth visually (see Appendix E).With quantitative metrics for all parameter maps with SSIM (structural similarity index measure) >0.83, PSNR (peak signal-to-noise ratio) >23,    MAE (mean absolute error) <=0.5 and NRMSE (normalized root mean squared error) <0.2 (see Table 1).

Discussion
We present a GAN-based AI modeling approach for generating DSCtype perfusion parameter maps (Tmax, CBF, CBV, MTT, and TTP) directly from native TOF-MRA images.Our method offers a non-invasive alternative to DSC imaging for the assessment of cerebral hemodynamics in patients with acute stroke and steno-occlusive disease.By utilizing a 3D adaptation of a pix2pix GAN architecture, our model effectively synthesized realistic perfusion parameter maps, demonstrating a strong correlation with the ground truth parameter maps derived from DSC images.Our results indicate that TOF-MRA-derived perfusion maps can serve as a meaningful alternative to DSC imaging in certain clinical scenarios to guide patient stratification in cerebrovascular disease.A TOF-MRA based approach would save scanning time and costs, avoid contrast-agent related health risks, and be applicable in cases where contrast administration or longer acquisition times may be contraindicated.
In acute stroke, the visual similarity between real and generated images seems promising.However, due to the lack of current benchmark values for this novel application it is challenging to quantitatively evaluate the results.Additionally, the moderate quantitative overlap in the segmentation-based analysis and two cases of flipped lesion locations indicate that generating clinically relevant hypoperfused areas with a Tmax value above 6 clearly needs further refinement.Furthermore, we have not analyzed the segmentation overlaps for other perfusion parameters, as this was not within the scope of our exploratory study, and should be the focus of future studies.Here, we would like to emphasize that our study serves as an exploratory analysis, a proof-ofconcept.Further validation in the context of patient stratification in acute stroke is crucial, both from a clinical as well as from a technical point of view.
In chronic steno-occlusive disease, our method has the potential of becoming a screening tool allowing repeated assessments of hemodynamics over time.Traditional imaging-based screening techniques focus only on the morphological aspects of the vessels, which may not always provide a comprehensive assessment of hemodynamics.Furthermore, these imaging techniques are associated with ionizing radiation exposure, the need for contrast agents that may cause allergic reactions, nephrotoxicity or have relatively high costs (Nogueira et al., 2018).The noninvasive and dynamic nature of our proposed method offers an alternative to these current techniques, as it can provide real-time information without the aforementioned disadvantages.This approach could facilitate serial monitoring of patients with dynamic chronic steno-occlusive disease, enabling clinicians to better assess disease progression.This could allow for early intervention potentially improving overall patient outcomes.However, also for this use case further research is needed to validate this method.
Clinically, major acute stroke studies have employed DSC-MRI or CT perfusion to assess perfusion parameters and identify patients who may benefit from reperfusion therapies (Nogueira et al., 2018;Campbell et al., 2015;Albers et al., 2018).Current guidelines allow stratification of patients based on advanced perfusion imaging under certain conditions (Berge et al., 2021;Turc et al., 2019).While our methodology is currently dependent on TOF-MRA, the general concept of utilizing vessel information to generate perfusion maps can be adapted to the CTA domain and contribute to the development of patient stratification strategies that are more broadly and easily applicable.
As mentioned in our results, the hypoperfused area for two patients appeared on the contralateral side relative to the real image.One explanation for this phenomenon could be that the TOF MRA modality provides limited tissue-perfusion related information in some patients.We hypothesize that our model might mainly rely on detecting vessel occlusions in the input image to then model the hypoperfused area downstream in the supplied arterial territories.The limited performance in detecting vessel occlusions in these patients might have misled the model.Also, the correlation between TOF MRA and collateral flow might be limited in certain cases leading to this behaviour.This limitation is not completely unexpected.However, we deem it highly promising how well a generative AI method was able to model perfusion maps from TOF-MRA alone.This limitation might be alleviated through the introduction of additional input modalities such as clinical parameters or other imaging sequences such as DWI or FLAIR to provide better tissue perfusion related context to the model.Given the exploratory nature of our study, and the black box nature of AI models attempts to explain this phenomenon and measures to mitigate it must remain speculative.
From a technical point of view, we envision several paths how the clinically relevant Tmax lesion generation performance can be enhanced in future studies.Next to testing different GAN architectures, it is possible to add additional clinically relevant information in a multimodal modeling approach.The GAN model could yield more accurate and clinically relevant results by integrating clinical features, such as patient demographics, stroke severity, and vascular risk factors.To further increase the overlap of the generated and real maps, segmentation masks of hypoperfused areas based on clinically relevant perfusion threshold values can be provided as additional input to the model to focus the training on the hypoperfused areas.Additionally, other imaging modalities like diffusion-weighted imaging (DWI) and fluidattenuated inversion recovery (FLAIR) could be integrated to enhance the model's ability to account for individual patient characteristics.This multi modal approach has already been shown to improve performance in medical imaging segmentation tasks across multiple image modalities (Pandey et al., 2023).Another recent example is MedFusionGAN which fused CT and (MRI) image sequences to generate images with CT bone structure and MRI soft tissue contrasts (Safari et al., 2023).Taken together, the integration of multimodal data could also aid in the development of more personalized and precise patient stratification strategies in AIS, ultimately improving patient outcomes in the context of acute stroke management.
There are several limitations to this study.First, the sample size of the PEGASUS dataset was relatively small compared to the Heidelberg dataset, which may have affected the model's performance in generalizing to a broader patient population.Second, due to differing perfusion processing techniques for both datasets the performance of the chronic stroke model trained on PEGASUS data could have been impacted by bias when pretrained on data of acute stroke patients from the Heidelberg dataset.Third, the study utilized retrospective and mono-centric data, which could introduce potential biases.Fourth, the vascular imaging was performed using only TOF-MRA.Adaptation to CTA-based generation of corresponding CT perfusion maps is feasible but needs further exploration.Lastly, due to computational constraints we did not translate more recent deep learning architectures to 3D such as DP-GAN or image-to-image latent diffusion models (Li et al., 2022;Saharia et al., 2022).
In conclusion, our study demonstrates that a 3D GAN model can accurately generate perfusion parameter maps from native TOF-MRA images.Our results highlight the importance of vascular status for patient stratification and may pave the way for a non-invasive alternative to contrast agent-based imaging for the assessment of cerebral hemodynamics in patients with cerebrovascular disease.Future studies might consider exploring different model architectures to combat some of the shortcomings of a conventional pix2pix GAN.Due to its adversarial nature, traditional GANs are known to be prone to instabilities (Welfert et al., 2023).However, many recent advances have been made to potentially address these shortcomings.Some better known techniques during training have shown to improve the performances of GANs, including feature matching, minibatch discrimination, historical averaging, one-sided label smoothing, and virtual batch normalization (Salimans et al., 2016).Another option could be the application of different GAN architectures like the Wasserstein GAN which was specifically designed to reduce instability of the model (Arjovsky et al., 2017).Furthermore, different architectures like Transformers or State space models have been shown to be more sensitive to long-range context in medical images and might better utilize perfusion information across the whole brain (Dalmaz et al., 2022;Atli et al., 2024).Additionally, the concrete impact of accurate co-registration on model performance is not very well explored and could have a negative impact on model performance, for example by image blurring.In the future, methods like CoCycleReg could be explored which incorporate co-registration and model training into a single step to ensure optimal training conditions (Lian et al., 2022).Finally, larger and more diverse datasets could lead to better robustness and generalizability.However, due to the high level of data privacy, acquiring such data in great amounts is uniquely challenging.Nevertheless, overall larger cohorts and, ultimately, external and prospective evaluations to validate the findings and explore the clinical utility of TOF-MRA-derived perfusion maps should be considered.

Data sharing statement
At the current time-point the imaging data cannot be made publicly accessible due to data protection.

Fig. 1 .
Fig. 1.Proposed 3D GAN architecture.The generator takes in 3D TOF-MRA volumes as an input and creates perfusion parameter maps such as Tmax from it.The discriminator's task is to differentiate between the true TOF-MRA and the true Tmax or the generated Tmax.Tmax: time-to-maximum.TOF-MRA: Time-of-flight Magnetic Resonance Angiography.GAN: generative adversarial network.

F
.Lohrke et al.

Fig. 2 .
Fig. 2. Generated Tmax (bottom row) from TOF-MRA images (top row) compared to the DSC-derived Tmax (middle row) for the acute stroke dataset.The example in the first column shows an image with the highest performance, the ones in the second and third column average performance and the last one the overall lowest performance.Example images for the figure were chosen and ranked by the authors based on visually assessed overlap/similarity of the perfusion maps.For all categories, however, our model was able to capture the relevant hypoperfused area.Tmax: time-to-maximum.TOF-MRA: Time-of-flight Magnetic Resonance Angiography.

Fig. 3 .
Fig. 3. Generated parameter maps CBF, CBV, MTT and TTP (bottom row) from TOF-MRA images (left) compared to the DSC-derived maps (top row) for the acute stroke dataset.The generated perfusion maps show high similarity to the ground truth.CBF: cerebral blood flow.CBV: cerebral blood volume.MTT: Mean transit time.TTP: time-to-peak.TOF-MRA: Time-of-flight Magnetic Resonance Angiography.

Fig. 4 .
Fig. 4. Segmentation-based overlap analysis of real and generated perfusion maps.From top to bottom the Dice coefficients comparing the manual segmentations (in white) are 0.76, 0.40 and 0.37.The analysis depicts the discrepancy between the visually assessed similarity and the quantified overlap of real and generated lesions with a Tmax>6.Tmax: time-to-maximum.

Table 1
Mean performance metrics for evaluating the similarity between the ground truth and the synthesized parameter maps for the acute stroke and PEGASUS dataset.The standard deviations across patients are shown in brackets.