Skip to main content

Advertisement

SpringerLink
Log in

Longitudinal Structural MRI Data Prediction in Nondemented and Demented Older Adults via Generative Adversarial Convolutional Network

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Alzheimer’s disease (AD) is the most common cause of dementia and threatens the health of millions of people. Early stage diagnosis of AD is critical for improving clinical outcomes and longitudinal magnetic resonance imaging (MRI) data collection can be used to monitor the progress of each patient. However, missing data is a common problem in longitudinal AD studies. The main factors come from subject dropouts and failed scans. This hinders the acquisition of longitudinal sequences that consist of multi-time-point magnetic resonance (MR) images at relatively uniform intervals. In this paper, we present a generative adversarial convolutional network to predict missing structural MRI data. In particular, we include multiple MRI scans as a temporal sequence collected at different times and determine the spatio-temporal relationship between the different scans in the proposed network. We adopt residual bottlenecks in the generator to decrease parameter values and deepen the network. In order to make full use of the longitudinal information, our discriminator classifies not only real MR images from generated MR images, but also fake sequences from real sequences in which the longitudinal MR images for all time points come from the dataset, only the last MR image comes from the generator. Results of our experiment show that our method performs more accurately for the longitudinal structural MRI data prediction of a brain afflicted with AD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Wang X, Shen D, Huang H (2016) Prediction of memory impairment with MRI Data: a longitudinal study of Alzheimer’s disease. Proc Int Conf Med Image Comput Computer Assist Intervent 2016:273–281

    Google Scholar 

  2. Havaei M, Guizard N, Chapados N et al (2016) Hemis: hetero-modal image segmentation. Proc Int Conf Med Image Comput Computer Assist Intervent 2016:469–477

    Google Scholar 

  3. Varsavsky T, Eaton-Rosen Z, Sudre CH et al (2018) PIMMS: permutation invariant multi-modal segmentation. Deep Learn Med Imag Analys Multi Learn Clini Dec Sup 2018:201–209

    Google Scholar 

  4. Chartsias A, Joyce T, Giuffrida MV et al (2017) Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans Med Imag 2017:803–814

    Google Scholar 

  5. Ramani A, Jensen JH, Helpern JA (2006) Quantitative MR imaging in Alzheimer disease. Radiology 2006:26–44

    Article  Google Scholar 

  6. Jack CR, Petersen RC, Xu YC et al (1999) Prediction of AD with MRI-based hippocampal volume in mild cognitive impairment. Neurology 1999:1397

    Article  Google Scholar 

  7. Visser PJ, Scheltens P, Verhey FRJ et al (1999) Medial temporal lobe atrophy and memory dysfunction as predictors for dementia in subjects with mild cognitive impairment. J Neurol 1999:477–485

    Article  Google Scholar 

  8. Convit A, De Asis J, De Leon MJ et al (2000) Atrophy of the medial occipitotemporal, inferior, and middle temporal gyri in non-demented elderly predict decline to Alzheimer’s disease. Neurobiol Ag 2000:19–26

    Article  Google Scholar 

  9. Killiany RJ, Gomez-Isla T, Moss M et al (2000) Use of structural magnetic resonance imaging to predict who will get Alzheimer’s disease. Ann Neurol 2000:430–439

    Article  Google Scholar 

  10. Chételat G, Landeau B, Eustache F et al (2005) Using voxel-based morphometry to map the structural changes associated with rapid conversion in MCI: a longitudinal MRI study. Neuroimage 2005:934–946

    Article  Google Scholar 

  11. Apostolova LG, Dutton RA, Dinov ID (2006) Conversion of mild cognitive impairment to Alzheimer disease predicted by hippocampal atrophy maps. Arch Neurol 2006:693–699

    Article  Google Scholar 

  12. Devanand DP, Pradhaban G, Liu X et al (2007) Hippocampal and entorhinal atrophy in mild cognitive impairment: prediction of Alzheimer disease. Neurology 2007:828–836

    Article  Google Scholar 

  13. Hämäläinen A, Tervo S, Grau-Olivares M et al (2007) Voxel-based morphometry to detect brain atrophy in progressive mild cognitive impairment. Neuroimage 2007:1122–1131

    Article  Google Scholar 

  14. Whitwell JL, Shiung MM, Przybelski SA et al (2008) MRI patterns of atrophy associated with progression to AD in amnestic mild cognitive impairment. Neurology 2008:512–520

    Article  Google Scholar 

  15. Fan Y, Batmanghelich N, Clark CM et al (2008) Spatial patterns of brain atrophy in MCI patients, identified via high-dimensional pattern classification, predict subsequent cognitive decline. Neuroimage 2008:1731–1743

    Article  Google Scholar 

  16. Rekik I, Li G, Wu G et al (2015) Prediction of Infant MRI Appearance and Anatomical Structure Evolution Using Sparse Patch-Based Metamorphosis Learning Framework. Proc Int Conf Workshop on Patch-based Tech Med Imag 2015:197–204

    Article  Google Scholar 

  17. Meng Y, Li G, Rekik I et al (2017) Can we predict subject-specific dynamic cortical thickness maps during infancy from birth? Hum Brain Mapp 2017:2865–2874

    Article  Google Scholar 

  18. Niethammer M, Huang Y, Vialard FX (2011) Geodesic regression for image time-series. Proc Int Conf Med Image Comput Computer Assist Intervent 2011:655–662

    Google Scholar 

  19. Pathan S, Hong Y (2018) Predictive image regression for longitudinal studies with missing data. [Online]. Available: https://arxiv.org/abs/1808.07553

  20. Chen LC, Papandreou G, Kokkinos I et al (2014) Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs. [Online]. Available: https://arxiv.org/abs/1412.7062

  21. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Proc Adv Neural Inf Process Syst 2012:1097–1105

    Google Scholar 

  22. Huang Y, Shao L, Frangi AF (2017) Simultaneous super-resolution and cross-modality synthesis of 3D medical images using weakly-supervised joint convolutional sparse coding. Proc IEEE Conf Comput Vis Pattern Recognit 2017:6070–6079

    Google Scholar 

  23. Nie D, Trullo R, Lian J et al (2018) Medical image synthesis with deep convolutional adversarial networks. IEEE Trans Biomed Eng 2018:2720–2730

    Article  Google Scholar 

  24. Goodfellow I, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial networks. Adv Neur Info Proc Sys 2014:2672–2680

    Google Scholar 

  25. Zhu JY, Park T, Isola P et al (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. Proc IEEE Conf Comput Vis 2017:2223–2232

    Google Scholar 

  26. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. [Online]. Available: https://arxiv.org/abs/1511.06434

  27. Denton E, Chintala S, Szlam A et al (2015) Deep generative image models using a laplacian pyramid of adversarial networks. Proc Adv Neural Inf Process Syst 2015:1486–1494

    Google Scholar 

  28. Isola P, Zhu JY, Zhou T et al (2017) Image-to-image translation with conditional adversarial networks. Proc IEEE Conf Comput Vis Pattern Recognit 2017:1125–1134

    Google Scholar 

  29. Wolterink JM, Leiner T, Viergever MA et al (2016) Generative adversarial networks for noise reduction in low-dose CT. IEEE Trans Med Imag 2016:2536–2545

    Google Scholar 

  30. Reed S, Akata Z, Yan X et al (2016) Generative adversarial text to image synthesis. [Online]. Available: http://proceedings.mlr.press/v48/reed16.pdf

  31. Yang LC, Chou SY, Yang YH (2017) MidiNet: A convolutional generative adversarial network for symbolic-domain music generation. [Online]. Available: https://arxiv.org/abs/1703.10847

  32. Mathieu M, Couprie C, LeCun Y (2015) Deep multi-scale video prediction beyond mean square error. [Online]. Available: https://arxiv.org/abs/1511.05440

  33. Gauthier J (2014) Conditional generative adversarial nets for convolutional face generation. In: Proc Class Project for Stanford CS231N, 2014, pp. 2, 2014

  34. Karacan L, Akata Z, Erdem A et al (2016) Learning to generate images of outdoor scenes from attributes and semantic layouts. [Online]. Available: url arXiv:1612.00215

  35. Olut S, Sahin YH, Demir U et al (2018) Generative adversarial training for MRA image synthesis using multi-contrast MRI. Proc Int Workshop Pred Intell Med 2018:147–154

    Google Scholar 

  36. Yu B, Zhou L, Wang L et al (2019) Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Trans Med Imag 2019:1750–1762

    Article  Google Scholar 

  37. Dar SUH, Yurt M, Karacan L et al (2019) Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Trans Med Imag 2019:2375–2388

    Article  Google Scholar 

  38. Kazeminia S, Baur C, Kuijper A et al (2018) GANs for medical image analysis. [Online]. Available: https://arxiv.org/abs/1809.06222

  39. Moeskops P, Veta M, Lafarge MW et al (2017) Adversarial training and dilated convolutions for brain MRI segmentation. Deep Learn Med Imag Analys Multi Learn Clini Dec Sup 2017:56–64

    Google Scholar 

  40. Xue Y, Xu T, Zhang H et al (2018) Segan: adversarial network with multi-scale L1 loss for medical image segmentation. Neuroinformatics 2018:383–392

    Article  Google Scholar 

  41. Izadi S, Mirikharaji Z, Kawahara J et al (2018) Generative adversarial networks to segment skin lesions. Proc IEEE Conf Int Sym Biomed Imaging 2018:881–884

    Google Scholar 

  42. Marcus DS, Fotenos AF, Csernansky JG et al (2010) (2010) Open access series of imaging studies: longitudinal MRI data in nondemented and demented older adults. J Cognit Neurosci 2010:2677–2684

    Article  Google Scholar 

  43. Smith SM, Jenkinson M, Woolrich MW et al (2004) Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 2004:S208–S219

    Article  Google Scholar 

  44. Woolrich MW, Jbabdi S, Patenaude B et al (2009) Bayesian analysis of neuroimaging data in FSL. Neuroimage 2009:S173–S186

    Article  Google Scholar 

  45. Jenkinson M, Beckmann CF, Behrens TEJ et al (2012) Fsl. Neuroimage 2012:782–790

    Article  Google Scholar 

  46. Sharma A, Hamarneh G (2019) Missing MRI pulse sequence synthesis using multi-modal generative adversarial network. IEEE Trans Med Imag 2019:1170–1183

    Google Scholar 

  47. Çiçek Ö, Abdulkadir A, Lienkamp SS et al (2016) 3D U-Net: learning dense volumetric segmentation from sparse annotation. Proc Int Conf Med Image Comput Computer Assist Intervent 2016:424–432

    Google Scholar 

  48. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. Proc IEEE Conf Comput Vis Pattern Recognit 2016:770–778

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (No.61671367), National Natural Science Foundation of China(No.42101380), The project was supported by the Open Research Fund of National Earth Observation Data Center (No. NODAOP2021007) and the Natural Science Foundation of Shaanxi Province (youth) (No. 2021JQ-324). Thanks for the scholarship from China Scholarship Council (CSC 2021).

Author information

Authors and Affiliations

  1. School of Information and Communications Engineering, Xi’an Jiaotong University, Xi’an, China

    Liyao Song & Jiancun Fan

  2. Key Laboratory of Spectral Imaging Technology, Xi’an Institute of Optics and Precision Mechanics of Chinese Academy of Sciences, Xi’an, China

    Quan Wang, Haiwei Li & Bingliang Hu

Authors
  1. Liyao Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haiwei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiancun Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bingliang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Quan Wang, Haiwei Li or Jiancun Fan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, L., Wang, Q., Li, H. et al. Longitudinal Structural MRI Data Prediction in Nondemented and Demented Older Adults via Generative Adversarial Convolutional Network. Neural Process Lett 55, 989–999 (2023). https://doi.org/10.1007/s11063-022-10922-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10922-6

Keywords

Search

Navigation