Abstract
Machine learning has emerged as a powerful technique for processing large and complex datasets. Recently it has been utilized for both improving the accuracy and accelerating the computational speed of electronic structure theory. In this chapter, we provide the theoretical background of both density functional theory, the most widely used electronic structure method, and machine learning on a generally accessible level. We provide a brief overview of the most impactful results in recent times. We, further, showcase how machine learning is used to advance static and dynamic electronic structure calculations with concrete examples. This chapter highlights that fusing concepts of machine learning and density functional theory holds the promise to greatly advance electronic structure calculations enabling unprecedented applications for in-silico materials discovery and the search for novel chemical reaction pathways.
Part of this chapter has been reproduced from Ref. [1] under a CC BY 4.0 license, https://creativecommons.org/licenses/by/4.0/.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fiedler L, Shah K, Bussmann M, Cangi A (2022) A deep dive into machine learning density functional theory for materials science and chemistry
Pedersen JK, Batchelor TAA, Bagger A, Rossmeisl J (2020) High-entropy alloys as catalysts for the CO\(_{2}\) and CO reduction reactions. ACS Catal 10(3):2169–2176
Ye W, Chen C, Wang Z, Chu I-H, Ong SP (2018) Deep neural networks for accurate predictions of crystal stability. Nat Commun 9(1):3800
Xiong S, Li X, Wu X, Yu J, Gorbatov OI, Di Marco I, Kent PRC, Sun W (2020) A combined machine learning and density functional theory study of binary Ti–Nb and Ti–Zr alloys: stability and Young’s modulus. Comput Mater Sci 184:109830
Meftahi N, Klymenko M, Christofferson AJ, Bach U, Winkler DA, Russo SP (2020) Machine learning property prediction for organic photovoltaic devices. npj Comput Mater 6(1):166
Salvador CAF, Zornio BF, Miranda CR (2020) Discovery of low-modulus Ti–Nb–Zr alloys based on machine learning and first-principles calculations. ACS Appl Mater Interfaces 12(51):56850–56861
Wilkins DM, Grisafi A, Yang Y, Lao KU, DiStasio RA, Ceriotti M (2019) Accurate molecular polarizabilities with coupled cluster theory and machine learning. Proc Natl Acad Sci USA 116(9):3401–3406
St. John PC, Guan Y, Kim Y, Kim S, Paton RS (2020) Prediction of organic homolytic bond dissociation enthalpies at near chemical accuracy with sub-second computational cost. Nat Commun 11(1):2328
Eremin RA, Zolotarev PN, Golov AA, Nekrasova NA, Leisegang T (2019) Ionic transport in doped solid electrolytes by means of DFT modeling and ML approaches: a case study of Ti-doped KFeO\(_{2}\). J Phys Chem C 123(49):29533–29542
Wu H, Lorenson A, Anderson B, Witteman L, Wu H, Meredig B, Morgan D (2017) Robust FCC solute diffusion predictions from ab-initio machine learning methods. Comput Mater Sci 134:160–165
Juneja R, Singh AK (2020) Unraveling the role of bonding chemistry in connecting electronic and thermal transport by machine learning. J Mater Chem A Mater Energy Sustain 8(17):8716–8721
Palizhati A, Zhong W, Tran K, Back S, Ulissi ZW (2019) Toward predicting intermetallics surface properties with high-throughput DFT and convolutional neural networks. J Chem Inf Model 59(11):4742–4749
Rhone TD, Chen W, Desai S, Torrisi SB, Larson DT, Yacoby A, Kaxiras E (2020) Data-driven studies of magnetic two-dimensional materials. Sci Rep 10(1):15795
Gartner TE, Zhang L, Piaggi PM, Car R, Panagiotopoulos AZ, Debenedetti PG (2020) Signatures of a liquid–liquid transition in an ab initio deep neural network model for water. Proc Natl Acad Sci USA 117(42):26040–26046
Sosso GC, Bernasconi M (2019) Harnessing machine learning potentials to understand the functional properties of phase-change materials. MRS Bull 44(09):705–709
Sosso GC, Miceli G, Caravati S, Behler J, Bernasconi M (2012) Neural network interatomic potential for the phase change material GeTe. Phys Rev B 85(17):174103
Smith JS, Nebgen BT, Zubatyuk R, Lubbers N, Devereux C, Barros K, Tretiak S, Isayev O, Roitberg AE (2019) Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning. Nat Commun 10(1):2903
Gabardi S, Baldi E, Bosoni E, Campi D, Caravati S, Sosso GC, Behler J, Bernasconi M (2017) Atomistic simulations of the crystallization and aging of GeTe nanowires. J Phys Chem C 121(42):23827–23838
Bartók AP, Payne MC, Kondor R, Csányi G (2010) Gaussian approximation potentials: the accuracy of quantum mechanics, without the electrons. Phys Rev Lett 104(13):136403
Thompson AP, Swiler LP, Trott CR, Foiles SM, Tucker GJ (2015) Spectral neighbor analysis method for automated generation of quantum-accurate interatomic potentials. J Comput Phys 285:316–330
Wood MA, Cusentino MA, Wirth BD, Thompson AP (2019) Data-driven material models for atomistic simulation. Phys Rev B 99(18):184305
Shapeev AV (2016) Moment tensor potentials: a class of systematically improvable interatomic potentials. Multiscale Model Simul SIAM Interdiscip J 14(3):1153–1173
Podryabinkin EV, Shapeev AV (2017) Active learning of linearly parametrized interatomic potentials. Comput Mater Sci 140:171–180
Snyder JC, Rupp M, Hansen K, Müller K-R, Burke K (2012) Finding density functionals with machine learning. Phys Rev Lett 108(25):1079–7114
Li L, Snyder JC, Pelaschier IM, Huang J, Niranjan U-N, Duncan P, Rupp M, Müller K-R, Burke K (2016) Understanding machine-learned density functionals. Int J Quant Chem 116(11):819–833
Dick S, Fernandez-Serra M (2020) Machine learning accurate exchange and correlation functionals of the electronic density. Nat Commun 11(1):3509
Kirkpatrick J, McMorrow B, Turban DHP, Gaunt AL, Spencer JS, Matthews AGDG, Obika A, Thiry L, Fortunato M, Pfau D, Castellanos LR, Petersen S, Nelson AWR, Kohli P, Mori-Sánchez P, Hassabis D, Cohen AJ (2021) Pushing the frontiers of density functionals by solving the fractional electron problem. Science 374(6573):1385–1389
Yeo BC, Kim D, Kim C, Han SS (2019) Pattern learning electronic density of states. Sci Rep 9(1):5879
Brockherde F, Vogt L, Li L, Tuckerman ME, Burke K, Müller K-R (2017) Bypassing the Kohn–Sham equations with machine learning. Nat Commun 8(1):872
Chandrasekaran A, Kamal D, Batra R, Kim C, Chen L, Ramprasad R (2019) Solving the electronic structure problem with machine learning. npj Comput Mater 5(1):22
Ellis JA, Fiedler L, Popoola GA, Modine NA, Stephens JA, Thompson AP, Cangi A, Rajamanickam S (2021) Accelerating finite-temperature Kohn–Sham density functional theory with deep neural networks. Phys Rev B 104(3):035120
Zhang J, Terayama K, Sumita M, Yoshizoe K, Ito K, Kikuchi J, Tsuda K (2020) NMR-TS: de novo molecule identification from NMR spectra. Sci Technol Adv Mater 21(1):552–561
Sumita M, Yang X, Ishihara S, Tamura R, Tsuda K (2018) Hunting for organic molecules with artificial intelligence: molecules optimized for desired excitation energies. ACS Cent Sci 4(9):1126–1133
Tian L, Yu W (2021) Effects of cluster expansion on the locations of phase transition boundary as a first step to quantify uncertainty in first principles statistical mechanics framework. Comput Mater Sci 186:110050
Choudhary K, Tavazza F (2019) Convergence and machine learning predictions of Monkhorst–Pack k-points and plane-wave cut-off in high-throughput DFT calculations. Comput Mater Sci 161:300–308
Grumbach MP, Hohl D, Martin RM, Car R (1994) Ab initio molecular dynamics with a finite-temperature density functional. J Phys Condens Matter 6(10):1999–2014
Alavi A, Kohanoff J, Parrinello M, Frenkel D (1994) Ab initio molecular dynamics with excited electrons. Phys Rev Lett 73(19):2599–2602
Alavi A, Parrinello M, Frenkel D (1995) Ab initio calculation of the sound velocity of dense hydrogen: implications for models of Jupiter. Science (New York, NY) 269(5228):1252–1254
Marzari N, Vanderbilt D, Payne MC (1997) Ensemble density-functional theory for ab initio molecular dynamics of metals and finite-temperature insulators. Phys Rev Lett 79(7):1337–1340
Alonso JL, Castro A, Echenique P, Polo V, Rubio A, Zueco D (2010) Ab initio molecular dynamics on the electronic Boltzmann equilibrium distribution. New J Phys 12(8):083064
Abedi A, Maitra NT, Gross EKU (2010) Exact factorization of the time-dependent electron-nuclear wave function. Phys Rev Lett 105(12):123002
Samanta A, Morales MA, Schwegler E (2016) Exploring the free energy surface using ab initio molecular dynamics. J Chem Phys 144(16):164101
Born M, Oppenheimer R (1927) Zur quantentheorie der molekeln. Ann Phys 389(20):457–484
Abedi A, Maitra NT, Gross EKU (2012) Correlated electron-nuclear dynamics: exact factorization of the molecular wavefunction. J Chem Phys 137(22):22A530
Seth SK, Banerjee S, Kar T (2010) Crystal structure and DFT calculations of andrographiside. J Mol Struct 965(1–3):45–49
Fang Z, Zhao Y, Wang H, Wang J, Zhu S, Jia Y, Cho J-H, Guan S (2019) Influence of surface charge density on ligand-metal bonding: a DFT study of NH\(_{3}\) and HCOOH on Mg (0 0 0 1) surface. Appl Surf Sci 470:893–898
Civalleri B, Doll K, Zicovich-Wilson CM (2006) Ab initio investigation of structure and cohesive energy of crystalline urea. J Phys Chem B 111(1):26–33
Duan Y-H, Wu Z-Y, Huang B, Chen S (2015) Phase stability and anisotropic elastic properties of the Hf–Al intermetallics: a DFT calculation. Comput Mater Sci 110:10–19
Yahia MB, Lemoigno F, Beuvier T, Filhol J-S, Richard-Plouet M, Brohan L, Doublet M-L (2009) Updated references for the structural, electronic, and vibrational properties of TiO\(_{2}\)(B) bulk using first-principles density functional theory calculations. J Chem Phys 130(20):204501
Eustace DA, McComb DW, Craven AJ (2010) Probing magnetic order in EELS of chromite spinels using both multiple scattering (FEFF8.2) and DFT (WIEN2k). Micron 41(6):547–553
Pantazis DA, Orio M, Petrenko T, Zein S, Bill E, Lubitz W, Messinger J, Neese F (2009) A new quantum chemical approach to the magnetic properties of oligonuclear transition-metal complexes: application to a model for the tetranuclear manganese cluster of photosystem II. Chem Eur J 15(20):5108–5123
Andriyevsky B, Pilz T, Yeon J, Halasyamani PS, Doll K, Jansen M (2013) DFT-based ab initio study of dielectric and optical properties of bulk Li\(_{2}\)B\(_{3}\)O\(_{4}\)F\(_{3}\) and Li\(_{2}\)B\(_{6}\)O\(_{9}\)F\(_{2}\). J Phys Chem Solids 74(4):616–623
Mauri F, Louie SG (1996) Magnetic susceptibility of insulators from first principles. Phys Rev Lett 76(22):4246–4249
La Porta FA, Gracia L, Andrés J, Sambrano JR, Varela JA, Longo E (2014) A DFT study of structural and electronic properties of ZnS polymorphs and its pressure-induced phase transitions. J Am Ceram Soc 97(12):4011–4018
Zhang G, Musgrave CB (2007) Comparison of DFT methods for molecular orbital eigenvalue calculations. J Phys Chem A 111(8):1554–1561
Schwalbe S, Hahn T, Liebing S, Trepte K, Kortus J (2018) Fermi–Löwdin orbital self-interaction corrected density functional theory: ionization potentials and enthalpies of formation. J Comput Chem 39(29):2463–2471
Dittmer A, Izsák R, Neese F, Maganas D (2019) Accurate band gap predictions of semiconductors in the framework of the similarity transformed equation of motion coupled cluster theory. Inorg Chem 58(14):9303–9315
Khrapkovskii GM, Tsyshevsky RV, Chachkov DV, Egorov DL, Shamov AG (2010) Formation enthalpies and bond dissociation enthalpies for C1–C4 mononitroalkanes by composite and DFT/B3LYP methods. J Mol Struct THEOCHEM 958(1–3):1–6
Zhan C-G, Nichols JA, Dixon DA (2003) Ionization potential, electron affinity, electronegativity, hardness, and electron excitation energy: molecular properties from density functional theory orbital energies. J Phys Chem A 107(20):4184–4195
Hernández-Haro N, Ortega-Castro J, Martynov YB, Nazmitdinov RG, Frontera A (2019) DFT prediction of band gap in organic-inorganic metal halide perovskites: an exchange-correlation functional benchmark study. Chem Phys 516:225–231
Barhoumi M, Sfina N, Said M (2021) Bandgap energy and dielectric function of GaOBr monolayer using density functional theory and beyond. Solid State Commun 329:114261
Daghash SM, Servio P, Rey AD (2019) Structural properties of sH hydrate: a DFT study of anisotropy and equation of state. Mol Simul 45(18):1524–1537
Pribram-Jones A, Pittalis S, Gross EKU, Burke K (2014) Thermal density functional theory in context. In: Graziani F, Desjarlais MP, Redmer R, Trickey SB (eds) Lecture notes in computational science and engineering, vol 96. Springer International Publishing, Cham, pp 25–60
Baldsiefen T, Cangi A, Gross EKU (2015) Reduced-density-matrix-functional theory at finite temperature: theoretical foundations. Phys Rev A At Mol Opt Phys 92(5):052514
Mermin ND (1963) Stability of the thermal Hartree–Fock approximation. Ann Phys 21(1):99–121
Hohenberg P, Kohn W (1964) Inhomogeneous electron gas. Phys Rev 136(3B):B864–B871
Kohn W, Sham LJ (1965) Self-consistent equations including exchange and correlation effects. Phys Rev 140(4A):A1133–A1138
Ceperley DM, Alder BJ (1980) Ground state of the electron gas by a stochastic method. Phys Rev Lett 45(7):566–569
Perdew JP, Burke K, Ernzerhof M (1996) Generalized gradient approximation made simple. Phys Rev Lett 77(18):3865
Sun J, Ruzsinszky A, Perdew JP (2015) Strongly constrained and appropriately normed semilocal density functional. Phys Rev Lett 115(3):036402
Becke AD (1993) A new mixing of Hartree–Fock and local density-functional theories. J Chem Phys 98(2):1372–1377
Lee C, Yang W, Parr RG (1988) Development of the Colle–Salvetti correlation-energy formula into a functional of the electron density. Phys Rev B 37(2):785–789
Toulouse J (2021) Review of approximations for the exchange-correlation energy in density-functional theory. arXiv:2103.02645 [cond-mat, physics:physics]
Mermin ND (1965) Thermal properties of the inhomogeneous electron gas. Phys Rev 137(5A):A1441–A1443
Pittalis S, Proetto CR, Floris A, Sanna A, Bersier C, Burke K, Gross EKU (2011) Exact conditions in finite-temperature density-functional theory. Phys Rev Lett 107(16):163001
Karasiev VV, Chakraborty D, Shukruto OA, Trickey SB (2013) Nonempirical generalized gradient approximation free-energy functional for orbital-free simulations. Phys Rev B 88(16):161108
Graziani F, Desjarlais MP, Redmer R, Trickey SB (2014) Frontiers and challenges in warm dense matter, vol 96. Springer International Publishing
Brown EW, DuBois JL, Holzmann M, Ceperley DM (2013) Exchange-correlation energy for the three-dimensional homogeneous electron gas at arbitrary temperature. Phys Rev B 88(8):081102
Karasiev VV, Sjostrom T, Dufty J, Trickey SB (2014) Accurate homogeneous electron gas exchange-correlation free energy for local spin-density calculations. Phys Rev Lett 112(7):076403
Dornheim T, Groth S, Bonitz M (2018) The uniform electron gas at warm dense matter conditions. Phys Rep Rev Sect Phys Lett 744:1–86
Ramakrishna K, Dornheim T, Vorberger J (2020) Influence of finite temperature exchange-correlation effects in hydrogen. Phys Rev B 101(19):195129
Jones JE (1924) On the determination of molecular fields. II. From the equation of state of a gas. Proc R Soc Lond A 106(738):463–477
Daw MS, Baskes MI (1984) Embedded-atom method: derivation and application to impurities, surfaces, and other defects in metals. Phys Rev B 29(12):6443–6453
Daw MS, Baskes MI (1983) Semiempirical, quantum mechanical calculation of hydrogen embrittlement in metals. Phys Rev Lett 50(17):1285–1288
Baskes MI (1987) Application of the embedded-atom method to covalent materials: a semiempirical potential for silicon. Phys Rev Lett 59(23):2666–2669
Tersoff J (1988) New empirical approach for the structure and energy of covalent systems. Phys Rev B 37(12):6991–7000
Brenner DW (1990) Empirical potential for hydrocarbons for use in simulating the chemical vapor deposition of diamond films. Phys Rev B 42(15):9458–9471
Pettifor DG, Oleinik II (1999) Analytic bond-order potentials beyond Tersoff–Brenner. I. Theory. Phys Rev B 59(13):8487–8499
Stuart SJ, Tutein AB, Harrison JA (2000) A reactive potential for hydrocarbons with intermolecular interactions. J Chem Phys 112(14):6472–6486
van Duin ACT, Dasgupta S, Lorant F, Goddard WA (2001) ReaxFF: a reactive force field for hydrocarbons. J Phys Chem A 105(41):9396–9409
Yu J, Sinnott SB, Phillpot SR (2007) Charge optimized many-body potential for the Si/SiO\(_{2}\) system. Phys Rev B 75(8):085311
Smith JS, Isayev O, Roitberg AE (2017) ANI-1, a data set of 20 million calculated off-equilibrium conformations for organic molecules. Sci Data 4(1):170193
Chan H, Narayanan B, Cherukara M, Loeffler TD, Sternberg MG, Avarca A, Sankaranarayanan SKRS (2021) BLAST: bridging length/timescales via atomistic simulation toolkit. MRS Adv 6(2):21–31
Lubbers N, Smith JS, Barros K (2018) Hierarchical modeling of molecular energies using a deep neural network. J Chem Phys 148(24):241715
Schütt KT, Sauceda HE, Kindermans P-J, Tkatchenko A, Müller K-R (2018) SchNet—a deep learning architecture for molecules and materials. J Chem Phys 148(24):241722
Zhang L, Han J, Wang H, Car R, Weinan E (2018) Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Phys Rev Lett 120(14):143001
Huan TD, Batra R, Chapman J, Krishnan S, Chen L, Ramprasad R (2017) A universal strategy for the creation of machine learning-based atomistic force fields. npj Comput Mater 3(1)
Runge E, Gross EKU (1984) Density-functional theory for time-dependent systems. Phys Rev Lett 52(12):997–1000
Enkovaara J, Rostgaard C, Mortensen JJ, Chen J, Dułak M, Ferrighi L, Gavnholt J, Glinsvad C, Haikola V, Hansen HA et al (2010) Electronic structure calculations with GPAW: a real-space implementation of the projector augmented-wave method. J Phys Condens Matter 22(25):253202
Tancogne-Dejean N, Oliveira MJT, Andrade X, Appel H, Borca CH, Le Breton G, Buchholz F, Castro A, Corni S, Correa AA, De Giovannini U, Delgado A, Eich FG, Flick J, Gil G, Gomez A, Helbig N, Hübener H, Jestädt R, Jornet-Somoza J, Larsen AH, Lebedeva IV, Lüders M, Marques MAL, Ohlmann ST, Pipolo S, Rampp M, Rozzi CA, Strubbe DA, Sato SA, Schäfer C, Theophilou I, Welden A, Rubio A (2020) Octopus, a computational framework for exploring light-driven phenomena and quantum dynamics in extended and finite systems. J Chem Phys 152(12):124119
The ELK code. http://elk.sourceforge.net/
Castro A, Marques MAL, Rubio A (2004) Propagators for the time-dependent Kohn–Sham equations. J Chem Phys 121(8):3425–3433
Pueyo AG, Marques MAL, Rubio A, Castro A (2018) Propagators for the time-dependent Kohn–Sham equations: multistep, Runge–Kutta, exponential Runge–Kutta, and commutator free magnus methods. J Chem Theory Comput 14(6):3040–3052
Bloch F (1929) Über die quantenmechanik der elektronen in kristallgittern. Z Phys 52(7):555–600
Martin RM (2020) Electronic structure. Basic theory and practical methods. Cambridge University Press, Cambridge
Oliphant N, Bartlett RJ (1994) A systematic comparison of molecular properties obtained using Hartree–Fock, a hybrid Hartree–Fock density-functional-theory, and coupled-cluster methods. J Chem Phys 100(9):6550–6561
Dunning Jr TH (1989) Gaussian basis sets for use in correlated molecular calculations. I. The atoms boron through neon and hydrogen. J Chem Phys 90(2):1007–1023
Woon DE, Dunning TH (1994) Gaussian basis sets for use in correlated molecular calculations. IV. Calculation of static electrical response properties. J Chem Phys 100(4):2975–2988
Woon DE, Dunning TH (1993) Gaussian basis sets for use in correlated molecular calculations. III. The atoms aluminum through argon. J Chem Phys 98(2):1358–1371
Koput J, Peterson KA (2002) Ab initio potential energy surface and vibrational-rotational energy levels of X2\(\Sigma \) + CaOH. J Phys Chem A 106(41):9595–9599
Balabanov NB, Peterson KA (2005) Systematically convergent basis sets for transition metals. I. All-electron correlation consistent basis sets for the 3d elements Sc–Zn. J Chem Phys 123(6):064107
Wilson AK, Woon DE, Peterson KA, Dunning TH (1999) Gaussian basis sets for use in correlated molecular calculations. IX. The atoms gallium through krypton. J Chem Phys 110(16):7667–7676
Siegbahn PEM (1980) Generalizations of the direct CI method based on the graphical unitary group approach. II. Single and double replacements from any set of reference configurations. J Chem Phys 72(3):1647–1656
Lischka H, Shepard R, Brown FB, Shavitt I (2009) New implementation of the graphical unitary group approach for multireference direct configuration interaction calculations. Int J Quant Chem 20(S15):91–100
Liu B, Yoshimine M (1981) The alchemy configuration interaction method. I. The symbolic matrix method for determining elements of matrix operators. J Chem Phys 74(1):612–616
Saxe P, Fox DJ, Schaefer HF, Handy NC (1982) The shape-driven graphical unitary group approach to the electron correlation problem. Application to the ethylene molecule. J Chem Phys 77(11):5584–5592
Saunders VR, van Lenthe JH (1983) The direct CI method. A detailed analysis. Mol Phys 48(5):923–954
Werner H-J, Reinsch E-A (1982) The self-consistent electron pairs method for multiconfiguration reference state functions. J Chem Phys 76(6):3144–3156
Werner H-J (1987) Matrix-formulated direct multiconfiguration self-consistent field and multiconfiguration reference configuration-interaction methods. In: Special volume in memory of Ilya Prigogine, vol 69, pp 1–62
Werner H-J, Knowles PJ (1991) A comparison of variational and non-variational internally contracted multiconfiguration-reference configuration interaction calculations. Theoret Chim Acta 78(3):175–187
Siegbahn PEM (1984) A new direct CI method for large CI expansions in a small orbital space. Chem Phys Lett 109(5):417–423
Knowles PJ, Handy NC (1984) A determinant based full configuration interaction program. Comput Phys Commun 54:75
Dunning TH, Hay PJ (1977) Gaussian basis sets for molecular calculations. In: Methods of electronic structure theory. Springer, pp 1–27
Møller C, Plesset MS (1934) Note on an approximation treatment for many-electron systems. Phys Rev 46(7):618–622
Suellen C, Freitas RG, Loos P-F, Jacquemin D (2019) Cross-comparisons between experiment, TD-DFT, CC, and ADC for transition energies. J Chem Theory Comput 15(8):4581–4590
Wagner LK, Bajdich M, Mitas L (2009) QWalk: a quantum Monte Carlo program for electronic structure. J Comput Phys 228(9):3390–3404
Kim J, Baczewski AD, Beaudet TD, Benali A, Bennett MC, Berrill MA, Blunt NS, Borda EJL et al (2018) QMCPACK: an open source ab initio quantum Monte Carlo package for the electronic structure of atoms, molecules and solids. J Phys Condens Matter 30(19):195901
Barker JA (1979) A quantum-statistical Monte Carlo method; path integrals with boundary conditions. J Chem Phys 70(6):2914–2918
Militzer B, González-Cataldo F, Zhang S, Driver KP, Soubiran F (2021) First-principles equation of state database for warm dense matter computation. Phys Rev E 103(1):013203
Prakash A, Chitta K, Geiger A (2021) Multi-modal fusion transformer for end-to-end autonomous driving. In: 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR), June 2021. IEEE, pp 7077–7087
Jumper J, Evans R, Pritzel A, Green T, Figurnov M, Ronneberger O, Tunyasuvunakool K, Bates R, Žídek A et al (2021) Highly accurate protein structure prediction with AlphaFold. Nature 596(7873):583–589
Schwartz MD (2021) Modern machine learning and particle physics. Harv Data Sci Rev
Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S (2020) Language models are few-shot learners. arXiv:2005.14165 [cs]
Pilania G (2021) Machine learning in materials science: from explainable predictions to autonomous design. Comput Mater Sci 193:110360
Mitchell TM (1997) Machine learning, 1st edn. McGraw-Hill Education, New York
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction, 2nd edn. Bradford Books, Cambridge, MA
Murphy KP (2022) Probabilistic machine learning: an introduction. MIT Press
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, vol 27. Curran Associates, Inc
Rasmussen CE, Williams CKI (2005) Gaussian processes for machine learning. In: Adaptive computation and machine learning. The MIT Press, Cambridge, MA
Rokach L, Maimon OZ (2008) Data mining with decision trees: theory and applications. World Scientific
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297
MacQueen J et al (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, vol 1. Oakland, CA, pp 281–297
Breiman L (2001) Random forests. Mach Learn 45(1):5–32
Friedman JH (2001) Greedy function approximation: a gradient boosting machine. Ann Stat 29(5):1189–1232
Dong G, Liu H (2018) Feature engineering for machine learning and data analytics, 1st edn. CRC Press Inc, USA
Ruder S (2017) An overview of gradient descent optimization algorithms. arXiv:1609.04747 [cs]
Yan X, Su XG (2009) Linear regression analysis. Theory and computing. World Scientific
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodol) 58(1):267–288
Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55–67
Ouyang R, Curtarolo S, Ahmetcik E, Scheffler M, Ghiringhelli LM (2018) SISSO: a compressed-sensing method for identifying the best low-dimensional descriptor in an immensity of offered candidates. Phys Rev Mater 2(8):083802
Vovk V (2013) Kernel ridge regression. In: Schölkopf B, Luo Z, Vovk V (eds) Empirical inference. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 105–116
Hilbe JM (2009) Logistic regression models. Taylor & Francis
Hornik K (1991) Approximation capabilities of multilayer feedforward networks. Neural Netw 4(2):251–257
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16 \(\times \) 16 words: transformers for image recognition at scale. arXiv:2010.11929
Chen MX, Firat O, Bapna A, Johnson M, Macherey W, Foster G, Jones L, Schuster M, Shazeer N, Parmar N et al (2018) The best of both worlds: combining recent advances in neural machine translation. In: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), NIPS’17, Dec 2018. Association for Computational Linguistics, Red Hook, NY, pp 6000–6010
Julie EG, Robinson YH, Jaisakthi SM (2021) Handbook of deep learning in biomedical engineering and health informatics. Biomedical engineering: techniques and applications. Apple Academic Press
Mathuriya A, Bard D, Mendygral P, Meadows L, Arnemann J, Shao L, He S, Karna T, Moise D, Pennycook SJ et al (2018) CosmoFlow: using deep learning to learn the universe at scale. In: SC18: international conference for high performance computing, networking, storage and analysis, Nov 2018. IEEE, pp 819–829
Rosenblatt F (1957) The perceptron: a perceiving and recognizing automaton (project PARA). Technical report 85-460-1. Cornell Aeronautical Laboratory
Nwankpa C, Ijomah W, Gachagan A, Marshall S (2018) Activation functions: comparison of trends in practice and research for deep learning. arXiv:1811.03378 [cs]
Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th international conference on machine learning, pp 807–814
Gonzalez S, Miikkulainen R (2020) Improved training speed, accuracy, and data utilization through loss function optimization. In: 2020 IEEE congress on evolutionary computation (CEC), July 2020. IEEE, pp 1–8
Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Lechevallier Y, Saporta G (eds) Proceedings of COMPSTAT’2010. Physica-Verlag HD, Heidelberg, pp 177–186
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323(6088):533–536
Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. J Mach Learn Res 12(61):2121–2159
Kingma DP, Ba J (2017) Adam: a method for stochastic optimization. arXiv:1412.6980
Practical methods of optimization | Wiley online books. https://onlinelibrary.wiley.com/doi/book/10.1002/9781118723203
Liu DC, Nocedal J (1989) On the limited memory BFGS method for large scale optimization. Math Progr 45(1):503–528
Minsky M, Papert SA (2017) Perceptrons. An introduction to computational geometry, expanded, subsequent edn. The MIT Press, Cambridge, MA
Dai Z, Liu H, Le Q, Tan M (2021) CoAtNet: marrying convolution and attention for all data sizes. In: Advances in neural information processing systems 34 pre-proceedings. NeurIPS
Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Shen D, Wu G, Suk H-I (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19(1):221–248
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical image computing and computer-assisted intervention—MICCAI 2015. Lecture notes in computer science. Springer International Publishing, Cham, pp 234–241
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, vol 25. Curran Associates, Inc
Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2009) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80
Miles C, Gonzalez AS, Battaglia P, Xu R, Cranmer K, Spergel D, Ho S (2020) Discovering symbolic models from deep learning with inductive biases. In: Advances in neural information processing systems, vol 33. Curran Associates Inc, Red Hook, pp 17429–17442
Park CW, Kornbluth M, Vandermause J, Wolverton C, Kozinsky B, Mailoa JP (2021) Accurate and scalable graph neural network force field and molecular dynamics with direct force architecture. npj Comput Mater 7(1):1–9
Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, vol 27. Curran Associates, Inc
Hewamalage H, Bergmeir C, Bandara K (2021) Recurrent neural networks for time series forecasting: current status and future directions. Int J Forecast 37(1):388–427
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707
Guan S, Loew M (2020) Analysis of generalizability of deep neural networks based on the complexity of decision boundary. In: 2020 19th IEEE international conference on machine learning and applications (ICMLA), Dec 2020. IEEE, pp 101–106
Cheng K, Wang N, Li M (2021) Interpretability of deep learning: a survey. In: Meng H, Lei T, Li M, Li K, Xiong N, Wang L (eds) Advances in natural computation, fuzzy systems and knowledge discovery. Advances in intelligent systems and computing. Springer International Publishing, Cham, pp 475–486
Duvenaud D (2014) Automatic model construction with Gaussian processes. Thesis, University of Cambridge
Lee J, Bahri Y, Novak R, Schoenholz SS, Pennington J, Sohl-Dickstein J (2018) Deep neural networks as Gaussian processes. arXiv:1711.00165
Kuss M, Pfingsten T, Csato L, Rasmussen CE (2005) Approximate inference for robust Gaussian process regression. Technical report 136. Max Planck Institute for Biological Cybernetics
Candela JQ, Rasmussen C (2005) A unifying view of sparse approximate Gaussian process regression. J Mach Learn Res
Halton JH (1960) On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals. Numer Math 2(1):84–90
Markidis S (2021) The old and the new: can physics-informed deep-learning replace traditional linear solvers? Front Big Data 4
Hastie T, Tibshirani R, Friedman J (2016) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York, NY
Murphy KP (2012) Machine learning: a probabilistic perspective, illustrated edn. The MIT Press, Cambridge, MA
Bishop CM (2006) Pattern recognition and machine learning. In: Information science and statistics. Springer-Verlag, New York
Nielsen MA (2015) Neural networks and deep learning. Determination Press, San Francisco, CA
Goodfellow I, Bengio Y, Courville A (2016) Deep learning book, illustrated edn. The MIT Press, Cambridge, MA
Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117
Mehta P, Bukov M, Wang C-H, Day AGR, Richardson C, Fisher CK, Schwab DJ (2019) A high-bias, low-variance introduction to machine learning for physicists. Phys Rep Rev Sect Phys Lett 810:1–124
Carleo G, Cirac I, Cranmer K, Daudet L, Schuld M, Tishby N, Vogt-Maranto L, Zdeborová L (2019) Machine learning and the physical sciences. Rev Mod Phys 91(4):045002
Varoquaux G, Buitinck L, Louppe G, Grisel O, Pedregosa F, Mueller A (2015) Scikit-learn. Machine learning without learning the machinery. GetMobile Mob Comput Commun 19(1):29–33
Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Kopf A, Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) PyTorch: an imperative style, high-performance deep learning library. In: Advances in neural information processing systems, vol 32. Curran Associates, Inc
Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Mané D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Viégas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X (2015) TensorFlow: large-scale machine learning on heterogeneous systems. Software. Available from tensorflow.org
Falcon W, The PyTorch Lightning Team (2019) PyTorch lightning, 3
Chollet F et al (2015) Keras. https://github.com/fchollet/keras
Foreman-Mackey D (2015) George: Gaussian process regression. Astrophysics source code library, p ascl:1511.015
Upchurch P, Gardner J, Pleiss G, Pless R, Snavely N, Bala K, Weinberger K (2017) Deep feature interpolation for image content changes. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), vol 31, July 2017. IEEE
Mahmoud CB, Anelli A, Csányi G, Ceriotti M (2020) Learning the electronic density of states in condensed matter. Phys Rev B 102(23):235130
Tsubaki M, Mizoguchi T (2020) Quantum deep field: data-driven wave function, electron density generation, and atomization energy prediction and extrapolation with machine learning. Phys Rev Lett 125(20):206401
Cangi A, Ellis JA, Fiedler L, Kotik D, Modine NA, Oles V, Popoola GA, Rajamanickam S, Schmerler S, Stephens JA, Thompson AP (2021) MALA
Wood MA, Thompson AP (2018) Extending the accuracy of the SNAP interatomic potential form. J Chem Phys 148(24):241721
Cusentino MA, Wood MA, Thompson AP (2020) Explicit multielement extension of the spectral neighbor analysis potential for chemically complex systems. J Phys Chem A 124(26):5456–5464
Thompson AP, Aktulga HM, Berger R, Bolintineanu DS, Brown WM, Crozier PS, in’t Veld PJ, Kohlmeyer A, Moore SG, Nguyen TD, Shan R, Stevens MJ, Tranchida J, Trott C, Plimpton SJ (2022) LAMMPS—a flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales. Comput Phys Commun 271:108171
Giannozzi P, Baseggio O, Bonfà P, Brunato D, Car R, Carnimeo I, Cavazzoni C, de Gironcoli S, Delugas P, Ruffino FF, Ferretti A, Marzari N, Timrov I, Urru A, Baroni S (2020) Quantum ESPRESSO toward the exascale. J Chem Phys 152(15):154105
Giannozzi P, Baroni S, Bonini N, Calandra M, Car R, Cavazzoni C, Ceresoli D, Chiarotti GL, Cococcioni M, Dabo I, Corso AD, de Gironcoli S, Fabris S, Fratesi G, Gebauer R, Gerstmann U, Gougoussis C, Kokalj A, Lazzeri M, Martin-Samos L, Marzari N, Mauri F, Mazzarello R, Paolini S, Pasquarello A, Paulatto L, Sbraccia C, Scandolo S, Sclauzero G, Seitsonen AP, Smogunov A, Umari P, Wentzcovitch RM (2009) QUANTUM ESPRESSO: a modular and open-source software project for quantum simulations of materials. J Phys Condens Matter 21(39):395502
Giannozzi P, Andreussi O, Brumme T, Bunau O, Nardelli MB, Calandra M, Car R, Cavazzoni C, Ceresoli D, Cococcioni M, Colonna N, Carnimeo I, Dal Corso A, de Gironcoli S, Delugas P, DiStasio RA, Ferretti A, Floris A, Fratesi G, Fugallo G, Gebauer R, Gerstmann U, Giustino F, Gorni T, Jia J, Kawamura M, Ko H-Y, Kokalj A, Küçükbenli E, Lazzeri M, Marsili M, Marzari N, Mauri F, Nguyen NL, Nguyen H-V, Otero-de-la Roza A, Paulatto L, Poncé S, Rocca D, Sabatini R, Santra B, Schlipf M, Seitsonen AP, Smogunov A, Timrov I, Thonhauser T, Umari P, Vast N, Wu X, Baroni S (2017) Advanced capabilities for materials modelling with Quantum ESPRESSO. J Phys Condens Matter 29(46):465901
Fiedler L, Cangi A (2022) LDOS/SNAP data for MALA: beryllium at 298 K, Feb 2022
Berendsen HJC, Postma JPM, van Gunsteren WF, DiNola A, Haak JR (1984) Molecular dynamics with coupling to an external bath. J Chem Phys 81(8):3684–3690
Morishita T (2000) Fluctuation formulas in molecular-dynamics simulations with the weak coupling heat bath. J Chem Phys 113(8):2976–2982
Dal Corso A (2014) Pseudopotentials periodic table: from H to Pu. Comput Mater Sci 95:337–350
Lejaeghere K, Bihlmayer G, Björkman T, Blaha P, Blügel S, Blum V, Caliste D, Castelli IE, Clark SJ, Dal Corso A, de Gironcoli S, Deutsch T, Dewhurst JK, Di Marco I, Draxl C, Dułak M, Eriksson O, Flores-Livas JA, Garrity KF, Genovese L, Giannozzi P, Giantomassi M, Goedecker S, Gonze X, Grånäs O, Gross EKU, Gulans A, Gygi F, Hamann DR, Hasnip PJ, Holzwarth NAW, Iuşan D, Jochym DB, Jollet F, Jones D, Kresse G, Koepernik K, Küçükbenli E, Kvashnin YO, Locht ILM, Lubeck S, Marsman M, Marzari N, Nitzsche U, Nordström L, Ozaki T, Paulatto L, Pickard CJ, Poelmans W, Probert MIJ, Refson K, Richter M, Rignanese G-M, Saha S, Scheffler M, Schlipf M, Schwarz K, Sharma S, Tavazza F, Thunström P, Tkatchenko A, Torrent M, Vanderbilt D, van Setten MJ, Van Speybroeck V, Wills JM, Yates JR, Zhang G-X, Cottenier S (2016) Reproducibility in density functional theory calculations of solids. Science 351(6280):aad3000. https://www.science.org/doi/pdf/10.1126/science.aad3000
Fiedler L, Hoffmann N, Mohammed P, Popoola GA, Yovell T, Oles V, Ellis JA, Rajamanickam S, Cangi A (2022) Training-free hyperparameter optimization of neural networks for electronic structures in matter. Mach learn: sci technol 3(4):045008. https://dx.doi.org/10.1088/2632-2153/ac9956
Hooke R, Jeeves TA (1961) “Direct search” solution of numerical and statistical problems. J ACM 8(2):212–229
Lewis RM, Torczon V, Trosset MW (2000) Direct search methods: then and now. J Comput Appl Math 124(1):191–207
Akiba T, Sano S, Yanase T, Ohta T, Koyama M (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, KDD’19, Anchorage, AK. Association for Computing Machinery, New York, NY, pp 2623–2631
Mellor J, Turner J, Storkey A, Crowley EJ (2021) Neural architecture search without training. eprint: 2006.04647v1
Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L (2021) Physics-informed machine learning. Nat Rev Phys 3(6):422–440
Raissi M, Perdikaris P, Karniadakis GE (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Comput Phys 378:686–707
Hennigh O, Narasimhan S, Nabian MA, Subramaniam A, Tangsali K, Rietmann M, Ferrandis JDA, Byeon W, Fang Z, Choudhry S (2020) NVIDIA SimNet\(^{\text{TM}}\): an AI-accelerated multi-physics simulation framework. arXiv:2012.07938 [physics]
Cho K, van Merrienboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder-decoder approaches. arXiv:1409.1259 [cs, stat]
Fiedler L, Modine NA, Schmerler S, Vogel DJ, Popoola GA, Thompson AP, Rajamanickam S, Cangi A (2023) Predicting electronic structures at any length scale with machine learning. npj Comput Mater 9:115. https://doi.org/10.1038/s41524-023-01070-z
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Fiedler, L., Shah, K., Cangi, A. (2023). Machine-Learning for Static and Dynamic Electronic Structure Theory. In: Qu, C., Liu, H. (eds) Machine Learning in Molecular Sciences. Challenges and Advances in Computational Chemistry and Physics, vol 36. Springer, Cham. https://doi.org/10.1007/978-3-031-37196-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-37196-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-37195-0
Online ISBN: 978-3-031-37196-7
eBook Packages: Chemistry and Materials ScienceChemistry and Material Science (R0)