Introduction

To appreciate the distinct features and innovative character of interatomic potentials obtained with machine learning techniques, let us briefly highlight the key aspects of classical potentials or force fields. In both machine learning and classical approaches, the motivation is the same, namely atomistic simulations of large systems, sampling the configurational space and dynamic processes in a statistically meaningful way. In the absence of any better way, such interatomic potentials were originally fitted to empirical data such as vibrations of molecules and the structure of crystalline solids [1, 2], elastic coefficients, thermal expansion coefficients, and thermodynamic properties of fluids. This led to ground-breaking work by pioneers such as Aneesur Rahman [3]. A major driver in the development of force fields in the 1980’s was the desire to perform molecular simulations of DNA and proteins and their interaction with drug molecules. This led to the development of force fields such as ECEPP by the group of Harald Scheraga at Cornell University, the CHARMM force field in the group of Martin Karplus at Harvard, the AMBER forcefield by the group of Peter Kollman at the University of California in San Francisco, and the OPLS force field from the group of Jorgensen at Yale University. In this context, first versions of the consistent force field (CFF) were developed by Lifson, Hagler and Dauber [4]. An interesting approach was pursued in the group of Norman Allinger by including electronic aspects in a force field called MMP2 [5].

These classical force fields are based on a deep chemical understanding of the bonding and intermolecular interactions in molecular and bio-molecular systems. The various parameters were obtained largely by fitting to experimental data such as known interatomic distances and vibrational frequencies. A characteristic feature of these force fields is the atom-typing, i.e., the assignment of specific force field parameters based on a topological analysis of the environment of each atom. To a certain extent, a generalization of this idea is found in the present machine-learned potentials. In the 1980’s, developers of molecular quantum chemistry programs based on Hartree–Fock theory had achieved the analytical computation of first and second derivatives of the total energy. At the same time, computers such as the CRAY-1 had become powerful enough to solve the Hartree–Fock equations for molecular systems of sufficient size representing the key chemical groups occurring in organic and biochemical molecules. This led to the idea of creating large and consistent training sets for fitting force field parameters using ab initio data, namely forces and second derivatives of small molecules in various deformed conformations [6]. This concept was implemented in the form of a large industrial consortium led by a San Diego based start-up company. The use of ab initio data was the basis for fitting the strong intramolecular interactions. However, the weak intermolecular interactions still required fitting to experimental data such as densities of molecular crystals and molecular liquids. The result of this effort was a valence force field called CVFF [6].

The adequate description of bonding in metals requires interaction terms beyond simple pair-wise interactions. This fact led to the development of empirical N-body potentials by Finnis and Sinclair [7]. Parameters for this simple but efficient potential could be obtained from fitting to experimental data including lattice constants, cohesive energy, and elastic moduli. During the same time, important conceptual work was pursued by David Pettifor in the development of bond order potentials [8]. Compared with the development of first and second derivatives in Hartree–Fock based quantum chemistry programs, this important feature was implemented in solid state and molecular programs using density functional theory, thus enabling the exploration of the energy hypersurface of systems such as metals, semiconductors, and ceramics. As a result, the development of classical interatomic potentials for metals and non-metallic inorganic materials shifted increasingly from fitting to experimental data such as lattice parameters, cohesive energies, elastic coefficients, and phonon dispersions to the fitting of ab initio data.

For metals and metal alloys, the effective medium theory [9] and the related embedded atom method (EAM) [10] became highly efficient and useful approaches to describe the properties of metals and alloys such as radiation damage in Fe–Cr alloys [11], the diffusion of vacancies and self-interstitials, the dynamics of dislocations, and the migration and trapping of H interstitials [12]. EAM potentials such as those for the study of H in Zr were developed by fitting energies, forces, and stress tensors from ab initio calculations [13]. The physical interpretation of the interaction terms in EAM potentials allows calibration with experimental data. This convenient aspect of EAM potentials was used, for example, to correct the small error in the density predicted by standard density functional theory (DFT) calculations [13]. As will be discussed below, MLPs do not offer such a direct possibility for compensating intrinsic DFT errors, thus requiring more creative approaches to accomplish this task. The work with EAM potentials also revealed the limitations of this approach. For example, the accurate description of metallurgically important yet subtle properties such as stacking fault energies requires a high degree of flexibility in the representation of interatomic interactions, which may be beyond the capabilities of EAM potentials. Furthermore, the application of EAM potentials to alloys turned out to be difficult, possibly because the embedding function and electron densities are only those of the elements and so often it is only the pair potential that can be tweaked.

Extensions of the EAM potentials by including angular terms have been explored and successfully applied [14]. However, the inclusion of atoms such as oxygen and nitrogen leads to the difficult question of charge transfer. Furthermore, the description of reactive systems, i.e., the making and breaking of bonds, poses tremendous challenges for interatomic potentials. To this end, concepts of polarization and charge equilibration have been introduced into force fields and have led to forms such as charge optimized many body (COMB) potentials and reactive force fields such as ReaxFF have been introduced [15]. It is probably fair to say that in the hands of experts, these approaches can be very useful, but their general applicability is littered with difficulties and pitfalls. One of the reasons is the interdependence of parameters, which are conceptually separated into different interaction terms, but in reality, are coupled in a highly non-linear manner.

Ionic potentials building on the pioneering concepts of Erwin Madelung [16], Max Born and Joseph Mayer [2] turned out to be extremely useful for the description of oxides and other ionic materials. Using only a few parameters, an amazing richness of phenomena can be captured [17, 18]. For example, only three parameters, namely an effective charge and two parameters describing the repulsion between O atoms are sufficient to capture the thermal expansion of Li2O and the diffusion coefficients of O including the super-ionic pre-melting, and the melting temperature [19].

Common to all the above approaches is the description of interatomic interactions in terms of a relatively small set of physics-based terms such as the length and strength of chemical bonds between pairs of atoms described by a Morse potential, the electrostatic interactions between atomic charges formulated as Coulomb interaction, and long-range van der Waals interactions with a characteristic 1/r6 dependence.

The greatest limitation to creating predictive classical potentials with quantum mechanical fidelity, as described above, is the inability to fit the complicated, multivariate potential energy surface of the electronic structure with a small set of parameters and simple analytic functions. Consequently, the need to create predictive quantum-based force fields has motivated the use of machine learning, which is well suited to fitting complicated multidimensional functions of many-body systems, albeit requiring a larger parameter set. Supporting the validity of the MLP approach is the “universal approximation theorem” [20], which suggests a neural network can be developed to represent to a good approximation the electronic structure’s multivariate potential energy surface within a training domain.

The innovation of machine learning

In a remarkable paper published in 1998, Gassner et al. demonstrated for the case of Al3+ ions in water that ‘the advantages of a neural network type potential function as a model-independent and “semiautomatic” potential function outweigh the disadvantages in computing speed and lack of interpretability’ [21]. Surprisingly, this idea did not immediately spread across the field and it was almost ten years later when Manzhos and Carrington combined the high-dimensional model representation idea of Rabitz and coworkers with neural-network fitting to build a multidimensional potential for the potential energy surface of water molecules [22, 23], and Behler and Parrinello demonstrated a generalized neural-network representation of high-dimensional potential-energy surfaces for the case of bulk Si [24]. In 2010, Bartók et al. introduced the Gaussian Approximation Potential and showed the power of this approach by computing bulk and surface properties of diamond, silicon, germanium, and phonon dispersions of iron, achieving unprecedented fidelity in reproducing the ab initio results [25].

Recognizing the innovative character of machine learning approaches to generate interatomic potentials, the field started to expand rapidly leading to a variety of implementations and promising applications. This evolution also revealed serious challenges and obstacles such as finding the most efficient protocol to create training sets, selecting the best mathematical form of representing the potential, and performing the fit of the large number of parameters. The use of MLPs in molecular dynamics simulations revealed aberrations such as sudden fusion of atoms and obviously unphysical breaking of bonds, which called for careful extensions of the respective training sets to inform the potential about such situations. Over time, experience with constructing training sets and generating MLPs has grown and the power and promise of MLPs by far outweigh the early concerns.

The key features of MLPs and their similarities with classical force fields are:

  1. 1.

    The total energy of a system is expressed as a sum of the contributions from each atom. While this is also true for potentials like EAM, it is not for valence force fields using parameters for bond-angles (3-body) and dihedrals (4-body) terms as well electrostatic terms.

  2. 2.

    The energy of each atom only depends on the atom type and position of the neighboring atoms within a relatively short radius. In other words, the interactions of an atom with its surroundings are assumed to be of a local nature. Again, this is similar to classical force fields, but in contrast to typical organic force fields, there is no concept of bonds between atoms.

  3. 3.

    The energy of each atom is described by an expansion in generic functions. These functions are translationally and rotationally invariant, and they do not have a direct physical interpretation.

  4. 4.

    The coefficients of these functions are determined either by a regression method or by a neural network using a training set that provides energies, forces, and stress tensors for a comprehensive set of systems that sample the configurational space of interest.

  5. 5.

    These primary fitting data are typically obtained from quantum mechanical calculations. The errors due to the approximations in the ab initio calculations are inherited by the resulting MLP.

  6. 6.

    Once created, MLPs cannot easily be calibrated with experimental data.

These key features have remarkable consequences, namely:

  1. 1.

    By construction, there is no concept of chemical bonds in the above constructed MLPs. In other words, MLPs are reactive force fields.

  2. 2.

    MLPs do not consider atomic charges or charge transfer between atoms. Solely, the geometric and chemical environment of an atom determines its energy. There is no concept of metallic, covalent, or ionic bonds. The absence of atomic charges is a remarkable feature and, amazingly, MLPs can describe ionic crystals very well. This is also a limitation, since long-range electrostatic interactions, for example due to charged point defects in an oxide or ions in a liquid, will require a special treatment.

  3. 3.

    Since MLPs are lacking physical insight, they do not per se prevent atoms from getting very close or systems to break into fragments unless explicitly trained.

  4. 4.

    A rather large number of systems and configurations are needed to train an MLP. This can amount to thousands of structures to inform an MLP about the hypersurface of the system of interest.

  5. 5.

    In principle, MLPs can be combined with classical force fields. For example, it is possible to add short-range repulsive terms to prevent atoms getting too close during molecular dynamics simulations. Along the same lines, it is also possible to use MLPs to capture the difference between a classical potential and DFT.

  6. 6.

    While MLPs are based on a single and uniform source, namely, quantum mechanical calculations, information about specific systems, such as experimental data, are not included in the fitting process, since this would spoil the consistency of the training set. In fact, an attempt to mix quantum mechanically computed total energies and forces with experimental data such as elastic coefficients would constitute a mismatch in the types of data.

The overall workflow of the creation and use of an MLP is illustrated in Fig. 1.

Figure 1
figure 1

Workflow of construction, optimization, and use of machine-learned potentials.

Full size image

Since the early seminal papers, the field has seen a lot of developments and refinements. In particular, Behler has put forward approaches based on Neural Network Potentials (NNPs) by introducing different sets of symmetry functions used to represent the local atomic environments, which are represented in terms of pairwise distances between all atoms of a local cluster, as well as different activation functions for the neural network [26,27,28,29]. These developments have also been discussed in more recent reviews [30, 31], whereas the early developments in this field have been summarized by Handley and Popelier [32].

An alternative route was taken by regression methods as first developed Bartók et al. by introducing the Gaussian Approximation Potential (GAP) [25]. Instead of using pairwise atomic distances to describe local atomic structures these authors preferred to work with combinations of radial functions and spherical harmonics, which they transformed into bispectrum coefficients allowing them to ensure the rotational symmetry of the local representation in addition to translational and permutation symmetry [33, 34]. Later on, this approach was adopted by Thompson and coworkers, while introducing the Spectral Neighbor Analysis Potential (SNAP) [35,36,37] and quadratic SNAP (qSNAP) [38]. However, while Bartók et al. relied on Bayesian regression, Thompson et al. preferred linear regression. Shapeev and coworkers proposed a yet different method, the Moment Tensor Potential (MTP), which is based on a representation of the local atomic environments in terms of harmonic polynomials [39,40,41]. Finally, Drautz introduced the Atomic Cluster Expansion (ACE), which turned out to be more general than the previous approaches and to include these as limiting cases [42,43,44,45,46,47]. Once generated, most of these MLPs can be used with LAMMPS, which is nowadays the standard code for classical molecular dynamics simulations [48].

Beyond the methods of determining the functional form and the parameters of MLPs a variety of approaches have been developed to improve efficiency in the creation of training sets. In this respect, learning-on-the-fly is a special case. One of the pioneers of this approach was Alesandro de Vita who demonstrated this approach using classical potentials [49] as well as for MLPs [50]. On-the-fly-learning was also implemented in the MTP scheme, where it was successfully applied to the study of vacancy diffusion in Al, Mo, and Si [40]. A particularly efficient realization of this concept was accomplished in the group of Georg Kresse and implemented in the Vienna Ab initio Simulation Package (VASP) [51,52,53]. In this approach, a molecular dynamics simulation is started using initially DFT to determine the energies and forces needed to propagate the system. The ab initio data are used to generate on the fly an MLP which is then used to evolve the molecular dynamics (MD) trajectory. A Bayesian inference is used to determine the need for expanding the training and to update the MLP. The net result is a dramatic acceleration of the molecular dynamics simulation while remaining very close to the DFT trajectories. This approach has been successfully applied to Zr and ZrO2 [54,55,56].

A related approach to overcome the limitations of static training sets created before the actual process of generating an MLP is active learning, which as indicated in Fig. 1 allows to dynamically extend the training set as the calculations using a previously generated MLP proceed [41, 47, 57, 58]. This requires in each step measures to check an MLP for going beyond its range of reliability. In the simplest case this means to perform a similarity check of the structures created in the course of an MD simulation using the MLP against the structures of the training set used to generate the MLP. In case that the similarity check fails, the new structure is included in the training set and a new MLP is generated.

While generating MLPs solely from quantum mechanical calculations without inclusion of other sources of information as, e.g., experimental data has the advantage of guaranteeing consistency of training sets, it comes at the same time with the disadvantage of being bound to the approximations underlying those calculations. Specifically, as well-known, the limitations of the local and semi-local approximations to density functional theory turn out to be insufficient for the treatment, e.g., of semiconductors and insulators as well as materials with localized electronic states. Approaches with higher-level approximations exist but come with higher computational demands. A particularly elegant method to keep the additional effort at a minimum is Δ-learning as recently introduced by several authors [59,60,61,62] and combined with an on-the-fly-learning approach by Kresse and coworkers [56]. This approach is based on the assumption that the difference of the energies arising from calculations using a low-level approximation as, e.g., generalized gradient approximation (GGA) and a high-level approximation as, e.g., metaGGA, hybrid functional, or RPA varies much more smoothly across the configurational space spanned by a training set than the low-level energies themselves. As a consequence, it should be sufficient to perform only low-level calculations for the full training set while restricting the high-level calculations to a much smaller set. If the latter is not a strict subset of the full training set, additional inexpensive low-level calculations must be performed for this set. Finally, two different MLPs are generated, one from the low-level results for the full training set and another one from the difference between the low-level and high-level results obtained for the smaller set, and these two MLPs are combined to a Δ-learned high-level MLP. The efficiency and validity of this approach have been demonstrated for ZrO2 [56] (see Fig. 2).

Figure 2
figure 2

Demonstration of the accuracy of Δ-learning as applied to (a) monoclinic, (b) tetragonal, and (c) cubic ZrO2 [56]. Phonon dispersions computed with SCAN exchange correlation functionals are well reproduced with a machine-learned force field (MLFF) generated with a SCAN-based training set and a PBE training set adjusted with Δ-learning to SCAN (SCANΔ). The Δ-learning function was generated with a subset of 100 simulations recomputed with SCAN. The MLFF-SCANΔ is illustrated to precisely reproduce the MLFF-SCAN results.

Full size image

Recently, the generation of MLPs for magnetic systems came into focus. In particular, magnetic MLPs were sought to study magnetization dynamics and spin–lattice interaction at elevated temperatures. Several approaches to magnetic MLPs have been discussed in the literature so far. Dragoni et al. demonstrated that the GAP approach can successfully describe the energetic and thermodynamics of ferromagnetic α-iron including the energetics of point defects [63]. The potential was also able to reproduce the Bain path connecting the bcc, bct, and fcc structures. Novikov et al. developed a magnetic Moment Tensor Potential (MTP) and applied it to the ferromagnetic and paramagnetic states of bcc Fe [64]. While their method is applicable only to collinear spin arrangements, the magnetic SNAP technique proposed by Nikolov et al. is based on an extended Heisenberg model with parameters taken from additional ab initio spin-spiral calculations [65]. Yet, the underlying pairwise spin interaction can result in limitations, especially for disordered systems. This is different in the method proposed by Eckhoff and Behler [66, 67], which includes the full spin structure within each local atomic cluster. The same is true for the magnetic version of the Atomic Cluster Expansion [43], which has been successfully applied to the investigation of bcc and fcc Fe [68].

Basic concepts of MLPs

From a bird’s eye view, generation of MLPs is most easily understood in terms of two parts, which efficiently access and process the information stored in the training set. To this end, one often distinguishes a descriptor, which describes atomic structures, and a regressor, which maps the structural information to energies, forces, and stresses [39].

The descriptor is usually based on dividing the atomic structure, be it a molecule or a periodic structure, into local atomic environments comprising a few tens of atoms. This situation is sketched in Fig. 3, where a local cluster centered about the central atom is highlighted. Within each local atomic environment, the atomic structure is described in terms of the (relative) atomic positions and the respective atom types. This description can be written as

$${B}_{i,k} ={B}_{i,k} \left(\left\{{{\varvec{r}}}_{ji}, {A}_{j}, j = 1,\dots , {N}_{i}\right\}\right), i = 1,\dots , N,$$

where the index i designates the central atom of a cluster, while the index j enumerates all atoms in the cluster of atom i. The vector rji designates the relative atomic position of atom j in the cluster with respect to the central atom i in sphercial coordinates. Alternatively, the atomic structure in a local cluster can be specified in terms of all pairwise distances rjl in the cluster. While Neural Network potentials often prefer to work with pairwise distances, regression methods often employ a representation in terms of spherical coordinates. The parameters Aj indicate the atomic species. N and Ni denote the number of all atoms in the system and of all atoms in the cluster surrounding atom i, respectively. Finally, the index k labels different descriptor functions, which in the context of neural networks are often called symmetry functions and which could be combinations of radial functions and spherical harmonics.

Figure 3
figure 3

Illustration of the environment-dependence of atomic energies for a system with periodic boundary conditions. The energy contribution of the atom at the center of the blue circle depends on the positions of all the atoms within the dashed spherical boundary that is defined by its cutoff radius.

Full size image

Following the basic idea underlying the descriptor, the regressor divides the energies, forces, and stress tensor component values into contributions from all local atomic environments, expresses these quantities in terms of the descriptors, and determines the parameters entering these expressions, which in the simplest case given below are the expansion coefficients of a linear combination. Specifically, the total system energy is written as,

$$E= \sum_{i}{E}_{i},$$

where again i labels the different atoms. Within each atomic cluster, the energy is then expressed in terms of the local atomic descriptors Bi,k. In the simple case of a linear-regression scheme this energy dependence would assume the form

$${E}_{i}={\beta }_{0}+\sum_{k=1}^{K}{\upbeta }_{k}{B}_{i,k}$$

The corresponding formula for the forces reads as

$${{\varvec{F}}}_{j }= -{\nabla }_{j}E= -{\nabla }_{j}\sum_{i}{E}_{i}=-\sum_{i}{\sum _{k=1}^{K}}{\upbeta }_{k}{\nabla }_{j}{\mathrm{B}}_{i,k}$$

Of course, the complexity of the approach is hidden in the functional form of the coefficients Bi,k.

The general paradigm presented above is adopted by all existing approaches, which are distinguished by their descriptors of the atomic structures and their regressors, i.e., their approaches for determining the parameters characterizing the MLP, although NNPs follow a different route to determine the parameters entering the MLP.

Applications of MLPs

In general, the breathtaking progress in the development of new approaches and tools enabling generation of efficient and accurate MLPs has been complemented by an ever-increasing number of applications to different kinds of materials including multinary metallic alloys, semiconductors, oxides, and liquids. Of course, this is due in a large part to the fact that MLPs are general in terms of the tractable systems and free of any notions like atomic charges, bonds, or magnetic moments. MLPs have been demonstrated to work well for large length and time scale MD simulations of 109 atoms for 1 ns [69]. Atomistic simulations with MLPs are found to reliably predict material phenomena and material properties of interest to scientific and engineering applications. Such properties include elastic moduli, phase transitions, phase diagrams, thermal conductivity, radiation damage, dislocations behavior, fracture, diffusion, lattice expansion, and phonon spectra [40, 54,55,56, 70,71,72,73,74,75,76,77,78,79,80,81]. Excellent overviews of the field focusing on both advances in method development and applications have been recently presented by Schmidt et al. as well as by Zuo et al. [82, 83].

The use of MLPs for the study of metallurgical phenomena, as exemplified in Fig. 4 and Fig. 5, has evolved quickly because traditional interatomic potentials of the EAM and MEAM type have often been unable to deal with important classes of problems. For instance, (i) few EAM potentials properly represent the structure and motion of the crucial screw dislocations in body-centered-cubic metals, (ii) EAM/MEAM potentials are often not quantitatively accurate for metal alloys due to the way alloying effects can be included, and (iii) such potentials often struggle to show physical behavior at the tips of sharp cracks. All these examples, and more, precisely encompass the kinds of systems and problems that are at the core of critical metallurgical issues in technologically important alloys.

Figure 4
figure 4

Adapted from Ref. [84]

Atomistic simulation of strengthening in Al-6xxx alloys by nanoscale Mg5Si6 β” precipitates having a 22-atom monoclinic unit cell, using a near-chemically-accurate NNP for the Al-Mg-Si system [84]. Only non-fcc atoms in a slice of crystal containing the dislocation slip plane are shown. For the same precipitate and orientation, an Al edge dislocation (gray band) cuts (top) through the precipitate when sheared to the right but loops (bottom) around the precipitate when sheared to the left. The asymmetry arises due to the asymmetry of the precipitate shearing energy and the sign of the residual misfit stresses in the precipitate. Accurate simulations require accurate representation of (i) the Al matrix and its dislocations, (ii) the precipitate lattice, elastic, and shearing properties, and (iii) the precipitate/Al interfaces. Execution at scales relevant to experiments (precipitate spacing of ~300A, in full 3d samples) is far beyond the capabilities of first-principles methods. These features make MLPs essential.

Full size image
Figure 5
figure 5

(a) Competing slip systems for <a> Burgers vector screw dislocation motion in hcp Zr. A Zr NNP predicts (b) the prismatic dislocation to be the preferred (stable) system, (c) the pyramidal dislocation to be metastable with energy 7 meV/A higher than the prism screw, and (not shown) the basal dislocation unstably transforming into the prism core. All these essential features of plastic slip in hcp Zr agree with experiments. In small DFT cells, the prism -pyramidal energy difference is only 3.2+/-1.6 meV/A and the NNP predicts 3.8 meV/A in good agreement but both much smaller than the large-size value of 7 meV/A, demonstrating the need to study these problems at scales far beyond what is feasible in DFT. Color contours show the Nye Tensor of the plastic displacement associated with each atomistic core structure. Adapted from [76].

Full size image

A key feature in MLP development for metals is an appropriate scope of training structures. Metallurgical performance is controlled by the motion and interaction of defects (dislocations, cracks, grain boundaries, vacancies, precipitates) in the material. Thus, training structures that are limited to bulk properties (lattice constants, cohesive energies, elastic constants, phonon spectra) plus a few simple defects (vicinal surface energies, vacancy formation energy, possible a stacking fault energy) are usually not at all sufficient to create an MLP that is “transferable” to complex crystalline defects, their interactions, and their behavior in alloys. The form of the MLP (neural network, GAP, SNAP, moment-tensor potential, etc.) may be less important than the database of training structures, although each type of MLP has advantages and disadvantages in terms of flexibility, generality, accuracy, and computational cost.

Of course, listing all published work on applications of MLPs to real materials would be far beyond this Introduction and so we mention only a few selected publications with several illustrated in Fig. 6. Of these applications a large focus is given to metals and metallurgy where MLPs have been highly successful. Nonetheless, the types of computations such as phonons, thermodynamics, and phase stability are applicable to all materials.

Figure 6
figure 6

(a) HCP (top) vs BCC (bottom) MD predicted phonons at 300 K and 1188 K, respectively, using a GAP by Qian et al. [72]. The BCC phonons illustrated with spectral energy density (SED) analysis exhibit large anharmonicities that correspond to a phase instability at lower temperatures. (b) Statistical performance of multinary SNAPs of Li et al. for metallurgy studies (reproduced from [90] according to the Creative Commons License (https://creativecommons.org/licenses/by/4.0/)). (c) Exemplar strong defect prediction performance of a GAP for W that Byggmästar et al. developed for radiation damage studies [73]. The DFT accurate GAP for W was applied to radiation damage cascade simulations to identify a preference for spherical void formation due to radiation damage. (d) Thermal expansion of the Eshet et al. NNP for Na illustrated as strongly trending with experiment [86].

Full size image

Seko et al. used compressed sensing techniques to reproduce energies, forces, and stress tensors, thus enabling the prediction of properties such as lattice constants and phonon dispersions for ten elemental metals, namely Ag, Al, Au, Ca, Cu, Ga, In K, Li, and Zn [85]. Eshet et al. used a neural network approach to create an MLP to simulate the high-pressure behavior of solid and liquid sodium [86] and Jose et al. demonstrated the performance of neural network potentials for solid copper [87]. Kruglov et al. simulated the density of solid and liquid Al and computed the phonon density of states, entropy, and melting temperature of solid Al in excellent agreement with experimental data and results from ab initio calculations [88], whereas Botu et al. constructed an MLP for elemental Al using only forces calculated for reference atomic structures including bulk Al with lattice expansion and contraction, surfaces, isolated clusters, point defects, grain boundaries, and dislocations resulting in several millions of local configurations [89]. Zong et al. used an MLP to calculate the elastic and vibrational properties as well as the phase stability of Zr [71], whereas Qian et al., using a Gaussian Approximation Potential, focused especially on the β-phase of Zr, which displays unstable phonon modes at 0 K, and were able to attribute this instability to a double well of the potential energy surface with a maximum at the equilibrium bcc structure, which vanishes at high temperatures due to dynamical averaging of the low-symmetry minima at elevated temperatures [72]. Radiation damage and defects in tungsten were studied by Byggmästar et al. also using GAP, who found excellent agreement of surface properties and defect energies with the DFT results [73].

Worth mentioning is also a study by Li et al., who demonstrated the successful application of MLPs to quaternary metallic alloys [90, 91]. Using a “physically-informed neural network”, which combines machine learning with an imposition of physically-motivated forms for underlying functions appearing in the potential, Mishin et al. were able to accurately capture the structure and motion of the key screw dislocation in bcc Ta [92], while the GAP potential for Fe [93] and the Moment Tensor Potential for W [94] showed similar physical results. These, and other studies, are enabling some new understanding of plasticity in this important new class of alloys where widely used EAM potentials show the usual artifacts for screw dislocations. A neural network approach to hexagonally close-packed Ti and Zr elemental metals showed broadly good behavior for the many different dislocation slip systems (basal, prismatic, pyramidal) in hcp crystals [95]. Using a neural network framework and putting strong emphasis on an extensive and metallurgically-relevant training dataset to develop a series of MLPs of increasing alloy complexity (Al, Al–Cu, Al–Mg–Si, Al–Mg–Cu, Al–Mg–Cu–Zn) Marchand and Curtin were able to capture dislocations, solutes, precipitates, etc. that are now enabling atomic-scale study of technological Al alloys (Al-2xxx, Al-5xxx, Al-6xxx, and Al-7xxx families) [96]. Neural network potentials for Mg [75] and Zr [76] revealed unexpected plastic flow behavior that rationalizes experimental observations [97].

While initial applications of MLPs focused on metals and alloys, the approach has more recently also been applied to a range of other systems including solids, liquids, and molecules. Deringer and Csányi generated a Gaussian Approximation Potential for amorphous carbon from ab initio molecular dynamics trajectories of liquid structures at temperatures up to 9000 K [98], whereas Bartók et al. used the same approach to train a model of Si surfaces, which reproduced the 2 × 1 reconstruction of the Si(001) surface with the correct tilt angle of the surface dimers as well as the famous Si(111)− 7 × 7 reconstruction, which are inaccessible to the traditional classical interatomic potentials or forcefields [62]. Shaidu et al. demonstrated the ability of a neural network potential to calculate elastic and vibrational properties of diamond, graphite, and graphene as well as the phase stability and structures of a wide range of crystalline and amorphous phases in good agreement with ab initio data [70], while Babaei et al. investigated phonon transport in crystalline Si with and without vacancies likewise using GAP and found very good agreement with results obtained from DFT calculations but much larger errors when classical forcefields were applied [74]. Ghasemi et al. presented an efficient approach to capture long-range electrostatic effects in clusters of NaCl and to combine them with short-range terms based on local ordering [99].

Among applications to molecules and molecular systems is work by Jose et al., who demonstrated the performance of neural network potentials for methanol molecules [87] as well as by Brockherde et al. for small molecular systems including the internal proton transfer in malonaldehyde (C3O2H4) [100]. MLPs have also been used to study proton transport reactions in large molecules. Jinnouchi et al. developed an MLP trained with on-the-fly ab initio MD to study proton conductivity mechanisms via charge transfer (see Fig. 7 for illustrative example in liquid H2O) in dry and hydrated Nafion polymer systems [101]. Water, arguably the most important liquid on earth, has also been a focus of MLP studies [102, 103]. Zhang et al. have computed the phase diagram of H2O as a function of pressure and temperature using a combination of deep neural network MLPs, capturing the major liquid and solid phases to within reasonable quantitative agreement with experiment [77]. More recently, Singraber et al. performed molecular dynamics simulations for water using a high-dimensional neural network potential [104,105,106], whereas Ko et al. demonstrated the validity of such neural networks for a wide range of organic molecules and metal cluster [107].

Figure 7
figure 7

Illustration of proton charge transfer between water molecules, right molecule to left in inset, using a SNAP generated from GGA-BLYP ab initio MD calculations. The use of SNAP demonstrates that MLPs may be reactive in general not only the deep neural network potentials common for chemistry applications.

Full size image

Finally, phase stability was addressed by Sosso et al., who created a neural network potential, which allowed an accurate description of the liquid, crystalline, and amorphous phases of the phases change material GeTe [108] as well as by Artrith and Urban, who successfully constructed an MLP for TiO2 using an artificial neural network approach, which in addition to correctly describing the phase stability of rutile, anatase, and brookite allowed predictions about the stability of high-pressure columbite and baddeleyite phases [109]. Artrith et al. also predicted stability of amorphous structures in the system LixSi1-x using a total of about 45,000 reference structures to train an artificial neural network (ANN) potential [110].

Overview of this Focus Issue

The dynamic development of the field and the excitement of many researchers assimilating the new ideas and applying the various approaches is well reflected by this Focus Issue. A representative selection of methods for generating machine-learned potentials is employed to describe the properties of an assortment of material systems including metals and alloys, oxides, silicates, semiconductors, graphanol, liquid silicon, and superionic conductors. Owing to the still limited experience with the various approaches, some of the contribution put emphasis on comparing different methods or different parameter settings and their impact on the final results as well as on the efficiency of generating and applying the respective MLPs. Others present new approaches.

Meziere et al. focus on an efficient creation of training sets for machine-learned potentials [111]. In doing so, they present a small-cell training approach, which is based on the idea of systematically extending the training set starting from unit cells with only a few atoms and moving on to larger and larger cells. In each step the generated MLP is tested on the structures of the next-largest cell size and the process is stopped if the latter are successfully described by the actual MLP. The approach is applied to zirconium and its hydrides.

Zhou et al. reflect on the known transferability of physically motivated forcefields such as the Embedded Atom Method beyond the portion of configuration space defined by the training data and the difficulties of MLPs to provide a similar transferability and stability [112]. This leads them to propose a modification of a Neural Network Potential by including information from an EAM potential to bound the energies of the former. The validity of their approach is demonstrated with zinc as an example.

Being interested in proton exchange membranes operating a higher temperature and low humidities, Achar et al. generate a deep-learning neural network potential to investigate double-sided graphanol, the hydroxyl functionalized form of graphene, which is the hydrogenated version of graphene [113]. Generation of the potential involved active learning to improve its robustness on extending the configuration space. Extensive tests with systems of up to 30,000 atoms demonstrated the validity of the potential to predict, e.g., thermal fluctuations and self-diffusivities with high efficiency and near-DFT accuracy.

Liang et al. use a semi-automated workflow ranging from high-throughput DFT calculations, efficient fitting and validation procedures including active learning and uncertainty indication to demonstrate the accuracy of the Atomic Cluster Expansion (ACE) with a study on Pt–Rh nanoparticles, which play an important role in catalysis [114]. Applying the generated potential in MD simulation at elevated temperatures they are able to show that experimentally observed thick Pt shells covering a Rh core are kinetically stabilized rather than being thermodynamically stable.

Distinguishing between the local atomic environments (descriptors) and the models (regressors) as the main ingredients for generating MLPs, Rohskopf et al. focus on the impact of the latter on the validity of an MLP for use in subsequent calculations of materials properties [115]. In doing so, they use models of different complexity ranging from linear regression to non-linear neural networks and investigate solid and liquid phases of silicon, GaN, and the superionic conductor Li10Ge(PS6)2 (LGPS). This allows the authors to discuss the specific advantages of linear and non-linear models if combined with different descriptors.

Semba et al. perform ab initio MD simulations with on-the-fly learning to generate an MLP for the investigation of the impact of oxygen and hydrogen on the structural and electronic properties of interfaces between amorphous and crystalline silicon [116]. This approach enables them to examine in-gap states that could influence the performance solar cells including such interfaces.

Using a machine-learned potential, Sotskov et al. present an algorithm for on-lattice crystal structure prediction, which provides a route for efficient high-throughput discovery of multicomponent alloys [117]. The algorithm is based on the systematic addition of atoms on the lattice sites of a supercell and calculation of the total energies in each step using an MLP. The approach is validated with the binary, ternary, and quaternary alloys of Nb, Mo, Ta, and W, for which new stable structures are found.

With an application focus, Jain et al. use an Al–Mg–Si NNP to study the natural (room temperature) aging in the Al-6xxx alloys widely used in the automotive industry [118]. They reveal that the origin of significant retardation of natural aging is the trapping of vacancies (responsible for mediating transport of solute elements in the alloy) in small 10–14 atom Mg-Si clusters. While long speculated to be the case, direct experimental evidence is difficult and hence the use of a near-chemically-accurate NNP is enabling insight into critical aspects of alloy processing.

Hill and Mannstadt perform a systematic comparison of different machine-learned potentials by investigating predicted structures, thermal expansion coefficients, and ionic conductivities of α- and β-eucryptite, which attracted interest in the glass industry due to their very low thermal expansion [119]. Despite significant differences of the fit errors produced by different methods calculated cell parameters and thermal expansion coefficients are of similar quality.

Perspectives

Insight into mechanisms and the quantitative prediction of materials properties can be considered as main goals of computational materials science. To this end, reliable simulations are needed that reach from macroscopic engineering scales to atomic-scale phenomena. In this context, MLPs are beginning to play a critical role in taking the accuracy of ab initio quantum mechanical computations from the scale of models with a few hundred atoms and sampling of a few thousand configurations to the level of potentially millions of atoms in millions of configurations. The linear scaling of simulations with MLPs and their computational parallelization are additional benefits of great practical value.

Current applications of MLPs, as shown in the present Focus Issue, clearly demonstrate the exciting possibilities of this approach but they also expose a number of remaining challenges. It is probably fair to say that the representative sampling of the configuration space in the construction of training sets is of critical, if not the most urgent, importance. Possibly for this reason so far, the most successful applications of MLPs have been in areas where the configurational space is reasonably well confined. For example, considering dislocations or crack formations in an fcc metal, most of the atoms in models of such systems are in the atomic environment of the perfect lattice and only a relatively small number of atoms are in different surroundings. Hence, useful training sets can be constructed from unit cells of the dominant crystal structure augmented by models representing atoms at surfaces, dislocation cores, and grain boundaries. In addition, information about other lattice types, e.g., hcp and bcc is needed to train an MLP.

The situation is dramatically different considering, for example, organic liquids, and polymers. The absence of any translational periodicity is aggravated by the fact that there are numerous ways to break bonds. Hence, training an MLP for such systems is exceedingly challenging. Even if an MLP exhibits amazingly accurate reproduction of the reference energies and forces, practical applications of such an MLP in long molecular dynamics simulations as needed to compute liquid properties such as diffusion and viscosity encounter stability problems.

Generality and transferability of MLPs is another important issue which can be expected to drive research and development efforts. A recently published work [120] shows the development of a general neural network based MLP using over 30 million DFT calculations as training set. The universality of this approach seems to be promising, but the robustness and predictive power of this potential remains to be seen.

The accuracy of MLPs hinges on the quality of the underlying training set. While standard DFT calculations using exchange–correlation potentials such as the GGA in the form of the Perdew, Burke, Ernzerhof are extremely useful, in final analysis this type of approximation to the many-body problem can lead to significant errors in predicted properties, for example binding energies of molecules on surfaces. To overcome the difficulties originating from this dependence on the accuracy of the underlying quantum mechanical calculations, methods such as Δ-learning [56] are extremely interesting. At the same time, the need for highly accurate yet computationally efficient ab initio calculations provides a stimulus to advance the field of electronic structure theory.

Given the relatively early stages of MLP technology and its undeniable successes, we can expect that this field will be very active for many years to come. Systematic applications to an increasingly wider class of systems and applications will move the field forward. MLPs are poised to become an indispensable part in the arsenal of methods available to the materials modeling community. As editors of this Focus Issue we hope that the selection of articles will provide a better appreciation of the current capabilities as well as showing the challenges, thus stimulating further research of this exciting field.