Physics-Informed Boundary Integral Networks (PIBI-Nets): A Data-Driven Approach for Solving Partial Differential Equations

Partial differential equations (PDEs) are widely used to describe relevant phenomena in dynamical systems. In real-world applications, we commonly need to combine formal PDE models with (potentially noisy) observations. This is especially relevant in settings where we lack information about boundary or initial conditions, or where we need to identify unknown model parameters. In recent years, Physics-Informed Neural Networks (PINNs) have become a popular tool for this kind of problems. In high-dimensional settings, however, PINNs often suffer from computational problems because they usually require dense collocation points over the entire computational domain. To address this problem, we present Physics-Informed Boundary Integral Networks (PIBI-Nets) as a data-driven approach for solving PDEs in one dimension less than the original problem space. PIBI-Nets only require points at the computational domain boundary, while still achieving highly accurate results. Moreover, PIBI-Nets clearly outperform PINNs in several practical settings. Exploiting elementary properties of fundamental solutions of linear differential operators, we present a principled and simple way to handle point sources in inverse problems. We demonstrate the excellent performance of PIBI- Nets for the Laplace and Poisson equations, both on artificial datasets and within a real-world application concerning the reconstruction of groundwater flows.

networks, Data assimilation, Boundary integral equation, Point sources

Introduction and related work
The study of data-driven partial differential equations (PDEs) is an important area of research owing to its potential to provide insights into complex dynamical systems.In this paper, we focus on linear PDEs with constant coefficients which find many applications in areas such as in hydrogeology [Kingsbury (2018)], electromagnetics [Khan and Lowther (2022)] or fluid dynamics [Anderson et al. (2013)].Boundary conditions of PDE models often allow interesting scientific interpretations and ensure the uniqueness of the solution.However, in practice, boundary conditions are often unknown and need to be estimated from noisy measurements.Additionally, most PDEs used in practice have no analytic solutions and can only be solved with numerical methods [Xun et al. (2013)].Recent research has focused on datadriven methods to identify the underlying solution of dynamical systems [Buizza et al. (2022)].These methods integrate data and machine learning techniques to improve the accuracy of the PDE solution.Overall, the study of data-driven PDEs has many practical applications and is an active area of research in mathematics, physics, and engineering.

Data-driven approaches and data assimilation
Data-driven approaches and data assimilation are becoming increasingly important in various fields of science and engineering.In recent years, there has been a growing interest in integrating machine learning with data assimilation [Arcucci et al. (2021)].Physics-informed neural networks (PINNs) [Raissi et al. (2019)] incorporate the governing PDE as constraints during training, while leveraging data to improve accuracy and identify parameters.PINNs have been extensively explored for solving various forward and inverse problems, see e.g.Yang et al. (2021); Wang et al. (2022).Based on PINNs, several follow-up papers as well as review articles followed, such as Cuomo et al. (2022) and Sharma et al. (2023).On the other hand, interpolation techniques and Gaussian processes, as demonstrated in Kingsbury (2018) and Nussbaumer et al. (2021), are commonly used in data assimilation approaches, but often at the expense of neglecting information of the underlying PDE.However, most data-driven approaches require a dense grid over the entire computational domain.

Approaches with boundary integral equations
In contrast to PINN approaches that require dense collocation points over the entire domain, methods based on boundary integral equations only require dense collocation points at the boundary.This reduces the problem dimension by one.Both, classical methods such as the boundary element method (BEM) described in Steinbach (2007) and deep learning approaches [Sun et al. (2023), Lin et al. (2023)] require boundary conditions as a prior.However, in many real-world applications, boundary conditions are commonly not known and need therefore be estimated as in Villalpando-Vizcaino et al. (2021).Nonetheless, these methods cannot deal with data measurements.We present PIBI-Net as a data-driven approach based on integral equations to target this challenge.
In section 2 we provide a brief introduction to boundary integral equations, which constitute the theoretical foundation of the PIBI-Net.In particular, we present the boundary integral equations for the Laplace as well as the Poisson equation, and derive an analytical formula for point sources.We showcase PIBI-Net with synthetic data for the Laplace equation and with real-world data for the Poisson equation in section 3 Lastly, we compare PIBI-Net to PINNs in section 4 and summarise our findings in section 5.

PIBI-Net is based on the boundary integral representation of linear PDEs with constant coefficients
In this equation, L represents the PDE operator, u the solution, and f the source term.This section demonstrates PIBI-Net exemplified for the Laplace and Poisson equations In this case, , u is a real-valued mapping u : Ω ⊆ R n → R, and g represents the unknown boundary condition function with unknown parameters α, β ∈ R. The objective is to find a solution u that satisfies equation (2) within the domain Ω, given certain data measurements.
In this framework, it is therefore necessary to determine the boundary density function g.
In subsection 2.1 we recall some foundations from potential theory.Subsequently, we present the boundary integral equation in subsection 2.2, which applies to both the Laplace and the Poisson equations.Furthermore, we showcase how to handle point sources in inverse problems for the nonhomogeneous Poisson equation.Lastly, we introduce in subsection 2.3 our PIBI-Net model and provide a detailed description of its architecture.PIBI-Net enables us to learn the boundary density function g while assimilating the solution to data measurements.

Foundations from potential theory
In order to lay a foundation for the architecture of PIBI-Net, we review some key concepts from potential theory.According to the Malgrange-Ehrenpreis theorem [Malgrange (1956); Ehrenpreis (1954)], every linear PDE (1) with constant coefficients has a fundamental solution, which is commonly known as Green's function.The fundamental solution G for a linear partial differential operator L is defined as where δ denotes the Dirac's-delta function.In particular, for the Laplace or Poisson equation (2) in dimension d, the fundamental solution is given by where x, y ∈ Ω and ∥ • ∥ denotes the Euclidean norm.Fundamental solutions for various PDEs, including Laplace, Poisson, heat, wave, Stokes, and Helmholtz equations, are available in Steinbach (2007), Kesavan and Vasudevamurthy (1985) as well as in Misljenovic (1982).The gradient of the fundamental solution ( 4) with respect to y on the boundary ∂Ω can be analytically derived with the chain rule as (5) Let z and y denote points on the boundary ∂Ω, n represents the outwarddirected normal vector, and ∂Ω ( * ) ds y indicates the surface integral over the boundary of Ω.For the Laplace and Poisson equations, we recall from Steinbach (2007) the single layer potential defined as which exhibits the property on the boundary ∂Ω And the double layer potential is defined as with the property on the boundary ∂Ω given by The outer normal derivatives can be easily computed by the dot product as • n, where ( * ) stands for g or G.We use these two potentials in the next subsection to formulate the boundary integral representation.

Boundary integral equation for homogeneous and non-homogeneous
PDEs with point sources In the context of non-homogeneous PDEs, we introduce the source layer potential and present the boundary integral representation formula for the Laplace or Poisson equations (2).For non-homogeneous PDEs as the Poisson equation, an external force f needs to be considered.We define the source layer potential as Recalling Steinbach (2007), the boundary integral representation formula based on ( 4) and ( 6)-( 9) for any solution of the Laplace or Poisson equation ( 2) is given by In the case of the homogeneous Laplace equation, where the right-hand side in ( 1) is zero, the source layer potential vanishes.
A point source can be represented as a Dirac's delta function with a magnitude c 0 ∈ R >0 .Consequently, M point sources represented by f at locations y 0 1 , . . ., y 0 M can be expressed by a sum of Dirac's delta functions with magnitudes c 0 1 , . . ., c 0 M .Thus, the integral (10) over the entire domain Ω simplifies to the point source layer potential by using the properties of Dirac's delta function.In the implementation of PIBI-Net, we employ a multi-layer perceptron (MLP) to learn an approximation ĝ of the boundary density function and, if present and unknown, the parameters for f .The hat indicates the surrogate parameters and functions.The input variable x needs to be distinguished between points x ∂Ω on the boundary and points x Ω inside the domain.During training, x Ω contribute solely to the boundary integral representation (11), while the boundary points x ∂Ω serve as input to the MLP.The MLP is exchangeable with any suitable network, such as a fully connected neural network or a Siren-Net [Sitzmann et al. (2020)].Automatic differentiation in common deep learning frameworks like PyTorch [Paszke et al. (2019)] and TensorFlow [Abadi et al. (2016)] enables the MLP to learn both the boundary density function ĝ and its derivative.
Data measurements u data at x data can be located either inside the domain Ω or on the boundary.Consequently, the only points we need to consider during the training phase within the domain are data measurements.The points x ∂Ω on the boundary, that serves as inputs of the MLP during training, have three different purposes: First, if present, as data measurements at x data to ensure the data assimilation of the solution.Second, similar to PINNs, as collocation points x coll to enforce the surrogate of the boundary density ĝ at the boundary ∂Ω.And lastly, as integral points x int of the Monte Carlo integration to approximate the boundary integral representations (11).
In order to train PIBI-Net, the loss function L expressed by is minimised via a gradient-based optimiser like Adam [Kingma and Ba (2014)].Here, û denotes the surrogate of the PDE solution u(x) and λ ∈ R serves as a weighting parameter.We want to point out that the source term f needs to be learnt as an additional parameter only if it is not precisely known.In many real-world applications, we typically have prior knowledge about the number M of point sources and a general understanding of their approximate locations.However, in an inverse problem setting where the point sources are unknown, we only need to learn the magnitudes c 0 1 , . . ., c 0 M and, if necessary, their exact locations y 0 1 , . . ., y 0 M , as stated in (12).However, after the training procedure of PIBI-Net, we still need to compute the boundary integral representation defined by (11) at each point x for which we wish to evaluate the PIBI-Net solution.Despite the additional computation of the boundary integral representation, the use of Monte Carlo integration nevertheless ensures fast evaluation.

Experiments and Results
We showcase the power of the PIBI-Net in comparison to PINNs.For this purpose, we present three experiments to cover both, homogeneous and non-homogeneous PDEs with unknown point source terms.In subsection 3.1 we demonstrate PIBI-Net on a toy example with synthetic data distributed throughout the entire computational domain for the Laplace equation.In this experiment, we show that PIBI-Nets work as well accurately to assimilate data as PINNs.However, PIBI-Nets show even better results compared to PINNs when data is only available in some areas of the computational domain, which we present in subsection 3.2.To cover the relevance of solving data-driven non-homogeneous PDEs with point sources, we apply PIBI-Net to real-world measurements for groundwater flows in subsection 3.3.

Laplace Equation with a distributed data set over the entire domain
We compare PIBI-Net with PINNs for the two-dimensional Laplace equation based on 80 data measurements, and use the finite difference method as a ground truth reference on an equidistant mesh with mesh size ∆x = 0.2.We constructed the data set by 80 randomly sampled measurements from the solution obtained through the finite difference method on Laplace equation ∆u(x) = 0 in Ω := (−1, 1) 2 with the Dirichlet boundary conditions given as As in many real-world applications, we have only given data measurements and treat therefore the boundary conditions as unknowns.
We obtained the results in Figure 2 by training PIBI-Net with 1000 boundary points, which we divided equally into 500 collocation and 500 integration points.To provide a fair comparison, we trained the PINN with 1000 collocations points as well, but in this case sampled over the entire domain.For the PINN, we defined the loss function as ∥û PINN (x data )−u data ∥ 2 2 + λ physics ∥û PINN (x coll )∥ 2 2 , where ûPINN represents the solution obtained by the network and λ physics a weighting factor to balance both loss terms.
In many real-world applications, the actual computational boundary can often be chosen freely.In this experiment, one option would therefore be to take the edges of the box Ω.However, we found that a circle around the origin with a radius of 1.5 is a better choice, as it avoids singularities that would appear at the corners of Ω.In this way, all data measurements lie within our chosen circle.As MLP, we used in both approaches a fully connected neural network with three hidden layers consisting 512 neurons each.In PIBI-Net we achieved the best results with Exponential Linear Unit (ELU) from Clevert et al. (2015), and for the PINN approach with hyperbolic tangent activation functions.We trained the network with Adam [Kingma and Ba (2014)] as a gradient-based optimiser until it converged.For the loss function we used the mean squared error metric.To analyse and compare the accuracy of the two methods, we subtracted pixel-wise the ground truth from the PIBI-Net respectively the PINN solution.The resulting pixel-wise absolute error plot is shown in the last row of Figure 2. Comparing the mean and standard deviation from the pixel-wise absolute error of the two methods over five trained runs, we found that PIBI-Net with a mean error of 0.0348 ± 0.00453 is comparable in accuracy to PINNs with a mean error of 0.04455 ± 0.01138.Exemplified one of the trained runs in Figure 2, we achieved a maximum pixel-wise deviation with PIBI-Net as 0.6397, and with the PINN as 0.6443.

Laplace Equation with a clustered data set close to the domain boundary
In this section we use the same Laplace equation ( 14) as in subsection 3.1 as ground truth solution, but we only take 30 measurement points that are in the area close to the boundary.Thereof, we do not have data measurements in the center area of the domain.Figure 3 presents the results we obtained by the PIBI-Net as well as the PINN approach.We used the same network setup as previously.
As before in the previous toy example in subsection 3.1, we calculated the mean as well as the standard deviation of both approaches over five trained runs.For each training phase, we sampled new data measurements.PIBI-Net achieved an accuracy with a mean error of 0.0269±0.00028,while PINNs had a mean error of 0.1458±0.00269.Figure 3 shows one of the trained runs.

Ground water-level measurements in Memphis aquifer in Steady-state
Steady-state flow of groundwater in a homogeneous aquifer can be described through the Laplace equation.However, in real-world scenarios it is challenging to find areas where this equation is fulfilled owing to the presence of pump wells in many places.In order to handle pump wells correctly, we need to include additional points source terms, which leads to the Poisson equation ( 2).Groundwater level altitudes becomes therefore important for studying historical water-level changes.
In our real-world experiment, we simulated the altitudes of the potentiometric surface of water for the Memphis area, Tennessee US, in fall 2000 based on 64 well measurements.The dataset is based on the water-level altitude measurements presented in Kingsbury (2018).
We used the same training settings as in the previous toy examples from subsection 3.1 but with ReLU activation functions.We added additional learning parameters for the pump wells as sink terms to the experiments for groundwater level altitudes shown in Figure 4, as they are unknown.
In particular, with the assumption that there is a pump well in each well field presented in Kingsbury (2018), we learned 2 × 8 additional parameters with the MLP, one for the location and one for the magnitude of each pump well.We initialised the locations with one of the wells inside.We used the conjugated gradient method to calculate the streamlines and indicate the groundwater flows in Figure 4.The first column shows the results with PIBI-Net in contrast to the PINN in the second column.For these calculations, all data measurements were used.We add the magnitude and the location of the pump wells as additional learning parameters to the approaches.

Discussion
We compared PIBI-Net to PINNs for the Laplace equation in sections 3.1 and 3.2 and showcased the performance of PIBI-Net for real-world observations in subsection 3.3.We have shown that PIBI-Net achieves at least comparable results in terms of accuracy to PINNs.However, we found some advantages as well as limitations.
• We have shown in Figure 3 that PIBI-Net outperforms PINNs in settings, where data is only available in areas close to the boundary.This might be relevant in real-world settings with sparse data sets.
• Since PIBI-Net only requires first-order derivatives, it enables us to use a broader range of MLP models with suitable activation functions.
• Adjusting the lambda hyperparameter in PIBI-Net is still needed to balance between the data and collocation loss terms.This can be challenging and time-consuming.However, this adjustment is also required in the PINN approach.
• As in other data-driven approaches, the solution of PIBI-Net relies strongly on the quality of the observations.If the observations do not cover the important regions for the dynamics, the problem might be ill-posed.Therefore, data-driven approaches cannot compete with solutions that contain more information through boundary conditions.Consequently, neither the PIBI-Net nor the PINN approach can cover, for example, the lower left corner because of missing observations.
• Since data points can lie on the boundary, and both collocation and integral points are located on the boundary ∂Ω as well, a careful selection is required to prevent singularities when they are too close to each other.Our experience has shown that a regular grid over the boundary shows the best results.We assigned every second point as an integral point and the rest as collocation points.By rotating the points slightly in each iteration, this approach can still add some level of randomness.
• As the integral calculation of PIBI-Net is based on the Monte Carlo integration, a reasonable amount of integration points is needed to obtain valid results.On one hand, too few integral points would lead to a wrong calculation of the integral.On the other hand, too many integral points could lead to singularities, as the distance between the collocation and integral points goes to zero.In our experiments, we found that the results with 1000 boundary points evenly divided into 500 collocation and 500 integral points give already good results.
• In our real-world experiment to simulate groundwater altitudes, we presented a setting where the reformulation of point sources given by equation ( 12) is useful and can be used in any physics-informed deep learning approach.However, in the case of non-homogeneous PDEs with non-point sources, we would lose PIBI-Net's advantage of the dimension reduction by one, as the integral needs to be solved over the entire domain.

Conclusion
We have shown that PIBI-Net as a data-driven deep learning approach delivers comparable results in accuracy to a PINN, while requiring only collocation points at the boundary.Even more, PIBI-Net outperforms PINNs in settings where data is clustered in regions close to the boundary, but missing in the middle area of the computational domain.This brings an enormous advantage in many problem settings, as most data-driven real-world applications are in three or four dimensions.PIBI-Net can be used for all dynamical systems that are based on a linear PDE with constant coefficients.
In the two-dimensional toy example, we have shown that the mean absolute error of the PIBI-Net solution falls within the same error range as from the PINN approach.Additionally, in the real-world steady-state groundwater application, we demonstrated that PIBI-Net can also deal with inverse problems by learning the location and magnitude of point sources.For non-point sources, however, an integral over the entire domain must still be solved.With Monte Carlo integration, one would lose the dimension reduction advantage of PIBI-Net.Therefore, further research needs to be done in this direction.
2.3.Physics-Informed Boundary Integral Network (PIBI-Net)PIBI-Net is a deep learning architecture that incorporates observations into a linear PDE with constant coefficients.Its construction is based on the boundary integral representation (11) and the simplified point source layer potential (12) as illustrated in Figure1.

Figure 2 :
Figure 2: Visualisation of the 2D Toy example for solving the Laplace equation using 80 measurements randomly sampled over entire domain Ω from the ground truth solution based on (14).The data measurements are indicated as black dots.We compare PIBI-Net listed in the first column to PINNs in the last column.We display the three-dimensional representations (top row), the projection to the x 1 = x 2 plane (middle row) and the pixel-wise absolute errors with respect to the ground truth (bottom row).

Figure 3 :
Figure 3: Visualisation of the 2D Toy example for solving the Laplace equation (14) using 30 measurements randomly sampled in the area close to the boundary ∂Ω.

Figure 4 :
Figure 4: Visualisation of the water head contour lines based on the well measurements indicated as black dots.The black lines indicate the streamlines of the groundwater flows.The first column shows the results with PIBI-Net in contrast to the PINN in the second column.For these calculations, all data measurements were used.We add the magnitude and the location of the pump wells as additional learning parameters to the approaches.