Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions

doi:10.1016/j.acha.2005.07.005

Applied and Computational Harmonic Analysis

Volume 21, Issue 1, July 2006, Pages 31-52

https://doi.org/10.1016/j.acha.2005.07.005 Get rights and content

Abstract

We describe a simple scheme, based on the Nyström method, for extending empirical functions f defined on a set X to a larger set $\bar{X}$ . The extension process that we describe involves the construction of a specific family of functions that we term geometric harmonics. These functions constitute a generalization of the prolate spheroidal wave functions of Slepian in the sense that they are optimally concentrated on X. We study the case when X is a submanifold of $R^{n}$ in greater detail. In this situation, any empirical function f on X can be characterized by its decomposition over the intrinsic Fourier modes, i.e., the eigenfunctions of the Laplace–Beltrami operator, and we show that this intrinsic frequency spectrum determines the largest domain of extension of f to the entire space $R^{n}$ . Our analysis relates the complexity of the function on the training set to the scale of extension off this set. This approach allows us to present a novel multiscale extension scheme for empirical functions.

References (13)

M. Abramowitz et al.
Handbook of Mathematical Functions
(1965)
N. Aronszajn
Theory of reproducing kernels
Trans. Amer. Math. Soc.
(1950)
Y. Bengio, J.-F. Paiement, P. Vincent, Out-of-sample extensions for LLE, isomap, MDS, eigenmaps, and spectral...
M. Belkin, P. Nyoigi, V. Sindhwani, On manifold regularization, in: Proc. 10th Int. Workshop on Artificial Intelligence...
H. Wackernagel
Multivariate Geostatistics
(2003)
I.J. Schoenberg
Metric spaces and completely monotone functions
Ann. of Math. (2)
(1938)

There are more references available in the full text version of this article.

Cited by (215)

Nonlinear dimensionality reduction then and now: AIMs for dissipative PDEs in the ML era
2024, Journal of Computational Physics
This study presents a collection of purely data-driven workflows for constructing reduced-order models (ROMs) for distributed dynamical systems. The ROMs we focus on, are data-assisted models inspired by, and templated upon, the theory of Approximate Inertial Manifolds (AIMs); the particular motivation is the so-called post-processing Galerkin method of Garcia-Archilla, Novo and Titi. Its applicability can be extended: the need for accurate truncated Galerkin projections and for deriving closed-formed corrections can be circumvented using machine learning tools. When the right latent variables are not a priori known, we illustrate how autoencoders as well as Diffusion Maps (a manifold learning scheme) can be used to discover good sets of latent variables and test their explainability. The proposed methodology can express the ROMs in terms of (a) theoretical (Fourier coefficients), (b) linear data-driven (POD modes) and/or (c) nonlinear data-driven (Diffusion Maps) coordinates. Both Black-Box and (theoretically-informed and data-corrected) Gray-Box models are described; the necessity for the latter arises when truncated Galerkin projections are so inaccurate as to not be amenable to post-processing. We use the Chafee-Infante reaction-diffusion and the Kuramoto-Sivashinsky dissipative partial differential equations to illustrate and successfully test the overall framework.
UQpy v4.1: Uncertainty quantification with Python
2023, SoftwareX
This paper presents the latest improvements introduced in Version 4 of the UQpy, Uncertainty Quantification with Python, library. In the latest version, the code was restructured to conform with the latest Python coding conventions, refactored to simplify previous tightly coupled features, and improve its extensibility and modularity. To improve the robustness of UQpy, software engineering best practices were adopted. A new software development workflow significantly improved collaboration between team members, and continuous integration and automated testing ensured the robustness and reliability of software performance. Continuous deployment of UQpy allowed its automated packaging and distribution in system agnostic format via multiple channels, while a Docker image enables the use of the toolbox regardless of operating system limitations.
From partial data to out-of-sample parameter and observation estimation with diffusion maps and geometric harmonics
2023, Computers and Chemical Engineering
A data-driven framework is presented, that enables the prediction of quantities, either observations or parameters, given sufficient partial data. The framework is illustrated via a computational model of the deposition of Cu in a Chemical Vapor Deposition (CVD) reactor, where the reactor pressure, the deposition temperature and feed mass flow rate are important process parameters that determine the outcome of the process. The sampled observations are high-dimensional vectors containing the outputs of a detailed CFD steady-state model of the process, i.e. the values of velocity, pressure, temperature, and species mass fractions at each point in the discretization. A machine learning workflow is presented, able to predict out-of-sample (a) observations (e.g. mass fraction in the reactor), given process parameters (e.g. inlet temperature); (b) process parameters, given observation data; and (c) partial observations (e.g. temperature in the reactor), given other partial observations (e.g. mass fraction in the reactor). The proposed workflow relies on two manifold learning schemes: Diffusion Maps and the associated Geometric Harmonics. Diffusion Maps are used for discovering a reduced representation of the available data, and Geometric Harmonics for extending functions defined on the discovered manifold. In our work a special use case of Geometric Harmonics is formulated and implemented, which we call Double Diffusion Maps, to map from the reduced representation back to (partial) observations and process parameters. A comparison of our manifold learning scheme to the traditional Gappy-POD approach is provided: ours can be thought of as a “Gappy DMAPs” approach. The presented methodology is easily transferable to application domains beyond reactor engineering.
Double Diffusion Maps and their Latent Harmonics for scientific computations in latent space
2023, Journal of Computational Physics
We introduce a data-driven approach to building reduced dynamical models through manifold learning; the reduced latent space is discovered using Diffusion Maps (a manifold learning technique) on time series data. A second round of Diffusion Maps on those latent coordinates allows the approximation of the reduced dynamical models. This second round enables mapping the latent space coordinates back to the full ambient space (what is called lifting); it also enables the approximation of full state functions of interest in terms of the reduced coordinates. In our work, we develop and test three different reduced numerical simulation methodologies, either through pre-tabulation in the latent space and integration on the fly or by going back and forth between the ambient space and the latent space. The data-driven latent space simulation results, based on the three different approaches, are validated through (a) the latent space observation of the full simulation through the Nyström Extension formula, or through (b) lifting the reduced trajectory back to the full ambient space, via Latent Harmonics. Latent space modeling often involves additional regularization to favor certain properties of the space over others, and the mapping back to the ambient space is then constructed mostly independently from these properties; here, we use the same data-driven approach to construct the latent space and then map back to the ambient space.
Data-driven control of agent-based models: An Equation/Variable-free machine learning approach
2023, Journal of Computational Physics
We present an Equation/Variable free machine learning (EVFML) framework for the control of the collective dynamics of complex/multiscale systems modeled via microscopic/agent-based simulators. The approach obviates the need for construction of surrogate, reduced-order models. The proposed implementation consists of three steps: (A) from high-dimensional agent-based simulations, machine learning, in particular, non-linear manifold learning (Diffusion Maps (DMs)) identifies a set of coarse-grained variables that parametrize the low-dimensional manifold on which the emergent/collective dynamics evolve. The out-of-sample extension and pre-image problems, i.e. the construction of non-linear mappings from the high-dimensional input space to the low-dimensional manifold and back, are solved by coupling DMs with the Nyström extension and Geometric Harmonics, respectively; (B) having identified the manifold and its coordinates, we exploit the Equation-free approach to perform numerical bifurcation analysis of the emergent dynamics; then (C) based on the previous steps, we design data-driven embedded wash-out controllers that drive the agent-based simulators to their intrinsic, imprecisely known, emergent open-loop unstable steady-states, thus demonstrating that the scheme is robust against numerical approximation errors and modeling uncertainty. The efficiency of the framework is illustrated by controlling emergent unstable (i) traveling waves of a deterministic agent-based model of traffic dynamics, and (ii) equilibria of a stochastic financial market model with mimesis.
Nonlinear Manifold Learning Determines Microgel Size from Raman Spectroscopy
2024, arXiv

View all citing articles on Scopus

¹: Now with Google Inc.

View full text

Geometric harmonics: A novel tool for multiscale out-of-sample extension of empirical functions

Abstract

Handbook of Mathematical Functions

Theory of reproducing kernels

Trans. Amer. Math. Soc.

Multivariate Geostatistics

Metric spaces and completely monotone functions

Ann. of Math. (2)