TOWARDS OVERCOMING THE CURSE OF DIMENSIONALITY IN PREDICTIVE MODELLING AND UNCERTAINTY QUANTIFICATION

This invited presentation summarizes new methodologies developed by the author for performing high-order sensitivity analysis, uncertainty quantification and predictive modeling. The presentation commences by summarizing the newly developed 3 rd -Order Adjoint Sensitivity Analysis Methodology (3 rd -ASAM) for linear systems, which overcomes the “curse of dimensionality” for sensitivity analysis and uncertainty quantification of a large variety of model responses of interest in reactor physics systems. The use of the exact expressions of the 2 nd -, and 3 rd -order sensitivities computed using the 3 rd -ASAM is subsequently illustrated by presenting 3 rd -order formulas for the first three cumulants of the response distribution, for quantifying response uncertainties (covariance, skewness) stemming from model parameter uncertainties. The use of the 1 st - , 2 nd -, and 3 rd -order sensitivities together with the formulas for the first three cumulants of the response distribution are subsequently used in the newly developed 2 nd /3 rd - BERRU-PM (“Second/Third-Order Best-Estimated Results with Reduced Uncertainties Predictive Modeling”), which aims at overcoming the curse of dimensionality in predictive modeling. The 2 nd /3 rd -BERRU-PM uses the maximum entropy principle to eliminate the need for introducing a subjective user-defined “cost functional quantifying the discrepancies between measurements and computations.” By utilizing the 1 st -, 2 nd - and 3 rd -order response sensitivities to combine experimental and computational information in the joint phase-space of responses and model parameters, the 2 nd /3 rd - BERRU-PM generalizes the current data adjustment/assimilation methodologies. Even though all of the 2 nd - and 3 rd -order are comprised in the mathematical framework of the 2 nd /3 rd -BERRU-PM formalism, the computations underlying the 2 nd /3 rd -BERRU-PM require the inversion of a single matrix of dimensions equal to the number of considered responses, thus overcoming the curse of dimensionality which would affect the inversion of hessian and higher-order matrices in the parameter space.


INTRODUCTION
This invited presentation summarizes ongoing work aimed at overcoming the "curse of dimensionality" in sensitivity analysis, uncertainty quantification and predictive modeling.The computation of higher-order sensitivities by conventional methods is limited in practice by the socalled "curse of dimensionality," since the number of such large-scale computations increases exponentially with the order of the response sensitivities.For the exact computation of the 1 st -and 2 ndorder response sensitivities to parameters, the "curse of dimensionality" has been overcome by the Second-Order Adjoint Sensitivity Analysis Methodology (2 nd -ASAM) conceived and developed by Cacuci [1].Generalizing the 2 nd -ASAM, Section 2 of this works presents the Third-Order Adjoint Sensitivity Analysis Methodology (3 rd -ASAM) for linear systems which computes efficiently and exactly expressions of the 3 rd -order functional derivatives ("sensitivities") of a scalar-valued system response with respect to all of the parameters that characterize the respective system.Such responses are often encountered when representing mathematically detector responses and reaction rates in reactor physics problems.The 1 st -, 2 nd -, and 3 rd -order sensitivities computed using the 3 rd -ASAM are incorporated in the new formulas, presented in Section 3.1, for computing the first four cumulants of the response distribution in the phase-space of model parameters.Section 3.2 presents a new mathematical formalism for [predictive modeling, which will be called the "Second/Third-Order Best-Estimated Results with Reduced Uncertainties Predictive Modeling (2 nd /3 rd -BERRU-PM)."Set in the joint phase-space of model responses and parameters, the 2 nd /3 rd -BERRU-PM incorporates experimental and computational information, including the complete set of 1 st -, 2 nd -, and 3 rd -order sensitivities of model responses to model parameters.Thus, the 2 nd /3 rd -BERRU-PM extends the "BERRU Predictive Modeling" methodology presented in [2], thereby generalizing and including, as particular cases, similar formulas used in data adjustment methodologies [3], Bayesian linear statistics [4], and data assimilation [5,6] methodologies.The 2 nd /3 rd -BERRU-PM uses the maximum entropy (MaxEnt) principle to eliminate the need for introducing and "minimizing" a user-chosen "cost functional quantifying the discrepancies between measurements and computations."Incorporating correlations, including those between the imprecisely known model parameters and computed model responses, the 2 nd /3 rd -BERRU-PM also provides a quantitative metric, constructed from sensitivity and covariance matrices, for determining the degree of agreement among the various computational and experimental data and helping eliminate discrepant information.Section 4 summarizes the significance of this work's novel results in the quest to overcome the curse of dimensionality in sensitivity analysis, uncertainty quantification and predictive modeling.

THIRD-ORDER ADJOINT SENSITIVITY ANALYSIS METHODOLOGY (3 rd -ASAM) FOR LINEAR SYSTEMS
The system's response considered in this work is a scalar-valued nonlinear functional of the forward state-function (e.g., the particle flux in a neutron/gamma transport equation . Such functionals represent detector responses and/or reaction rates in reactor physics problems, and will be denoted in inner-product form as , where ; S Κ ΅ denotes a suitably differentiable function of its arguments.The inner product has the form , and the forward function Κ ¡ is the solution of the following operator equations, in which the equality hold in the weak (distributional) sense: The quantities appearing in Eq. ( 1) are defined as follows: (i) dimensional column vector whose components are the physical system's imprecisely known parameters, which are subject to uncertainties; (ii) † 1 , , \ denotes the x N -dimensional phase-space position vector, defined on a phase-space domain denoted as denotes a u N -dimensional column vector whose components represent the system's dependent variables (also called "state functions"); (iv
Due to space limitations, only the final form of the 1 st -, 2 nd -, and 3 rd -order response sensitivities will be presented in this work.Thus, the 1 st -order response sensitivity , has the following expression: where 1 P denotes the bilinear concomitant on w: x , and the 1 st -level adjoint function ȥ [ is the solution of the following 1 st -Level Adjoint Sensitivity System (1 st -LASS): , where the subscript "A" indicates "adjoint".The expression of the 2 nd -order sensitivity of with respect to parameters i D and j D is as follows: ; , (5) subject to adjoint boundary conditions denoted as ; ; ; ; , .
The exact computation of all of the partial second-order sensitivities requires at most N D large-scale (adjoint) computations using the 2 nd -LASS, rather than

2
O N D large-scale computations as would be required by forward methods.If the 2 nd -LASS is solved N D -times, the "off-diagonal" 2 nd -order mixed sensitivities R Κ Μ Μ ¡ Μ ¡ ΅ will be computed twice, using distinct 2 nd -level adjoint functions, thereby providing an independent intrinsic numerical verification that the 1 st -and 2 nd -order sensitivities are computed accurately.In practice, it is useful to prioritize the computation of the 2 ndorder sensitivities by using the rankings of the relative magnitudes of the 1 st -order sensitivities as a "priority indicator": the larger the magnitude of the relative 1 st -order sensitivity, the higher the priority for computing the corresponding 2 nd -order sensitivities.Also, since exactly-vanishing 1 st -order sensitivities may indicate critical points of the response in the phase-space of model parameters, it is also of interest to compute the 2 nd -order sensitivities that correspond to vanishing 1 st -order EPJ Web of Conferences 247, 00002 (2021) PHYSOR2020 https://doi.org/10.1051/epjconf/202124700002sensitivities.Information provided by the 1 st -order sensitivities might indicate which 2 nd -order sensitivities could be neglected.The 3 rd -order partial sensitivity of the response to the model parameters has the following expression: ; ; ; ; , , 1,..., ; 1,..., ; 1,..., , where 3 P denotes the bilinear concomitant on w: x , and the 3 rd -level adjoint functions ; ; , The boundary conditions for the adjoint functions Note that solving Eq. ( 7) would be performed by using the same forward and adjoint solvers (i.e., computer codes) as used for solving the original forward and adjoint systems, except that the right-sides of the respective solvers would have different as "sources," thereby minimizing any need for new code development.Using the 3 rd -LASS enables the specific computation of the 3 rd -order sensitivities in the priority order set by the user, so that only the important 3 rd -order partial sensitivities Information provided by the first-and second-order sensitivities might indicate which 3 rd -order sensitivities could be neglected.The expressions provided above in Eqs. ( 2) through (9) are particular cases of those provided in [7].

2 nd /3 rd -BERRU-PM: "SECOND/THIRD-ORDER BEST-ESTIMATED RESULTS WITH REDUCED UNCERTAINTIES PREDICTIVE MODELING WITH UNCERTAINTY ANALYSIS IN THE JOINT PHASE-SPACE OF RESPONSES AND PARAMETERS
Consider a computational model that comprises a total of N D parameters, denoted as i D , 1 V , of i D , is defined as follows: between two parameters i D and j D , is defined as follows (iii) The 3rd-order moment, 3 ijk P , of the multivariate parameter distribution function p Į , and the 3rd-order parameter correlation, ijk t , respectively, are defined as follows: The 4th-order moment, 4 ijkl P , of the multivariate parameter distribution function p Į , and the 4th-order parameter correlation, ijkl q , respectively, are defined as follows:

Third-Order Expressions for the Expected Value, Covariance and Skewness of Response Multivariate Distributions in Parameter Space
Considering response sensitivities up to 3 rd -order, it has been shown [1] that the expressions of the first three cumulants of the distribution in the phase-space of model parameters of r N computed (i) The expected (mean) value, denoted as (ii) The covariance, denoted as https://doi.org/10.1051/epjconf/202124700002 (iv) The third-order cumulant for three responses, wD wD wD wD w wD ¦¦¦ ¦¦¦¦ The first-order sensitivities contribute the leading terms to the 2 nd -, 3 rd -, and 4 th -order moments of the response distribution, thus providing the leading contributions to the responses covariances, skewness, and kurtosis.Therefore, obtaining the exact and complete set of first-order sensitivities of responses to model parameters is of paramount importance for any analysis of a computational model.The 2 ndorder sensitivities contribute the leading correction terms to the response's expected value (causing it to differ from the response's computed value), and also contribute to the response variances, covariances, and skewness.In particular, if the parameters follow a normal (Gaussian) multivariate distribution, the 2 nd -order sensitivities contribute the leading terms to the response's 3 rd -order cumulant.Thus, neglecting the 2 nd -second-order response sensitivities to normally distributed parameters would nullify the 3 rd -order response correlations and hence would nullify the skewness of a response.The 3 rd -order sensitivities contribute the 3 rd -and 4 th -order (in parameter standard deviations) terms to the responses' mean values and covariances.The above formulas will be incorporated into the 2 nd /3 rd -BERRU PM framework presented in the next Sub-section.

2 nd /3 rd -BERRU-PM Framework
Consider that r N system responses that have been experimentally measured.For the mathematical derivations to follow, it is convenient to denote each measured response as m i r , and to consider them as being the components of the r N -dimensional column vector m r defined as . The covariance matrix of measured responses is defined as Į , given by Eq. (10), are considered to be the components of the following vector of "expected values of the computed response" . The response covariances defined in Eq. ( 11) are considered to be the components of a The covariances between the computed responses and the model parameters defined in Eq. ( 12) are considered to be the components of an . The a priori information about the model parameters, computed and measured responses provided in the foregoing can be conveniently summarized by considering that the physical system under consideration is described mathematically by a obeying an unknown joint multivariate distribution function p , Į U in the combined phase-space of responses and parameters, having and expected value denoted as and a covariance matrix denoted as C and defined below.This information can be used in conjunction with the "maximum entropy (MaxEnt)" principle to obtain the following "MaxEnt" approximation, denoted as Expressions such as shown in Eq. ( 14) can be evaluated to a high degree of accuracy (a priory controlled) by using the saddle-point method (also called Laplace approximation, or steepest descent method).
Since the components of the vector c E r , and the components of the matrices r C and r D C contain 2nd-order and 3rd-order sensitivities, the formulas presented in Eqs. ( 15) and ( 16) generalize all of the previous formulas of this type found in data assimilation/assimilation procedures published to date (which contain at most first-order sensitivities), including the state-of-the-art "data adjustment" methodologies [3] used in reactor physics.The second-order moments of the posterior distribution p , Į U comprise the covariances between the best estimated response, which are denoted as be r C , the covariances between the best-estimate parameters, which are denoted as be D C , and the covariances between the best-estimate parameters and responses, which are denoted as be r D C .The expression of the "best-estimate" posterior parameter covariance matrix be r C for the best-estimate responses be r is derived by using the saddle-point method in conjunction with the results given in Eqs. ( 14) and (15) to obtain: As indicated in Eq. ( 17), the initial covariance matrix m C is multiplied by the matrix C , which means that the variances contained on the diagonal of the best-estimate matrix be r C will be smaller than the experimentally measured variances contained in m C .Hence, the addition of new experimental information has reduced the predicted best-estimate response variances in be r C by comparison to the measured variances contained a priori in m C .Since the components of the matrix r C contain 2 nd -order and 3 rd -order sensitivities, the formula presented in Eq. ( 17) generalizes all of the previous formulas of this type found in data assimilation/assimilation procedures published to date (which contain at most first-order sensitivities).
The expression of the "best-estimate" posterior parameter covariance matrix be D C for the best-estimate parameters be Į is derived by using the saddle-point method in conjunction with the results given in Eqs.(14) and Eq. ( 16) to obtain: C , and r C contain 2nd-order and 3rd-order sensitivities, the formula presented in Eq. ( 18) generalizes all of the previous formulas of this type found in data assimilation/assimilation procedures published to date (which contain at most first-order sensitivities).The expression of the "best-estimate" posterior parameter covariance matrix be D C for the best-estimate parameters be Į and best-estimate responses be r is derived similarly to obtain: As before, since the components of the matrices r D C , and r C contain 2 nd -order and 3 rd -order sensitivities, the formula presented in Eq. ( 19) generalizes all of the previous formulas of this type found in data assimilation/assimilation procedures published to date (which contain at most first-order sensitivities).It is important to note from the results shown in Eqs. ( 15 Although the expressions presented in Eqs.(15) through (19) look superficially similar to the expressions presented in [7,19].This superficial similarity stems from the fact that the 2 nd /3 rd -BERRU-PM methodology only includes means and covariances of the model parameters and computed and measured responses.However, contrary to previously published works, the 2 nd /3 rd -BERRU-PM methodology presented in in Eqs.(15) through ( 19) comprises all of the 2 nd -and 3 rdorder sensitivities of the computed responses with respect to the model parameters.None of the works published thus far include the complete set of 2 nd -and 3 rd -order sensitivities.Consequently, the 2 nd /3 rd -BERRU-PM methodology comprises, as particular cases, the previously published formulas used in predictive modeling [1], data adjustment [2], Bayesian linear statistics [3], and data assimilation [4,5].

CONCLUSIONS
This work has presented the Third-Order Adjoint Sensitivity Analysis Methodology (3 rd -ASAM) for linear systems, which enables the efficient computation of the exact expressions of the 1 st -, 2 nd -, and 3 rd -order functional derivatives ("sensitivities") of a scalar-valued system response to all the parameters characterizing the respective system.Such responses are often encountered when representing mathematically detector responses and reaction rates in reactor physics problems.The 3 rd -ASAM extends the 2 nd -ASAM [1] in the quest to overcome the "curse of dimensionality" in

Μ
can be represented in operator form as i ,...,N D .Since the values of the parameters i D are determined experimentally they can be considered to behave stochastically, obeying a multivariate probability distribution function denoted as p D Į , which is seldom known in practice.The cumulants of p D Į are defined as follows: EPJ Web of Conferences 247, 00002 (2021) PHYSOR2020 https://doi.org/10.1051/epjconf/202124700002(i) The mean value of a parameter: .,N , are as follows: D A and is given by the following expression EPJ Web of Conferences 247, 00002 (2021) PHYSOR2020 the measured responses are considered to be the components of the vector m E r , defined as

EPJ
Web of Conferences 247, 00002 (2021) PHYSOR2020 https://doi.org/10.1051/epjconf/202124700002 and positive definite.Therefore, the subtraction indicated in Eq. (18) implies that the components of the main diagonal of be D C must have smaller values than the corresponding elements of the main diagonal of D C .In this sense, the introduction of new computational and experimental information has reduced the best-estimate parameter variances on the diagonal of be D C .Since the components of the matrices D C , r D ) through (19) that the computation of the best estimate parameter and response values, together with their corresponding best-estimate covariance matrices, only requires the computation of is computationally very advantageous, since r N N D (i.e., the number of responses is much less than the number of model parameters) in the overwhelming majority of practical situations.The minimum value, indicator for evaluating the degree of consistency of the given parameters and responses.
column vector; the components of are operators acting on ΅ and ¡ ; (v) †