Sensitivity of stress testing metrics to estimation risk, account behaviour and volatility for credit defaults

Abstract One approach to stress testing the amount of capital required by a bank for credit risk is to use parameterised account level models with credit application characteristics, behavioural characteristics and macroeconomic factors as predictors. The standard methodology underestimates the amount of capital required because it fails to include uncertainty over the model parameters, over the future trajectory of behavioural variables and over volatility. We provide a methodology for estimating the magnitudes of these additional losses and so a methodology to gain a more accurate estimate of the amount of capital required.


Introduction
Every major bank undertakes an annual stress test to indicate whether it has enough capital to protect depositors in the event of a rare but plausible severe negative shock to the value of its assets. These tests are required under the Basel Accords (Pillars I and II) and are implemented by most western banking supervisors including the Federal Reserve, the European Banking Authority and the Bank of England. Major banks also stress test their portfolios to examine the adequacy of their computed economic capital under their internal capital adequacy assessment process (ICAAP). For regulatory capital the national Banking Supervisor supplies a small number of macroeconomic scenarios to a bank and asks it to predict both the amount of capital it will have under these scenarios in a 2 À 5 year horizon and its risk weighted assets (RWA) (see BCBS (2006BCBS ( , 2010). The ratio of the former to the latter (the capital adequacy ratio) must exceed a figure given by the regulator. Typically, statistical methods are used to make the predictions but recently, machine learning methods have been proposed (see Petropoulos et al., 2020, Brummelhuis andLuo (2019), Gogas et al. (2018) and Kupiec (2018)). Specifically, for credit risk, there are at least two broad methodologies that are followed to compute the RWA and so capital required. For regulatory capital, the bank is asked to compute the amount of capital it must hold according to the Vasicek formula (see, for example, Siemsen and Vilsmeier: 2018). The amount is essentially the amount of unexpected losses defined as the difference between the 99.9th percentile of the distribution of expected losses, the Value-at-Risk (VaR) and the mean of that distribution. An alternative methodology is to simulate a large number of scenarios, for example of macroeconomic values for each macroeconomic variable and compute the expected loss distribution and so the VaR from that distribution.
Both methodologies for predicting the VaR required for a portfolio of loan accounts have at least three weaknesses. First, they assume that the parameterised models that are used hold for the population; there is no uncertainty over the parameters. Second, there is no uncertainty over the future values of the behavioural variables which play a vital role in credit scoring models. Third, there is no residual error in the prediction. Clearly omission of these sources of uncertainty results in underestimation of the VaR for a loan portfolio and so of the amount of capital required. This could mean that depositors appear to be adequately protected when in fact they are not. Our article suggests ways to make the scenario approach to stress testing that is used by financial regulators more accurate in terms of the amount of capital needed by banks to protect against plausible but extreme adverse events.
In this article, we contribute to the literature in three ways. First, we give a methodology to assess the sensitivity of the VaR of a loss distribution for a credit card portfolio to the uncertainty over the future values of (a) behavioural variables in a credit scoring model used as inputs to the loss distribution, (b) the parameters of the credit scoring model and (c) residual values of the scoring model. Second we show the relative sizes of these amounts of uncertainty for a portfolio of credit card accounts. Third we show the sensitivity of the amount of capital required relative to the mean that is implied by each type of risk in terms of the VaR and expected shortfall values. We find that the underestimation of the amount of capital required is substantial and therefore that regulators may need to consider seeking estimates of these values from banks in annual stress tests.
There is a large literature on sensitivity analysis (see Borgonovo and Plischke (2016) for a review of methods and Pesenti et al. for an alternative approach). We are interested in measuring the sensitivity of VaR to each of the risks we have outlined. Considering each of the specific sources of risk, and starting with that associated with future values of behavioural variables, we are unaware of any papers that evaluate the implications of uncertainty over their future values in a credit scoring model for metrics of the predicted loss distribution.
In contrast, there is a substantial literature on model parameter estimation risk. Two streams of literature are most closely related to this article. 1 One stream looks at methods to reduce model uncertainty. They include Bayesian model averaging (BMA) (Henry & Kok, 2013;Siemsen & Vilsmeier, 2018) and BMA and ridge regression (Hofmarcher et al. (2014)). The second stream of literature considers methods for evaluating the implications of different sources of prediction uncertainty. Misina and Tessier (2008) consider a model relating industry sector default rates to macroeconomic variables. Using a Cholesky decomposition to retain the correlations between the macroeconomic variables with simulated values of the macroeconomic variables and error terms, they illustrate the differences between the loss distribution if the model is linear compared with non-linear, thus arguing uncertainty over the correct form can have major implications for the VaR. Jacobs et al. (2015) proposed Bayesian regression to take into account parameter uncertainty. But, their model took aggregate banking sector charge-off rates as the measure of loss rather than using account level data and included only three macroeconomic variables. Bignozzi and Tsanakas (2016) consider model uncertainty, parameter uncertainty and residual risk. They show that a Bayesian approach with a non-informative prior removes residual estimation risk if the model selected is correct. Using different Bayesian approaches, for example choose the model with the highest posterior weight, they argue the amount of residual estimation risk indicates the amount of model risk. Wang et al. (2020) estimated Bayesian models of PD distributions at account level using application, behavioural and macroeconomic variables and compared the implied VaR with that from a frequentist model. Garcia-Cespedes and Moreno (2016) suggest uncertainty over probability of default (PD) estimation can be computed from the variance of the estimates and they illustrate using estimates of the PD in different corporate credit ratings. They find that the randomness of PD, rather than the fixed value in the Basel Accord, increases the probabilities of extreme losses but the VaR increases by much less than the uncertainty over the PD. A further paper that considers uncertainty over behavioural model parameters is by Bakoben et al. (2020) who propose a method to cluster accounts by vector autoregression (VAR) parameters of behavioural equations taking into account uncertainty as to their values in the clustering algorithm. They then estimate cross sectional scoring models using cluster membership or the degree of cluster membership uncertainty as the covariate, but they do not consider the implications of parameter uncertainty for the future expected loss distribution or VaR and their application is relatively simple.
The closest paper to ours is Gross and Poblacion (2019). They decompose prediction uncertainty about required Common Equity Tier 1 (CET1), at the level of the bank, using data for 75 banks across 16 countries into model uncertainty (34%), coefficient uncertainty (26%) and residual uncertainty (40%). However Gross and Poblacion consider only bank level data rather than account level data and do not consider a stress test of a credit scoring model. They also do not consider the implications of stressing behavioural variables which are important at account level but are not normally included at bank level.
None of the papers in the literature give a methodology for measuring the implications for the VaR of a portfolio of the three types of risk in a credit scoring model that we are interested in here: behavioural variable value risk, parameter uncertainty and residual uncertainty. Whilst there are several methods for estimating parameter uncertainty they have not been applied to credit scoring models to predict the amount of capital necessary to cover this risk. 2 Importantly, as far as we are aware, there is no paper that decomposes prediction uncertainty for a credit scoring model into different sources and shows the contribution of each to VaR. That is the aim of our article.
Section 2 describes the baseline hazard model that forms the credit scoring model we will be assuming. Section 3 explains the methodology: the use of the Poisson-Binomial distribution to measure volatility risk, the use of multiple generated trajectories using account level vector auto-regressions to gain a measure of the risk of not knowing future values of the behavioural variables, and the distribution of the maximum likelihood estimator of the parameters to measure the risk of mis-estimation. Section 4 shows an implementation for a credit card portfolio and Section 5 concludes.

Dynamic models for credit risk data
Let us consider a portfolio of n credit card accounts, and denote by q i, s the default probability for account i at time s (from the opening date), assuming that this account is still active just prior to this time. In this article, we assume that time is measured in months. The probabilities q i, s are driven by a number of factors including application variables, behavioural variables and macroeconomic variables. A standard way to quantify this dependence is via a discrete-time survival model (also refereed to as a dynamic model) as follows In this expression, U i is a one-row matrix of timeindependent variables associated with account i (these are often refereed to as application variables), and Z i, is the one-row matrix of dynamic or time-dependent variables on account i. These time-dependent variables often include variables measuring the behaviour of the accounts (behavioural variables), as well as macro-economic variables. Interactions between variables can also be appended to the U i 's and Z i, 's. To complete the definition, a and b are unknown coefficients to be estimated, h 0, is a baseline function and g is the logit link function,.
In practice, the baseline function is often smoothed and this allows one to reduce irregular fluctuations from one month to the next. Various smoothing methods can be used. Typical examples include the use of polynomial-type functions (Bellotti & Crook, 2013) as well as spline basis functions (Djeundje & Crook, 2018); our preferred method is to use B-splines basis functions (De Boor, 1978). The foundation and attractiveness of this approach are discussed by Eilers and Marx (1996); more recent exposures of the benefit of this approach in the credit risk context can be found in Crook (2018, 2019a).
Using B-splines, the baseline function can be expressed as h 0, s ¼ BðsÞa, where BðsÞ is a one-row matrix of B-splines at time s, and a is a vector of unknown coefficients to be estimated. In practice, using too many B-splines leads to over-fitting. Conversely, using too few splines results in poor fit. An attractive way of solving this dilemma is to allow a rich set of B-splines and then apply a penalty on adjacent elements of the coefficient vector a to achieve smoothing (O'Sullivan, 1986;Eilers & Marx, 1996). We prefer this method in this article.
Within credit risk, dynamic models of the form (1) have been investigated in the recent literature, starting with sub-models that include application or behavioural variables only, through to larger models involving macroeconomic variables (Djeundje & Crook, 2019a, 2019b. Various extensions have also been considered and implemented successfully including varying coefficients models (Djeundje & Crook, 2019a) and multi-state intensity models, some with random effects to account for heterogeneity (Lando & Skødeberg, 2002;Djeundje & Crook, 2018).
In this article, we show and describe how the framework of discrete survival models can be used to assess and quantify the impact of three sources of uncertainty when stress testing credit defaults.

Mis-estimation, volatility. and behavioural risks
Consider a portfolio of credit card accounts over multiple years. We denote by t o the current calendar time. Thus, we assume that data is observed only up to t o Á These data include application, behavioural and macroeconomic variables, as well as the survival times of the accounts that have already defaulted. We are interested in constructing and investigating the distribution of the loss random variable over a set future time horizon of length d; for example, one month, six months, one year, etc.
Consider an account i still active at current calendar time t o . The loss associated with this account over the future horizon ðt o , t o þ d can be expressed as where EAD i, ð0, d and LGD i, ð0, d are the exposure and loss given default associated with account i during the future horizon ðt o , t o þ d, and y i, ð0, d is an indicator variable taking value 1 if account i defaults during the future horizon ðt o , t o þ d, and 0 otherwise. Our prime interest in this work is to assess the impact of a number of sources of uncertainty involved in the estimation of the risk of default. Thus, if one assumes a 100% LGD and constant EAD for all accounts as in Bellotti and Crook (2013), the aggregated loss (relative to the constant EAD) associated with the future time horizon where R o represents the set of accounts still active at the beginning of the future horizon (i.e., the current calendar time t o ).
If we can generate a large sample from the entire distribution of the aggregated losses, then economic capital can be derived as VaR less the mean (or ES less the mean). For example, if L ¼ fl 1 , :::, l N g is a reasonable sample generated from the loss distribution, the a%-level VaR relative to the expected loss, which we denote by VaR a , can be estimated by Before we proceed, it is imperative to first identify the main components or contributors of the loss distribution. Let us denote by p i, ð0, d the probability that account i defaults during the future time horizon ðt o , t o þ d, given that this account is still active at the opening time t o . That is, whereq i, t o þk are the conditional monthly default probabilities for account i at future calendar time t o þ k: The expected value and variance of the aggregated loss are given by Hence, the structure of the loss distribution is driven by the default probabilitiesq i, t o þk over the future horizon ðt o , t o þ d, in conjunction with the sample volatility or noise due to the composition of the portfolio being analysed. Note that the true values of these default probabilitiesq i, t o þk are unknown themselves, but they can be predicted by applying the fitted parameters from model (1) to the corresponding variables over the future horizon. Hence, a number of components contributing to the distribution of the credit default loss can be identified, including model risk, behavioural risk, macroeconomic risk, mis-estimation risk, volatility risk, etc.
We focus on the measurement and comparison of behavioural, mis-estimation and volatility risks only. By behavioural risk we mean the risk of not knowing future values of behavioural variables. By mis-estimation risk we mean the risk of not knowing the process generating the population values of the parameters. By volatility risk we mean the risk of not knowing if an account will actually default even if we can predict the probability of default perfectly.
In particular in Section 3.1, we show how to quantify the impact of volatility via the Poison-Binomial distribution. In Section 3.2, we use the distribution of the maximum likelihood estimator of the parameters to measure mis-estimation. In Section 3.3, we measure behavioural risk through account-level vector auto-regressions. Section 4 then illustrates, in terms of stress metrics, the magnitude of different combinations of these three types of risk within a global sensitivity analysis framework (Borgonovo & Plischke, 2016).

Volatility risk and quantification
The aggregated loss L ð0, d is a random variable consisting of a sum of Bernoulli random variables with not-all-equal individual success probabilities, p i, ð0, d , i ¼ 1, :::, n: Thus, conditional on the values of the cumulated probabilities p i, ð0, d , the aggregated loss variable L ð0, d follows the so-called Poisson-Binomial (PB) distribution which we denote by Hence, for a credible risk management strategy, capital must be held against the fact that each portfolio is made up of many individuals' accounts, irrespective of whether the true values of the default probabilities for these accounts are known or not.
We propose to quantify the contribution of the portfolio-volatility on stress metrics (such as VaR or ES) based on the PB distribution (7). Specifically, we simulate prospective losses using (7) conditional on the individual probabilities p i, ð0, d and then use these simulated losses to estimate the stress metrics. More details on this are given in given in Sections 4.4 and 4.5.
The PB distribution itself is not new. It was first considered by (Neyman, 1939) and an early application was by (Barrett, 2007) in the context of criminal and civil courts. But more recent investigations of the properties of this distribution can be found in Daskalakis et al. (2015), Hong (2013), or Chen and Liu (1997). In particular, Daskalakis et al. (2015) explore the sample complexity of learning from PB distributions, while the other two papers explore different methods of computing and sampling for PB distributions. In this work, we sample from any given PB distribution based on the Fourier transform of its characteristic function as in Hong (2013).

Mis-estimation risk and quantification
In practice, for each account still active at current calendar time t o , its conditional monthly default probabilities over the future horizon ðt o , t o þ d are unknown. They must be predicted using the fitted model. Hence uncertainty exists over the portfolio's underlying default rates, since these can only be estimated to a degree of confidence linked to the size and richness of the data. Part of this uncertainty is reflected in the estimation process of the model parameters a, a and b (where a represents the vector of parameters used to model the baseline function h 0, ).
For notational convenience, let us denote by h the joint vector of all parameters. That is, Also, letĥ denote the maximum likelihood estimator of h and HðĥÞ denotes the Hessian matrix evaluated at the maximum likelihood estimate (MLE) of h: It is well known from the theory of MLE thatĥ is efficient and asymptotically normal: We can use expression (9) with MLE plugged in, to generate plausible alternative values of the parameters of the model, which can be used in turn to calculate alternative plausible default probabilities and to quantify the amount of capital that should be held against the risk of mis-estimation. More details on this are provided in Sections 4.3 and 4.5. A similar approach was used in the context of longevity risk insurance by Richards (2016) to quantify the cost of mis-estimation for capital adequacy and solvency.

Behavioural risk and quantification
Accounts are assumed to be observed from opening date up to current calendar time t o only. Thus, the values of the behavioural variables beyond t o are unknown. In order to generate plausible values of default probabilities p i, ð0, d , we first need to generate plausible monthly values of the underlying behavioural variables (for each account still active at time t o ) over the future horizon ðt o , t o þ d: One possibility is to consider each behavioural variable in isolation and then sample plausible future values based on past observed values, separately for each account. However, behavioural variables are potentially correlated with one another. Thus, it is unrealistic to generate plausible future values of each behavioural variable in isolation.
In this work, we propose to generate plausible future values of the behavioural variables through account-level vector-auto-regressive processes. Hence, for a given account i still active at the calendar time t o , we consider models of the form where Z i, s represents the vector of behavioural processes associated with account i at duration time s, a o, i is the vector of intercepts, p is the lag, H i, k is the matrix of coefficients associated with lagged values of these processes, e i, s is the vector of error terms, and R i is the covariance matrix R i : In particular, if p ¼ 1 and elements within H i, 1 are set to the identity matrix, then Equation (10) simplifies to a multivariate random walk with drift vector a o, i , in which case the correlation between the behavioural processes are measured through the correlation matrix R i alone. Similarly, if the H i, k are diagonal, then Equation (10) simplifies to a set of correlated or uncorrelated auto-regressive processes, depending of the structure imposed on R i : But in general, Equation (10) allows each behavioural process to affect the prediction of its counterparts through the coefficients matrices H i, k and covariance matrix R i : Models of the form of (10) can be fitted using observed data up to calendar time t o . In this article, these models are fitted separately for each account via the method of least squares. The fitted parameters are then used to simulate plausible values of the behavioural processes for each account over the future horizon ðt o , t o þ s: These simulated values are used subsequently in conjunction with the regression coefficients estimated from model (1) to generate prospective default probabilities and to construct the loss distribution. Details about the exact forms of models (10) implemented in this article are presented in Section 4 below.

Data and base survival model
The data that motivated this work is a sample of about 50,000 credit card accounts from a major bank in the UK. The accounts in the sample were opened from 2002 through to 2011.
This dataset comprises a number of application variables as well as behavioural variables; the list of variables used in this analysis is summarised in the first column of Table 1. In particular, four behavioural variables are used in this investigation, including the Credit limit (denoted by CL), the Repayment amount (denoted by PAY), the percentage of time spent with one outstanding payment (denoted by DEL), and the Proportion of the credit limit that is drawn (denote by PDR). An illustration of three of these behavioural variables for a typical account is shown in Figure 1.
We define an account as being in default if and when it became three months in arrears. The three missed payments are not required to have been missed consecutively. To ensure a consistent definition over time we computed a minimum payment each month using constant parameters over time. This definition is not the same as that used in practice by the data provider.
For illustration, we set the current calendar date t o to December 2010. Thus, the base survival model (1) is fitted using data available from 2002 through to December 2010. The accounts still active at the end of December 2010 form the stress-testing set. We can use this stress-testing set to quantify the impact of different sources of uncertainty, going forward. Various future horizons can be considered; the illustrations in this article correspond to a twelve months future horizon (i.e., d ¼ 12).
The starting point is to fit the base survival model (1). The fitted coefficients are shown in Table  1. The coefficients have the expected signs and are consistent with the literature. Thus, holding other variables constant, the higher the credit limit, the greater the percentage of duration time the account has been one payment behind and the higher the proportion of the limit that is drawn the higher the PD. Also the higher is the repayment amount the lower the PD. Older account holders are seen to have lower PDs and those with more cards of types C and D have a higher PD. More discussion of the magnitude and significance of most of these coefficients can be found in Crook (2018, 2019a). Our aim in this article is the quantification of uncertainty.
In the next few sections, we shall focus on the assessment of the three sources of uncertainty described in Section 3. We use four scenarios to investigate and compare the impact of these sources over the future time-period ðDec2010 À Dec2011: As we shall see, each source can be switched on/off when estimating the aggregated loss distribution.

Scenario1: Behavioural risk only
In this first scenario, we consider behavioural risk alone. To quantify its impact, we generate N ¼ 5000 plausible trajectories for each behavioural variable and for each account (in the stress-testing set) over the future twelve months (i.e., January to December 2011). These generated behavioural trajectories are then used in conjunction with the MLE of the regression coefficients from the base survival model to estimate plausible monthly PDs (over the future horizon) and subsequently N candidates for the cumulative probabilities p i, ðDec2010, Dec2011 for each account. The application variables were set at their observed values in the data. A sample of N plausible aggregated losses can then be derived using Equation (6), and used to construct the empirical loss distribution, as well as to estimate the VaR according to Formula (4).
To generate future trajectories of the behavioural variables, we consider three alternative variants of vector autoregressive models for the behavioural processes.
First, we consider multivariate random walks with drift terms, which we outline as follows This is a very simplified version of model (10), and e i, s ¼ ðe i, s, 1 , e i, s, 2 , e i, s, 3 , e i, s, 4 Þ: In this reduced model, the correlation between the behavioral variables is captured by the ð4 Â 4Þ covariance matrices R i, s ¼ covarðe i, s Þ: For each account, the drift vector a can be estimated using the method of least squares, and an estimate of the covariance matrix R i, s derived from the residuals. For each account we simulate by choosing values of e i, s ¼ ðe i, s, 1 , e i, s, 2 , e i, s, 3 , e i, s, 4 Þ randomly to predict each behavioural variable one month ahead. These predicted values are used as inputs to generate future predictions. We follow this simulation method in all of the following models.
Second, we consider correlated autoregressive processes for the behavioural processes as follows In this case, the matrices of coefficients H i, 1 are diagonal with unknown parameters to be estimated on the diagonal. The interaction between the individual behavioural processes are captured by the diagonal elements of H i, 1 and the covariance matrix V i : An illustration of simulated paths by applying this process to a typical account is shown in Figure 2.
Third, we consider a full vector autoregressive model of order one. That is In this case, H i, 1 is a full 4 Â 4 matrix filled with the b's, and the structure of the interactions between behavioural processes is more complex and flexible compared to models (11) and (12).
We quantify the size of the behavioural risk in terms of the VaR and ES relative to the mean of the predicted losses. The latter are the aggregated values of those losses predicted by the base hazard model, (1), where the observed values of the application variables and the predicted values of the behavioural variables are substituted in. Among the three model structures (11), (12), and (13) for the behavioural processes, the multivariate random walks are the easiest to implement. However, this structure can struggle to capture certain important features seen in the data. In contrast, the full vector autoregressive process is more flexible and can adjust to capture more complex associations between the behavioural processes. However, it can become computationally intensive as the number of accounts increases. High order vector autoregressive processes were considered but were found to be too complex for the vast majority of accounts, especially those with a small number of observations or low variability.
An illustration of the aggregated loss (corresponding to a 12 months horizon) subject to behavioural risk alone is shown in Figure 3. To create this graphic, plausible trajectories of the underlying behavioural processes over the future horizon were jointly simulated based on the correlated autoregressive processes in Equation (12). Similar distributions can be constructed based on correlated random walk models (11) or the full vector autoregressive models (13).
A comparison of the VaR and ES relative to the mean when using these three processes to quantify behavioural risk is shown in Table 2. This highlights the sensitivity of risk measures and economic capital with respect to the choice of the behavioural process models. In particular, modelling behavioural patterns using the simplest multivariate random walk (11) leads to more uncertainty compared to the other two models. This is expected since the second and third processes (12) and (13) are extensions of (11), and so would be expected to model the data more accurately. What is especially interesting is that the VaR and ES of the autocorrelated process (12) and the VAR model (13) are very similar; there is little benefit from using a VAR model (13).

Scenario2: Mis-estimation risk only
Under this scenario, we consider the impact of misestimation risk alone as follows.
(i) Upon fitting the base survival model (1), the Hessian matrix is estimated and used to simulate N alternative plausible vectors for the regression parameters h via distribution (9). (ii) The behavioural processes are jointly forecasted over twelve months using correlated auto-regressive models (12) at account level. (iii) Each simulated vector of regression parameters from (i) is applied (to the application variables and central forecasts of the behavioural variable from (ii)) yielding a sample of N monthly default probabilities over 12 months for each account. Finally, these monthly probabilities are used to compute the cumulated probabilities p i, ðJan2011, Dec2011 and to generate a sample of N potential loses.
An illustration of the negative inverse of the Hessian matrix (i.e., the covariance matrix ofĥ) is shown in Figure 4. We find the values of VaR and ES required for mis-estimation risk are 6.5% and 7.3% of the expected values, respectively. These are a little less than those due to behavioural risk.

Scenario3: volatility risk only
Under this scenario, we consider the impact of volatility risk alone. To generate the cumulated Plausible values of the underlying behavioural processes over the future horizon were jointly simulated based on correlated auto-regressive processes; see Equation (12). Table 2. Assessment of behavioural risk in terms of VaR and ES over a 12-months horizon relative to the mean. The VaR 99:5 values are estimated based on a sample of 5000 losses, and the the loss samples were computed based on simulated values of the behavioural processes, in conjunction with the MLE of the vector of regression parameters h: The underlying simulated behavioural processes over the 12 months horizon were generated using three different forms of vector auto-regressive processes; see Equations (11)(12) and (13).
Model for behavioural processes: predicted probabilities p i, ðJan2011, Dec2011 and hence the potential losses, we proceed as follows: The MLE estimateĥ of the vector of regression parameters is obtained by fitting the base survival model (1). The behavioural processes are jointly forecasted over a 12-months horizon using vector autoregressive models, separately for each account. The MLEĥ is applied (unto the application variables and forecasted behavioural variables) yielding (a) predicted values of the monthly default probabilities over 12 months and (b) the cumulated probabilities p i, ðJan2011, Dec2011 : Potential losses are then simulated based on these cumulated probabilities within the Poisson-Binomial distribution (7).
A quantification of the volatility risk in terms of VaR and ES is presented in the third row of Table 3. This suggests that the volatility risk is much higher than either that associated with behavioural or misestimation risk.

Scenario4: volatility, mis-estimation and behavioural risks
In this scenario, all three sources of uncertainty (i.e., volatility, mis-estimation and behavioural risks) are switched on, through the following steps.
(i) Upon fitting the base survival model (1), the normal distribution (9) is used to simulate a plausible alternative vector of regression parameters. (ii) For each account, upon fitting the vector auto-regressive model (12) to its behavioural data, a path is simulated of each behavioural variable over the twelve month horizon. (iii) For each account, the simulated vector generated in (i) is applied to the application variables and the simulated behavioural paths generated in (ii). This yields a potential path of monthly default probabilities over twelve months, and subsequently the cumulated probabilities p i, ðJan2011, Dec2011 : (iv) These cumulated probabilities are then used to generate a potential loss through the Poisson-Binomial distibution (7). (v) Steps (i)-(iv) above are repeated N times, yielding a sample of N potential losses.
A summary of the resulting aggregated risk is shown in the bottom row of Table 3    sum of the values for each individual risk component because the risk sources are not independent.

Risk assessment by employment class
So far, the assessment of uncertainty has been presented at portfolio level. In practice, a portfolio is often made up of different business segments, each segment having a different composition and can cause different risk to the lender. To illustrate the differences between applicant segments we undertake an assessment of the risk separately for each of three levels of employment. We follow the same methodology as above. A summary is shown in Table 4. Table 4 shows that for all five segments the capital required for volatility risk is much higher than for the other two risks. But the segments differ noticeably in the amount of each type of risk they are associated with. The employed segment has considerably higher behavioural and mis-estimation risk than the other segments and the employed segment and that for which there is no data available have higher volatility risk than those in the other risk segments. Overall, the retired or unemployed segment has the lowest volatility and behavioural risk and the lowest overall risk from these three sources. If all other sources of risk were the same for all segments these results suggest that the amount of capital that needs to be held for the employed segment and the segment where no details are available is larger than that for those retired or unemployed or self employed or students and so these former segments would yield, ceteris paribus, a lower return on equity when lent to.

Discussion and conclusions
Many regulators require banks to adopt model risk management principles. For example the UK's Prudential Regulation Authority (PRA) states that banks must engage in "appropriate testing of models to take into account potential limitations, assess their robustness and stability over time, and across a variety of economic and market conditions" (Prudential Regulation Authority, 2018, pp. 8). The PRA does not specify exactly how uncertainty over the future values of behavioural variables, uncertainty over model parameters and uncertainty over the occurrence of default given the predicted PD, should be analysed. In this article, we have developed a framework for the quantification of the impact of these three sources of uncertainty in the prediction of the VaR for a credit portfolio and so the amount of capital a bank is required to hold for that portfolio. There are, of course, many other sources of uncertainty in the computation of required capital, for example the state of the macroeconomy. However, in this article, we are concentrating on different types of modelling risk, conditional on the state of the macroeconomy. The sources of model risk we are considering are typically omitted from the calculation of the VaR of a retail credit portfolio. In this case the behavioural variables take on fixed values as do the PD model parameters, and so does the predicted PD for each account. The aggregated loss across all accounts (given by Equation 3) takes on a single value; conditional on the macroeconomy, there is no additional uncertainty. In this case the VaR in Equation (4) has the value 1. Comparing the values of the VaR under each scenario from Table 3 with the value of 100% we can see the increase in VaR and so capital if the three sources of uncertainty are incorporated rather than ignored. Thus, in the portfolio we considered, if all three sources of uncertainty are incorporated, the value of capital to be held will increase by up to 30% relative to if they are omitted. Our article contributes to prudential policy by offering a methodology to incorporate all three sources of risk in the VaR calculation for a portfolio of credit products. Given the potential under-capitalisation if this is not done, this is an important message to lenders and to regulators. We offer a method that regulators might consider advancing to lenders to protect depositors and others and the methodology might be wisely adopted by lenders when computing their economic capital. Parts of the methodology may also be incorporated into the the prediction of expected cash flows for loans for which risk has increased under IFRS9 (and similarly CICL). Under IFRS9 (International Accounting Standards Board, 2013) expected cash flows from loans may be computed using PD probabilities in each future month for which a loan is outstanding and these can be gained from a survival model. The Standard requires that a range of possible outcomes is assessed and their probabilities of occurrence. A survival model of PD may include behavioural variables (as we have shown). So, the range of PDs may be gained following the methods in our article: by simulation of survival model parameters, simulation of behavioural variables values and of the occurrence of defaults given the PD. Of course, other sources of uncertainty would also be included such as possible changes in macroeconomic variables.
We have concentrated on uncertainty associated with PD modelling. A limitation of our approach is that we have not considered uncertainty over the future states of the macroeconomy-we have concentrated on model risk. A second limitation is that the residuals from the VAR that is used to model the behavioural variables may not include enough values or range of values to represent the population distributions. This means the analyst should use as long a time series of the behavioural variables as possible. A further limitation is that aggregate loss is the product of PD, LGD and EAD (Equation 2) and we have concentrated only on the former. However, a similar approach may be applied to these other two terms since there would be uncertainty over the estimation model parameters, and uncertainty as to the future values of behavioural variables which might enter both models. In principle, uncertainties over these variables should be included in the estimation of the VaR.
On the other hand, our method has a number of strengths. One of these is that one can follow it to gain the VaR at a time horizon of the analyst's choice. This follows from our use of a survival model for PD prediction and a VAR to product the future values of the behavioural variables. This means that the method could inform managerial decisions over different time horizons: for example one year, as in Basel or five years. Different stakeholders may be interested in these predictions over different horizons: certain institutions may be more interested in a five year horizon and some depositors only in a one year horizon. Notes 1. A further group of papers considers worst outcomes; that is, theoretical upper bounds on VaR from model risk (Cont, 2006;Embrechts et al., 2013;Puccetti & Ruschendorf, 2012;Bernard et al., 2017;Bernard 2017;and Glasserman & Xu, 2014. 2. In the context of credit risk, Jacobs et al. (2015) and Gross and Poblacion (2019) give methods for estimating parameter uncertainty, but they do not do so for a credit scoring model and hence not for capital prediction for a specific credit portfolio. Wang et al. (2020) do give a method for estimating parameter uncertainty but do not decompose predicted loss into the three types of risk we are interested in.

Disclosure statement
No potential conflict of interest was reported by the authors.