Elsevier

Journal of Econometrics

Volume 154, Issue 1, January 2010, Pages 85-100
Journal of Econometrics

Stochastic model specification search for Gaussian and partial non-Gaussian state space models

https://doi.org/10.1016/j.jeconom.2009.07.003Get rights and content

Abstract

Model specification for state space models is a difficult task as one has to decide which components to include in the model and to specify whether these components are fixed or time-varying. To this aim a new model space MCMC method is developed in this paper. It is based on extending the Bayesian variable selection approach which is usually applied to variable selection in regression models to state space models. For non-Gaussian state space models stochastic model search MCMC makes use of auxiliary mixture sampling. We focus on structural time series models including seasonal components, trend or intervention. The method is applied to various well-known time series.

Introduction

State space models are widely used in time series analysis to deal with processes which gradually change over time. Model specification, however, is a challenge for these models as one has to specify which components to include and to decide whether they are fixed or time-varying. For state space models, like for many other complex models, this often leads to testing problems which are non-regular from the view-point of classical statistics. Thus, a classical approach toward model selection which is based on hypothesis testing such as a likelihood ratio test or information criteria such as AIC or BIC cannot be easily applied, because it relies on asymptotic arguments based on regularity conditions that are violated in this context.

Consider, for example, modelling a time series y=(y1,,yT) through the dynamic linear trend model, defined for t=1,,T as: yt=μt+εt,εtN(0,σε2), where μt follows a random walk with a random drift starting from unknown initial values μ0 and a0: μt=μt1+at1+ω1t,ω1tN(0,θ1),at=at1+ω2t,ω2tN(0,θ2). A typical specification problem arising for this model is to decide if the drift at is time-varying rather than constant. However, testing θ2=0 versus θ2>0 results in a non-regular testing problem, because the null hypothesis lies on the boundary of the parameter space. A similar specification problem is deciding which components are present in this time series model. For instance, is it necessary to include a dynamic drift term at or should at be removed because the level μt follows a simple random walk? This is another non-regular problem, because again the null hypothesis can be rephrased as testing a0=θ2=0.

The Bayesian approach is, in principle, able to deal with such non-regular testing problems. Suppose that K different models M1,,MK are considered to be candidates for having generated the time series y. In a Bayesian setting each of these models is assigned a prior probability p(Mk) and the goal is to derive the posterior model probability p(Mk|y) for each model Mk,k=1,,K.

There are basically two strategies to cope with the challenge associated with computing the posterior model probabilities. The traditional approach dating back to Jeffreys (1948) and Zellner (1971) determines the posterior model probabilities of each model separately by using Bayes’ rule, p(Mk|y)p(y|Mk)p(Mk), where p(y|Mk) is the marginal likelihood for model Mk. An explicit expression for the marginal likelihood exists only for conjugate problems like linear regression models with normally distributed errors, whereas for more complex models numerical techniques are required. For Gaussian state space models, marginal likelihoods have been estimated using methods such as importance sampling (Frühwirth-Schnatter, 1995, Durbin and Koopman, 2000), Chib’s estimator (Chib, 1995), numerical integration (Shively and Kohn, 1997) and bridge sampling (Frühwirth-Schnatter, 2001). Recently, Frühwirth-Schnatter and Wagner (2008) considered estimation of the marginal likelihood for non-Gaussian state space models and demonstrated that the resulting estimators can be pretty inaccurate.

The modern approach to Bayesian model selection is to apply model space MCMC methods by sampling jointly model indicators and parameters, using e.g. the reversible jump MCMC algorithm (Green, 1995) or the stochastic variable selection approach (George and McCulloch, 1993, George and McCulloch, 1997). The stochastic variable selection approach is commonly applied to model selection for regression models and aims at identifying non-zero regression effects, but it is useful far beyond this problem. It allows parsimonious covariance modelling for longitudinal data as shown by Smith and Kohn (2002) and covariance selection in random effects models as shown by Chen and Dunson (2003) and Frühwirth-Schnatter and Tüchler (2008).

Shively et al. (1999) present a variable selection approach to non-parametric regression using priors for the unknown functions that are expressed in state space form, however, they did not deal explicitly with time series models. In the present paper we show that such a variable selection approach is useful for dealing with model selection problems in more general state space models.

To perform stochastic model specification search for the dynamic linear trend model defined in (1), (2), (3), for instance, we introduce three binary stochastic indicators in such a way that the unconstrained model corresponds to setting all indicators equal to 1. Reduced model specifications result by setting certain indicators equal to 0. One of those models, for instance, is the local level model, where the drift component at completely disappears:μt=μt1+ω1t,ω1tN(0,θ1). Another interesting special case is the linear trend model, where yt=μ0+ta0+εt,εtN(0,σε2). The practical implementation of this approach is innovative in two respects. First, we employ a new prior for the process variances of the state space model by assuming that the square root of each process variance follows a normal distribution centered at 0. It is well-known that variable selection is, in general, sensitive to the choice of the prior, see e.g. Fernández et al. (2001). We show both for simulated as well as for real data that this prior is less influential on posterior inference when the true process variance is close to 0 than the usually applied inverted Gamma prior. This is in line with Gelman (2006) who came to similar conclusions for the related random effects model.

Second, we derive an MCMC method for Gaussian as well as partially Gaussian state space models that performs stochastic model specification search by sampling the indicators simultaneously with the models parameters. The sampler is based on a non-centered parameterization of the state space model which generalizes previous work in this area such as Pitt and Shephard (1999) and Frühwirth-Schnatter (2004). In combination with the normal prior on the square root of each process variance this leads to a Gibbs sampler that is easily implemented. This is in contrast to Shively et al. (1999) who consider a data-based prior which is motivated by the BIC criterion and leads to a sampling scheme where numerical integration has to be performed for each sweep of the MCMC scheme in order to sample indicators and parameters jointly.

To implement this approach for non-Gaussian state space modelling of time series of count data or binary, categorical, or multinomial data we make use of auxiliary mixture sampling (Frühwirth-Schnatter and Wagner, 2006, Frühwirth-Schnatter and Frühwirth, 2007, Frühwirth-Schnatter et al., forthcoming) which is a simple MCMC method for estimating a broad class of discrete-valued models.

Throughout the paper we focus on structural time series models including seasonal components, trend and an intervention effect and apply the method to various well-known time series.

Section snippets

The dynamic linear trend model

Our method is based on a non-centered parameterization of the dynamic linear trend model which is discussed in the next subsection.

The parsimonious basic structural model

In the basic structural model, a seasonal component is added to the dynamic linear trend model discussed in Section 2, see e.g. Harvey (1989):st=st1stS+1+ω3t,ω3tN(0,θ3),yt=μt+st+εt,εtN(0,σε2), where μt is the same as in (2), (3) and S is the number of seasons. The initial seasonal pattern is given by s0=(sS+1,,s0) with sS+1++s0=0. In addition to the model specification problems discussed in Section 2, a decision has to be made if a seasonal pattern is present and if this pattern is

Model selection for non-Gaussian state space models

The investigations in Shively et al. (1999) show that variable selection in state space models is also feasible for binary data. In this section we show how the variable selection approach developed in Sections 2 The dynamic linear trend model, 3 Extension to the basic structural model for Gaussian state space model may be extended to non-normal state space models using auxiliary mixture sampling (Frühwirth-Schnatter and Wagner, 2006, Frühwirth-Schnatter and Frühwirth, 2007, Frühwirth-Schnatter

Concluding remarks

The model space MCMC approach discussed in this paper could be easily adapted to other state space models. An important extension we are investigating currently is searching for fixed and time-varying coefficients in a regression model.

It is possible to extend our approach to several other non-Gaussian time series models. The method works for any partial Gaussian state space model in the sense of Shephard (1994), i.e. for any state space model that is conditionally Gaussian given a set of

Acknowledgements

We thank the anonymous reviewers and the associate editor for numerous useful suggestions and comments.

References (38)

  • C. Czado et al.

    State space mixed models for longitudinal observations with binary and binomial responses

    Statistical Papers

    (2008)
  • P. De Jong et al.

    The simulation smoother for time series models

    Biometrika

    (1995)
  • J. Durbin et al.

    Time series analysis of non-Gaussian observations based on state space models from both classical and Bayesian perspectives

    Journal of the Royal Statistical Society, Series. B

    (2000)
  • J. Durbin et al.

    Time Series Analysis by State Space Methods

    (2001)
  • S. Frühwirth-Schnatter

    Data augmentation and dynamic linear models

    Journal of Time Series Analysis

    (1994)
  • S. Frühwirth-Schnatter

    Bayesian model discrimination and Bayes factors for linear Gaussian state space models

    Journal of the Royal Statistical Society, Ser. B

    (1995)
  • S. Frühwirth-Schnatter

    Fully Bayesian analysis of switching Gaussian state space models

    Annals of the Institute of Statistical Mathematics

    (2001)
  • S. Frühwirth-Schnatter

    Efficient Bayesian parameter estimation for state space models based on reparameterizations

  • Frühwirth-Schnatter, S., Frühwirth, R., Held, L., Rue, H., 2009. Improved auxiliary mixture sampling for hierarchical...
  • Cited by (0)

    View full text