Bayesian Model Comparison of Structural Equation Models

Abstract

Structural equation modeling is a multivariate method for establishing meaningful models to investigate the relationships of some latent (causal) and manifest (control) variables with other variables. In the past quarter of a century, it has drawn a great deal of attention in psychometrics and sociometrics, both in terms of theoretical developments and practical applications (see Bentler and Wu, 2002; Bollen, 1989; Jöreskog and Sörbom, 1996; Lee, 2007). Although not to the extent that they have been used in behavioral, educational, and social sciences, structural equation models (SEMs) have been widely used in public health, biological, and medical research (see Bentler and Stein, 1992; Liu et al. 2005; Pugesek et al., 2003 and references therein). A review of the basic SEM with applicants to environmental epidemiology has been given by Sanchez et al. (2005).

Appendix: Full Conditional Distributions

The conditional distributions required by the Gibbs sampler in the posterior simulation of the integrated model will be presented in this appendix. We use p(∙) to denote the conditional distribution if the context is clear, and note that (Y _obs, W _obs, U _mis, O) = U.

• (i) p(V ǀ9, a, Y _obs, W _obs, U _mis, Ω₁, Ω₂, O) = p(V Čθ,U,Ω₁,Ω₂): This conditional distribution is equal to a product of p(v _gǀθ, U _g, Ω_1g, ω_2g) with g = 1,…, G. For each gth term in this product, its conditional distribution is $N\left[ {\mu _g^*,\sum _g^* } \right],$ where

  $$\mu _g^* = \sum _g^* \left\{ {{\mathbf{\psi }}_1^{ - 1} \sum\limits_{i = 1}^{N_g } {\left[ {{\mathbf{u}}_{gi} - {\mathbf{A}}_1 {\mathbf{c}}_{ugi} - \Lambda _1 \omega _{1gi} } \right] + {\mathbf{\psi }}_2^{ - 1} \left[ {{\mathbf{A}}_2 {\mathbf{c}}_{\upsilon g} + \Lambda _2 \omega _{2g} } \right]} } \right\},{\text{and}}\sum _g^* = \left({N_g {\mathbf{\psi }}_1^{ - 1} + {\mathbf{\psi }}_2^{ - 1} } \right)^{ - 1}.$$

  ((31))

• (ii) p(Ω₁ǀθ, α, Y _obs, W _obs, U _mis, Ω₂, V, O) = p(Ω₁ǀθ, U, Ω₂, V) $= \mathop \prod \limits_{g = 1}^G \mathop \prod \limits_{i = 1}^{N_g }$ p(ω_1gi ǀθ, v _g , ω_2g , u _gi ), where p(ω_1gi ǀθ, v _g , ω_2g , u _gi ) is proportional to

  $$\begin{array}{l}\exp \left[ - \frac{1} {2}\left\{ {{\mathbf{\xi }}_{1gi}^T {\mathbf{\Phi }}_1^{ - 1} {\mathbf{\xi }}_{1gi} + \left[ {{\text{u}}_{gi} - {\mathbf{v}}_g - {\mathbf{A}}_1 {\mathbf{c}}_{ugi} - {\mathbf{\Lambda }}_1 \omega _{1gi} } \right]^T {\mathbf{\psi }}_1^{ - 1}} \right.\right.\\ \qquad\qquad\times \left[ {{\mathbf{u}}_{gi} - {\mathbf{v}}_g - {\mathbf{A}}_1 {\mathbf{c}}_{ugi} - {\mathbf{\Lambda }}_1 \omega _{1gi} } \right] + \left[ {\eta _{1gi} - {\mathbf{B}}_1 {\mathbf{c}}_{1gi} - \prod _1 \eta _{1gi} - {\mathbf{\Gamma }}_1 {\mathbf{F}}_1 \left({{\mathbf{\xi }}_{1gi} } \right)} \right]^T\\ \qquad\qquad\left.\left.\times {\mathbf{\psi }}_{1\delta }^{ - 1} \left[ {\eta _{1gi} - {\mathbf{B}}_1 {\mathbf{c}}_{1gi} - {\mathbf{B}}_1 {\mathbf{c}}_{1gi} - \prod _1 \eta _{1gi} - {\mathbf{\Gamma }}_1 {\mathbf{F}}_1 \left({{\mathbf{\xi }}_{1gi} } \right)} \right] \right\} \right]. \end{array}$$

  ((32))

• (iii) p(Ω₂ǀθ, α, Y _obs, W _obs, U _mis, Ω₁, V, O): This distribution has very similar form as in p(Ω.₁ ǀ·) and (32), hence is not presented to save space.

• (iv) p(α, Y _obsǀθ, W _obs, U _mis, Ω₁, Ω₂, V, O): To deal with the situation with little or no information about these parameters, the following noninformation prior distribution is used: p(α_k) = p(a _k,2,…, a_k,bk−1) C, k = _{1,…, s}, where C is a constant. As (α, Y _g) is independent with (α, Y _h) for g ≠ h, and that Ψ 1 is diagonal, we have

  $$p\left({\alpha,{\mathbf{Y}}| \cdot } \right) = \mathop \prod \limits_{g = 1}^G p\left({\alpha _h,{\mathbf{Y}}_g | \cdot } \right) = \mathop \prod \limits_{g = 1}^G \mathop \prod \limits_{k = 1}^s p\left({\alpha _k,{\mathbf{Y}}_{gk} | \cdot } \right),$$

  ((33))

  where Y _{gk = [ygk1,…, ygkNg]}. Let ψ_1k be the kth diagonal element of Ψ 1, vgk be the kth element of v _g, and Λ _1k be the kth row of Λ₁, and I _a (y) be an indicator function with value 1 if y is A and zero otherwise, p(α,Yǀ·) can be obtained from (33) and

  $$p\left({\alpha _k,y_{gki} | \cdot } \right) \propto \mathop \prod \limits_{i = 1}^{N_g } \Phi ^* \left\{ {\psi _{1k}^{ - 1/2} \left[ {y_{gki} - \upsilon _{gk} - \Lambda _{1k}^T \omega _{1gi} } \right]} \right\}I_{(\alpha _{k,z_{gki}},\alpha _{_{k,z_{gki} }} + 1]} \left({y_{gki} } \right).$$

  ((34))
  $$p\left({\omega _{gik,{\text{obs}}} |\theta,\omega _{1gi},d_{gik,{\text{obs}}} } \right) \sim \left\{ {\begin{array}{*{20}c} {N\left[ {{\mathbf{\Lambda '}}_{1k} \omega _{1gi},\psi _{1k} } \right]I_{\left({ - \infty,0} \right)} \left({\omega _{gik,{\text{obs}}} } \right),{\text{if}}d_{gik,} {\text{obs = 0}}} \\ {N\left[ {{\mathbf{\Lambda '}}_{1k} \omega _{1gi},\psi _{1k} } \right]I_{\left({0,\infty } \right)} \left({\omega _{gik,{\text{obs}}} } \right),{\text{if}}d_{gik,} {\text{obs = 1,}}} \\ \end{array} } \right.$$

  ((35))

  where n _g,k is the number of d _gki,obs in D _k,obs and D _k,obs is the kth row of D _obs.

  $$\left[ {{\mathbf{u}}_{gi,{\text{mis}}} |\theta,\omega _{1gi} } \right]\mathop = \limits^D N\left[ {{\mathbf{v}}_g + {\mathbf{A}}_{1i,{\text{mis}}} {\mathbf{c}}_{ugi} + \Lambda _{1i,{\text{mis}}} \omega _{1gi},{\psi }_{1i}, {\text{mis}}} \right],$$

  ((36))

  where A _1i,misand ι_1i,mis are submatrices of A ₁ and ι₁with rows that correspond to observed components deleted, and Ψ1i,mis is a submatrix of Ψ1 with the appropriate rows and columns deleted. (vii) p(θǀα, Y _obs, W _obs, Umis, Ω₁, Ω₂, V, O) = p(θǀ U, Ω₁, Ω₂):Letθ₁ be the vector of unknown parameters in A ₁, Λ₁, and Ψ₁, θ_1ω be the vector of unknown parameters in Π ₁, Γ ₁, Φ ₁, and Ψ _1δ, θ2 be the vector of unknown parameters in A ₂, Λ₂, and Ψ2, and θ_2ω be the vector of unknown parameters in Π₂, σ₂, Φ₂, and Ψ2δ.

For θ1, the following commonly used conjugate type prior distributions are used:

$$\begin{array}{*{20}c} {p\left({\psi _{1k}^{ - 1} } \right)\mathop = \limits^D Gamma\left[ {\alpha _{01\varepsilon k} } \right],} & {p\left({\Lambda _{1k} |\psi _{1k} } \right)} \\ \end{array} \mathop = \limits^D N\left[ {{\mathbf{\Lambda }}_{01k},\psi _{1k} {\mathbf{H}}_{01yk} } \right],$$

((37))

where ${\mathbf{A}}_{1k}^T,{\mathbf{\Lambda }}_{1k}^T$ are the row vectors that contain the unknown parameters in the kth row of A1 and A1, respectively;$\alpha _{01\varepsilon k,} \beta _{01\varepsilon k,} {\mathbf{A}}_{01k,} {\mathbf{\Lambda }}_{01k,} {\mathbf{H}}_{01k,}$ and H _01yk are given hyper-parameters values. For k ≠ h, it is assumed that (ψ_1k, Λ_{1 k}) and (ψ_1h, Λ_1h) are independent. Let U¢ = {u _gi—v _g—A ₁ c _ugi;i = 1,…, N _g, g = 1,…, G and ${\mathbf{U}}_k^{*^T }$be the kth row of U, Ω₁ ={σ_1gi; i =1… N _g g = 1…G}

$$\begin{array}{*{20}c} {p\left({\psi _{1k}^{ - 1} | \cdot } \right)\mathop = \limits^D Gamma\left[ {2^{ - 1} N_g + \alpha _{01\varepsilon k},\beta _{1\varepsilon k} } \right]{\text{and}}} & {p\left({\Lambda _{1k} |\psi _{1k}, \cdot } \right)} \\ \end{array} \mathop = \limits^D N\left[ {{\mathbf{m}}_{1k},\psi _{1k} \sum _{1k} } \right].$$

((38))

Let C _u = {c _ugi; i = 1,…, N _g, g = 1,…, G}, Û = {u _gi—vg—Λ₁ω_1gi;i = 1,…, N _g, g = 1,…, G}, and ${\mathbf{\tilde U}}_k^T$ be the feth row of ${\mathbf{\tilde U}},\tilde \sum _{1k} = \left({{\mathbf{H}}_{01k}^{ - 1} + {\mathbf{C}}_u {\mathbf{C}}_u^T } \right)^{ - 1},{\mathbf{\tilde m}}_{1k} = \tilde \sum _{1k} \left({{\mathbf{H}}_{01k}^{ - 1} {\mathbf{A}}_{01k} + {\mathbf{C}}_u {\mathbf{\tilde U}}_k } \right).$

For θ₁ω, it is assumed that Φ1 is independent of (Λ_1ω, :_1s), where Λ_1o = $\left({{\mathbf{B}}_1^T,\prod _1^T,{\mathbf{\Gamma }}_1^T } \right)^T.$Also, (Λ_1ωk;, ω_1δk) and (Λ_{1ωh, ω1δh}) are independent, where Λ_1ωk and ψ_1δk are the kth row and diagonal element of Λ _1ω and ΨΨ 1,δ, respectively. The associated prior distribution of Φ₁ is $p\left({{\mathbf{\Phi }}_1^{ - 1} } \right)\mathop = \limits^D W\left[ {{\mathbf{R}}_{01,\rho 01,q12} } \right],$ where W[∙, ∙, q12] denotes the q 12-dimensional Wishart distribution, ρ₀₁ and the positive definite matrix R ₀₁ are given hyper-parameters. Moreover, the prior distribution of ψ_1δk and Λ_1ωak are

$$\begin{array}{*{20}c} {p\left({\psi _{1\delta k} } \right)\mathop = \limits^D Gamma\left[ {\alpha _{01\delta k},\beta _{01\delta k} } \right]{\text{and}}} & {p\left({\Lambda _{1\omega k} |\psi _{1\delta k} } \right)} \\ \end{array} \mathop = \limits^D N\left[ {{\mathbf{\Lambda }}_{01\omega k},\psi _{1\delta k} {\mathbf{H}}_{01\omega k} } \right],$$

((39))

where α_01δk, Λ_01ωk, and H _01ωk are given hyper-parameters. Let E ₁ = (η_1g1,…, η_1gNg); g = 1,…, G},${\mathbf{E}}_{1k}^T$be the kth row of E ₁, ϩ₁ = {(ξ _1g1,…, ξ _1gNg); g = 1,…, G} and${\mathbf{F}}_1^* = \left\{ {\left({{\mathbf{F}}_1^* \left({\xi _{1g1} } \right), \ldots,{\mathbf{F}}_1^* \left({\xi _{1gNg} } \right)} \right);g = 1, \ldots,G} \right\},$, in which ${\mathbf{F}}_1^* \left({\xi _{1g1} } \right) = \left({\eta _{1gi}^T,{\mathbf{F}}_1 \left({\xi _{1gi} } \right)^T } \right)^T,i = 1, \ldots,N_g,$, i = 1,…, N _g, and it can be shown that for k = 1,…, q ₁₁,

$$\begin{array}{*{20}c} {p\left({\psi _{1\delta k} | \cdot } \right)\mathop = \limits^D Gamma\left[ {2^{ - 1} N_g + \alpha _{01\delta k},\beta _{1\delta k} } \right]{\text{,}}} & {p\left({{\mathbf{\Lambda }}_{1\omega k} |\psi _{1\delta k}, \cdot } \right)} \\ \end{array} \mathop = \limits^D N\left[ {{\mathbf{m}}_{1\omega k},\psi _{1\delta k} \sum _{1\omega k} } \right],$$

((40))

where $\sum _{1\omega k} = \left({{\mathbf{H}}_{01\omega k}^{ - 1} + {\mathbf{F}}_1^* {\mathbf{F}}_1^{*^T } } \right)^{ - 1},{\mathbf{m}}_{1\omega k} \left({{\mathbf{H}}_{01\omega k}^{ - 1} {\mathbf{\Lambda }}_{01\omega k} + {\mathbf{F}}_1^* {\mathbf{E}}_{1k} } \right),$ and$\beta _{1\delta k} = \beta _{01\delta k} + \frac{1} {2}\left({{\mathbf{E}}_{1k}^T {\mathbf{E}}_{1k} - {\mathbf{m}}_{{\text{l}}\omega {\text{k}}}^T \sum _{{\text{l}}\omega {\text{k}}}^{ - 1} {\mathbf{m}}_{{\text{l}}\omega {\text{k}}} + \Lambda _{01\omega k}^T {\mathbf{H}}_{01\omega k} } \right).$be the inverted Wishart distribution, the conditional distribution relating to Φ₁ is given by

$$p\left({\Phi _1 |\Xi _1 } \right)\mathop = \limits^D IW\left[ {\left({\Xi _1 \Xi _1^T + {\mathbf{R}}_{01}^{ - 1} } \right),\sum\limits_{g = 1}^G {N_g + \rho _{01},q_{12} } } \right].$$

((41))

Conditional distributions involved in θ₂ are derived similarly on the basis of the following independent conjugate type prior distributions: for k = 1,…, p, and

$$\begin{array}{l} {\begin{array}{ll} {p\left({\psi _{1\varepsilon k}^{ - 1} } \right)\mathop = \limits^D Gamma\left[ {\alpha _{02\varepsilon k},\beta _{02\varepsilon k} } \right],} & {p\left({\Lambda _{2k} |\psi _{2k} } \right)} \\ \end{array} \mathop = \limits^D N\left[ {{\mathbf{\Lambda }}_{02k},\psi _{2k} {\mathbf{H}}_{02yk} } \right],} \\ {p\left({{\mathbf{A}}_{2k} } \right)\mathop = \limits^D N\left[ {{\mathbf{A}}_{02k},{\mathbf{H}}_{02k} } \right],k = 1, \ldots,p,} \\ \end{array} $$

where ${\mathbf{A}}_{2k}^T$ and ${\mathbf{\Lambda }}_{2k}^T$ are the vectors that contain unknown parameters in the kth rows of A2 and Λ₂, respectively; $\alpha _{02\varepsilon k,} \beta _{02k,} {\mathbf{A}}_{02k,} {\mathbf{\Lambda }}_{02k,} {\mathbf{H}}_{02k,}$ and ${\mathbf{H}}_{02yk}$ are given hyperparameters.

Similarly, conditional distributions involved in θ_2ω are derived on the basis of the following conjugate type distributions: for k = 1,…, q21,

$$\begin{array}{*{20}c} {\begin{array}{*{20}c} {p\left({\psi _{2\delta k}^{ - 1} } \right)\mathop = \limits^D Gamma\left[ {\alpha _{02\delta k},\beta _{02\delta k} } \right],} & {p\left({\Lambda _{2\omega k} |\psi _{2\delta k} } \right)} \\ \end{array} \mathop = \limits^D N\left[ {{\mathbf{\Lambda }}_{2\omega k},\psi _{2\delta k} } \right]\mathop = \limits^D N\left[ {{\mathbf{\Lambda }}_{02\omega k},\psi _{2\delta k} {\mathbf{H}}_{02\omega k} } \right],} \\ {p\left({\Phi _2^{ - 1} } \right)\mathop = \limits^D W\left[ {{\mathbf{R}}_{02},\rho _{02},q_{22} } \right],} \\ \end{array} $$

where${\mathbf{\Lambda }}_{2\omega } = \left({B_2^T,\prod _2^T,{\mathbf{\Gamma }}_2^T } \right)^T$ and ${\mathbf{\Lambda }}_{2\omega k}$ is the vector that contains the unknown parameters in the _kth row of Λ_2ω. As these conditional distributions are similar to those in (38), (40) and (41), they are not presented to save space.