Bayesian model selection for a linear model with grouped covariates

Abstract

Model selection for normal linear regression models with grouped covariates is considered under a class of Zellner’s $g$-priors. The marginal likelihood function is derived under the proposed priors, and a simplified closed-form expression is given assuming the commutativity of the projection matrices from the design matrices. As illustration, the marginal likelihood functions of the balanced $q$-way ANOVA models, either solely with main effects or with all interaction effects, are calculated using the closed-form expression. The performance of the proposed priors in model comparison problems is demonstrated by simulation studies on two-way ANOVA models and by two real data studies.

Appendices

Appendix A: Proof of Theorem 1

Note that

$$\begin{aligned} m(\varvec{y}\mid \varvec{g})= & {} \int _{0}^{\infty }{ \frac{1}{\sigma ^2} \frac{1}{(2\pi \sigma ^2)^{n/2}} \prod _{j=1}^{m}{\left[ \frac{|\varvec{X}_j'\varvec{X}_j|^{1/2}}{(2\pi g_j \sigma ^2)^{p_j/2}}\right] }} \nonumber \\&\int {\exp {\left\{ -\frac{1}{2\sigma ^2} [ (\varvec{y}-\varvec{X}\varvec{\beta })'(\varvec{y}-\varvec{X}\varvec{\beta })+ \varvec{\beta }'\varvec{M}\varvec{\beta }]\right\} }\mathrm{d}\varvec{\beta }\mathrm{d}\sigma ^2}. \end{aligned}$$

If we write $\tilde{\varvec{\beta }} = (\varvec{X}'\varvec{X}+\varvec{M})^{-1}\varvec{X}'\varvec{y},$

$$\begin{aligned} (\varvec{y}-\varvec{X}\varvec{\beta })'(\varvec{y}-\varvec{X}\varvec{\beta }) +\varvec{\beta }'\varvec{M}\varvec{\beta }= (\varvec{\beta }-\tilde{\varvec{\beta }})'(\varvec{X}'\varvec{X}+\varvec{M}) (\varvec{\beta }-\tilde{\varvec{\beta }})+\varvec{y}'\varvec{R}\varvec{y}, \end{aligned}$$

where $\varvec{R}$ is defined by (11). Therefore,

$$\begin{aligned} m(\varvec{y}\mid \varvec{g})= & {} \int _{0}^{\infty } \frac{\exp {\left( -\frac{1}{2\sigma ^2}\varvec{y}'\varvec{R}\varvec{y}\right) }}{(2\pi )^{\frac{n-p_0}{2}} (\sigma ^2)^{\frac{n-p_0}{2}+1} |\varvec{X}'\varvec{X}+\varvec{M}|^{\frac{1}{2}}} \prod _{j=1}^{m} {\left( \frac{|\varvec{X}_j'\varvec{X}_j|^{1/2}}{g_j^{p_j/2}}\right) } \mathrm{d}(\sigma ^2) \nonumber \\= & {} \frac{1}{(2\pi )^{\frac{n-p_0}{2}} |\varvec{X}'\varvec{X}+\varvec{M}|^{\frac{1}{2}}} \prod _{j=1}^{m} {\left( \frac{|\varvec{X}_j'\varvec{X}_j|^{1/2}}{g_j^{p_j/2}}\right) } \Gamma \left( \frac{n-p_0}{2}\right) \left( \frac{2}{\varvec{y}'\varvec{R}\varvec{y}}\right) ^{\frac{n-p_0}{2}}. \end{aligned}$$

This proves (10).

Appendix B: Proof of Theorem 2

To prove Theorem 2, we first derive some of the necessary results in the following lemma.

Lemma 2

Suppose that (13) holds.

• (a) Both $\varvec{P}_{\varvec{\gamma }}$ in (14) and $\varvec{A}_{\varvec{\gamma }}$ in (15) are projection matrices.

• (b) For any $\varvec{\gamma }\ne \varvec{\gamma }^*\subseteq \{0,1,\ldots ,m\}$, we have $\varvec{A}_{\varvec{\gamma }}\varvec{A}_{\varvec{\gamma }^*}=\mathbf {0}$.

• (c) We have the expression for the determinant,

  $$\begin{aligned} \left| \varvec{I}_n+\sum _{j=1}^{m}{g_j(\varvec{I}_n-\varvec{P}_0)\varvec{P}_j}\right| = \prod _{\varvec{\gamma }\in \varvec{\Gamma }}{\left( 1+ \sum _{j \in \varvec{\gamma }}{g_j}\right) ^{p_{\varvec{\gamma }}}}. \end{aligned}$$

  (22)

• (d) We have the expression for the inverse,

  $$\begin{aligned} \left[ \varvec{I}_n+\sum _{j=1}^{m}{g_j(\varvec{I}_n-\varvec{P}_0)\varvec{P}_j} \right] ^{-1} = \varvec{I}_n+\sum _{\varvec{\gamma }\in \varvec{\Gamma }} {u_{\varvec{\gamma }}(\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\varvec{\gamma }}}, \end{aligned}$$

  (23)

  where $u_{\varvec{\gamma }}$ is defined as in (18).

• (e) $u_{\varvec{\gamma }}$ defined in (18) satisfies the following property: for any $\varvec{\gamma }_0\in \varvec{\Gamma }$,

  $$\begin{aligned} \sum _{\emptyset \ne \varvec{\gamma }\subseteq \varvec{\gamma }_0}{u_{\varvec{\gamma }}} =-1+\frac{1}{1+\sum _{j\in \varvec{\gamma }_0}{g_j}}. \end{aligned}$$

  (24)

Proof

Parts (a) and (b) are easy. For Part (c), the identity matrix $\varvec{I}_n$ can be decomposed as $\varvec{I}_n=\sum _{\varvec{\gamma }\subseteq \{0,\ldots ,m\}}{\varvec{A}_{\varvec{\gamma }}}$. Since $(\varvec{I}_n-\varvec{P}_0)\varvec{P}_j=\sum _{\varvec{\gamma }\in \varvec{\Gamma }:j\in \varvec{\gamma }}{\varvec{A}_{\varvec{\gamma }}}$, we know that

$$\begin{aligned} \varvec{I}_n+\sum _{j=1}^{m}{g_j(\varvec{I}_n-\varvec{P}_0)\varvec{P}_j}= \sum _{\varvec{\gamma }\notin \varvec{\Gamma }}{\varvec{A}_{\varvec{\gamma }}}+ \sum _{\varvec{\gamma }\in \varvec{\Gamma }}{\varvec{A}_{\varvec{\gamma }}} \left( 1+\sum _{j:j \in \varvec{\gamma }}{g_j}\right) . \end{aligned}$$

$\forall \varvec{\gamma }\subseteq \{0,\ldots ,m\}$, $\varvec{A}_{\varvec{\gamma }}$ is idempotent and symmetric, whose eigenvalues are $p_{\varvec{\gamma }}$ $1$’s and $(n-p_{\varvec{\gamma }})$ $0$’s. Therefore, there is an $n\times p_{\varvec{\gamma }}$ matrix $\varvec{B}_{\varvec{\gamma }}$ (if $p_{\varvec{\gamma }}=0$, we let $\varvec{B}_{\varvec{\gamma }}$ be a null matrix) such that $\varvec{A}_{\varvec{\gamma }}=\varvec{B}_{\varvec{\gamma }}\varvec{B}_{\varvec{\gamma }}'$ and $\varvec{B}_{\varvec{\gamma }}'\varvec{B}_{\varvec{\gamma }}=\varvec{I}_{p_{\varvec{\gamma }}}$. Note that for $\varvec{\gamma }^*\ne \varvec{\gamma }$, $\varvec{B}_{\varvec{\gamma }}'\varvec{B}_{\varvec{\gamma }^*}=\mathbf {0}_{p_{\varvec{\gamma }}\times p_{\varvec{\gamma }^*}}$. Further, if $\varvec{\gamma }\in \varvec{\Gamma }$, write $\varvec{C}_{\varvec{\gamma }}=\sqrt{1+\sum _{j \in \varvec{\gamma }}{g_j}}\varvec{B}_{\varvec{\gamma }};$ if $\varvec{\gamma }\notin \varvec{\Gamma }$, define $\varvec{C}_{\varvec{\gamma }}=\varvec{B}_{\varvec{\gamma }}.$ We then combine all $\varvec{C}_{\varvec{\gamma }}$’s side-by-side into an $n\times n$ matrix $\varvec{C}$ and get $\bigg |\varvec{I}_n+\sum _{j=1}^{m}{g_j(\varvec{I}_n-\varvec{P}_0)\varvec{P}_j}\bigg |=|\varvec{C}\varvec{C}'|=|\varvec{C}'\varvec{C}|$, and (22) follows by noting that $\varvec{C}'\varvec{C}$ is a block diagonal matrix with the diagonal parts being $(1+\sum _{j \in \varvec{\gamma }}{g_j})\varvec{I}_{p_{\varvec{\gamma }}}$ if $\varvec{\gamma }\in \varvec{\Gamma }$, and $\varvec{I}_{p_{\varvec{\gamma }}}$, otherwise.

For Part (d), note that

$$\begin{aligned} (\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\varvec{\gamma }}(\varvec{I}_n-\varvec{P}_0)\varvec{P}_j= \left\{ \begin{array}{ll} (\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\varvec{\gamma }}, &{}\quad \mathrm{if}~~j\in \varvec{\gamma }, \\ (\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\varvec{\gamma }\cup \{j\}}, &{}\quad \mathrm{if}~~j\notin \varvec{\gamma }. \end{array} \right. \end{aligned}$$

Consider the product of $(\varvec{I}_n\!+\!\sum _{j\!=\!1}^{m}{g_j(\varvec{I}_n\!-\!\varvec{P}_0)\varvec{P}_j})$ and $(\varvec{I}_n\!+\sum _{\varvec{\gamma }\in \varvec{\Gamma }}{u_{\varvec{\gamma }} (\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\varvec{\gamma }}})$, the coefficient before each term $(\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\varvec{\gamma }}$ should be zero. We use mathematical induction to prove (23). For $(\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\{j\}}$, we have $g_j+u_{\{j\}}+g_ju_{\{j\}}=0.$ This implies that $u_{\{j\}}=-g_j/(1+g_j)$, so (18) holds when $|\varvec{\gamma }|=1$. For $(\varvec{I}_n-\varvec{P}_0)\varvec{P}_{\varvec{\gamma }}$ with $|\varvec{\gamma }|=k\ge 2$, if (18) holds for $|\varvec{\gamma }|=k-1$, we have $u_{\varvec{\gamma }}+\sum _{j\in \varvec{\gamma }}{g_j(u_{\varvec{\gamma }}+u_{\varvec{\gamma }\setminus \{j\}})}=0$, which implies that

$$\begin{aligned} u_{\varvec{\gamma }}=-\frac{\sum _{j\in \varvec{\gamma }} {g_ju_{\varvec{\gamma }\setminus \{j\}}}}{1+\sum _{j\in \varvec{\gamma }}{g_j}}. \end{aligned}$$

The conclusion (18) also holds for $|\varvec{\gamma }|=k$. Thus, (23) and (18) are proved.

For Part (e), without loss of generality, we only prove (24) for $\varvec{\gamma }_0=\{1,\ldots ,k\}$. In fact,

$$\begin{aligned} \sum _{\emptyset \ne \varvec{\gamma }\subseteq \{1,\ldots ,k\},}{u_{\varvec{\gamma }}}= & {} \sum _{j_1}{ \left( \frac{-g_{j_1}}{1+g_{j_1}} \left( 1+\sum _{j_2}{\frac{-g_{j_2}}{1+g_{j_1}+g_{j_2}} \left( \cdots \left( 1+\sum _{j_k}{\frac{-g_{j_k}}{1+g_{j_1}+\cdots +g_{j_k}}} \right) \cdots \right) } \right) \right) } \\= & {} \sum _{j_1}{ \left( \frac{-g_{j_1}}{1+g_{j_1}} \left( \cdots \left( 1+\sum _{j_{k-1}}{\frac{-g_{j_{k-1}}}{1+g_{j_1}+\cdots +g_{j_{k-1}}}} \left( \frac{1+g_{j_1}+\cdots +g_{j_{k-1}}}{1+g_1+\cdots +g_k} \right) \right) \cdots \right) \right) }\\= & {} \sum _{j_1} \left( \frac{-g_{j_1}}{1+g_{j_1}} \left( 1+\sum _{j_2}{\frac{-g_{j_2}}{1+g_{j_1}+g_{j_2}}} \left( \cdots \left( \frac{1+g_{j_1}+\cdots +g_{j_{k-2}}}{1+g_1+\cdots +g_k} \right) \cdots \right) \right) \right) . \end{aligned}$$

By the induction, we have

$$\begin{aligned} \sum _{\emptyset \ne \varvec{\gamma }\subseteq \{1,\ldots ,k\}}{u_{\varvec{\gamma }}}= & {} \sum _{j_1}{ \left( \frac{-g_{j_1}}{1+g_{j_1}} \frac{1+g_{j_1}}{1+g_1+\cdots +g_k} \right) }\\= & {} \frac{-(g_1+\cdots +g_k)}{1+g_1+\cdots +g_k} ~=~-1+ \frac{1}{1+g_1+\cdots +g_k}. \end{aligned}$$

The lemma is proved. $\square$

Now we are ready to prove Theorem 2. To calculate $\varvec{R}$, we write

$$\begin{aligned} \varvec{D}=\left( \begin{array}{ll} \varvec{I}_{p_0} &{} \mathbf {0} \\ -(\varvec{X}^{*})'\varvec{X}_0(\varvec{X}_0'\varvec{X}_0)^{-1} &{} \varvec{I}_{p_1+\cdots +p_m} \end{array} \right) , \end{aligned}$$

(25)

where $\varvec{X}^*$ is defined in (8). Then $\varvec{R}= \varvec{I}_n-\varvec{X}\varvec{D}'[\varvec{D}(\varvec{X}'\varvec{X})\varvec{D}'+\varvec{D}\varvec{M}\varvec{D}']^{-1}\varvec{D}\varvec{X}'$. We get $[\varvec{D}(\varvec{X}'\varvec{X})\varvec{D}'+\varvec{D}\varvec{M}\varvec{D}']^{-1}= \mathrm{diag}( (\varvec{X}_0'\varvec{X}_0)^{-1}, ~ (\varvec{X}^{*'}(\varvec{I}_n-\varvec{P}_0)\varvec{X}^*+\varvec{M}_1)^{-1})$. Define $\tilde{\varvec{X}}=(\varvec{I}_n-\varvec{P}_0)\varvec{X}^*,$ then $\varvec{R}=(\varvec{I}_n-\varvec{P}_0)-\tilde{\varvec{X}}(\tilde{\varvec{X}}'\tilde{\varvec{X}}+\varvec{M}_1)^{-1}\tilde{\varvec{X}}'$. Use the fact that for invertible matrices $\varvec{\Phi }$ and $\varvec{\Delta }$, $(\varvec{\Phi }+\varvec{\omega }\varvec{\Delta }\varvec{\omega }')^{-1}=\varvec{\Phi }^{-1}-\varvec{\Phi }^{-1}\varvec{\omega }(\varvec{\Delta }^{-1}+\varvec{\omega }'\varvec{\Phi }^{-1}\varvec{\omega })^{-1}\varvec{\omega }'\varvec{\Phi }^{-1}$, and define $\varvec{O}=\tilde{\varvec{X}}\varvec{M}_1^{-1}\tilde{\varvec{X}}'$, then

$$\begin{aligned} \tilde{\varvec{X}}(\tilde{\varvec{X}}'\tilde{\varvec{X}}+\varvec{M}_1)^{-1}\tilde{\varvec{X}}' = \varvec{O}-\varvec{O}(\varvec{I}_n+\varvec{O})^{-1}\varvec{O}=\varvec{I}_n-(\varvec{I}_n+\varvec{O})^{-1}. \end{aligned}$$

(26)

Also,

$$\begin{aligned}&\varvec{I}_n+\varvec{O}= \varvec{I}_n+(\varvec{I}_n-\varvec{P}_0)\varvec{X}^*\varvec{M}_1^{-1}\varvec{X}^{*'}(\varvec{I}_n-\varvec{P}_0) \nonumber \\&=\varvec{I}_n+(\varvec{I}_n-\varvec{P}_0)\left( \sum _{j=1}^{m}{g_j\varvec{P}_j}\right) (\varvec{I}_n-\varvec{P}_0) ~ =~\varvec{I}_n+\sum _{j=1}^{m}{g_j(\varvec{I}_n-\varvec{P}_0)\varvec{P}_j}.\qquad \end{aligned}$$

(27)

Applying (27) and (23) to (26), we get $\tilde{\varvec{X}}(\tilde{\varvec{X}}'\tilde{\varvec{X}}+\varvec{M}_1)^{-1}\tilde{\varvec{X}}' \!=\!-\sum _{\varvec{\gamma }\in \varvec{\Gamma }}{u_{\varvec{\gamma }}(\varvec{I}_n\!-\!\varvec{P}_0)\varvec{P}_{\varvec{\gamma }}}$. This proves (17). Next, we calculate $|\varvec{X}'\varvec{X}+\varvec{M}|$. For $\varvec{D}$ defined in (25), $|\varvec{X}'\varvec{X}+\varvec{M}|=|\varvec{D}\varvec{X}'\varvec{X}\varvec{D}'+\varvec{D}\varvec{M}\varvec{D}'| = |\varvec{X}_0'\varvec{X}_0|\,|\tilde{\varvec{X}}' \tilde{\varvec{X}}+ \varvec{M}_1|$. Using the identity $|\varvec{\omega }\varvec{\Delta }\varvec{\omega }'+\varvec{\Phi }|=|\varvec{\Delta }|\,|\varvec{\Phi }|\,|\varvec{\Delta }^{-1}+\varvec{\omega }'\varvec{\Phi }^{-1}\varvec{\omega }|,$ we have

$$\begin{aligned} |\varvec{X}'\varvec{X}+\varvec{M}|= & {} |\varvec{X}_0'\varvec{X}_0|\,|\varvec{M}_1|\,|\varvec{I}_n +\tilde{\varvec{X}} \varvec{M}_1^{-1} \tilde{\varvec{X}}'| \nonumber \\= & {} \prod _{j=0}^{m} {|\varvec{X}_j'\varvec{X}_j|} \left( \prod _{j=1}^{m} {g_j^{-p_j}}\right) \left| \varvec{I}_n+ \sum _{j=1}^{m} {g_j(\varvec{I}_n-\varvec{P}_0) \varvec{P}_j}\right| . \end{aligned}$$

By Part (c) of Lemma 2,

$$\begin{aligned} |\varvec{X}'\varvec{X}+\varvec{M}|=\prod _{j=0}^{m} {|\varvec{X}_j'\varvec{X}_j|} \left( \prod _{j=1}^{m} {g_j^{-p_j}} \right) \prod _{\varvec{\gamma }\in \varvec{\Gamma }} {\left( 1+\sum _{j \in \varvec{\gamma }} {g_j}\right) ^{p_{\varvec{\gamma }}}}. \end{aligned}$$

(28)

The conclusion (16) follows by plugging (28) into (10). The theorem is proved.

Appendix C: Proof of Theorem 3

For each $\varvec{\gamma }\in \varvec{\Gamma }$, define $\xi _{\varvec{\gamma }}=\xi (\varvec{\gamma })=\bigcap _{\xi \in \varvec{\gamma }}{\xi }$. We first show that for any $\varvec{\gamma }\in \varvec{\Gamma }$,

$$\begin{aligned} \varvec{A}_{\varvec{\gamma }}\ne \mathbf {0} \Rightarrow \varvec{\gamma }=\{\tau :\tau \supseteq \xi (\varvec{\gamma })\}. \end{aligned}$$

(29)

In fact, by the definition of $\xi (\varvec{\gamma })$, $\forall \tau \in \varvec{\gamma },~\xi (\varvec{\gamma }) \subseteq \tau$. On the other hand, if $\exists \tau \supseteq \xi (\varvec{\gamma }) ~ \mathrm{s.t.}~\tau \notin \varvec{\gamma }$, then $(\varvec{I}_n-\varvec{P}_{\tau })\varvec{P}_{\xi _{\varvec{\gamma }}}=\mathbf {0}_{n\times n}$ in $\varvec{A}_{\varvec{\gamma }}$. So (29) holds. Therefore, $\forall \varvec{A}_{\varvec{\gamma }}\ne \mathbf {0}$,

$$\begin{aligned} \varvec{A}_{\varvec{\gamma }} =\varvec{P}_{\xi _{\varvec{\gamma }}}\prod _{\tau \subsetneq \xi _{\varvec{\gamma }}} {(\varvec{I}_n-\varvec{P}_{\tau })} = \prod _{\tau \subsetneq \xi _{\varvec{\gamma }}} {(\varvec{P}_{\xi _{\varvec{\gamma }}}-\varvec{P}_{\tau \cap \xi _{\varvec{\gamma }}})} = \prod _{j\in \xi _{\varvec{\gamma }}} {(\varvec{P}_{\xi _{\varvec{\gamma }}}-\varvec{P}_{\xi _{\varvec{\gamma }}\setminus \{j\}})}, \end{aligned}$$

so $p_{\varvec{\gamma }}=\sum _{\tau \subseteq \xi _{\varvec{\gamma }}} {(-1)^{|\xi _{\varvec{\gamma }}|-|\tau |}p_{\tau }}$. In (16),

$$\begin{aligned} \prod _{\varvec{\gamma }\in \varvec{\Gamma }} {\left( 1+\sum _{j \in \varvec{\gamma }}{g_j}\right) ^{-p_{\varvec{\gamma }}/2}}= \prod _{\emptyset \ne \xi \subseteq \{1,\ldots ,q\}} {\left( 1+\sum _{\tau \supseteq \xi }{g_{\tau }}\right) }^{-\frac{1}{2} \sum _{\tau ^*\subseteq \xi } {(-1)^{|\xi |-|\tau ^*|}p_{\tau ^*}}}. \end{aligned}$$

Next, we need to calculate $\varvec{R}$ in this case. In (17), $\varvec{P}_{\varvec{\gamma }}=\prod _{\xi \in \varvec{\gamma }}{\varvec{P}_{\xi }} =\varvec{P}_{\xi _{\varvec{\gamma }}}$. Therefore, in (17),

$$\begin{aligned} \sum _{\varvec{\gamma }\in \varvec{\Gamma }} {u_{\varvec{\gamma }}(\varvec{I}_n-\varvec{P}_{\varvec{\phi }})\varvec{P}_{\varvec{\gamma }}}= \sum _{\emptyset \ne \tau \subseteq \{1,2,\ldots ,q\}} {\left( (\varvec{I}_n-\varvec{P}_{\varvec{\phi }})\varvec{P}_{\tau } \sum _{\varvec{\gamma }:\xi (\varvec{\gamma })=\tau }{u_{\varvec{\gamma }}}\right) }. \end{aligned}$$

The theorem will be proved given the following lemma.

Lemma 3

For nonempty $\tau \subseteq \{1,2,\ldots ,q\}$, define $U'_{\tau }=\sum _{\varvec{\gamma }\in \varvec{\Gamma }:\xi (\varvec{\gamma })=\tau }{u_{\varvec{\gamma }}}$. Then we have

$$\begin{aligned} \sum _{\tau ^*\supseteq \tau }{U'_{\tau ^*}}= & {} -1+\frac{1}{1+\sum _{\tau ^*\supseteq \tau }{g_{\tau ^*}}}.\end{aligned}$$

(30)

$$\begin{aligned} U'_{\tau }= & {} \left\{ \begin{array}{ll} \sum _{\tau ^*\supseteq \tau } {(-1)^{|\tau ^*|-|\tau |} \frac{1}{1+\sum _{\tau ^{**}\supseteq \tau ^*}{g_{\tau ^{**}}}}}, &{}\quad \text{ if } \,\,\tau \subsetneq \{1,\ldots ,q\}, \\ -1+\frac{1}{1+g_{\{1,\ldots ,q\}}}, &{}\quad \text{ if } \,\,\tau = \{1,\ldots ,q\}. \end{array} \right. \end{aligned}$$

(31)

Proof

For (30), note that for any $\varvec{\gamma }\in \varvec{\Gamma }$, $\xi (\varvec{\gamma })\supseteq \tau \Leftrightarrow \varvec{\gamma }\subseteq \{\tau ^*:\tau ^*\supseteq \tau \}$. Therefore, using Lemma 2(e), we get

$$\begin{aligned} \sum _{\tau ^*\supseteq \tau }{U'_{\tau ^*}}= \sum _{\emptyset \ne \varvec{\gamma }\subseteq \{ \tau ^*:\tau ^*\supseteq \tau \}} {u_{\varvec{\gamma }}} = -1+\frac{1}{1+\sum _{\tau ^*\supseteq \tau }{g_{\tau ^*}}}. \end{aligned}$$

For (31), we use mathematical induction. If $\tau =\{1,\ldots ,q\}$, (31) is exactly (30). If $\tau =\{1,\ldots ,q-1\}$ , from (30), we have $U'_{\{1,\ldots ,q\}}+U'_{\{1,\ldots ,q-1\}} =-1+1/(1+g_{\{1,\ldots ,q-1\}}+g_{\{1,\ldots ,q\}})$, which implies that

$$\begin{aligned} U'_{\{1,\ldots ,q-1\}}= \frac{1}{1+g_{\{1,\ldots ,q-1\}}+g_{\{1,\ldots ,q\}}}- \frac{1}{1+g_{\{1,\ldots ,q\}}}. \end{aligned}$$

This proves that (31) holds for $\tau$ with $|\tau |=q-1$. Clearly, (30) implies a recursive formula,

$$\begin{aligned} U'_{\tau }= -1+\frac{1}{1+\sum _{\tau ^*\supseteq \tau }{g_{\tau ^*}}}- \sum _{\tau ^*\supsetneq \tau }{U'_{\tau ^*}}. \end{aligned}$$

Suppose (31) holds for $\tau$ with $|\tau |=k+1$, then

$$\begin{aligned} U'_{\{1,\ldots ,k\}}= & {} -1+\frac{1}{1+\sum _{\tau \supseteq \{1,\ldots ,k\}} {g_{\tau }}}- \sum _{\tau \supsetneq \{1,\ldots ,k\}}{U'_{\tau }} \nonumber \\= & {} \frac{1}{1+\sum _{\tau \supseteq \{1,\ldots ,k\}} {g_{\tau }}}- \sum _{\tau \supsetneq \{1,\ldots ,k\}} {\left( \sum _{\tau ^*\supseteq \tau } {(-1)^{|\tau ^*|-|\tau |} \frac{1}{1+\sum _{\tau ^{**}\supseteq \tau ^*} {g_{\tau ^{**}}}}} \right) } \nonumber \\= & {} \frac{1}{1+\sum _{\tau \supseteq \{1,\ldots ,k\}} {g_{\tau }}}\\&- \sum _{\tau ^*\supsetneq \{1,\ldots ,k\}} \left( \frac{1}{1+\sum _{\tau ^{**}\supseteq \tau ^*} {g_{\tau ^{**}}}}\left( \sum _{\tau :\tau ^*\supseteq \tau \supsetneq \{1,\ldots ,k\}} {(-1)^{|\tau ^*|-|\tau |}}\right) \right) \nonumber \\= & {} \frac{1}{1+\sum _{\tau \supseteq \{1,\ldots ,k\}} {g_{\tau }}}+ \sum _{\tau ^*\supsetneq \{1,\ldots ,k\}} {\left( \frac{1}{1+\sum _{\tau ^{**}\supseteq \tau ^*} {g_{\tau ^{**}}}} (-1)^{|\tau ^*|-k} \right) } \nonumber \\= & {} \sum _{\tau ^*\supseteq \{1,\ldots ,k\}} {\left( \frac{1}{1+\sum _{\tau ^{**}\supseteq \tau ^*} {g_{\tau ^{**}}}} (-1)^{|\tau ^*|-k} \right) }. \end{aligned}$$

Therefore, (31) holds for $\tau$ with $|\tau |=k$. Repeat this procedure recursively, we can show that (31) holds for any nonempty $\tau \subsetneq \{1,\ldots ,q\}$. The lemma is proved. $\square$