Minimum $\phi$-Divergence Estimation in Constrained Latent Class Models for Binary Data

Abstract

The main purpose of this paper is to introduce and study the behavior of minimum $\phi$-divergence estimators as an alternative to the maximum-likelihood estimator in latent class models for binary items. As it will become clear below, minimum $\phi$-divergence estimators are a natural extension of the maximum-likelihood estimator. The asymptotic properties of minimum $\phi$-divergence estimators for latent class models for binary data are developed. Finally, to compare the efficiency and robustness of these new estimators with that obtained through maximum likelihood when the sample size is not big enough to apply the asymptotic results, we have carried out a simulation study.

Appendix

Remark 1

We are going to develop the calculations for ${{\partial p(\mathbf{y_{\nu }}, \varvec{\lambda }, \varvec{\eta })\over \partial \lambda _{\alpha }}}$ and ${{\partial p(\mathbf{y_{\nu }}, \varvec{\lambda }, \varvec{\eta })\over \partial \eta _{\beta }}}.$

For ${{\partial p(\mathbf{y_{\nu }}, \varvec{\lambda }, \varvec{\eta })\over \partial \lambda _{\alpha }}}$ note that

$$\begin{aligned} p(\mathbf{y_{\nu }}, \varvec{\lambda }, \varvec{\eta }) \!&= \! \sum _{j=1}^m w_j \prod _{i=1}^k \left( {exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r \!+\! c_{ji}}\right) \over 1 \!+\!exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r \!+\! c_{ji}}\right) }\right) ^{y_{\nu i}} \left( 1\!-\! {exp \left( { \sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) \over 1 +exp \left( { \sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) } \right) ^{1-y_{\nu i}} \\&= \sum _{j=1}^m w_j \prod _{i=1}^k {exp \left( y_{\nu i}\left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) \right) \over 1 +exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) } \end{aligned}$$

Now,

$$\begin{aligned} {\partial \left( {exp \left( y_{\nu i}\left( { \sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) \right) \over 1 +exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) } \right) \over \partial \lambda _{\alpha }}&= {exp \left( y_{\nu i}\left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) \right) q_{ji\alpha }\over 1 +exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) }\\&\quad \times \left[ y_{\nu i} - {exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) \over 1 +exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) }\right] \\&= {exp \left( y_{\nu i}\left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) \right) \over 1 +exp \left( {\sum \nolimits _{r=1}^t q_{jir} \lambda _r + c_{ji}}\right) } q_{ji\alpha }\left( y_{\nu i} - p_{ji}\right) , \end{aligned}$$

whence

$$\begin{aligned} {\partial p(\mathbf{y_{\nu }}, \varvec{\lambda }, \varvec{\eta })\over \partial \lambda _{\alpha }} = \sum _{j=1}^m w_j \left[ Pr (\mathbf{y_{\nu }} | P_{\nu }\in C_j ) \sum _{i=1}^k q_{ji\alpha } (y_{\nu i}- p_{ji})\right] , \, \, \alpha =1,\ldots , t. \end{aligned}$$

Similarly,

$$\begin{aligned} p(\mathbf{y_{\nu }}, \varvec{\lambda }, \varvec{\eta }) = \sum _{j=1}^m {exp ({ \sum \nolimits _{r=1}^u v_{jr} \eta _r + d_{j}})\over {\sum \nolimits _{j=1}^m exp (\sum \nolimits _{r=1}^u v_{jr} \eta _r + d_{j})}} Pr (\mathbf{y_{\nu }} | P_{\nu }\in C_j ). \end{aligned}$$

Now,

$$\begin{aligned} {\partial \left( {exp \left( {\sum \nolimits _{r=1}^u v_{jr} \eta _r + d_{j}}\right) \over {\sum \nolimits _{h=1}^m exp \left( \sum \nolimits _{r=1}^u v_{hr} \eta _r + d_{h}\right) }} \right) \over \partial \eta _{\beta } }&= {exp \left( {\sum \nolimits _{r=1}^u v_{jr} \eta _r + d_{j}}\right) \over {\sum \nolimits _{h=1}^m exp \left( \sum \nolimits _{r=1}^u v_{hr} \eta _r + d_{h}\right) }}\\&\times \left[ v_{j\beta } - {{\sum \nolimits _{h=1}^m exp (\sum _{r=1}^u v_{hr} \eta _r + d_{h}})v_{h\beta }\over {\sum \nolimits _{h=1}^m exp \left( \sum \nolimits _{r=1}^u v_{hr} \eta _r + d_{h}\right) }}\right] \\&= w_j \left[ v_{j\beta } - \sum _{h=1}^m w_h v_{h\beta }\right] , \end{aligned}$$

whence

$$\begin{aligned} {\partial p(\mathbf{y_{\nu }}, \varvec{\lambda }, \varvec{\eta })\over \partial \eta _{\beta }} = \sum _{j=1}^m w_j Pr (\mathbf{y_{\nu }} | P_{\nu }\in C_j )\left[ v_{j\beta } - \sum _{h=1}^m w_h v_{h\beta } \right] ,\, \, \beta =1,\ldots , u. \end{aligned}$$

Proof of Theorem 1

Let $l^{2^k}$ be the interior of the $2^k$-dimensional unit cube; then, the interior of $\Delta _{2^k}$ is contained in $l^{2^k}.$ Let $W$ be a neighborhood of $(\varvec{\lambda }_0, \varvec{\eta }_0 ),$ the true value of the unknown parameter $(\varvec{\lambda }, \varvec{\eta }),$ on which

has continuous second partial derivatives. Let

$$\begin{aligned} \mathbf{F}:=(F_1,\ldots , F_{t+u}): l^{2^k} \times W \rightarrow \mathbb {R}^{t+u} \end{aligned}$$

whose components $F_j,\, j=1,\ldots , t+u$ are defined by

$$\begin{aligned} F_j(\tilde{p}_1,\ldots , \tilde{p}_{2^k}; \lambda _1,\ldots , \lambda _t; \eta _1,\ldots , \eta _u ):= {\partial D_{\phi }({\tilde{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta })) \over \partial s_j},\, j=1,\ldots , t+u, \end{aligned}$$

where $s_j$ is defined in (8).

It holds

$$\begin{aligned} F_j(p_1(\varvec{\lambda }_0 , \varvec{\eta }_0 ),\ldots , p_{2^k}(\varvec{\lambda }_0 , \varvec{\eta }_0 );\lambda _1^0,\ldots , \lambda _t^0; \eta _1^0,\ldots , \eta _u^0)=0,\, \forall j=1,\ldots , t+u \end{aligned}$$

due to

$$\begin{aligned} {\partial D_{\phi } ({\tilde{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta }))\over \partial \lambda _{\alpha }}&= \sum _{\nu =1}^{2^k} \left\{ \phi \left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) - {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \phi '\left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) \right\} {\partial p_{\nu }( \varvec{\lambda }, \varvec{\eta })\over \partial \lambda _{\alpha }} ,\, \alpha =1,\ldots , t.\\ {\partial D_{\phi } ({\hat{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta }))\over \partial \eta _{\beta }}&= \sum _{\nu =1}^{2^k} \left\{ \phi \left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) - {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \phi '\left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) \right\} {\partial p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial \eta _{\beta }} ,\, \beta =1,\ldots , u. \end{aligned}$$

In the following we shall rewrite the two previous expressions by

$$\begin{aligned} {\partial D_{\phi } ({ \tilde{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta }))\over \partial s_{j}},\, j=1,\ldots , t+u. \end{aligned}$$

Since

$$\begin{aligned} {\partial \over \partial s_r}\left( {\partial D_{\phi } ({ \tilde{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta }))\over \partial s_{j}} \right)&= -\sum _{\nu =1}^{2^k} \phi '\left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })^2} {\partial p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial s_{r}} {\partial p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial s_{j}} \\&+ \sum _{\nu =1}^{2^k} \phi ''\left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })^2} {\partial p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial s_{r}} {\partial p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial s_{j}} {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })}\\&+ \sum _{\nu =1}^{2^k} \phi '\left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })^2} {\partial p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial s_{r}} {\partial p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial s_{j}} \\&+ \sum _{\nu =1}^{2^k} {\partial ^2 p_{\nu }(\varvec{\lambda }, \varvec{\eta })\over \partial s_{r} s_j} \left\{ \phi \left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) - \phi '\left( {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })} \right) {\tilde{p}_{\nu }\over p_{\nu }(\varvec{\lambda }, \varvec{\eta })}\right\} , \end{aligned}$$

and denoting $\pi _{\nu }=p_{\nu }(\varvec{\lambda }_0 , \varvec{\eta }_0), \nu =1,\ldots , 2^k,$ the $(t+u)\times (t+u)$ matrix $\mathbf{J}_\mathbf{F}$ associated with function $\mathbf{F}$ at point $(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), (\varvec{\lambda }_0 , \varvec{\eta }_0 ))$ is given by

$$\begin{aligned} {\partial \mathbf{F}\over \partial (\varvec{\lambda }_0 , \varvec{\eta }_0 )}&= \left( {\partial \mathbf{F}\over \partial (\varvec{\lambda }, \varvec{\eta })} \right) _{({\tilde{\mathbf{p}}}, (\varvec{\lambda }, \varvec{\eta }))= (\pi _1,\ldots , \pi _{2^k}; \lambda _1^0,\ldots , \lambda _t^0; \eta _1^0,\ldots , \eta _u^0)} \\&= \left( \left( {\partial \over \partial s_r}\left( {\partial D_{\phi } ({\tilde{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta }))\over \partial s_{j}} \right) \right) _{\mathop {r=1,\ldots , t+u}\limits ^{j=1,\ldots , t+u}} \right) _{({\tilde{\mathbf{p}}}, (\varvec{\lambda }, \varvec{\eta })) =(\pi _1,\ldots , \pi _{2^k}; \lambda _1^0,\ldots , \lambda _t^0; \eta _1^0,\ldots , \eta _u^0)} \\&= \phi ''(1) \left( \sum _{l=1}^{2^k} {1\over p_l(\varvec{\lambda }_0 , \varvec{\eta }_0 )}{\partial p_l(\varvec{\lambda }_0 , \varvec{\eta }_0 )\over \partial s_{r}} {\partial p_l(\varvec{\lambda }_0 , \varvec{\eta }_0 )\over \partial s_{j}} \right) _{\mathop {r=1,\ldots , t+u}\limits ^{j=1,\ldots , t+u}} \end{aligned}$$

To get the last expression we are using that $\phi (1)=\phi '(1)=0.$ Recall that if $\mathbf{B}$ is a $p\times q$ matrix with $rank(\mathbf{B})=p$ and $\mathbf{C}$ is a $q\times s$ matrix with $rank(\mathbf{C})=q,$ then $rank(\mathbf{BC})=p.$ Taking

$$\begin{aligned} \mathbf{B}= \mathbf{J}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T, \, \, \mathbf{C}=\mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)}^{-{1\over 2}}, \end{aligned}$$

it follows that $\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T=\mathbf{BC}$ has rank $t+u$ applying the fourth condition of Birch. Also,

$$\begin{aligned} rank (\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0))=rank (\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T)=rank (\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0))=t+u. \end{aligned}$$

Therefore, the $(t+u)\times (t+u)$ matrix ${\partial \mathbf{F}\over \partial (\varvec{\lambda }_0 , \varvec{\eta }_0)}$ is nonsingular at $(\pi _1, \ldots , \pi _{2^k}; \lambda _1^0,\ldots , \lambda _t^0; \eta _1^0,\ldots , \eta _u^0)$.

Applying the Implicit Function Theorem, there exists a neighborhood $U$ of $(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), (\varvec{\lambda }_0 , \varvec{\eta }_0 ))$ such that the matrix $\mathbf{J}_\mathbf{F}$ is nonsingular (in our case $\mathbf{J}_\mathbf{F}$ at $(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), (\varvec{\lambda }_0 , \varvec{\eta }_0 ))$ is positive definite and then it is continuously differentiable). Also, there exists a continuously differentiable function

$$\begin{aligned} \tilde{\varvec{\theta }}:A\subset l^{2^k} \rightarrow \mathbb {R}^{t+u} \end{aligned}$$

such that $\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)\in A$ and

$$\begin{aligned} \left\{ ({\tilde{\mathbf{p}}}, (\varvec{\lambda }, \varvec{\eta }))\in U : \mathbf{F}({\tilde{\mathbf{p}}}, (\varvec{\lambda }, \varvec{\eta }))=0\right\} =\left\{ ({\tilde{\mathbf{p}}}, \tilde{\varvec{\theta }}({\tilde{\mathbf{p}}})): { \tilde{\mathbf{p}}}\in A \right\} . \end{aligned}$$

(12)

We can observe that $\tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0))$ is an argmin of

$$\begin{aligned} \psi (\varvec{\lambda }, \varvec{\eta }):= D_{\phi }(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), \mathbf{p}(\varvec{\lambda }, \varvec{\eta })) \end{aligned}$$

because $\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)\in A$ and then

$$\begin{aligned} \mathbf{F}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), \tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0))) = {\partial D_{\phi }(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), \mathbf{p}(\tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)))) \over \partial (\varvec{\lambda }, \varvec{\eta })} = \mathbf{0}. \end{aligned}$$

On the other hand, applying (12),

$$\begin{aligned} (\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), \tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0))) \in U, \end{aligned}$$

and then $\mathbf{J}_\mathbf{F}$ is positive definite at $(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), \tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0))).$ Therefore,

$$\begin{aligned} D_{\phi }(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), \mathbf{p}(\tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)))) = \inf _{(\varvec{\lambda }, \varvec{\eta })\in \Theta } D_{\phi }(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0), \mathbf{p}(\varvec{\lambda }, \varvec{\eta })), \end{aligned}$$

and by the $\phi$-divergence properties $\tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0))= (\varvec{\lambda }_0 , \varvec{\eta }_0 )^T,$ and

$$\begin{aligned} {\partial \mathbf{F}\over \partial \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) } - {\partial \mathbf{F} \over \partial (\varvec{\lambda }_0 , \varvec{\eta }_0)} {\partial (\varvec{\lambda }_0 , \varvec{\eta }_0 )\over \partial \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)} =\mathbf{0}. \end{aligned}$$

Further, we know that

$$\begin{aligned} {\partial \mathbf{F} \over \partial (\varvec{\lambda }_0 , \varvec{\eta }_0 )} = \phi ''(1) \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0) \end{aligned}$$

and we shall establish later that the $(t+u)\times 2^k$ matrix ${\partial \mathbf{F} \over \partial \varvec{\pi }}$ is

$$\begin{aligned} {\partial \mathbf{F} \over \partial \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)} = \phi '' (1) \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 )}^{-{1\over 2}}. \end{aligned}$$

(13)

Therefore, the $(t+u)\times 2^k$ matrix ${\partial (\varvec{\lambda }_0 , \varvec{\eta }_0 )\over \partial \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) }$ is

$$\begin{aligned} {\partial (\varvec{\lambda }_0 , \varvec{\eta }_0 )\over \partial \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) } = (\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0))^{-1} \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 )}^{-{1\over 2}}. \end{aligned}$$

The Taylor expansion of the function $\tilde{\varvec{\theta }}$ around $\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)$ yields

$$\begin{aligned} \tilde{\varvec{\theta }}({\tilde{\mathbf{p}}}) = \tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)) + \left( {\partial \tilde{\varvec{\theta }}({\tilde{\mathbf{p}}}) \over {\tilde{\mathbf{p}}}}\right) _{{ \tilde{\mathbf{p}}}=\varvec{\pi }} ({\tilde{\mathbf{p}}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)) + o(\Vert {\tilde{\mathbf{p}}} -\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) \Vert ). \end{aligned}$$

As $\tilde{\varvec{\theta }}(\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)) = (\varvec{\lambda }_0 , \varvec{\eta }_0 )^T,$ we obtain from here

$$\begin{aligned} \tilde{\varvec{\theta }}({\tilde{\mathbf{p}}})&= (\varvec{\lambda }_0 , \varvec{\eta }_0 )^T+ (\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0))^{-1} \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 }^{-{1\over 2}} ({ \tilde{\mathbf{p}}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0))\\&+ o(\Vert { \tilde{\mathbf{p}}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) \Vert ). \end{aligned}$$

We know that ${\hat{\mathbf{p}}}{\overset{a.s.}{\longrightarrow }} \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 ),$ so that ${\hat{\mathbf{p}}}\in A$ and, consequently, $\tilde{\varvec{\theta }}({\hat{\mathbf{p}}})$ is the unique solution of the system of equations

$$\begin{aligned} {\partial D_{\phi }( {\hat{\mathbf{p}}}, \mathbf{p} ( \tilde{\varvec{\theta }}({\hat{\mathbf{p}}})))\over s_j} =0,\, j=1,\ldots , t+u, \end{aligned}$$

and also $( {\hat{\mathbf{p}}}, \tilde{\varvec{\theta }}({\hat{\mathbf{p}}}))\in U.$ Therefore, $\tilde{\varvec{\theta }}({\hat{\mathbf{p}}})$ is the minimum $\phi$-divergence estimator, $\hat{\varvec{\theta }}_{\phi }$, satisfying the relation

$$\begin{aligned} \hat{\varvec{\theta }}_{\phi }&= (\varvec{\lambda }_0 , \varvec{\eta }_0 )^T+ (\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0))^{-1} \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 )}^{-{1\over 2}} ({\hat{\mathbf{p}}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 ))\\&+ o(\Vert {\hat{\mathbf{p}}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 )\Vert ). \end{aligned}$$

Finally, we are going to establish (13). We compute the $(i,j)$-th element of the $(t+u)\times 2^k$ matrix ${\partial \mathbf{F}\over \partial \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) }.$

$$\begin{aligned} {\partial \over \partial p_i} \left( {\partial D_{\phi } ({\tilde{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta }))\over \partial s_{j}} \right)&= {\partial \over \partial p_i} \left( \sum _{l=1}^{2^k} \left\{ \phi \left( {\tilde{p}_{l} \over p_l(\varvec{\lambda }, \varvec{\eta })} \right) - \phi ' \left( {\tilde{p}_{l}\over p_l(\varvec{\lambda }, \varvec{\eta })} \right) {\tilde{p}_{l}\over p_l(\varvec{\lambda }, \varvec{\eta })}\right\} {\partial p_l(\varvec{\lambda }, \varvec{\eta })\over \partial s_j} \right) \\&= {1\over p_i(\varvec{\lambda }, \varvec{\eta })} \left( -{p_i\over p_i (\varvec{\lambda }, \varvec{\eta })} \phi ''\left( {p_i\over p_i (\varvec{\lambda }, \varvec{\eta })}\right) \right) {\partial p_i(\varvec{\lambda }, \varvec{\eta })\over \partial s_j} \end{aligned}$$

and for $(\pi _1,\ldots , \pi _{2^k}; \lambda _1^0,\ldots , \lambda _t^0; \eta _1^0,\ldots , \eta _u^0)$ we have

$$\begin{aligned} {\partial \over \partial p_i} \left( {\partial D_{\phi } ({ \tilde{\mathbf{p}}}, \mathbf{p}(\varvec{\lambda }, \varvec{\eta }))\over \partial s_{j}} \right) = {1\over p_i (\varvec{\lambda }_0 , \varvec{\eta }_0 )} \phi ''\left( 1\right) {\partial p_i(\varvec{\lambda }_0 , \varvec{\eta }_0 )\over \partial s_j}. \end{aligned}$$

Since $\mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)=\mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0 )}^{-{1\over 2}} \mathbf{J}(\varvec{\lambda }_0 , \varvec{\eta }_0),$ then (13) holds. $\square$

Proof of Theorem 2

Applying the previous theorem, it holds

$$\begin{aligned} \sqrt{N}(\hat{\varvec{\theta }}_{\phi } -(\varvec{\lambda }_0 , \varvec{\eta }_0)^T)&= \left( \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0 )^T \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0) \right) ^{-1} \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0) \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)}^{-{1\over 2}} \sqrt{N} (\hat{\mathbf{p}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0))\\&+\, \sqrt{N} \, \, \, o(\Vert \hat{\mathbf{p}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) \Vert ). \end{aligned}$$

Note that

$$\begin{aligned} \sqrt{N} \, \, \, o(\Vert \hat{\mathbf{p}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) \Vert ) = o_p(1). \end{aligned}$$

On the other hand, as $\hat{\mathbf{p}}$ is the sample proportion, we can apply the Central Limit Theorem to conclude

$$\begin{aligned} \sqrt{N} (\hat{\mathbf{p}} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)) {\overset{L}{\longrightarrow }} \mathcal{N}(\mathbf{0} , \varvec{\Sigma }_{\mathbf{p}(\varvec{\lambda }_0, \varvec{\eta }_0)} ), \end{aligned}$$

where $\varvec{\Sigma }_{\mathbf{p}(\varvec{\lambda }_0, \varvec{\eta }_0)}$ is given by

$$\begin{aligned} \varvec{\Sigma }_{\mathbf{p}(\varvec{\lambda }_0, \varvec{\eta }_0)} = \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)} - \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0) \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T. \end{aligned}$$

Therefore, it follows

$$\begin{aligned} \sqrt{N}(\hat{\varvec{\theta }}_{\phi } -(\varvec{\lambda }_0 , \varvec{\eta }_0 )) {\overset{L}{\longrightarrow }} \mathcal{N}(\mathbf{0} , \varvec{\Sigma }^*), \end{aligned}$$

where $\varvec{\Sigma }^*$ is given by

$$\begin{aligned} \varvec{\Sigma }^* = \left( \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0 )^T \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0) \right) ^{-1} - \mathbf{B} \mathbf{B}^T \end{aligned}$$

with $\mathbf{B}:=\left( \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0 )^T \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0) \right) ^{-1} \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)}^{1\over 2}.$

It is not difficult to see that

$$\begin{aligned} \mathbf{D}_{\mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)}^{{1\over 2}} \mathbf{A}(\varvec{\lambda }_0 , \varvec{\eta }_0)=0, \end{aligned}$$

whence $\mathbf{B}=\mathbf{0}$ and the result holds. $\square$

Proof of Theorem 3

Using Theorem 2, it suffices to apply the delta method. Then, we can conclude that

$$\begin{aligned} \sqrt{N}(\mathbf{p}( \hat{\varvec{\theta }}_{\phi })- \mathbf{p} (\varvec{\lambda }_0, \varvec{\eta }_0 )) {\overset{L}{\longrightarrow }} \mathcal{N}(\mathbf{0} , \nabla \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)^T\left( \mathbf{A}(\varvec{\lambda }_0, \varvec{\eta }_0 )^t \mathbf{A}(\varvec{\lambda }_0, \varvec{\eta }_0 ) \right) ^{-1} \nabla \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)). \end{aligned}$$

Now, as $\nabla \mathbf{p}(\varvec{\lambda }_0 , \varvec{\eta }_0)= \mathbf{J}(\varvec{\lambda }_0 , \varvec{\eta }_0)$, the theorem is proved. $\square$