A sequential distance-based approach for imputing missing data: Forward Imputation

Abstract

Missing data recurrently affect datasets in almost every field of quantitative research. The subject is vast and complex and has originated a literature rich in very different approaches to the problem. Within an exploratory framework, distance-based methods such as nearest-neighbour imputation (NNI), or procedures involving multivariate data analysis (MVDA) techniques seem to treat the problem properly. In NNI, the metric and the number of donors can be chosen at will. MVDA-based procedures expressly account for variable associations. The new approach proposed here, called Forward Imputation, ideally meets these features. It is designed as a sequential procedure that imputes missing data in a step-by-step process involving subsets of units according to their “completeness rate”. Two methods within this context are developed for the imputation of quantitative data. One applies NNI with the Mahalanobis distance, the other combines NNI and principal component analysis. Statistical properties of the two methods are discussed, and their performance is assessed, also in comparison with alternative imputation methods. To this purpose, a simulation study in the presence of different data patterns along with an application to real data are carried out, and practical hints for users are also provided.

Appendix

In FIP, the requirement: $n_{k} \ge p$ for each $k=0,1, \ldots , K$ is not binding. For a given k, suppose that $n_{k} < p$, and consider the correlation matrix $\mathbf {R}_{k}$ computed from the complete matrix ${\mathbf {X}}_{k}$ (similar arguments would hold for the variance-covariance matrix $\varvec{\Sigma }_{k}$). Let $\mathbf {Z}_{k}$ be the standardized matrix derived from ${\mathbf {X}}_{k}$. It follows that: $\mathbf {R}_{k} = \frac{1}{n_{k}} \mathbf {Z}_{k}^t \mathbf {Z}_{k}$, for which: ${\mathrm{{rank}{(}}}\mathbf {R}_{k}) \le n_{k} < p$. Now, consider the $n_{k} \times n_{k}$ product-matrix: $\mathbf {P}_{k} = \mathbf {Z}_{k} \mathbf {Z}_{k}^t$. Standard results of matrix algebra ensure that $\mathbf {P}_{k}$ and $\mathbf {R}_{k}$ have the same rank. In particular, as $\mathbf {Z}_{k}$ has zero-mean columns, it always holds that: $\mathbf {Z}_{k} \mathbf {Z}_{k}^t \mathbf {1}= 0 \cdot \mathbf {1}= \mathbf {0}$, where $\mathbf {1}$ is a vector of $n_{k}$ ones, so that $\mathbf {P}_{k}$ always admits a zero eigenvalue. Therefore, ${\mathrm{{rank}{(}}}\mathbf {P}_{k}) = {\mathrm{{rank}{(}}}\mathbf {R}_{k}) \le n_{k} -1$. Moreover, $n_{k} \mathbf {R}_{k}$ has the same non-null eigenvalues of $\mathbf {P}_{k}$, with (at least) extra $p - n_{k} + 1$ eigenvalues equal to zero.

Let $\eta ^{(k)}_{s}$ be a non-null eigenvalue of $\mathbf {P}_{k}$, and $\mathbf {v}^{(k)}_{s}$ the corresponding normalized eigenvector ($s = 1, \ldots , n^{\prime }_k$ $\le n_{k} -1$, where: $n^{\prime }_k = {\mathrm{{rank}{(}}}\mathbf {P}_{k})$). By definition, $\mathbf {P}_{k} \mathbf {v}^{(k)}_{s} = \eta ^{(k)}_{s} \mathbf {v}^{(k)}_{s}$. Pre-multiplying both members by $\mathbf {Z}_{k}^t$ leads to: $n_{k} \mathbf {R}_{k} \mathbf {Z}_{k}^t \mathbf {v}^{(k)}_{s} = \eta ^{(k)}_{s} \mathbf {Z}_{k}^t \mathbf {v}^{(k)}_{s}$, from which it is apparent that: $\mathbf {Z}_{k}^t \mathbf {v}^{(k)}_{s} = \varvec{\xi }^{(k)}_{s}$ is the s-th p-dimensional eigenvector of $n_{k} \mathbf {R}_{k}$. Finally, from the fact that: ${\varvec{\xi }^{(k)}_{s}}^{t} \varvec{\xi }^{(k)}_{s} = {\mathbf {v}^{(k)}_{s}}^{t} \mathbf {P}_{k} \mathbf {v}^{(k)}_{s} = \eta ^{(k)}_{s} {\mathbf {v}^{(k)}_{s}}^{t} \mathbf {v}^{(k)}_{s} = \eta ^{(k)}_{s}$, it derives that the s-th normalized eigenvector of $n_{k} \mathbf {R}_{k}$, which is also an eigenvector for $\mathbf {R}_{k}$, is given by: $\varvec{\omega }^{(k)}_{s} = \varvec{\xi }^{(k)}_{s} / \sqrt{\eta ^{(k)}_{s}}$, while the s-th eigenvalue of $\mathbf {R}_{k}$ is: $\lambda ^{(k)}_{s} = \eta ^{(k)}_{s} / n_{k}$, for $s= 1, \ldots , n^{\prime }_k \le n_{k} -1 < p$.