Smooth manifold extraction in high-dimensional data using a deep model

Abstract

Manifold is considered to be the explicit form of data, so the smoothness of manifold is related to data dimensionality. Data becomes sparse in the high-dimensional space, which hardly affords sufficient information. Thus, it is a challenge for smooth manifold extraction from the data existing in high-dimensional space. To address this issue, here proposes a deep model of having three-hidden layers for smooth manifold extraction. Our thought is originate from the view of the optimal transportation mass theory. Because high-dimensional data resides around a low-dimensional manifold, we can reconstruct a lower dimensional manifold in high-dimensional space. To guarantee the quality of the reconstructed manifold, the sampling condition is used in order to reconstruct a discrete surface that can converge to an original surface. Meanwhile, the loss function derived by Brenier theorem minimizes the error between the original data distribution and the reconstructed data distribution. In addition, to promote the generalization ability of our model, the neurons in the hidden layers are turned off with the probability manner just during training. Experimental results show our method outperforms the state-of-the-art methods in smooth manifold extraction. We find that as for a deep model, the manner of turning off some neurons using probability carries more weights in improving the smoothness of manifold extraction than investing the effort of simply stacking hidden layers. Moreover, the manner of turning off some neurons using probability also mitigates over-fitting to a certain extent. Our finding also suggests that for high-dimensional space, the results of manifold extraction using the model possessing a deep architecture basic paradigm are superior to that of using the state-of-the-art methods.