Nonlinear Alignment and Its Local Linear Iterative Solution

Zhang, Sumin; Ma, Zhengming; Zhan, Zengrong

doi:https://doi.org/10.1155/2016/7196043

Mathematical Problems in Engineering

On this page

Abstract Introduction Related Works Experimental Results Conclusions Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2016 | Article ID 7196043 | https://doi.org/10.1155/2016/7196043

Nonlinear Alignment and Its Local Linear Iterative Solution

Sumin Zhang,^1,2Zhengming Ma ,¹and Zengrong Zhan³

Academic Editor: Erik Cuevas

Received16 Mar 2016

Revised26 Jul 2016

Accepted03 Aug 2016

Published07 Sept 2016

Abstract

In manifold learning, the aim of alignment is to derive the global coordinate of manifold from the local coordinates of manifold’s patches. At present, most of manifold learning algorithms assume that the relation between the global and local coordinates is locally linear and based on this linear relation align the local coordinates of manifold’s patches into the global coordinate of manifold. There are two contributions in this paper. First, the nonlinear relation between the manifold’s global and local coordinates is deduced by making use of the differentiation of local pullback functions defined on the differential manifold. Second, the method of local linear iterative alignment is used to align the manifold’s local coordinates into the manifold’s global coordinate. The experimental results presented in this paper show that the errors of noniterative alignment are considerably large and can be reduced to almost zero within the first two iterations. The large errors of noniterative/linear alignment verify the nonlinear nature of alignment and justify the necessity of iterative alignment.

1. Introduction

Several papers published in Science in 2000 started the research on manifold learning [1, 2]. From then on, manifold learning has made great progress and produced many representative algorithms as well as a lot of improvements around these representative algorithms, such as ISOMAP [2], Locally Linear Embedding (LLE) [1, 3], Hessian LLE (HLLE) [4], Local Tangent Space Alignment (LTSA) [5], Laplacian Eigenmaps (LE) [6], Diffusion Maps [7], and Maximum Variance Unfolding (MVU) [8]. Manifold learning as a way of machine learning has achieved good performance in many applications of machine learning.

It may be worth noting that although many machine learning algorithms claim to be manifold learning algorithms, they seem to have nothing to do with the topological manifolds defined in mathematics, not to mention the differential manifolds. The manifold learning algorithm proposed in this paper is based on the mathematical characteristics of topological/differential manifolds. This kind of algorithms can be divided into two stages: local homeomorphism and alignment. The mathematical foundation of local homeomorphism is based on the definition of topological manifolds, while the mathematical foundation of alignment is based on the characteristics of differential manifolds. In local homeomorphism, a manifold is divided into a finite number of overlapped local regions. Each local region is homeomorphic to an open set of Euclidean space. The local regions are called patches of manifold and the open sets homeomorphic to the patches are called local coordinates of patches or local coordinates of manifold directly. In alignment, the local coordinates are aligned in Euclidean space to form an area corresponding to the manifold. The area is called global coordinate of manifold. This paper only focuses on alignment; that is to say, we assume that the local coordinates have already been obtained during local homeomorphism and under this assumption we only study how to derive global coordinates from local coordinates.

The remaining sections are arranged as follows. In Section 2, some related works are reviewed briefly. In Section 3, the mathematical foundations of manifold learning are laid. In Section 4, the local nonlinear relation between the global and local coordinates is deduced mathematically. In Section 5, the local linear iterative alignment (LLIA) solution to nonlinear alignment is proposed. In Section 6 the experimental results are presented. In Section 7, some conclusions are given.

There are three kinds of alignment in manifold learning: local coordinate alignment, patch alignment, and manifold alignment. The alignment in the algorithm proposed in this paper belongs to local coordinate alignment.

The local coordinate alignment is based on the mathematical definition of manifold. According to the mathematical definition of manifold, a manifold can be divided into a number of overlapped patches and each patch is homeomorphic to an open set of Euclidean space. The open sets are called local coordinates of manifold. The local coordinate alignment is to align the local coordinates in Euclidean space to form a larger open set which will be correspondent to the manifold globally. This larger open set is called global coordinate of manifold. The local coordinate alignment can be done by aligning local coordinates one by one on the light of geometrical intuition [9–11] or by turning alignment into an eigenvalue-solving problem with differential geometry [5, 12–15].

The patch alignment [16] is more algebraic than geometrical. Before alignment, the given data have to be divided into overlapped patches , . The patch alignment is then formulated as follows:where is called local pattern of , . How to calculate from is dependent on different algorithms. For example, in LE algorithm [6],where is the similarity between and , and such that the and components of are and , respectively; the other components are . More complicated schemes to calculate based on the local similarity, local linearity, local geometry, and so on in can be found in [17–19].

In supervised or semisupervised learning, besides the inherent attributes of data; there are discriminative labels assigned to data. Many recent researches in patch alignment try to incorporate the discriminative labels into the calculation of . For example, in MPA algorithm [20], the neighbours of are first divided into two groups: have the same label with , while have different labels from or have no label at all; then where , , . More schemes in this respect can be founded in [20, 21].

The manifold alignment is involved in a number of high-dimensional datasets which are taken from different manifolds. These datasets are all reduced to a low-dimensional Euclidean space and aligned according to certain rules. For example, let and be two different datasets and let and be the results of dimension reduction of and ; then the framework of LE-like manifold alignment can be expressed as follows:where is the similarity between and , the similarity between and , and the similarity between and . If and are not matching points between and , . How to determine the matching points and how to calculate the similarity are application-dependent. More complicated manifold learning algorithms can be founded in [22–27].

3. Mathematical Foundations of Manifold Learning

Manifold learning can be divided into two stages: local homeomorphism and alignment. In this section we will elaborate the mathematical foundations of local homeomorphism and alignment.

3.1. Mathematical Foundations of Local Homeomorphism

Let be a Hausdorff topological space; if, for all , there is a neighbourhood of such that is homeomorphic to an open set of -dimensional Euclidean space , then is called an -dimensional topological manifold (see Figure 1). Now let be the homeomorphic mapping between and ; is called a chart of and is called an atlas of .

In manifold learning, the neighbourhood is called a patch of and the open set is called the local coordinate of . Furthermore, let and such that for all , . It is clear that is also a homeomorphic mapping between and another open set . An important feature of which will facilitate the mathematical deduction of nonlinear alignment is that . Hereafter, all the local homeomorphic mappings used in this paper are assumed to have this feature.

Furthermore, the set is an open cover of ; that is, . If is compact, then there must be a finite subset of such that this finite subset is also an open cover of [28]. More specifically, if is a -dimensional compact manifold, then there must be a finite number of points and the corresponding neighbourhoods such that(1) is homeomorphic to an open set of -dimensional Euclidean space , .(2) is a finite open cover of ; that is, .

In practice, the given data are always assumed to be taken from a compact manifold and have the neighbourhoods which are locally homeomorphic to the open sets of Euclidean space. The aim of manifold learning during the stage of local homeomorphism is to find these neighbourhoods and then derive the local coordinates of these neighbourhoods. It is clear that the existence of the neighbourhoods and their local coordinates is guaranteed by the mathematical definition of topological manifold. At present, the most commonly used method of finding the neighbourhoods is the K Nearest Neighbours (KNN) method and the most commonly used method of deriving the local coordinates of neighbourhoods is the so-called tangent space method, that is, the local PCA method [5, 29].

3.2. Mathematical Foundations of Alignment

Let be a topological manifold and an atlas of ; if, for any two charts and such that , the mapping is differentiable, then is called differential manifold (see Figure 2). In the rest part of this paper the manifolds are all assumed to be differential manifolds.

Let be a function defined on the differential manifold . Generally speaking, cannot be differentiated directly on . In order to define the differentiability of on , we have to define its local pullback function first. Now let . According to the definition of manifold, there must be a neighbourhood of such that is homeomorphic to an open set of Euclidean space , where is the dimension of . Let be the homeomorphic mapping between and , the so-called local pullback function of is then defined as (see Figure 3).

Note that, in the theory of topological spaces, being homeomorphic means being identical. Also note that manifold is a kind of topological space. Therefore, when limited within the local regions and , can be regarded as the same with its local pullback function .

Furthermore, according to the definition of differential manifold, if is differentiable, so is the function . Therefore, if the function is replaced with in , then the differentiability of is completely dependent on . This is the reason why the differentiability of within the local region is defined by the differentiability of its local pullback function within the local region . More specifically, by making use of , that is, , the gradient vector and Hessian matrix of at can be expressed as follows:

If is differentiable, can be even Taylor-expanded in the neighbourhood of . In fact, for all , where . The differentiation of local pullback functions lays the mathematical foundation of alignment.

4. Nonlinear Alignment

In manifold learning, it is always assumed that an -dimensional manifold can be mapped to an open set of the -dimensional Euclidean space . The open set is called the global coordinate of . The global coordinate of a manifold is the target of manifold learning. The aim of alignment is to derive the manifold’s global coordinate from its local coordinates.

Now let be the mapping between the manifold and its global coordinate . For all , is also called the global coordinate of , where is the ith component function of , .

Obviously, the component functions of can be regarded as the functions defined on the manifold and can be locally pulled back to the -dimensional Euclidean space (see Figure 4). According to (6), deduced in Section 3.2, the component functions can be Taylor-expanded in the neighbourhood of : for all ,where is the local pullback function of from into , . Equation (7) can be rewritten into a matrix form:where . In (8), is the global coordinate of , while is the local coordinate of . Equation (8) establishes the local relationship between and . Generally speaking, this relationship is nonlinear.

5. The Local Linear Iterative Solution to Nonlinear Alignment

In algorithm, a matrix is given, where the column vectors of are assumed to be taken from a -dimensional, compact, and differential manifold which is embedded into the -dimensional Euclidean space , where ; the manifold learning algorithms want to find a matrix such that is the global coordinate of , . Under such circumstances, we construct our algorithm of local linear iterative alignment.

5.1. Linearization of Nonlinear Alignment

We first locally linearize the nonlinear relation of alignment deduced in Section 4. For each data point , let be the neighbourhood of , where is the selection matrix in which the element of the th column is ; other elements are , . It is noted that the neighbourhood includes itself. Let be the centre of . From now on, is regarded as the neighbourhood of , no longer the neighbourhood of .

Let be the global coordinate of , that is, , where , and the local coordinate of , that is, , where and is the local homeomorphic mapping between and . According to (8) derived in Section 4, one haswhere is the local pullback function of from to , .

Equation (9) is a nonlinear equation. From the perspective of computational mathematics, a nonlinear equation can be solved by linear iteration. Therefore (9) has to be first linearized:where is the linear part of . The error incurred from the linearization will be reduced again and again during the linear iteration.

Equation (10) can be rewritten in matrix form:where is the centralizing matrix, . In geometry, (11) means that , the global coordinate of the patch , can be locally approximated by translation, rotation, and scaling of , the local coordinate of the patch .

5.2. Local Linear Iterative Alignment (LLIA) Algorithm

In the proposed LLIA algorithm, the global coordinate is approximated iteratively:In the above iteration the initial iterative value is locally set to be the local coordinates , while the other iterative values are derived from the last iterative values based on the following local linear relation of alignment:where , . The deduction of from is as follows:where is the solution to the following problem:

The algorithm framework of iterative alignment is as follows.

The Beginning of Iteration Calculate : Calculate : Calculate : Calculate : Calculate : Calculate : If , then and go back to calculate

The End of Iteration.

6. Experimental Results

Here the experimental results of 8 manifold learning algorithms on the toy and real-world data are presented. The 8 manifold learning algorithms are the proposed local linear iterative alignment (LLIA) algorithm, LLE, HLLE, LTSA, Diffusion Map, MVU, ISOMAP, and LE. The last 7 manifold learning algorithms are implemented by using MANI, a widely used platform of manifold learning which can be downloaded from internet [30].

6.1. Toy Data

The toy data used in the experiments are Trefoil, Punctured Sphere, and Toroidal Helix. These toy data are also produced by using MANI. Figures 5, 6, and 7 show the experimental results of 8 manifold learning algorithms on three toy datasets. At present there are no widely accepted quantitative evaluation criteria in manifold learning [31]. The evaluations of manifold learning algorithms are commonly based on visual perception.

At the top of Figure 5, an image of Trefoil and some coloured data taken from Trefoil are presented. Since Trefoil is constructed by twisting a planar coil into a cubic body, a good manifold learning algorithm should be able to recover the planar coil from the given coloured data and keep the order of colours unchanged. It can be seen from Figure 5 that the proposed LLIA algorithm does a good job in this respect.

The so-called Punctured Sphere is a spherical surface whose top has been cut off. The colour of the given data shown in Figure 6 indicates the position relationship between the given data. The aim of manifold learning is to produce an overhead view of Punctured Sphere on a plane. It can be seen from Figure 6 that the proposed algorithm achieves much better results than other manifold learning algorithms.

Toroidal Helix is something like that shown at the top of Figure 7. Although Toroidal Helix is much more complicated than Trefoil, it is also constructed by twisting a planar coil into a cubic body. The aim of manifold learning is also to recover the planar coil from the given coloured data. In doing so, the order of data colour must be kept unchanged because the order of data colour indicates the position relationship of data which should not be damaged during dimension reduction. It can be seen from Figure 7 that the performance of the proposed LLIA algorithm is one of the best among 8 manifold learning algorithms.

Table 1 shows the average alignment errors of the proposed LLIA algorithm during iterations. It can be seen from Table 1 that the initial alignment errors are considerably large and will be reduced sharply to almost zero during the first 2 iterations. The fact that the initial alignment errors are considerably large indirectly verifies the nonlinearity of alignment and justifies the necessity of iterative alignment. Furthermore the fact that the errors of iterative alignment will be reduced sharply to almost zero during the first 2 iterations means that the time complexity of the proposed LLIA algorithm is quite low.

6.2. Real-World Data

Figure 8 shows the experimental results of 8 manifold learning algorithms on the dataset of Frey Face. Frey Face is often used to test the effects of manifold learning algorithms in many academic literatures [1, 3, 32, 33]. The photos in Frey Face are all taken from the same person with different head poses and facial expressions. Therefore, although the dimension of photos is 560, the intrinsic dimension of photos is two: head pose and facial expression. In Figure 8, the photos are dimensionally reduced from 560 to 2 by using 8 different manifold learning algorithms, including the proposed LLIA algorithm. In the images of Figure 8, the facial expression changes along the vertical direction, while the head pose changes along the horizontal direction. It can be seen that the proposed LLIA algorithm achieves the best visual effect among the 8 manifold learning algorithms.

Figure 9 shows the experimental results on the dataset of plaster. Plaster is also a dataset often used to test the effects of manifold learning algorithms in many academic literatures [2, 33]. The photos of plaster are all taken from the same plaster from different shooting angle and under different shooting illumination. Therefore, although the dimension of photos is , the intrinsic dimension of photos is two: shooting angle and illumination. In Figure 9, the photos are dimensionally reduced from 4096 to 2 by using manifold learning algorithms. In the images of Figure 9, the shooting angle changes along the vertical direction, while the shooting illumination changes along the horizontal direction. Again, it can be seen that the proposed LLIA algorithm achieves the best visual effect among the 8 manifold learning algorithms.

As shown in Figures 5, 6, 7, 8, and 9, the effects of manifold learning algorithms are evaluated qualitatively, not quantitatively. At present, there are no widely accepted quantitative evaluation criteria in manifold learning [31]. However, the evaluations of manifold learning algorithms can be quantized indirectly by applying manifold learning algorithms to practical applications. In Table 2, the manifold learning algorithms are applied to the clustering application. USPS is an image dataset of handwritten numerals and has been used in many academic literatures of manifold learning [3, 34]. In Table 2, the images are first dimensionally reduced by using manifold learning algorithms and then clustered by using the K-means algorithm. The clustering precisions shown in Table 2 are then used to evaluate the effects of manifold learning algorithms indirectly. It can be seen from Figure 11 that the proposed LLIA algorithm is superior to LLE, HLLE, LTSA, and Diffusion Map but inferior to ISOMAP and LE. Note that both ISOMAP and LE belong to the so-called global preserving algorithms, while the proposed LLIA, LLE, HLLE, LTSA, and Diffusion Map belong to the so-called local preserving algorithms. Table 2 shows that, in the clustering of handwritten numerals, although the local preserving algorithms are inferior to the global preserving algorithms, the proposed LLIA achieves the best result among the local preserving algorithms.

6.3. Robustness Experiments

Figures 10 and 11 show the robustness experiments against the change of neighbourhood size on Toroidal Helix and Punctured Sphere. The experimental data as well as the experimental results of manifold learning algorithms (except LLIA) are provided by using the manifold learning platform MANI.

It can be seen in Figure 10 that, on Toroidal Helix, the proposed algorithm LLIA performs best among the listed manifold learning algorithms. The experimental results of other manifold learning algorithms deform very much along with the change of K. However, as shown in Figure 11, most of the listed manifold learning algorithms (including LLIA) perform stably on Punctured Sphere. Anyway, the proposed LLIA algorithm shows some robustness against the different neighbourhood sizes as well as the different manifolds.

Figures 12 and 13 show the robustness experiments against noises on Toroidal Helix and Punctured Sphere. The noises are added by using the manifold learning platform MANI. There are two Matlab sentences in MANI which can be used to add noises to the manifold data: noiseSigma = handles.noise; handles.X = handles.X + noiseSigma randn(N,3).In the above Matlab sentences, the variable noiseSigma controls the noise strength. In Figures 12 and 13, the noise strength has been changed from 0 to 18%.

It can be seen in Figure 12 that, on Toroidal Helix, the performance of proposed algorithm LLIA is quite robust against noises, while other manifold learning algorithms deteriorate quickly when the noises are strengthened steadily. On the other hand, on Punctured Sphere, ISOMAP, Diffusion Map, and the proposed LLIA perform stably against the change of noises. Anyway, it can be concluded from both Figure 12 and Figure 13 that the proposed LLIA algorithm is robust against noises on different manifolds and different noise strength.

7. Conclusions

(1)At present, many machine learning algorithms, including the famous LLE algorithm [1, 35], claim to be manifold learning algorithms. However, the manifolds mentioned in these algorithms seem to have nothing to do with the manifolds defined in mathematics. In contrast, the LLIA algorithm proposed on this paper is constructed based on the mathematical characteristics (local homeomorphism, local pullback, etc.) of differential manifolds.(2)The manifold learning can be divided into two stages: local homeomorphism and alignment. In local homeomorphism, a Taylor expansion between a patch and its local coordinate can be deduced by using one-parameter transformation group acting on manifolds [5, 36]. In this paper, a Taylor expansion between the global and local coordinates of a patch is also deduced by using the local pullback of functions defined on differential manifold. It should be pointed out that these two Taylor expansions are completely different. The former reveals the relation between the high-dimensional data and low-dimensional date, while the latter reveals the relation between two low-dimensional data.(3)In the proposed LLIA algorithm, the nonlinear problem is solved by using a linear iteration method. There are two problems to be addressed here: the convergence of iteration and the time complexity of algorithm. In computational mathematics, the linear iterative methods are often used to solve nonlinear problems and the convergences of iteration have been thoroughly discussed [37]. In practice, the experimental results presented in this paper show that the alignment errors would be considerably large if without iteration and will be reduced sharply during the first two iterations. This fact also implies that the time complexity of the proposed LLIA algorithm will be quite low because only few iterations are needed.(4)At present, the algorithms of local coordinate alignment can be divided into two categories: the one-time alignment and the gradual alignment. In the one-time alignment, the manifold’s global coordinate is obtained at one time by solving an eigenvalue problem [3, 5, 12–14], while in the gradual alignment, the manifold’s global coordinate grows up gradually from a patch’s local coordinate [9–11]. No matter which alignment is used, they are all based on the local linear model of alignment. The proposed LLIA algorithm is based on the nonlinear model of alignment and therefore belongs to neither one-time alignment nor gradual alignment.

Competing Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported in part by the Guangzhou Municipal Science and Technology Projects under Grant 2014J4100095.

References

S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” Science, vol. 290, no. 5500, pp. 2323–2326, 2000.
View at: Publisher Site | Google Scholar
J. B. Tenenbaum, V. de Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” Science, vol. 290, no. 5500, pp. 2319–2323, 2000.
View at: Publisher Site | Google Scholar
J. Chen, Z. M. Ma, and Y. Liu, “Local coordinates alignment with global preservation for dimensionality reduction,” IEEE Transactions on Neural Networks and Learning Systems, vol. 24, no. 1, pp. 106–117, 2013.
View at: Publisher Site | Google Scholar
D. L. Donoho and C. Grimes, “Hessian eigenmaps: locally linear embedding techniques for high-dimensional data,” Proceedings of the National Academy of Sciences of the United States of America, vol. 100, no. 10, pp. 5591–5596, 2003.
View at: Publisher Site | Google Scholar | MathSciNet
Z. Zhang and H. Zha, “Principal manifolds and nonlinear dimensionality reduction via tangent space alignment,” SIAM Journal on Scientific Computing, vol. 26, no. 1, pp. 313–338, 2004.
View at: Publisher Site | Google Scholar | MathSciNet
M. Belkin and P. Niyogi, “Laplacian eigenmaps for dimensionality reduction and data representation,” Neural Computation, vol. 15, no. 6, pp. 1373–1396, 2003.
View at: Publisher Site | Google Scholar
R. R. Coifman and S. Lafon, “Diffusion maps,” Applied and Computational Harmonic Analysis, vol. 21, no. 1, pp. 5–30, 2006.
View at: Publisher Site | Google Scholar | MathSciNet
K. Q. Weinberger and L. K. Saul, “Unsupervised learning of image manifolds by semidefinite programming,” International Journal of Computer Vision, vol. 70, no. 1, pp. 77–90, 2006.
View at: Publisher Site | Google Scholar
Z. Han, D.-Y. Meng, Z.-B. Xu, and N.-N. Gu, “Incremental alignment manifold learning,” Journal of Computer Science and Technology, vol. 26, no. 1, pp. 153–165, 2010.
View at: Publisher Site | Google Scholar
Z. M. Ma and J. Chen, “The huffman-like alignment in manifold learning,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 28, no. 4, Article ID 1451005, 2014.
View at: Publisher Site | Google Scholar
Y. Hou, P. Zhang, X. Xu, X. Zhang, and W. Li, “Nonlinear dimensionality reduction by locally linear inlaying,” IEEE Transactions on Neural Networks, vol. 20, no. 2, pp. 300–315, 2009.
View at: Publisher Site | Google Scholar
L. Yang, “Alignment of overlapping locally scaled patches for multidimensional scaling and dimensionality reduction,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 438–450, 2008.
View at: Publisher Site | Google Scholar
T. Zhang, X. Li, D. Tao, and J. Yang, “Local Coordinates Alignment (LCA): a novel manifold learning approach,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 22, no. 4, pp. 667–690, 2008.
View at: Publisher Site | Google Scholar
D. Zhao, “Formulating LLE using alignment technique,” Pattern Recognition, vol. 39, no. 11, pp. 2233–2235, 2006.
View at: Publisher Site | Google Scholar
Y. Zhan and J. Yin, “Robust local tangent space alignment via iterative weighted PCA,” Neurocomputing, vol. 74, no. 11, pp. 1985–1993, 2011.
View at: Publisher Site | Google Scholar
T. Zhang, D. Tao, X. Li, and J. Yang, “Patch alignment for dimensionality reduction,” IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9, pp. 1299–1313, 2009.
View at: Publisher Site | Google Scholar
X. You, W. Ou, C. L. C. Chen, Q. Li, Z. Zhu, and Y. Tang, “Robust nonnegative patch alignment for dimensionality reduction,” IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 11, pp. 2760–2774, 2015.
View at: Publisher Site | Google Scholar
Y. Chen, Z. Jin, J. Yin, and J. Zhu, “Dimensionality reduction via locally reconstructive patch alignment,” Optical Engineering, vol. 51, no. 7, Article ID 077208, pp. 397–407, 2012.
View at: Publisher Site | Google Scholar
Z. Lai, W. K. Wong, Y. Xu, C. Zhao, and M. Sun, “Sparse alignment for robust tensor learning,” IEEE Transactions on Neural Networks & Learning Systems, vol. 25, no. 10, pp. 1779–1792, 2014.
View at: Publisher Site | Google Scholar
J. Xu, S. Xie, and W. Zhu, “Marginal patch alignment for dimensionality reduction,” Soft Computing, 2015.
View at: Publisher Site | Google Scholar
D. Tao, L. Liang, L. Jin, and Y. Gao, “Similar handwritten Chinese character recognition using discriminative locality alignment manifold learning,” in Proceedings of the 11th International Conference on Document Analysis and Recognition (ICDAR '11), pp. 1012–1016, IEEE, Beijing, China, September 2011.
View at: Publisher Site | Google Scholar
H. T. Vu, C. Carey, and S. Mahadevan, “Manifold warping: manifold alignment over time,” in Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI '12), Toronto, Canada, July 2012.
View at: Google Scholar
C. Wang, B. Liu, H. Vu et al., Sparse Manifold Alignment, 2012.
C. Wang and S. Mahadevan, “Heterogeneous domain adaptation using manifold alignment,” in Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI '11), pp. 1541–1546, Barcelona, Spain, July 2011.
View at: Publisher Site | Google Scholar
C. Wang and S. Mahadevan, “Manifold alignment preserving global geometry,” in Proceedings of the 23rd International Joint Conference on Artificial Intelligence (IJCAI '13), Beijing, China, August 2013.
View at: Google Scholar
Z. Cui, H. Chang, S. Shan, and X. Chen, “Generalized unsupervised manifold alignment,” in Proceedings of the 28th Annual Conference on Neural Information Processing Systems (NIPS '14), pp. 2429–2437, December 2014.
View at: Google Scholar
D. Tuia, M. Volpi, M. Trolliet, and G. Camps-Valls, “Semisupervised manifold alignment of multimodal remote sensing images,” IEEE Transactions on Geoscience & Remote Sensing, vol. 52, no. 12, pp. 7708–7720, 2014.
View at: Publisher Site | Google Scholar
M. Hirsch, Differential Topology, Springer, Berlin, Germany, 1997.
Z. Zhang, J. Wang, and H. Zha, “Adaptive manifold learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 2, pp. 253–265, 2012.
View at: Publisher Site | Google Scholar
T. Wittman, “MANI: Manifold Learning Toolkit,” http://macs.citadel.edu/wittman/research.html.
View at: Google Scholar
J. Zhang, Q. Wang, L. He, and Z.-H. Zhou, “Quantitative analysis of nonlinear embedding,” IEEE Transactions on Neural Networks, vol. 22, no. 12, pp. 1987–1998, 2011.
View at: Publisher Site | Google Scholar
S. Roweis, L. Saul, and G. Hinton, “Global coordination of local linear models,” in Advances in Neural Information Processing Systems (NIPS), vol. 14, pp. 889–896, MIT Press, Cambridge, Mass, USA, 2001.
View at: Google Scholar
T. Lin and H. Zha, “Riemannian manifold learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 796–809, 2008.
View at: Publisher Site | Google Scholar
G. Hinton and S. Roweis, “Stochastic neighbor embedding,” in Proceedings of the Advances in Neural Information Processing Systems (NIPS ’02), vol. 15, pp. 857–864, 2002.
View at: Google Scholar
J. Chen and Z. M. Ma, “Locally linear embedding: a review,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 25, no. 7, pp. 985–1008, 2011.
View at: Publisher Site | Google Scholar | MathSciNet
W. M. Boothby, Introduction to Differentiable Manifolds and Riemannian Geometry, Academic Press, Orlando, Fla, USA, 2nd edition, 2002.
Y. Saad, Iterative Methods for Sparse Linear Systems, Society for Industrial and Applied Mathematics, Philadelphia, Pa, USA, 2nd edition, 2003.

Copyright

Copyright © 2016 Sumin Zhang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

909

Downloads

1185

Citations

Mathematical Problems in Engineering

Nonlinear Alignment and Its Local Linear Iterative Solution

Abstract

1. Introduction

2. Related Works

3. Mathematical Foundations of Manifold Learning

3.1. Mathematical Foundations of Local Homeomorphism

3.2. Mathematical Foundations of Alignment

4. Nonlinear Alignment

5. The Local Linear Iterative Solution to Nonlinear Alignment

5.1. Linearization of Nonlinear Alignment

5.2. Local Linear Iterative Alignment (LLIA) Algorithm

6. Experimental Results

6.1. Toy Data

6.2. Real-World Data

6.3. Robustness Experiments

7. Conclusions

Competing Interests

Acknowledgments

References

Copyright