Gaussian Mixture Model for 3-DoF orientations

Abstract

This paper presents learning and generalization algorithms for Gaussian Mixture Model (GMM) in order to accurately encode 3-DoF orientations and Euclidean variables in a common model. We employ correct displacement, integration and weighted averaging arithmetics for unit quaternions to adapt the learning and generalization methods of standard GMMs.

We validate the proposed method in three different applications, learning a 3-dimensional rotation matrix, learning reachable space of a robot, and learning the motion model from demonstrations. We show good experimental results compared to the state-of-the-art method.

Introduction

We propose a modified learning algorithm for Gaussian Mixture Models (GMM) that treats the fundamentally non-Euclidean geometry of the space of 3D-orientations in a principled way, in order to encode both, 3D-orientations and standard Euclidean variables in a common model. GMM employs a mixture of Gaussian probability density functions to model the statistic distribution of a multi-variate dataset, and it has been successfully applied to many robotic applications. One group of applications is concerned with learning robot motions from human demonstration. In this context GMMs were successfully applied to model reaching and moving [1], [2], [3], hitting [4], [5] or catching [6], [7] tasks. Another group of applications handles estimating a distribution of admissible grasping postures for an object and modeling the reachable space of a robot, i.e. the possible positions and orientations the robot’s hand can achieve [7]. By combining both acquired models, a robot can calculate if any graspable part of the object lies inside its reachable space.

A fundamental issue arising when we try to apply standard GMMs to orientation representations, is the non-Euclidean nature of SO (3), the space of rotations in ${\mathbb{R}}^3$. Instead of being flat, the orientation space is bounded and curved: After a rotation of $360°$, we are back at the origin. However, the formulation of the Gaussian probability distribution in standard GMMs assumes a Euclidean structure and correspondingly employs a Euclidean distance measure.

Most existing workexploits the fact that locally SO (3) is similar to ${\mathbb{R}}^3$. That is, for small rotation magnitudes, one can successfully employ standard Euclidean distances. For example, three of the four quaternion components [4] or scaled axis–angle representations [6] were used, without any modifications to the GMM learning algorithm.

In the same line of research, Feiten et al. [8], [9] introduced Mixtures of Projected Gaussians (MPG). MPG represents a distribution over SE (3), by a tangent point (as unit quaternions) and a Gaussian distribution over the projected space of orientation and translation. MPG provides outstanding performances in many researches [9], [10]. However, the projection of MPG [9] is unstable near the opposite orientation to the base orientation as shown in Fig. 1. Additionally, MPG doesn’t provide a regression method in the literature. Although MPG provides a way to fuse two projected Gaussians, the fusion is only valid when the angle between the normals of the tangent spaces is less than 15° [9]. Moreover, similar to the above researches [4], [6], they also exploit the fact that the projected space of SO (3) is locally similar to ${\mathbb{R}}^3$.

As these methods [4], [6], [9] use the Euclidean space EM [1], [11], [12] in order to model the non-Euclidean space dataset, they cannot exactly capture the distributions over SO (3). Hence it brings about a serious modeling error when it models a wide range of orientation distributions with a small number of Gaussians.

Beside the GMMs introduced above, there are other GMM extensions (such as Dirichlet process GMMs [13], Quantum GMMs [14], [15], etc.). However these methods also consider Euclidean space variables only, and are not suitable to model $\mathit{SO}\left(3\right)$.

For large rotation magnitudes, Gaussian process (GP) with a dual quaternion was introduced by [16]. They models the distribution of 6D rigid body poses using GP, and the poses are represented by a dual quaternion. As a kernel function of GP, they employ the quaternion distance measure [17] in order to compute one-dimensional distances between unit quaternions. However, the regression process is computationally expensive for a large dataset, as it use all the training points, and it recomputes the covariance matrix of the training points and a query point (c.f. one of our example uses 10⁷ training data points, see Section 3.2).

Another line of research for large rotation magnitudes, uses a more redundant, ${\mathbb{R}}^6$ representation, employing two columns of the corresponding 3 $\times$ 3 rotation matrix [7] (named TRGMM). Although it can model wide range orientations using standard GMMs, it has several limitations: more dimensions require more training samples for GMM learning; no measures are taken to ensure orthonormality of reconstructed orientation matrices; and the standard Euclidean distance between two orientations in ${\mathbb{R}}^6$ representation doesn’t exactly measure the distance in orientation space as shown in Fig. 1.

As a result of their fundamentally different structure, there exists no singularity-free, differentiable one-to-one mapping (a homeomorphism) between SO (3) and ${\mathbb{R}}^3$. Only if we consider higher-dimensional spaces, e.g. 3 $\times$ 3 rotation matrices, a smooth mapping exists. Additionally to a proper choice of representation, also some standard operations, e.g. the distance measure, need to be adapted in order to accurately handle the non-Euclidean structure of SO (3) in Gaussian distributions.

Alternatively, some direct methods were proposed to treatdistributions on the orientation manifold correctly, e.g. von Mises–Fisher [18], Bingham [19] and wrapped Gaussian [20] distributions.¹ These [18], [19] are antipodally symmetric probabilitydistributions on an unit hypersphere. They can model the distribution of SO (3) orientations (as unit quaternions) on a 4D hypersphere. However, it is computationally expensive (especially the integral in the algorithm). Furthermore, composing the distribution of orientation with position was not considered. In this paper, we propose an alternative approach by adapting the learning procedure of GMM in order to intrinsically and accurately treat the geometry of a chosen orientation representation and to encode the distributions of orientation and Euclidean space variables in a common model. We have chosen unit quaternions to represent orientations, because they provide convenient mathematical notation and constitute a minimal, though four-dimensional, smooth representation of SO (3) [22].

In aEuclidean space, the distance between two data points is the norm of the distance vector—computed by subtracting one data point from the other. Averaging of data points (to obtain the mean) is accomplished by summing up all vectors and dividing by their number. However, in a non-Euclidean space like SO (3), these operations are not well-defined anymore. For example, the mean of two unit quaternions, $q$ and $-q$, yields the null vector, which is not a unit quaternion anymore.

In a few applications [17], [23], the quaternion distance issue is resolved by employing the quaternion logarithm. Ude et al. [17], [23] employ it in order to express the distance vector from an initial orientation to a goal orientation for learning control of complex robot behaviors with DMPs. To perform weighted quaternion averaging, Markley et al. [24] proposed a robust and accurate method based on PCA.

These motivate the two core contributions of the present paper: (1) An extension of the standard GMM equations that makes them consistent with the underlying non-Euclidean geometry of the space of 3D-orientations (we name it as unit quaternions GMM; in short QGMM). (2) A comparison of the new algorithm (QGMM) with standard GMM on three different problems in robotics, showing that the proper treatment of the geometry of 3D-orientation space can lead to significant accuracy gains.

The remainder of this paper is organized as follows. Section 2 presents the summary of quaternion arithmetics and modified learning procedures of GMM. In Section 3, we evaluate the method in three different learning applications. Finally, we conclude with a summary and remaining issues in Section 4.