Elsevier

Pattern Recognition

Volume 48, Issue 8, August 2015, Pages 2418-2432
Pattern Recognition

Integral invariants for space motion trajectory matching and recognition

https://doi.org/10.1016/j.patcog.2015.02.029Get rights and content

Highlights

  • A meaningful definition of integral invariants for space trajectories is proposed.

  • We derive a novel estimation method of integral invariants for discrete trajectories.

  • A non-linear distance is defined to measure the similarity between trajectories.

  • Integral invariants show robustness to noise, occlusions, and transformations.

  • Experiments are carried out to examine their effectiveness and robustness in applications.

Abstract

Motion trajectories provide a key and informative clue in motion characterization of humans, robots and moving objects. In this paper, we propose some new integral invariants for space motion trajectories, which benefit effective motion trajectory matching and recognition. Integral invariants are defined as the line integrals of a class of kernel functions along a motion trajectory. A robust estimation of the integral invariants is formulated based on the blurred segment of noisy discrete curve. Then a non-linear distance of the integral invariants is defined to measure the similarity for trajectory matching and recognition. Such integral invariants, in addition to being invariant to transformation groups, have some desirable properties such as noise insensitivity, computational locality, and uniqueness of representation. Experimental results on trajectory matching and sign recognition show the effectiveness and robustness of the proposed integral invariants in motion trajectory matching and recognition.

Introduction

A motion trajectory, which records a sequence of moving positions of a tracked object, provides a compact and representative clue for motion characterization. It has been extensively studied for describing activities, behaviors, and motion patterns in different applications such as learning motion patterns [1], [2], human action recognition [6], [7], human–robot interaction [3], gesture recognition [4], [8], and trajectory retrieval [49]. As these applications suggest, motion trajectories play an important role in determining the contents of video, perceiving similar motion patterns, retrieving actions, and so on. Min et al. [6] employed motion trajectories tracked from some body joints as features input to a discriminant model for classifying human activities. Yang et al. [8] modeled a gesture recognition system using a time-delay neural network, where motion patterns are learned from hand trajectories. Oikonomopoulos et al. [9] also tracked hand trajectories to understand human actions based on the RVM discriminative model [10]. Apart from modeling human actions, motion trajectories of objects of interest are often utilized to build some activity models to understand and retrieve motion patterns in video surveillance [13], [49] and for information visualization [5]. Nevertheless, in most related work motion trajectories were often directly used in the raw data form with naïve processing. The raw data rely on the absolute positions of motions in a coordinate system, and are, therefore, ineffective in computation and are sensitive to noise. Not surprisingly, they will change under different viewpoints. Therefore, most space motion trajectory features cannot be captured directly by the raw data.

Shape description has received considerable attention in computer vision for shape matching and classification. In this regard, shape descriptors for describing object contours are closely related to our research. In [21], [18], Curvature scale space (CSS) was developed for shape matching. Curvatures of a shape contour at different scales are produced by convolving the shape contour with a series of Gaussian kernels in a coarse to fine manner, where the shape contour is deformed at varying scales, yielding undesirable distortions in the shape. By chain code [11], one can digitize a space curve in terms of relative direction changes of segmented lines. However the relative changes with respect to neighboring lines limited the use of chain code for complex space curves. Using algebraic curve, such as B-spline [24] and Bezier curve [12], a shape contour can be approximated through some key control points. These curve fitting methods show non-uniqueness when the sampling rate of motion trajectories varies or partial occlusions exist in a trajectory, because their approximation accuracies depend on those key control points. Shape context [16] was introduced to capture the histogram bins of neighboring points at each reference point of a curve using a log-polar weight kernel. As a local descriptor, shape context possesses rich invariant properties, and is capable of handling occlusions, but it is not the best way to describe a space trajectory due to its coarse distributions captured for a shape. Therefore, even though these shape descriptors have shown good performance in specific applications, they are incapable of fully capturing motion trajectory features in 3D Euclidean space (3D), due to their limited representation capability for simple shape contours (Chain code and shape context), sensitivity to noise (CSS and chain code) and non-uniqueness (B-spline). Several moment invariants for 3D curves under similarity transformations were derived in [47], but they are global descriptors with two limitations: high-order moments are sensitive to noise [48] and they cannot admit invariant to occlusions. In addition, transform functions based on Fourier and Wavelet [22], [23] extracted global features from a trajectory, but meanwhile the local features are lost. They are also not stable with respect to noise due to high-order Fourier coefficients involved. All of these shape or curve descriptors were initially constructed for simple planar shapes. They are thus insufficient and non-compact to semantically represent complex 3D space motion trajectories. Therefore, they cannot be extended straightforward to our research. As discussed above, a good descriptor for free-form motion trajectories is expected to be robust to noise and occlusions, and invariant to specific group transformations. Therefore, such a descriptor should necessarily satisfy a number of criteria, some of which are consistent with CSS [21]: uniqueness, invariance, noise resistance, and locality, and it needs to be applicable to both planar and space trajectories.

In this paper, we first review the related literature in invariants, and claim our contributions in invariant representation in Section 2 before proposing the definition and estimation of the new integral invariants in Section 3. In Section 4, we propose the similarity measure that allows warping motion trajectories with various temporal lengths onto each other. We conduct two experiments in Section 5 to show the properties, robustness of the proposed integral invariants through trajectory matching, and their effectiveness in sign recognition. Finally, we conclude the paper in Section 6.

Section snippets

Previous work on Invariants and our contributions

Invariants have played an important role for various applications in computer vision ranging from shape representation and matching [17] to object recognition [25] and gesture recognition [19]. Consequently plentiful features that are invariant to specific transformations (affine, similarity, Euclidean) have been investigated in [37], [38], [40], [41], [42]. Two invariant local descriptors related to our research are differential invariants and integral invariants, which have been investigated

Integral invariants

A space motion trajectory is a sequence of position vectors of a moving object in 3-dimesional Euclidean space, and we denote it with γ: IR3, parameterized by temporal sequence t:γ(t)={x(t),y(t),z(t)|t[a,b]},where [a,b]I is the time interval and we assume the motion trajectory γ in this paper is a regular curve, i.e., γ׳(t)0 at all t. Normally, a space motion trajectory can also be parameterized with respect to arc length s, γ(s)={x(s),y(s),z(s)}. Note that in practical scenarios motion

Similarity measure

In this section, we define a similarity measure between a pair of motion trajectories. A distance function between their integral invariants is defined to measure the similarity between them to match, cluster, and classify motion trajectories. Thanks to their invariance, the distance inherits invariant properties that are not affected by group transformations on the trajectories. Also, the robustness of the integral invariants makes the distance less sensitive to noise in practical scenarios.

Experiments

This section presents experiments that show the locality, noise robustness, and invariance properties of the integral invariants, resulting in an effective and robust description for motion trajectory representation, matching, and recognition. We conduct the following experiments on several datasets which contain a large variety of motion types and variations:

  • Trajectory matching: we first evaluate the effectiveness and robustness of the integral invariants by matching a number of pairs of

Conclusion

In this paper, we propose a class of new integral invariants as an effective and robust description for motion trajectories to achieve effective and robust motion trajectory matching and recognition. Compared with differential invariants, the integral invariants involving integration along a motion trajectory have a smoothing effect and, therefore, are less sensitive to noise without preprocessing the motion trajectory. Also, regarding the estimation of the integral invariants, we can control

Conflict of interest

None declared.

Acknowledgment

This work was supported by the Research Grants Council of Hong Kong (Project no. CityU 118613), and the National Natural Science Foundation of China (No. 61273286) and the Center of Robotics and Automation at CityU.

Zhanpeng Shao received his B.S. and M.S. degrees in mechanical engineering from Xi׳an University of Technology, Xi׳an, China, in 2004 and 2007, respectively. From 2007 to 2011, he was an embedded system engineer in EVOC Intelligent Technology Co., Ltd. Currently, he is a Ph.D. candidate in the Department of Mechanical and Biomedical Engineering at City University of Hong Kong. His research interests include pattern recognition, feature extraction, and robot vision.

References (46)

  • J. Moldenhauer, I. Boesnach, T. Beth, V. Wank, K. Bos, Analysis of human motion for humanoid robots, in: Proceedings of...
  • Z. Wang et al.

    Visual traffic jam analysis based on trajectory data

    IEEE Trans. Vis. Comput. Gr.

    (2013)
  • J. Min, R. Kasturi, Activity recognition based on multiple motion trajectories, in: Proceedings of the 17th...
  • S. Ali, A. Basharat, M. Shah, Chaotic invariants for human action recognition, in: Proceedings of IEEE International...
  • M.H. Yang et al.

    Extraction of 2D motion trajectories and its application to hand gesture recognition

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2002)
  • A. Oikonomopoulos, I. Patras, M. Pantic, N. Paragios, Trajectory-based representation of human actions, in: Proceedings...
  • C. Rao et al.

    View-invariant representation and recognition of actions

    Int. J. Comput. Vis.

    (2002)
  • B.T. Morris et al.

    Trajectory learning for activity understanding: unsupervised, multilevel, and long-term adaptive approach

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2011)
  • S. Manay et al.

    Integral invariants for shape matching

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2006)
  • B. Hong et al.

    Shape matching using multiscale integral invariants

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2015)
  • G. Mori et al.

    Efficient shape matching using shape contexts

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2005)
  • S. Belongie et al.

    Shape matching and object recognition using shape context

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2002)
  • F. Bashir, A. Khokhar, Curvature scale space based affine invariant trajectory retrieval, in: Proceedings of IEEE...
  • Cited by (34)

    • Symmetry-adapted representation learning

      2019, Pattern Recognition
      Citation Excerpt :

      Invariance to symmetries in pattern recognition and computational neuroscience is an old challenging problem [1–12]. More recently, in the context of machine learning, data symmetries have been used to derive data representations with the properties of equivariance and invariance [13–17] to unknown, symmetry-generating transformations, for example geometric transformations. These properties are reflected as structure in the representation atoms and can be explicitly used for reducing the complexity of downstream supervised learning.

    • Hierarchically linked infinite hidden Markov model based trajectory analysis and semantic region retrieval in a trajectory dataset

      2017, Expert Systems with Applications
      Citation Excerpt :

      After objects in a video are tracked by several methods (Bae, Kang, Liu, & Chung, 2016; Elbahri et al., 2016; Park, Lee, & Yoon, 2016), a trajectory of each object is described as a sequence of positions. Since Shao and Li (2015) indicated that trajectories contained concise characteristics of the movements, trajectory analysis has become an important part of video analysis. Among several types of latent semantics in trajectories, we focus on semantic regions (Kwon, Jin, Moon, Kang, & Park, 2016; Wang, Ma, Ng, & Grimson, 2011).

    • Gradient-layer feature transform for action detection and recognition

      2016, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      Particularly, for image based action detection cases, bag of words model is rather useful as seen in [26,27]. Plus, LDA model [12], ANN [28,29] and KNN [6,10] are also significant learning schemes for action detection and recognition. This section explains our approach.

    View all citing articles on Scopus

    Zhanpeng Shao received his B.S. and M.S. degrees in mechanical engineering from Xi׳an University of Technology, Xi׳an, China, in 2004 and 2007, respectively. From 2007 to 2011, he was an embedded system engineer in EVOC Intelligent Technology Co., Ltd. Currently, he is a Ph.D. candidate in the Department of Mechanical and Biomedical Engineering at City University of Hong Kong. His research interests include pattern recognition, feature extraction, and robot vision.

    Youfu Li received his B.S. and M.S. degrees in electrical engineering from Harbin Institute of Technology China. He obtained the Ph.D. degree in robotics from the Department of Engineering Science, University of Oxford, in 1993. From 1993 to 1995 he was a postdoctoral research staff in the Department of Computer Science at the University of Wales, Aberystwyth, UK. He joined City University of Hong Kong in 1995. His research interests include robot sensing, robot vision, 3D vision and visual tracking. He has served as an Associate Editor for the IEEE Transactions on Automation Science and Engineering, the IEEE Robotics and Automation Magazine and an Editor for CEB, IEEE International Conference on Robotics and Automation.

    View full text