Modified high-order neural network for invariant pattern recognition

https://doi.org/10.1016/j.patrec.2004.09.029Get rights and content

Abstract

A modification for high-order neural networks (HONN) is described. The proposed modified HONN takes into account prior knowledge of the binary patterns that must be learned. This significantly reduces hence computation time as well as memory requirements for network configuration and weight storage. An “approximately equal triangles” scheme for weight sharing is also proposed. These modifications enable the efficient computation of HONNs for image fields of greater that 100 × 100 pixels without any loss of pattern information.

Introduction

Invariant pattern recognition is known as a highly complex and difficult problem. Different techniques were developed to cope with this problem, which can be divided to two distinguished stages (Wood, 1996): invariant feature extraction and feature classification. An example of the former is the moment descriptors (Wong et al., 1995), and of the latter is the multilayer perceptron (Chaudhuri and Bhattacharya, 2000). In some algorithms, which implement neural networks, these two stages can be combined. Invariant pattern recognition using neural networks is a particularly attractive approach because of its similarity with biological systems.

Neural networks used for invariant pattern recognition are classified by the way that invariance is achieved (Barnard and Casasent, 1991): invariance by training, invariant feature spaces, or invariance by structure. Invariance by training is achieved by presenting the network a predefined set of patterns under different transformations, such as rotation (Wood, 1996). The second class, known as invariant feature spaces, works by classifying invariant features (Mahmoud and Mohamed, 2000). Invariance by structure is achieved by the structure of the neural network; good examples are the recognition (Fukushima, 2001) and high-order neural networks (HONN) (Spirkovska and Reid, 1992). This article will concentrate on the HONN and its modifications.

In third-order networks, which are a special case of the HONN, invariance is built into the network structure. This enables fast network learning with only one view of each pattern presented at the learning stage. However, the number of interconnections in the network grows exponentially, and this prevents the use of this method for image fields larger than 18 × 18 pixels (Spirkovska and Reid, 1992).

Several other solutions have been proposed to minimize the number of the HONN interconnections. Weight sharing by similar triangles (Spirkovska and Reid, 1992) reduces the number of individual weights and achieves translation, rotation and partial scale invariance. Weight sharing by “approximately similar triangles” (Perantonis and Lisboa, 1992, He and Siyal, 1999, Takano et al., 1994) further reduces the number of individual weights and achieves translation rotation, limited scale, and limited distortional invariance. Coarse coding (Spirkovska and Reid, 1993) reduces the input field by obtaining a few coarsely coded images from the original image. In non-fully interconnected HONN (Spirkovska and Reid, 1990), the number of interconnections is reduced by sub-sampling. A neural network that could learn high-order correlations was also proposed (Guler, 2001).

Except for coarse coding, all these methods partially solve the problem of the HONN interconnections, but still do not work with larger images. With the coarse coding scheme, on the other hand, very large input image fields can be computed, but the network loses its capacity to learn patterns after its size is reduced. Consequently, the research community in the field of invariant pattern recognition largely abandoned the HONN method.

In this paper, a modification for the third-order network is described. The proposed modification takes into account prior knowledge of the binary patterns that must be learned. By eliminating idle loops, the network achieves significant reductions in computation time as well as in memory requirements for network configuration and weight storage. Better recognition rates (compared to conventionally constructed networks with the same input image field) are attained by the introduction of a new “approximately equal triangles” scheme for weight sharing. The modified configuration enables efficient computation of image fields larger than 100 × 100 pixels without any loss of image information—an impossible task with any previously proposed algorithm.

Section snippets

HONN architecture

The output of a third-order network can be described by the following equation:yi=fabcwiabcxaxbxcwhere i is the output index, w is the weight associated with a particular triangle, y is the actual output, x is a binary input, and a, b, and c are the indices of the inputs.

A schematic description of this network is shown in Fig. 1.

In the training phase, a perceptron-like rule is used:Δwiabc=η(ti-yi)xaxbxcwhere t is the expected training output, y is the actual output, η is the learning rate,

The proposed modified HONN method

To improve the network efficiency, a modified HONN is proposed. This network is not intended for grayscale input patterns, so the input image is binary: pixels corresponding to an edge or contour of the object have the value “1” and all other pixels have the value “0”.

Using Eq. (1), each value of “0” gives a product of “0”. Therefore, only active triangles—those with all three pixels corresponding to an object edge or contour—can influence the result. In addition, the weights of inactive

Experimental results

This section presents experimental results for the modified HONN resources and compares them with the conventional HONN construction method. The simulations were performed on a Pentium 4 computer (1700 MHz), and the programs for both the modified HONN and conventional HONN were written in Visual C++. Seven different object classes with both 60 × 60 and 170 × 170 pixel patterns were used in the construction and recognition stages. These patterns (the contours of actual keys) are shown in Fig. 5. One

Conclusions

A modified high-order neural network for efficient invariant object recognition has been presented. The proposed modification achieves significant reductions in computation times and memory requirements. With the proposed modified HONN, large input patterns (greater than 100 × 100 pixels) are processed without extremely large computational resources. Computation times are improved by using prior knowledge of patterns in the learning stage to eliminate idle loops. By using the “approximately equal

Acknowledgement

The authors would like to thank the anonymous reviewers for their helpful notes.

References (14)

  • E. Barnard et al.

    Invariance and neural nets

    IEEE Trans. Neural Networks

    (1991)
  • B.B. Chaudhuri et al.

    Efficient training and improved performance of multilayer perceptron in pattern classification

    Neurocomputing

    (2000)
  • K. Fukushima

    Recognition of partly occluded patterns: A neural network model

    Biol. Cybernet.

    (2001)
  • M. Guler

    A model with an intrinsic property of learning higher order correlations

    Neural Networks

    (2001)
  • He, Z., 1999. Invariant Pattern Recognition with Higher-Order Neural Networks. M.Sc. thesis, School of Electrical and...
  • Z. He et al.

    Improvement on higher-order neural networks for invariant object recognition

    Neural Process. Lett.

    (1999)
  • I.K. Mahmoud et al.

    Invariant 2D object recognition using the wavelet modulus maxima

    Pattern Recogn. Lett.

    (2000)
There are more references available in the full text version of this article.

Cited by (55)

  • Geographical spatial analysis and risk prediction based on machine learning for maritime traffic accidents: A case study of Fujian sea area

    2022, Ocean Engineering
    Citation Excerpt :

    CNNs have representation learning and shift-invariant abilities to classify input information according to their hierarchical structure. Therefore, they are also called “shift invariant artificial neural networks” (SIANN) (Artyomov and Yadid Pecht, 2005). Long short term memory (LSTM) networks are cycle-time neural networks specially designed to solve long-term dependency problems of general recurrent neural network (RNN).

  • Exponential stability criterion of high-order BAM neural networks with delays and impulse via fixed point approach

    2018, Neurocomputing
    Citation Excerpt :

    In 1990, Narendra and Parthasarathy began to apply neural networks to system identification and control of nonlinear systems, specially high- order neural networks owing to their excellent approximation capabilities [11]. Besides, the proposed high-order neural networks for efficient invariant object recognition achieve significant reductions in computation times and memory requirements [25]. Furthermore, several authors have demonstrated the feasibility of using these architectures in many other applications such as function approximation [26,27], grammatical inference [28] and target detection [29].

  • Modular neural network programming with genetic optimization

    2011, Expert Systems with Applications
    Citation Excerpt :

    HONN’s nonlinear combination of inputs (Zurada, 1992), allows for easy capture of high order correlations and efficient nonlinear mapping (Abdelbar & Tagliarini, 1996; Tsai, 2009). HONNs have been applied widely in various research domains, especially to problems related to pattern recognition and function approximation (Artyomov & Yadid-Pecht, 2005; Foresti & Dolso, 2004; Rovithakis, Chalkiadakis, & Zervakis, 2004; Wang & Lin, 1995). References show the HONN as a powerful soft-computing approach.

View all citing articles on Scopus
View full text