ASSOCIATIVE MEMORIES NETWORK FOR FACE RECOGNITION AND OBJECT RECOGNITION

.


INTRODUCTION
An associative memory AM is a special kind of neural network that allows recalling one output pattern given an input pattern as a key that might be altered by some kind of noise (additive, subtractive or mixed). Several models of AMs are described in [1], [2], [3], [4], [5], [6], [7] and [8]. Most common application of these models is as a filter. In general, these models have several constraints which limit their applicability in complex problems such as object and face recognition. In particular, models described in [4], [5] and [6] cannot handle with mixed noise. Associative model presented in [7] and [8] is robust to mixed noise.
An association between input pattern x and output pattern y is denoted as ( ) K then W is auto-associative, otherwise it is hetero-associative. A distorted version of a pattern x to be restored will be denoted as x % . If an AM W is fed with a distorted version of k x and the output obtained is exactly k y , we say that recalling is perfect. These models only associate an input pattern with a unique output pattern which in some problems could be seen as a limitation. Suppose for example that instead of recall a face of a person associated to an input pattern you want to recall different faces of the person, probably took in different stages of his life, using the same input pattern, or any pattern related to the face. In order to solve these kind problems, we will need more than one associative memory. We will need a network of associative memories because an associative memory only recalls an output pattern and not a collection of patterns.
In this paper we present how a network of AMs can be used to recall nor just one pattern but several of them given an input pattern. Further more we shown how the adopted associative model is also useful in complex problems such as object and face recognition. In this proposal an association between input pattern x and a collection of output pattern Y is denoted as, ( ) The remaining of the paper is organized as follows. In sections 2 and 3 the associative model used in this research is described. In section 4 the proposed network of AMs is presented. In section 5 the experimental results obtained with the proposal are given. In section 6, finally, the conclusions and several directions for further research in this direction are presented.

DYNAMIC ASSOCIATIVE MODEL
The brain is not a huge fixed neural network, as had been previously thought, but a dynamic, changing neural network that adapts continuously to meet the demands of communication and computational needs [9]. This fact suggests that some connections of the brain could change in response to some input stimuli.
Humans, in general, do not have problems to recognize patterns even if these are altered by noise. Several parts of the brain interact together in the process of learning and recalling a pattern. For example, when we read a word the information enters the eye and the word is transformed into electrical impulses. Then electrical signals are passed through the brain to the visual cortex, where information about space, orientation, form and color is analyzed. After that, specific information about the patterns passes on the other areas of the cortex that integrate visual and auditory information. From here information passes through the arcuate fasiculus, a path that connects a large network of interacting brain areas; paths of this pathway connect language areas with other areas involving in cognition, association and meaning, for details see [10] and [11]. Based upon the above example we have defined in our model several interacting areas, one per association we would like the memory to learn. Also we have integrated the capability to adjust synapses in response to an input stimulus.
As we could appreciate from the previous example, before an input pattern is learned or processed by the brain, it is hypothesized that it is transformed and codified by the brain. In our model, this process is simulated using the following procedure recently introduced in }}} Output: Set of codified and de-codifying patterns.
This procedure allows computing codified patterns from input and output patterns denoted by x and y respectively; € x and € y are de-codifying patterns. Codified and de-codifying patterns are allocated in different interacting areas and d defines of much these areas are separated. On the other hand, d determines the noise supported by our model. In addition a simplified version of k x denoted by k s is obtained as: where mid operator is defined as When the brain is stimulated by an input pattern, some regions of the brain (interacting areas) are stimulated and synapses belonging to those regions are modified.
In our model, we call these regions active regions and could be estimated as follows: Once computed the codified patterns, the decodifying patterns and k s we can build the associative memory. Let fundamental set of associations (codified patterns). Synapses of associative memory W are defined as: After computed the codified patterns, the decodifying patterns, the reader can easily corroborate that any association can be used to compute the synapses of W without modifying the results. In short, building of the associative memory can be performed in three stages as: 1. Transform the fundamental set of association into codified and de-codifying patterns by means of previously described Procedure 1. 2. Compute simplified versions of input patterns by using equation 1. 3. Build W in terms of codified patterns by using equation 3.
As we had already mentioned, synapses could change in response to an input stimulus; but which synapses should be modified? For example, a head injury might cause a brain lesion killing hundred of neurons; this entails some synapses to reconnect with others neurons. This reconnection or modification of the synapses might cause that information allocated on brain will be preserved or will be lost, the reader could find more details concerning to this topic in [12] and [13].
This fact suggests there are synapses that can be drastically modified and they do not alter the behavior of the associative memory. In the contrary, there are synapses that only can be slightly modified to do not alter the behavior of the associative memory; we call this set of synapses the kernel of the associative memory and it is denoted by W K . In the model we can find two types of synapses: synapses that can be modified and do not alter the behavior of the associative memory; and synapses belonging to the kernel of the associative memory. These last synapses play an important role in recalling patterns altered by some kind of noise. Let n ∈ W K R the kernel of an associative memory W . A component of vector W K is defined as: According to the original idea of our proposal, synapses that belong to W K are modified as a response to an input stimulus. Input patterns stimulate some active regions, interact with these regions and then, according to those interactions, the corresponding synapses are modified. Synapses belonging to W K are modified according to the stimulus generated by the input pattern. This adjusting factor is denoted by w ∆ and can be computed as: where ar is the index of the active region. Finally, synapses belonging to W K are modified as: Once synapses of the associative memory have been modified in response to an input pattern, every component of vector y can be recalled by using its corresponding input vector x as: In short, pattern y can be recalled by using its corresponding key vector x or x % in six stages as follows: 1. Obtain index of the active region ar by using equation 2. 2. Transform k x using de-codifying pattern € ar x by applying the following transformation: € . The formal set of prepositions that support the correct functioning of this dynamic model can be found in [14].

MODIFIED DAM
In [15] the authors describe an interesting idea for recognizing faces based on the random selection of stimulating points (pixels), see figure 1.

ARCHITECTURE OF THE NETWORK
Classical AMs (see for example [1], [2], [3], [4], [5], [6], [7] and [8]) are able to recover a pattern (an image) from a noisy version of it. In their original form classical AMs are not useful when image is altered by image transformations, such as translations, rotations, and so on.
The network of AMs proposed in this paper is robust under some of these transformations. Taking advantage of this fact, we can associate different versions of an image (rotated, translated and deformed) to an image.
Our task is to propose a network of AMs aimed to associate an image with other images belonging to the same collection. In order to achieve this, first suppose we want to associate images belonging to a collection with an image of the same collection using an AM. A good solution could be to compute the average image of whole images belonging to the collection and then associate the average image with any image that belongs to the collection. The same solution can be applied to other collections. Once computed the average images from different collections and chosen the images to be associated, we can train the AM as was described in section 2.
Until this point the AM only can recover an association between a collection of input patterns X and output pattern y denoted as, is a collection of input patterns and r is the number of patterns belonging to collection X . This means that it can only be recovered the associated image using any image from a collection. However, we would like to get the inverse result; instead of recovering the associated image using any image from a collection, we would like to recover all the images belonging to the collection using any image of the collection. To achieve this goal we will train a network of AMs built as in previous sections. Each AM will associate all the images of a collection with one image of this collection. This implies that for recovering all images of a given collection, we would need r AMs, where r is the number of images belonging to the collection. The network architecture of AMs needed for recovering a collection of images is shown in Fig. 2.

Fig. 2 -Architecture of a network of AMs for recalling a collection of patters using an input pattern.
In order to train the network of r AMs, first of all we need to know the number of collections we want to recover. Training phase is done as follows: 1. Transform each image into a vector. Once trained the network of AMs, when is fed with any image of a collection, each AM will respond with an image that belongs to the collection. To recover a collection of images we just operate each AM with the input image as described in Section 3 (recalling phase).

Build n collections of images
Some important to remark is that once the collection of associative memories has been stimulated by an input pattern, we do not need any decision rule to determine the final class. Each associative memory will recall the corresponding output pattern which belongs to the same collection.

EXPERIMENTAL RESULTS
In this section the accuracy of the proposal is tested using two different benchmarks of images, see Fig. 3 and Fig. 4.

Fig. 4. (a-e) Collections of images taken from the Essex Collection of facial images.
From the first benchmark 20 complex images were grouped into 5 collections composed by 4 images. After that, we proceeded to train the network of AMs as was explained in Section 5. For this problem our network is composed of four AMs. It is important to say that the number of images composing a collection could by any, the only restrictions to guaranty perfect and robust recall is that patterns (images) satisfy propositions described in [14].
Once trained the network, five experiments were performed to test the accuracy of the proposal. The first four experiments use the original DAM and the last experiment uses the modified DAM. The first experiment consisted on recovering a collection of images using any image of the collection in order to verify how much robust is the proposal under image deformations. Second experiment consisted on recovering a collection of images using any image of the collection altered by mixed noise in order to verify how much robust is the proposal under deformations and noisy version of the images. In the third experiment each image of the training set was rotated (from 0 to 360). We then used them to fed the network of AMs in order to verify how much robust is the proposal under deformations and rotations. Finally for forth and five experiments the images of the training set were rotated (from 0 to 360) and translated, and then used them to fed the network of AMs in order to verify how much robust is the proposal under deformations, rotations and translations.
The accuracy of the proposal was of 100% in the first experiment. The five collections of images where perfectly recovered by using any image of the collection (20 images), in Table 1 are shown some results obtained in this experiment. Remember that we train the network of AMs with average images, so then; when we fed the network with an image of any collection this image could be seen as a deformed version of the average images.
The results provided by our proposal in this experiment show that the associative model used to train the network of AMs are robust under deformations. Something important to say is that if we use other associative models for training the network of AMs such as morphological or median AMs, the collections might not be correctly recovered due to they are not robust under this kind of transformations or deformations. Table 1. Some results obtained for the first experiment. As you can appreciate all sets of images were perfectly recovered.

Input image
Image recovered by each AM.
The accuracy of the proposal, in the second experiment, was of 100%. The five collections of images where recovered perfectly by using any image of the collection as input, even when these images were altered by noise (200 images). As you can see in Table 2, despite of the level of noise added to the images, the collections were correctly recovered. Despite of other associative models are robust to this kind of noise, they might not recover the all collections due to they are only robust under additive, subtractive and mixed noise but not to image transformations.
The accuracy of the proposal, in the third experiment, was also of 100%. The five collections of images where recovered perfectly even when rotated version of the images were used (700 images), see in Table 3. Some important to say is, to our knowledge, neither morphological AMs nor other classical models are robust under rotations. Due to we used simplified patterns using mid operator and due to this operator is invariant to rotations the accuracy of the proposal was of 100%.

Input image
Image recovered by each AM Table 3. Some results obtained for the third experiment. Despite of the noise added to the images and rotations, all sets of images were again correctly recovered.

Input image
Image recovered by each AM The accuracy of the proposal, in the third experiment, was also of 100%. The five collections of images where recovered perfectly even when rotated version of the images were used (700 images), see in Table 3. Some important to say is, to our knowledge, neither morphological AMs nor other classical models are robust under rotations. Due to we used simplified patterns using mid operator and due to this operator is invariant to rotations the accuracy of the proposal was of 100%.
The accuracy of the proposal, in the fourth experiment was of 40%. In this experiment with 700 images; with some images we recalled a collection, but with some other images (when patterns did not satisfied the proposition that guarantees robust recall) the collections were not recalled, see Table 4. However, the results obtained by our proposal are acceptable if they are compared with the results provided by order associative models (less of 10% of accuracy). Table 4. Some results obtained for the fourth experiment. With some images we recalled a collection, but with some other images (when patters did not satisfied the proposition that guarantees robust recall) the collections were not recalled.

Input image
Image recovered by each AM Finally, if we use the modified DAM the accuracy increases, in average, to more than 80%. In this experiment we selected different numbers of stimulation points, as in [15].
For the second benchmark 300 complex images of faces were grouped into 15 collections composed by 20 images. After that, we proceeded to train the network of AMs as was explained in Section 5. For this problem our network was composed of 15 AMs.
Once trained the network, two experiments were performed to test the accuracy of the proposal. The first experiment uses the original DAM and the second experiment uses the modified DAM. The first experiment consisted on recovering a collection of faces using any face of the collection in order to verify how much robust is the proposal under face recognition. In these experiments we did not include noise or affine transformation because of the different gesticulations in the faces.
The accuracy of the proposal using the original DAM was of 20%. In this experiment with 300 images; with some images we recalled a collection, but with some other images (when patterns did not satisfy the proposition that guarantees robust recall), due to the gesticulations and illumination changes, the collections were not recalled.
For the last experiment, using the modified DAM, the accuracy increased to more than 95%. These results were obtained with more than 100 stimulating points.
In general, the accuracy of the proposal with different banks of images (altered by noisy and rotated) was of 100%. This was due to the input patterns (the images) satisfy the propositions presented in [14]. If these patterns do not satisfy these propositions, as images used in experiment four (translated and rotated images), the accuracy of the proposal diminish. However, the results provided by our proposal using the modified DAM, upperformed the results provide by the original DAM and other associative models.

CONCLUSION
In this paper we have proposed a network of AMs. This network is useful for recalling a collection of output patterns using an input pattern as a key. The network is composed by several dynamic associative memories (DAM). This DAM is inspired in some aspects of human brain. The model, due to plasticity of its synapses and functioning, is robust under some transformations as rotation, translation and deformations.
In addition, we describe an algorithm for training a network of AMs codifying the images of a collection by using an average image. Once computed the average images we proceed to training the network of AMs.
The network is capable to recall a collection of images (patterns) even if images are altered by noise or suffer some deformations and rotations.
Due to the original DAM present some problems with translations and face recognition we modified the DAM using a random selection technique. This technique allow the DAM up-perform the results obtained with the original DAM.
Through several experiments we have shown the efficiency of the proposal. In the first three experiments the proposal provided an accuracy of 100%. Even when the images were altered with mixed noise and rotated, the network of AMs recovered the corresponding collection. When object in images suffer translations or the object are faces, the accuracy of the proposal diminished. This is because most of the patterns under this transformation do not satisfied the propositions that guarantee robust recall. By using the modified DAM the accuracy of the proposal increases to more than 80% with translated objects and almost the 100% with complex faces. In general, the results provided by our proposal up-performed the results provide by other associative models.
The performed experiments could be seen as an application in image retrieval problems. We could say that we have developed a small system able to recover a collection of images (previously organized), even in the presence of altered versions of the images.
Nowadays we are working and directed this research to solve real problems related with signal and image retrieval. We are focusing our efforts to propose new associative models able to associate and recall images under more complex transformations. Furthermore, this new models have to work with images of much more complicated objects such as flowers, animals, cars, etc.