Wavelet Network Implementation on an Inexpensive Eight Bit Microcontroller

Following the targeted word direction of Opto-and Nanoelectronics, the field of science and technology related to the development of new display technology and organic materials based on liquid crystals ones is meeting the task of replacing volume inorganic electro-optical matrices and devices. An important way in this direction is the study of promising photorefractive materials, conducting coatings, alignment layers, as well as electric schemes that allow the control of liquid crystal mesophase with good advantage. This book includes advanced and revised contributions and covers theoretical modeling for optoelectronics and nonlinear optics, as well as includes experimental methods, new schemes, new approach and explanation which extends the display technology for laser, semiconductor device technology, medicine, biotechnology, etc. The advanced idea, approach


Introduction
The approximation of general continuous functions by nonlinear networks is very useful for system modeling and identification.Such approximation methods can be used, for example, in black-box identification of nonlinear systems, signal processing, control, statistical data analysis, speech recognition, and artificial intelligence.Recently neural networks have been established as a general approximation tool for fitting nonlinear models from input/output data due to their ability of learning rather than complicated process functions (Gao, 2002).Their attractive property is the self-learning ability.A neural network can extract the system features from historical training data using the learning algorithm, requiring little or no a priori knowledge about the process (Patan, 2008).This is why during the past few years the nonlinear dynamic modelling of processes by neural networks has been extensively studied (Narendra & Parthasarathy, 1990;Nerrand et al., 1993;Levin, 1992;Rivals & Personnaz, 1996).In standard neural networks, the nonlinearities are approximated by superposition of sigmoidal functions (Cybenko, 1989).In the other hand, the wavelet theory has found many applications in function approximation, numerical analysis and signal processing.Though this attractive theory has offered efficient algorithms for various purposes, their implementations are usually limited to wavelets of small dimension.The reason is that constructing and storing wavelet basis of large dimension are of prohibitive cost.In order to handle problems of larger dimension, it is necessary to develop algorithms whose implementation is less sensitive to the dimension.And it is known that neural networks are powerful tools for handling problems of large dimension.Due to the similarity between wavelet decomposition and one-hidden-layer neural networks, the idea of combining both wavelets and neural networks has been proposed in various works (Zhang & Benveniste, 1992;Pati & Krishnaprasad, 1993;Hong, 1992;Bakshi & Stephanopoulos, 1993;Tsatsanis & Giannakis, 1993;et al., 1994;Delyon et al., 1995;Saad Saoud & Khellaf, 2009).For example, in (Zhang & Benveniste, 1992) wavelet network is introduced as a class of feedforward networks composed of wavelets, in (Pati & Krishnaprasad, 1993) the discrete wavelet transform is used for analyzing and synthesizing www.intechopen.comfeedforward neural networks, in (Hong, 1992) orthogonal wavelet bases are used for constructing wavelet-based neural network, and in (Saad Saoud & Khellaf, 2009) the dynamic wavelet networks is proposed and used to control the chemical reactor.Combining wavelets and neural networks can hopefully remedy the weakness of each other, resulting in networks with efficient constructive methods and capable of handling problems of moderately larger dimensions.Hence, we can say that the neural network and the wavelet network are capable of modeling non-linear systems.On the basis of supplied training data the neural or the wavelet networks learn the relationship between the process input and output.The data have to be examined carefully before they can be used as a training set for network methods.
The training sets consist of one or more input data and one or more output data (Roffel & Betlem, 2006).After the training of the network, a test-set of data should be used to verify whether the desired relationship was learned.These two operations (train and validate the network) are achieved generally by using the computer, the finding weights and bias are implemented either in the computer itself or through the implementation of the optimal network's parameters in the microcontroller (Gulbag et al., 2009;Cotton et al., 2008;Liung et al., 2003;Neelamegamand & Rajendran , 2005).One common drawback is that in both cases, in order to find the network's parameters precise calculations that are very processor intensive are required.This robust processing equipment can be expensive and rather large.In several applications such as adaptive control (Plett, 2003), or predictive control (Liu et al., 1998), we need to adapt the network's parameters in real time and in this case the computer is very important to adapt the parameters.These problems were overcome with the implementation of the whole neural network with its backpropagation algorithm in the microcontroller, which is proposed in our previous work (Saad Saoud & Khellaf, 2011).In this later work a multilayer neural network is trained and validated using a very inexpensive and low end microcontroller, but the problems of larger dimensions still exist.For this case the real implementation of the whole wavelet network into an inexpensive microcontroller is proposed in this study.The low end and inexpensive microcontroller PIC16F877A of Microchip trains and validates the wavelet network, and the well-known backpropagation algorithm is implemented to obtain the optimal network parameters.All the operations done by the microcontroller are shown through an alphanumeric liquid crystal display and several buttons are added in the embedded system which produces an ergonomic communication interface human/machine.The wavelet network takes more program memory place, for this reason the assembly language is preferred.The Continuous Stirred Tank Reactor (CSTR) system is chosen as a realistic nonlinear system to demonstrate the feasibility and the performance of the results found using the microcontroller.Several results will be presented in this chapter to give the reader more information about this field.A comparative study is made between the microcontroller and the computer.The chapter is organized as follows: After the description of the nonlinear dynamic system identification in general and by using the wavelet network in particular, the implementation of the backpropagation algorithm for the wavelet network in the microcontroller.A comparison between wavelet network based on the eight bit microcontroller and those based on the computer is presented.To illustrate how effectively the eight bit microcontroller can learn nonlinear dynamic models, results for a Continuous Stirred Tank Reactor are given.All the electronic tools, electrical schemes and the implemented algorithm are discussed.The chapter concludes with few final remarks.

Nonlinear dynamic system identification
The key problem in system identification is to find a suitable model structure within which a good model is to be found (Sjöberg et al. 1995).In general, the nonlinear dynamic system identification is the operation to determine a transformation operator T i , for some desired ε>0, so that (Efe & Kaynak, 1997): With T p (u) and T i (u) denote the system to be identified, which maps the compact set U ϵ R n and Y ϵ R m , and the identification model outputs respectively for the same input u.
The purpose is to find a class T i such that T p is represented by T i adequately well.The operator T p is defined by specific input-output pairs that are obtained from the inputs and the outputs of the system to be identified.Fig. 1 summarizes a system identification structure.
Several nonlinear identification models can be found in the literature.The wavelet networks prove their capabilities (Zhang & Benveniste, 1992).The most important aspect of a wavelet network based identification scheme is the determination of an adaptive algorithm that minimizes the difference between the actual plant and the outputs of the identified model by using a set of training pairs which represent the approximate behavior of the actual plant.

Fig. 1. Wavelet networks identification
As mentioned in the introduction, it exists a great similarity between the wavelet network and the one hidden layer neural networks which gives it the opportunity to be an attractive alternative to the feed forward neural network or the Radial Basis Function (RBF) networks (Khalaf, 2006).The use of the wavelet as an activate function gives power to the network because it reduces naturally the noise and hence the resulting network becomes more simple due to the linear output.In this work, another aspect is given to this powerful architecture, which is the implementation of the whole wavelet network with its backpropagation algorithm in the PIC microcontroller.Wavelet function is a waveform that has a limited duration and an average value of zero.
There exist several types of wavelet functions.In this work the Mexican Hat wavelet (Abiyev, 2003) given by the equation 2 is used as a transfer function. With: www.intechopen.com Features of Liquid Crystal Display Materials and Processes 90 The network output is simply the sum of the neuron's outputs of the hidden layer and it is given by the equation (3) as follows:   Where : ψ is wavelet activation function of the neuron with a translation t and a dilation d.The backpropagation algorithm is the most popular training method which is widely used in the neural network applications (Efe, 1996).The backpropagation algorithm is, however, very general and not limited to one-hidden-layer sigmoid neural network models.Instead, it could be applied to all network models (Sjöberg et al., 1995).
In this chapter we use this elegant technique to train the wavelet network.The method minimizes the performance or the so-called the cost function defined on the actual and desired outputs of the network by the equation (4).When updating each individual network parameter (weight, translation and dilation), the gradient information obtained from the differentiation of the cost function is used.As a matter of fact, we are looking for least mean squares.This can be attained by moving the network parameters vectors in a direction such that the performance function decreases.It is obvious that this direction is the negative gradient direction of the performance function.Assuming that the objective is to minimize this kind of performance function, the network parameters updating rule is given by the equations ( 6) to (9): With: As we can see from the above equations, the updating network parameters are performed by evaluating the gradient of the performance function with respect to each individual network parameter in the network.

PIC Implementation of the full wavelet network
In our previous paper (Saad Saoud & Khellaf, 2011) the neural network is implemented and trained using Microchip microcontroller (PIC16F876A).In this work, the extension of this card is used to train and validate another more powerful model which is the wavelet network.In this work, the microcontroller type is replaced by the PIC16F877A to give more power and utility to the final realized card.First of all, why did we choose this type of microcontroller?
The PIC microcontrollers not only are the most widely used and well known microcontrollers, they are also the best supported.In fact, PIC system design and programming has become a powerful specialization with a large number of professional and amateur specialists.There are hundreds of websites devoted to PIC-related topics.An entire cottage industry of PIC software and hardware has flourished around this technology (Sanchez & Canton, 2007).
In this chapter and like our previous work we keep the mid-range PIC microcontroller, but we move to an upgraded version: the PIC16F877A, to give the new designed card better extensions, such as, more inputs outputs giving the user more flexibility when using the card in the control or other real practical situations.
The wavelet network implementation is written in assembly language based on the MPLAB tool distributed freely by Microchip Company.A sample of the 9162 assembly lines taken as the principal program is shown in the figure 4. The wavelet network hexadecimal source (Wavelet.hex)can be found in Intech publisher.Like the old realized card, the microcontroller has to find the optimal wavelet network parameters by using the well known backpropagation algorithm based on the memorized input output pairs in the EEPROM memory.The photo of the realized card based on the PIC16F877A microcontroller is shown in figure 5.
As shown in the program sample, the loops (Loop 1 and Loop 2) are used to read the 500 pairs input-output.The "Loop 3" in Fig. 4 is simply used to train the network.It can be noticed that the program is very flexible, and it gives the designers the opportunity to change the iteration number, the number of the pairs of modeling data and also the type of the data itself, just they have to respect the input regressor.

Fig. 4. A sample program of the assembly wavelet network implementation
The process describes the reaction of two products, which are mixed and react to generate a compound A having a concentration C a (t), with the temperature of the mixture T(t).This reaction is exothermic.The generated heat acts to slow the reaction.The reaction is controlled by introducing a coolant flow rate q c (t), which helps to change the temperature and thereby the concentration.C a0 is the inlet feed concentration, q is the process flow rate, T 0 and T c0 are the inlet feed and coolant temperatures.All these values are assumed constant at nominal values.In the same way, k 0 , E/R, υ, k 1 , k 2 and k 3 are thermodynamic and chemical constants.
The nominal conditions for a product concentration C a = 0.1 mol/l are: T = 438.54K,q c = 103.41l/min.The data used for modeling can be found in (De Moor, 1998) and it is shown in figure 7. We choose randomly a sequence of 750 samples of the input-output from the whole given data.As we have made in our previous works (Saad Saoud & Khellaf, 2011;Saad Saoud et al., 2011), before using the data, we have to make several changes to be accepted by the network.The used data saved in the EEPROM passed through two necessary operations, first it has been normalized to 0.1 and 0.9 and second it is multiplied by 36408 to cover the maximum range and not exceeding the positive signed range of 16 bit (7FFF hex) of data.It should be mentioned that in very few cases, error values could be displayed on the LCD and this is due the floating point format difference between the computer and the PIC microcontroller (PIC 16F877A, 24 bit floating point format).In the microcontroller, all these operations will be reversed to have real values.Also it can be seen in the previous sample assembly language program that 500 samples were used to train the network and the rest of data is used for the network validation.At the beginning, the microcontroller trains the network to find the optimal parameters.All the parameters except the dilations were initialized randomly, hence the network dilations were initialized ones to avoid zeros in the denominators.These operations and the others are summarized in the following algorithm: 1. Initialize the weights and translations randomly small and the dilations by ones.2. Load the data 3. Calculate the wavelet network output using the equation (3) 4. Calculate the error using the equation ( 5) 5. Update the weights, the translations and the dilations with the equations ( 6) to (9). 6.If the E < ε or the number of the iteration is achieved, End the operation Else goto the step 2. To illustrate this algorithm and enter deeply into the microcontroller program memory we give the following example : Let's take for example the concentration C a =0.11944 mol/l.We have to normalize this value between 0.1 and 0.9, for this we make this operation: C a_Normalized = a_NxC a + b_N When the coefficients a_N and b_N are the normalization coefficients for the whole concentration data and in this case are : a_N=9.2937and b_N=-0.4638We can find simply C a_Normalized = 0.64625.And before saving this value in the EEPROM we have to multiply it by 36408.So the value that should be saved is 5BE8 hex.This operation will be done with all the taken samples.The microcontroller will make the inverse operations at each time it reads the values from the EEPROM.In the realized card shown in the figure 5, we can find, in addition to the microcontroller, the EEPROM 24C512, the power supply part and the alphanumeric Liquid Cristal Display (LCD).This last one has a capital importance, it is the bridge between the users and the embedded card and without it the user cannot manipulate the card.Its use gives the realized card power and simplicity.It displays all the necessary operations.For example, the user can start and see the training part of the network, the mean squared error will be displayed at each time the program finishes the three loops (Loop 1 to 3) in the sample program of the figure 4. The users can read the validated data memorized in the EEPROM at the end of the training part and at each time the system can be interrupted by pushing on the rest button.As shown in the electronic circuit (Fig. 6) it contains few components, and we can say that the deserved work can be done with three principal components: the microcontroller with its basic circuits, the memory and the LCD.The comparison between the obtained results using two instruments (the computer and the microcontroller) and based on the extensive practical results carried out within the course of this study is shown by figures 8 and 9 and it is given in the table 1.As remarked, the obtained results using the realized card is very close to the real results, and sometimes better than the results obtained using the computer especially in the validation part (Fig. 9).This is due to the well known problem of the wavelet network, which is the choice of the initial wavelet network parameters.These results give the microcontroller applications an expansion in the artificial intelligence field.On the other hand, when we want to compare the card based on the neural network which has been realized in our previous work and the card presented in this chapter, we can find several points.First, the assembly language line number is smaller in the wavelet network based card than the program based on the neural network.This difference gives the card an important consequence which is the time reduction 2.77 second which is greater in the card based on the neural network (Saad Saoud & Khellaf, 2011).These two important advantages are the result of the network architecture itself, because the wavelet network has a linear output which makes the calculations easier.Second, the new card gives the designer more simplicity of use, because the assembly program is written in the way that we can make a call of the standard subroutines, and we can also change at any time the network parameters such as the number of samples for training and validation or the number of iterations.We can say also that there are several disadvantages of the new card but not so important.The card cost is a little more expensive that the old card (Saad Saoud & Khellaf, 2011), and the wavelet network needs a special care during parameters initialization.

Moving to other LCD technologies
In this chapter, a direct Liquid Crystal Display (LCD) application is illustrated.Even though it is used as a slave of the PIC16F877A, it could be noted that without this tool the electronic card can not be used.This proves the usefulness of the liquid crystal display in the industrial life.This card is normally operated with any LCD type, which is actually in our work the Twisted Nematic (TN)-Display.But it can be improved with other display types such us the In-Plane Switching (IPS)-and Multidomain-Vertical-Alignment (MVA)-Displays.In this work, we take the reference (Willem den Boer, 2005), as an example to describe these modes.An often-quoted drawback of conventional TN LCDs has been their poor viewing angle behavior.Several dramatic improvements in viewing angle have been developed over the past few years.The most important ones are (Willem den Boer, 2005): The In-Plane-Switching (IPS) LC mode and the Multidomain-Vertical-Alignment (MVA) LC mode.

Wavelet network trained through
In both cases, the aim is to obtain a good viewing angle.The use of one of them gives the realized card another aspect, and improves the quality of the electronic system.Like all systems, moving to another type has advantages and disadvantages.We can cite here the principal disadvantages: - The cost of the card: the IPS and MVA types are more expensive than the simple TN display type.- The occupied space: on one hand and because the LCD is controlled by the microcontroller, and the IPS-, MVA-Display types are more sophisticated, they require more assembly instructions which make the program more complicated.On the other hand, bigger LCD occupies more place which means large card dimension.Hence, we can say, the choice of the LCD type is based on the practical situation in where the realized card will be implemented.

Conclusion
In this chapter, a direct practical application of the Liquid Cristal Display (LCD) is presented and proved by the real implementation of the full wavelet network with its backpropagation in a very inexpensive microcontroller.The realized cad is embedded and very simple to use by people having a small knowledge in the electronic and the assembly language programming.The embedded card is tested using the celebrated nonlinear system which is the CSTR chemical reactor.And in all cases, the programmer can at any time change the system to be modeled using the wavelet network respecting the input regressor and the data manipulation before saving it.In our future work, we try to use the realized card in real control situations and giving the architecture a dynamic aspect.

Appendix
In this appendix we try to give the reader of this chapter the tools to realize and test by himself the proposed card in this chapter.First, it is very important to realize the card, for this reason the card's PCB is in pdf format that can be found within this book (wavelet mask.pdf and wavelet card.pdf).Second, the users should implement the components like it is shown in figures 5 and 6.

Figures 2 Fig. 2 .
Figures 2 and 3 are the representation of the Mexican Hat wavelet function for different translations and dilations:

Fig. 5 .
Fig. 5.The realized card based on the PIC microcontroller using the wavelet network

Fig. 9 .
Fig. 9. Validation results of the simulated CSTR reactor using neural network based on the microcontroller and the computer.

Table 1 .
Comparison between the two strategies (computer and microcontroller)