An Accelerometer-Based Digital Pen for Handwritten Digit and Gesture Recognition

This paper presents an accelerometer-based digital pen for handwritten digit and gesture trajectory recognition applications. The digital pen consists of a triaxial accelerometer, a microcontroller, and an RF wireless transmission module for sensing and collecting accelerations of handwriting and gesture trajectories. The proposed trajectory recognition algorithm composes of the procedures of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. The algorithm is capable of translating time-series acceleration signals into important feature vectors. Users can use the pen to write digits or make hand gestures, and the accelerations of hand motions measured by the accelerometer are wirelessly transmitted to a computer for online trajectory recognition. The algorithm first extracts the time-and frequency-domain features from the acceleration signals and, then, further identifies the most important features by a hybrid method: kernel-based class separability for selecting significant features and linear discriminant analysis for reducing the dimension of features. The reduced features are sent to a trained probabilistic neural network for recognition. Our experimental results have successfully validated the effectiveness of the trajectory recognition algorithm for handwritten digit and gesture recognition using the proposed digital pen.


Introduction
Explosive growth of miniaturization technologies in electronic circuits and components has greatly decreased the dimension and weight of consumer electronic products, such as smart phones and handheld computers, and thus made them more handy and convenient. Due to the rapid development of computer technology, human-computer interaction (HCI) techniques have become an indispensable component in our daily life. Recently, an attractive alternative, a portable device embedded with inertial sensors, has been proposed to sense the activities of human and to capture his/her motion trajectory information from accelerations for recognizing gestures or handwriting.
A significant advantage of inertial sensors for general motion sensing is that they can be operated without any external reference and limitation in working conditions. However, motion trajectory recognition is relatively complicated because different users have different speeds and styles to generate various motion trajectories. Thus, many researchers have tried to narrow down the problem domain for increasing the accuracy of handwriting recognition systems.
Recently, some researchers have concentrated on reducing the error of handwriting trajectory reconstruction by manipulating acceleration signals and angular velocities of inertial sensors. However, the reconstructed trajectories suffer from various intrinsic errors of inertial sensors. Hence, many researchers have focused on developing effective algorithms for error compensation of inertial sensors to improve the recognition accuracy.
A pen type input device to track trajectories in 3-D space by using accelerometers and gyroscopes. An efficient acceleration error compensation algorithm based on zero velocity compensation was developed to reduce acceleration errors for acquiring accurate reconstructed trajectory. An extended Kalman filter with magnetometers micro inertial measurement unit (μIMU with magnetometers) was employed to compensate the orientation of the proposed digital writing instrument. If the orientation of the instrument was estimated precisely, the motion trajectories of the instrument were reconstructed accurately.

II. related work
Recently, some studies have focused on the development of digital pens for trajectory recognition and HCI applications. For instance, an alternative method of conventional tablet-based handwriting recognition has been proposed by Milner. In his system, two dual-axis accelerometers are mounted on the side of a pen to generate time-varying x-and y-axis acceleration for handwriting motion. The author employed an HMM with a bandpass filtering and a down-sampling procedure for classification of seven handwritten words. The best recognition rate is 96.2% when the number of states of the HMM is equal to 60.
The input device embedding a triaxial accelerometer and a triaxial gyroscope for online 3-D character gesture recognition. Fisher discriminant analysis was adopted, and different combinations of sensor signals were used to test the recognition performance of their device. When all six axes raw signals were used as inputs of the recognition system, the recognition rate was 93.23%. In addition, they proposed an ensemble recognizer consisting of three sub recognizers with the following signals as inputs: acceleration, angular velocity, and estimated handwriting trajectory. The recognition rate of the recognizer was 95.04%.
Similarly, a gesture recognition system consisting of a gesture input device, a trajectory estimation algorithm, and a recognition algorithm in 3-D space was proposed by Cho. The trajectory estimation algorithm based on an inertial navigation system was developed to reconstruct the trajectories of numerical digits and three hand gestures, and then, a Bayesian network was trained to recognize the reconstructed trajectories. The average recognition rate was 99.2%.
Zho proposed a μIMU for 2-D handwriting applications. They extracted the discrete cosine transform features from x-and y-axis acceleration signals and one angular velocity and used an unsupervised self-organizing map to classify 26 English alphabets and ten numerical digits. The recognition rate of 26 English alphabets and ten numerical digits achieved 64.38% and 80.8%, respectively.

III. HARDWARE DESIGN OF DIGITAL PEN
The digital pen consists of a Triaxial accelerometer shown RESEARCH PAPER in figure 1 (ADXL335, Analog Devices), a microcontroller (LPC2148 with a 12-b A/D converter), and a wireless transceiver (XBee ZNet 2.5/XBee PRO Znet 2.5 OEM). The schematic diagram of the pen-type portable device is shown in figure 2.  The triaxial accelerometer measures the acceleration signals generated by a user's hand motions. The microcontroller collects the analog acceleration signals and converts the signals to digital ones via the A/D converter. The wireless transceiver transmits the acceleration signals wirelessly to a personal computer (PC).
The ADXL335 is a low-cost capacitive micro machined accelerometer with a temperature compensation function and a g-select function for a full-scale selection of ±3g and is able to measure accelerations over the bandwidth of 1.5 kHz for all axes. The accelerometer's sensitivity is set from −3g to +3 g in this study. The LPC2148 integrates a high-performance 12-b A/D converter and an optimized signal cycle 25-MHz 32-bit microcontroller unit (MCU) on a signal chip. The output signals of the accelerometer are sampled at 100 Hz by the 12-b A/D converter. Then, all the data sensed by the accelerometer are transmitted wirelessly to a PC by an RF transceiver at 2.4-GHz transmission band with 1-Mb/s transmission rate. The overall power consumption of the digital pen circuit is 30 mA at 3.6 V.

IV. TRAJECTORY RECOGNITIN SYSTEM
The block diagram of the proposed trajectory recognition algorithm consisting of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction is shown in The motions for recognition include Arabic numerals and eight hand gestures. The acceleration signals of the hand motions are measured by a triaxial accelerometer and then preprocessed by filtering and normalization. Consequently, the features are extracted from the preprocessed data to represent the characteristics of different motion signals, and the feature selection process based on KBCS picks p features out of the original 24 extracted features. To reduce the computational load and increase the recognition accuracy of the classifier, we utilize LDA to reduce the dimension of the selected features. The reduced feature vectors are fed into a PNN classifier to recognize the motion to which the feature vector belongs. The detailed procedure of the proposed trajectory recognition algorithm as follows.
A. SIGNAL PREPROCESSING The raw acceleration signals of hand motions are generated by the accelerometer and collected by the microcontroller. Due to human nature, our hand always trembles slightly while moving, which causes certain amount of noise. The signal preprocessing consists of calibration, a moving average filter, a high-pass filter, and normalization.

B. FATURE GENERATION
The characteristics of different hand movement signals can be obtained by extracting features from the preprocessed x, y, and z axis signals, and we extract eight features from the triaxial acceleration signals, including mean, STD, VAR, IQR, correlation between axes ,MAD, rms, and energy .
When the procedure of feature generation is done, 24 features are then generated. Because the amount of the extracted features is large, we adopt KBCS to select most useful features and then use LDA to reduce the dimensions of features.

C. FEATURE SELECTION
Feature selection comprises a selection criterion and a search strategy. The adopted selection criterion is the KBCS which is originally developed by Wang.

D. FEATURE EXTRACTION
For pattern recognition problems, LDA is an effective feature extraction (or dimensionality reduction method) which uses a linear transformation to transform the original feature sets into a lower dimensional feature space. The purpose of LDA is to divide the data distribution in different classes and minimize the data distribution of the same class in a new space.
After feature extraction, these reduced features will be fed into the PNN classifier to recognize different hand movements.

E.CLASSIFIER CONSTRUCTION
The PNN first with enough training data, the PNN is guaranteed to converge to a Bayesian classifier, and thus, it has a great potential for making classification decisions accurately and providing probability and reliability measures for each classification. In addition, the training procedure of the PNN only needs one epoch to adjust the weights and biases of the network architecture. Therefore, the most important advantage of using the PNN is its high speed of learning. Typically, the PNN consists of an input layer, a pattern layer, a summation layer, and a decision layer as shown in  2) Layer 2: The second layer is the pattern layer, and the number of neurons in this layer is equal to N L . Once a pattern vector x from the input layer arrives, the output of the neurons of the pattern layer can be calculated as follows: (1) where xki is the neuron vector, σ is a smoothing parameter, d is the dimension of the pattern vector x, and φ ki is the output of the pattern layer.
3) Layer 3: The third layer is the summation layer. The contributions for each class of inputs are summed in this layer to produce the output as the vector of probabilities. Each neuron in the summation layer represents the active status of one class. The output of the kth neuron is (2) Where Ni is the total number of samples in the kth neuron.

4) Layer 4:
The fourth layer is the decision layer c(x) = arg max{ Pk (x) }, k = 1, 2,…….,m where m denotes the number of classes in the training samples and c(x) is the estimated class of the pattern x.
If the a priori probabilities and the losses of misclassification for each class are all the same, the pattern x can be classified according to the Bayes' strategy in the decision layer based on the output of all neurons in the summation layer.

V. EXPERIMENTAL RESULT
The effectiveness of trajectory recognition algorithm is validated by the following two experiments: Handwritten digit recognition Gesture recognition. The proposed trajectory recognition algorithm consists of the following procedures: acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction.
We used different combinations of feature selection and extraction methods and employed PNN to recognize handwritten digits and hand gestures. In addition, we compared the recognition results of the PNN trained by the features from different feature engineering methods with those of feedforward neural networks (FNNs).

A.HANDWRITTEN DIGIT RECOGNITION
Hold the digital pen to draw the trajectories of Arabic numerals and the pen tip must touch a table. The acceleration signals after the signal preprocessing procedure of the proposed trajectory recognition algorithm for the digit.
There were 11 significant features including corr xz , mean x , mean y , MADx, IQRx, rms x , corr xy , mean z , energy x , energy y , and energy z selected from 24 features by the KBCS. Finally, the dimension of the selected features was further reduced to nine by the LDA not only to ease the burden of computational load but also to increase the accuracy of classification.

B.GESTURE RECOGNITION
In the second method hold the pen to perform eight hand gestures in a 3-D space. The gestures are shown in Figure 5.10

RESEARCH PAPER
tal writing instrument (UDWI) by calibrating 2-D trajectories. 2) To obtain the accurate attitude angles by using the multiple camera calibration.
However, in order to recognize or reconstruct motion trajectories accurately, the aforementioned approaches introduce other sensors such as gyroscopes or magnetometers to obtain precise orientation. This increases additional cost for motion trajectory recognition systems as well as computational burden of their algorithms.
In order to reduce the cost of systems and simplify the algorithms, much research effort has been devoted to extract important features from time-series inertial signals are computed correlation coefficients of the absolute value of acceleration and the absolute value of the first and second derivatives of acceleration to form feature vectors.

VII. CONCLUSION
This seminar has presented a systematic trajectory recognition algorithm framework that can construct effective classifiers for acceleration-based handwriting and gesture recognition. The proposed trajectory recognition algorithm consists of acceleration acquisition, signal preprocessing, feature generation, feature selection, and feature extraction. With the reduced features, a PNN can be quickly trained as an effective classifier. In the experiments, we used 2-D handwriting digits and 3-D hand gestures to validate the effectiveness of the proposed device and algorithm. The overall handwritten digit recognition rate was 98%, and the gesture recognition rate was also 98.75%. This result encourages us to further investigate the possibility of using our digital pen as an effective tool for HCI applications.