Writer Identification Using Handwritten Cursive Texts and Single Character Words

Tobias Kutzner 1 , Carlos F. Pazmiño-Zapatier 2, Matthias Gebhard 1, Ingrid Bönninger 1, Wolf-Dietrich Plath 1 and Carlos M. Travieso 2,3,* 1 Institute of Medical Technology, Brandenburg University of Technology Cottbus, 01968 Senftenberg, Germany; Tobias.Kutzner@b-tu.de (T.K.); Matthias.Gebhard@b-tu.de (M.G.); Ingrid.Boenninger@b-tu.de (I.B.); Wolf-Dietrich.Plath@b-tu.de (W.-D.P.) 2 Instituto para el Desarrollo Tecnológico y la Innovación en Comunicaciones (IDeTIC), Universidad de Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain; carlos.pazmino101@alu.ulpgc.es 3 Departamento de Señales y Comunicaciones, Universidad de Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain * Correspondence: carlos.travieso@ulpgc.es; Tel.: +34-928-452-864


Introduction
The new and different kinds of sensors open up the option to modern societies to analyze and collect different data in all fields. For a high percentage of these data, the software systems have to guarantee the protection of data privacy, such as bank data, examination marks in schools and universities, data in medical diagnoses, data in presidential elections, and movement data in car-assist systems. Therefore, many security and encryption methods are developed.
However, software security systems are developed by humans and therefore they can be decrypted by humans. This is one reason for the increasing interest in using biometric methods in authentication systems in recent years. There are many systems to recognize persons by gait, fingerprint, iris, and voice analysis, etc. In particular, in this proposal, a writer verification is developed using their online handwriting information. To authenticate a system using a password, the user name and password are normally entered via keyboard.
If touch screen devices are used for authentication, the password can be entered by hand on the touch screen. In addition to the user name and password, the biometric information contained in the handwritten password can also be verified to make authentication more secure.
Therefore, the aim of this paper is the presentation of a robust writer verification system. Recent writer identification studies have focused on two main problems, the author profile and the author verification.
In Reference [1], authors propose a system that uses the topological pixel density and pixel distribution and the gradient feature Gradient Local Binary Patterns. At first, each feature is associated to one SVM classifier, then Sugeno's Fuzzy Integral with a set of SVMs is used. As test records, the three databases IAM, KHATT, and IAM + KHATT were used. The combined system reaches at least 4% better results than the individual methods.
In order to objectively compare the performance of writer identification and gender classification systems, a ICDAR2015 Competition on Signature Verification and Writer Identification for On-and Off-line Skilled Forgeries (SigWIcomp2015) using QUWI database was organized [2]. The competition received five writer identification tasks and eight gender classification tasks. The overall best results for writer identification are reported by Nuremberg method. The best results for gender classification are reached by the CVC method.
A system of classification of gender is proposed in Reference [2] based on the protocols used in three competitions, ICDAR 2013, ICDAR 2015 and ICFHR 2016. The most interesting aspect of these competitions was the use of a dataset with writing samples of the same person in Arabic and in English. The textual information that distinguishes between male and female handwriting is extracted from different combinations of Oriented Basic Image Features (oBIFs) and used to train a Super Vector Machine classifier. About the author verification, some references can be shown. In author verification systems, the author of the handwritten document is recognized. In Reference [3], authors present a textural-based method for writer identification. The General Pattern Run-Length Transformation is used for binary and gray scale images of the four public databases. The experimental results with an identification rate between 84.3% and 99.5% outperform the state-of-the-art approaches.
In Reference [4], counting methods, vote technique and supervised learning are proposed to identify if an author has written a given document or not. The counting method in combination with the vote technology delivers identification results of 70.7% but its effectiveness highly depends on the number of known documents.
Many recent studies are focused on signature verification [5][6][7]. One of the recent signature verification investigations [5] extracts texture features, Local Binary Patterns (LBP) and Uniform Local Binary Patterns (ULBP). As a classification algorithm, the Nearest Neighbor is used. The research work is based on a database of 6240 Bangla and Hindi signature datasets and 7800 skilled forgeries. From the results obtained, applying LBP and ULBP features were almost equal. Researchers neither could determine a significant difference using eight or 12 signatures by each writer. The K-Value of the Nearest Neighbor has an influence on the result quality, due to experimentally, the proposal reached a best accuracy of about 76%.
A previously published paper [6] proposes a two-stage score normalization method to minimize the lack of information of intra-user variability. In the first step, simple forgeries are detected, and in the second stage, more skilled forgeries. For the experiments, the MCYT and the SUSIG databases with 10 reference signatures were used. Simple forgeries were detected with an Equal Error Rate (EER) of 0.85% for the MCYT dataset and skilled forgeries with an ERR of 2.13% for SUSIG dataset.
Other researchers [8,9] have attracted attention to identify the writers by their handwritten characters or numbers. Characteristic points on the image are detected by structural analysis and The attention on geometrical and temporal features like writing velocity or acceleration was attracted by References [27][28][29][30][31]. It is known that signatures written by the same person differ slightly from time to time. Therefore, in References [7,28,32,33], only the part of less fluctuation is considered as the stable feature of handwriting in the signing process. The forward variance along the forward direction from the starting point and the backward variance from the end point along the backward direction are calculated. In order to extract time intervals with less variance, thresholds are used. The experiments performed on the MCYT-100 database (5.000 signatures from 100 persons) reached an Equal Error Rate of 4.49%.
The authors of Reference [31] use descriptors based on spatially and temporally neighboring strokes to classify into one of the three classes: text, figure and table. They use English (700 pages) and Japanese (1520) datasets written by 40 people.
Considerable attention on time features was attracted by References [32][33][34][35][36]. The distribution on the handwriting feature pairs age-time up and age-time down were analyzed in Reference [35] with the goal to obtain better knowledge about the correlation between handwriting and the age. The experiment base was the BIOSECURED database with handwriting samples of 400 people. The preliminary results of the research show that changes of feature value are not affected by aging but rather by the motor or cognitive disorder of elderly people.
Previous studies [37][38][39][40] have focused on time or stroke features of handwriting. A writer recognition system for touch-screen mobile devices was proposed in Reference [37] for non-Latin languages with a large set of characters. Therefore, the authors avoid complex time-intensive algorithms like Multilayer Perceptron, Support Vector Machine or Hidden Markov Model. To recognize strokes in a character, a small number of prototype of stroke shapes is used with weighted Dynamic Time Warping and a Look-Up Table. To analyze the topographic information of pen pressure during the writing process is the central idea of the research [39].
Previous studies [40,41] have focused on pressure or used pressure as an additional feature. Only a few researchers [23,41] have used geometrical and temporal features.
Nevertheless, a robust authentication system, which identifies the writers independent of their writing style, if they write their signature or if they write isolated characters, still has not been found.
In our previous [42][43][44][45][46][47][48] studies, the focus was on the writer verification of handwritten single character passwords. So far no attention has been paid to establish a relationship between writer verification block letter and cursive texts.
The main objective of this paper is to find a discriminative, strong and novel set of features, which can be used for writer verification/identification using handwritten words composed of single character as well as cursive text. This approach combines statistical, geometrica, and temporal features. Furthermore, this proposal wants to analyze the efficiency of different types of features. For the classification stage, the proposal uses Bayes Net, Naïve Bayes and Nearest Neighbor classifiers. The proposed system is evaluated by public databases, the Secure Password DB 150 [42] and the IAM Online English Handwritten Text Database [49] (public) in order to show its robustness. Finally, the proposal is compared with different references from the state-of-the-art methods. The innovation of this proposal is focused on its final set of features and its reduction feature, which is applied for two different kinds of handwritten data: cursive text and isolated characters. Besides, it reaches better results than the state-of-the-art methods.
The use of Dynamic Time Warping (DTW) on signature verification can be found in the state-of-the-art methods too with good results [32,50,51]. Its use on handwritten text is new, and it is an innovation of this proposal.
An important aspect is the analysis of the signature stability of handwritings, in particular, Reference [52] shows a literature review, where the signature stability is analyzed for the verification of handwritten strokes. It is the key to develop the actual current proposal.
Most of the references deal with identification, but this proposal suggests to analyze a word composed of isolated characters, as if it were a password, and to check the grade of discrimination for using as security applications, as is the access with login and password.
The paper is organized as follows. Section 2 of this proposal describes the material: the used databases, the raw data and their pre-processing. Section 3 describes some of the 67 used features. Section 4 contains the feature reduction and classification methods. The experimental methodology can be found in Section 5. Section 6 presents the reached results of each experiment and a comparison versus the state-of-the-art methods. Finally, Section 7 offers conclusions.

Materials
In order to have a robust proposal, the dataset has to play a very important role. Therefore, this proposal will use two datasets for the evaluation and validation of the proposed system. They are The SECURE PASSWORD dataset [42] with 150 users, which is an own dataset; and The IAM online handwriting dataset [49] with 220 writers, which is a public dataset.

Secure Password Dataset
The database contains 2792 handwritten passwords with eight single characters of 150 users. Hereinafter, this database will be called Secure Password-DB-150. It is composed of handwritten passwords and it was collected in Spain and Germany, in particular at University of Las Palmas de Gran Canaria (ULPGC) and Brandenburg University of Technology (BTU), between 2012 and 2016. Some authors designed and built this dataset. The secure password-DB-150 considers about 18 samples of each user. The handwritten passwords are single characters, written in one or more lines (see Figure 1). Most of the references deal with identification, but this proposal suggests to analyze a word composed of isolated characters, as if it were a password, and to check the grade of discrimination for using as security applications, as is the access with login and password.
The paper is organized as follows. Section 2 of this proposal describes the material: the used databases, the raw data and their pre-processing. Section 3 describes some of the 67 used features. Section 4 contains the feature reduction and classification methods. The experimental methodology can be found in Section 5. Section 6 presents the reached results of each experiment and a comparison versus the state-of-the-art methods. Finally, Section VII offers conclusions.

Materials
In order to have a robust proposal, the dataset has to play a very important role. Therefore, this proposal will use two datasets for the evaluation and validation of the proposed system. They are The SECURE PASSWORD dataset [42] with 150 users, which is an own dataset; and The IAM online handwriting dataset [49] with 220 writers, which is a public dataset.

Secure Password Dataset
The database contains 2792 handwritten passwords with eight single characters of 150 users. Hereinafter, this database will be called Secure Password-DB-150. It is composed of handwritten passwords and it was collected in Spain and Germany, in particular at University of Las Palmas de Gran Canaria (ULPGC) and Brandenburg University of Technology (BTU), between 2012 and 2016. Some authors designed and built this dataset. The secure password-DB-150 considers about 18 samples of each user. The handwritten passwords are single characters, written in one or more lines (see Figure 1). The passwords are written on a display of an Android smartphone, preprocessed by a Java program, and transferred to a server, where feature extraction, classification, writer identification, and verification follow (see Figure 2).  The passwords are written on a display of an Android smartphone, preprocessed by a Java program, and transferred to a server, where feature extraction, classification, writer identification, and verification follow (see Figure 2). Most of the references deal with identification, but this proposal suggests to analyze a word composed of isolated characters, as if it were a password, and to check the grade of discrimination for using as security applications, as is the access with login and password.

IAM Online Handwriting Dataset
The paper is organized as follows. Section 2 of this proposal describes the material: the used databases, the raw data and their pre-processing. Section 3 describes some of the 67 used features. Section 4 contains the feature reduction and classification methods. The experimental methodology can be found in Section 5. Section 6 presents the reached results of each experiment and a comparison versus the state-of-the-art methods. Finally, Section VII offers conclusions.

Materials
In order to have a robust proposal, the dataset has to play a very important role. Therefore, this proposal will use two datasets for the evaluation and validation of the proposed system. They are The SECURE PASSWORD dataset [42] with 150 users, which is an own dataset; and The IAM online handwriting dataset [49] with 220 writers, which is a public dataset.

Secure Password Dataset
The database contains 2792 handwritten passwords with eight single characters of 150 users. Hereinafter, this database will be called Secure Password-DB-150. It is composed of handwritten passwords and it was collected in Spain and Germany, in particular at University of Las Palmas de Gran Canaria (ULPGC) and Brandenburg University of Technology (BTU), between 2012 and 2016. Some authors designed and built this dataset. The secure password-DB-150 considers about 18 samples of each user. The handwritten passwords are single characters, written in one or more lines (see Figure 1). The passwords are written on a display of an Android smartphone, preprocessed by a Java program, and transferred to a server, where feature extraction, classification, writer identification, and verification follow (see Figure 2).

IAM Online Handwriting Dataset
To determine the possibility of using the same features for writer verification with handwritten single character words, the experiments work with another dataset in order to validate the proposal. That used dataset was the IAM online handwriting database with cursive texts for verification.
The database contains 1760 handwritten text examples of 220 people. The text contains between 40 and 60 words on one and more than one line. The IAM online handwriting database considers eight samples of each user. (see Figure 3). To determine the possibility of using the same features for writer verification with handwritten single character words, the experiments work with another dataset in order to validate the proposal. That used dataset was the IAM online handwriting database with cursive texts for verification.
The database contains 1760 handwritten text examples of 220 people. The text contains between 40 and 60 words on one and more than one line. The IAM online handwriting database considers eight samples of each user. (see Figure 3).

Data Adaptation
In order to use the proposed system, the IAM data have to be transformed into a defined format. A XML parser program is used to transform the IAM data to our format. For the transformed texts, the same procedure is applied to the handwritten passwords of the Secure Password-DB-150.

Methods for Feature Extraction
This section contains information about the feature extraction. In particular, this proposal includes a set of 67 features, which are 18 geometrical, 33 statistical, and 16 temporal features. All features have a real value. The list of these parameters is included in Table 1. The name of the parameter is composed by the type of parameter, underscore and the name of parameter; for statistical features, it is "STAT_parameter_name"; for geometrical features, it is "GEO_parameter_name"; and for temporal features, it is "TEMP_parameter_name". It will be used for the rest of the document. The most significant features will be described in the next subsections, avoiding to describe intuitive and easy parameters.

Data Adaptation
In order to use the proposed system, the IAM data have to be transformed into a defined format. A XML parser program is used to transform the IAM data to our format. For the transformed texts, the same procedure is applied to the handwritten passwords of the Secure Password-DB-150.

Methods for Feature Extraction
This section contains information about the feature extraction. In particular, this proposal includes a set of 67 features, which are 18 geometrical, 33 statistical, and 16 temporal features. All features have a real value. The list of these parameters is included in Table 1. The name of the parameter is composed by the type of parameter, underscore and the name of parameter; for statistical features, it is "STAT_parameter_name"; for geometrical features, it is "GEO_parameter_name"; and for temporal features, it is "TEMP_parameter_name". It will be used for the rest of the document. The most significant features will be described in the next subsections, avoiding to describe intuitive and easy parameters. The extraction of features begins with parameter sampling like point coordinates, time values and touch down and touch up information for the identification of the connected components (segments). Where signatures consist of coordinates (x i , y i ), i = [1, n] (n: number of signature points) and segments consist of coordinates (x i , y i ), i = [1, np] (np: number of segment points). These terms will be used in all of the following formulas for a more detailed description of the features. A new connected component begins whenever the pen or finger is placed down on the display for writing and ends when the pen or finger is lifted up again. The features generated per segment are summed up and divided by the number of segments of the entire signature.

Geometrical Features
The different geometrical features are calculated and the features we have developed ourselves are described in more detail in the next paragraphs.
• REGRESSION_LOWER_HORANGLE (  (1)); • REGRESSION_ON_HORANGLE (Table 1 #2): Sum of points at horizontal axis (Equation (2)); • CENTRAL_POINT (Table 1 #3): Number of the central point of the Signature in x direction (Equation (3)); • REGRESSION_UPPER_HORANGLE (Table 1 #4): Sum of points above horizontal axis (Equation (4)); • EUCLID (  (5)); where (x 1 ,y 1 ) is the first point of the segment and (x n ,y n ) is the last point of the segment (see Figure 4). • HORIZONTAL_ POINT_ANGLE (Table 1 #7): This angle is calculated from the horizontal corner between the terminal points of the segments of the whole signature. To calculate this angle, a point-displaced eight units to the left from the center of rectangle has to be selected in this implementation. (see Figure 5, Equation (6)). a = arctan • SPHI (Segment length distance relation; Table 1 #8): The length of the segment (Euclidean distance of all points) divided by the distance between the starting point and endpoint of the segment, (Equation (7)); • HYP_ANGLE (Table 1 #9): The feature HYP_ANGLE determines the angle between the first and last point of the segment concerning the hypotenuse of the segment (see Figure 6, Equation (8)); Table 1 #10): Determines the angle in relation to the regression straight of the segment (see Figure 7, Equation (9)); where n is the number of points in the segment and (x i , y i ) are the coordinates of point i.

•
Width height surface of a character sequence (see Figure 8) WORD_WIDTH (  • HORIZONTAL_ POINT_ANGLE (Table 1 #7): This angle is calculated from the horizontal corner between the terminal points of the segments of the whole signature. To calculate this angle, a point-displaced eight units to the left from the center of rectangle has to be selected in this implementation. (see Figure 5, Equation (6)).   • HORIZONTAL_ POINT_ANGLE (Table 1 #7): This angle is calculated from the horizontal corner between the terminal points of the segments of the whole signature. To calculate this angle, a point-displaced eight units to the left from the center of rectangle has to be selected in this implementation. (see Figure 5, Equation (6)).

Figure 5.
Example of horizontal point angle. • REG_ANGLE (Table 1 #10): Determines the angle in relation to the regression straight of the segment (see Figure 7, Equation (9)); where n is the number of points in the segment and (xi, yi) are the coordinates of point i. • Width height surface of a character sequence (see Figure 8) WORD_WIDTH (Table 1 #11) WORD_HEIGHT (

Statictical Features
The extracted different statistical features we have developed ourselves are described in more detail in the next paragraphs: • POINTS ( Standard derivation in x or y direction with number of all n points (see Equation (10));  • REG_ANGLE (Table 1 #10): Determines the angle in relation to the regression straight of the segment (see Figure 7, Equation (9)  • Width height surface of a character sequence (see Figure 8) WORD_WIDTH (Table 1 #11) WORD_HEIGHT (

Statictical Features
The extracted different statistical features we have developed ourselves are described in more detail in the next paragraphs: • POINTS ( Standard derivation in x or y direction with number of all n points (see Equation (10));   • Width height surface of a character sequence (see Figure 8) WORD_WIDTH (Table 1 #11) WORD_HEIGHT (

Statictical Features
The extracted different statistical features we have developed ourselves are described in more detail in the next paragraphs: • POINTS ( Standard derivation in x or y direction with number of all n points (see Equation (10));

Methods for Feature Reduction and Classification System
This section includes the description of feature reduction methods and classification system methods applied on this approach in order to show the comparison and robustness of different methods.

Methods of Feature Reduction
Three methods are used and described for this proposal: • Fisher Score: The Generalized Fisher Score [10] is a joint feature selection criterion, which aims at finding a subset of features and maximizes the lower bound of the traditional Fisher Score. It also resolves redundant problems in the feature selection process. The mathematical description of the Fisher Score is shown as below in Equation (29).
where, j-the j-th feature. i-the i-th class, which could be interpreted as the i-th subject in our test. n-the size of the instances for a certain class µ-the mean value for a certain class σ-the standard deviation for a certain class • Correlation analysis: Although there are many attributes which are correlated, this proposal removes attributes which have a correlation coefficient above ±0.9. For this purpose, all attributes will be compared with each other. However, before an attribute gets discarded, we compare the Fisher Score of both attributes. If the score of the actual attributing better than the other score of the compared attribute, then the attribute gets marked for removal. When all other attributes are tested and no other attribute is better than the actual, all marked attributes get removed. However, if there is one Attribute which is better than the actual attribute, then it gets removed and the marked ones get unmarked. • Info Gain Attribute Evaluation: At second, Information Gain Attribute Evaluation (IG) [44] is used for ranking. This ranker evaluates the worth of an attribute by measuring the information gain with respect to the class. The mathematical description of Information Gain Attribute Evaluation is shown as below in Equation (30).
In which, H(Y) is the entropy of Y. H(Y/X) is the entropy of Y after observing X. p(y) is the marginal probability density function for the random variable Y. p(y/x) is the conditional probability of y given x.
The same test procedure was performed for the extracted features of the handwritten signature from the IAM online handwriting database.

Methods of Classification Systems
• KNN: For KNN (value of k = 1 for all experiments), we use the distance between an instance and the centroid of a class as the decision threshold. To calculate the ROC-Curve, we determine the greatest distance over all test instances. Then we alter the threshold from 0 to this determined greatest value. When the distance of a test instance is greater than the current decision threshold, we refuse the user. For every altered threshold, we calculate the TPR and FPR. • Naïve Bayes and Bayes Net: For every instance, we calculate the class probability. If the probability is higher than the threshold, the user is accepted. As higher class probabilities occur more often than lower probabilities, we use a higher resolution for higher probabilities.

Experimental Methodology
The idea is to apply the proposal on two datasets with different kinds of data in order to show its robustness and innovation. One of them is ours (Secure Password DB 150), and the approach was developed on it. The second dataset is public, and it was used to validate the proposal (IAM online handwriting database). The experimental protocol implements two strategies, K-fold cross validation (Cross 10) method and hold-out cross validation (Hold-out 66%) method for three classifiers, in order to show the robustness of the proposal. Besides, it is implemented under identification and verification stages. The verification stage is measured by receiver operating characteristic (ROC) curves and the recognition stage is measured by the success percentage of the confusion matrix. All experiments are running in an i5 core processor using WEKA tool [54]. The computational time is shown and measured by the average (AVG); time is shown in seconds (s.).
During the experimental process, the implementation of all experiments was sequentially led. The first step was to check the accuracy of each type of feature; and finally, the accuracy for all features. The second step was to apply feature-reduction methods in order to improve the accuracy of the proposal. Transversally, they were implemented for the three classifiers and all strategies and stages.
About the specific experiments, the first experiments start with the analysis of the Secure Password DB 150. At first, the classification of the handwritten passwords by using the different feature types: geometrical, statistical and temporal features separately is carried out. Afterwards, we applied all 67 features for the writer verification.
About the quality measure, the approach applies ROC curves for estimating a threshold for discriminating genuine from imposter, because the experiments are based on a verification approach.
On the x-axis is the false positive rate (FPR) and on the y-axis the true positive rate (TPR). The FPR and TPR are given in percent and are calculated as shown below (see Equations (31) and (32)): where the variables are defined on the following confusion matrix (see Figure 9). The TPR told us how many of the genuine Users are correctly classified as a genuine User. The FPR told us how many imposters are classified incorrectly as genuine Users. For estimating a threshold, the proposal uses for KNN the distance and for Naïve Bayes and Bayes Net applies the class probability. The next step is to develop a procedure to choose the optimal threshold based on the ROC-Curve.
approach. On the x-axis is the false positive rate (FPR) and on the y-axis the true positive rate (TPR). The FPR and TPR are given in percent and are calculated as shown below (see Equations (31) and (32)): where the variables are defined on the following confusion matrix (see Figure 9). Since we have a multiclass problem, we need to simplify our dataset to a binary class problem. For generating the ROC-Curve, we worked with a real forgery dataset. Therefore, we have different writers for the same password. The password is randomly generated. The imposter knows the password. Thus, a real scenario where a forger tried to hack a password was implemented and tested.

Genuine (OUT)
For both methods with genuine users and imposters, the approach trains a classifier with 66% of genuine samples and the rest of the samples (34%) are used to test it; then, TPR is calculated. A tested sample is a true positive, when the classifier predicts the correct class label and the instance matches the acceptance rule. An imposter is a false positive, when the instance matches the acceptance rule, no matter if the predicted class label is the correct class label for the forged instance.
For another robustness test, we calculate the ROC-Curve applying a k-fold cross validation, in particular for K equal to 10. As a result, we plot the area between the maximum and minimum ROC-Curves and the average of all curves. The area is colored grey and the average is colored blue. The points on the blue line are the points we calculate for different thresholds. The horizontal red line marks the 90% TPR and the vertical red line marks the 10% FPR. In the results section, these figures can be observed.

Experiments and Results
The experiments evaluate the grade of discrimination for the type of features, for the set of all features and for the feature reduction of the whole feature set. The results show the grade of the robustness and its analysis.

Results for Different Feature Sets in Secure Password DB 150
The following tables show that the results first split to the feature type's geometrical, statistical, temporal, pressure and finally all features for the secure password DB 150.
• Geometrical features Table 2 shows results for geometrical features in the secure passwords. Naïve Bayes classifier achieves the best result for the K-fold cross validation method. It seems that the geometrical features show low results in comparison with the state-of-the-art methods. The computational times are good for application. • Statistical features In Table 3, it is shown that statistical features deliver better results than geometrical features. K-nearest neighbor classifier delivers the best result for classification. • Temporal features Table 4 shows that temporal features do not deliver the best results; the best classifier is the k-nearest neighbor classifier closely followed by Naïve Bayes classifier for cross validation. The time for the classification is the same as before. • Using all features In Table 5, it is shown that using all features contributes once again to an improvement of the classification results. Although geometrical and statistical features individually achieved not so good results, by the use of all features, the classification rate grows nearly about 2% for the best result. KNN protrudes again with the best result for the cross validation. Using all features has a negative effect on the average time for the classification. It is not surprising, either, however, that with fewer features, the system needs less time for classification. However, for the practical application, this time should be reduced under the retention of the good classification results. Table 5 shows some Fisher Scores for features. • Applying feature reduction with Fisher Score and Correlation analysis: The first step is to apply the Fisher Scores, and later, it is applied to a quartile of 30%. Every attribute which has a lower score than the 30% quantile is removed from the dataset. After that, the authors apply the correlation matrix. If the correlation of two attributes is bigger than ±0.9, authors remove the attribute with the lower score. The colored rows in grey mark the final chosen attributes (see Table 6), and then, the 37 attributes are selected.
In Table 7, it is demonstrated that after the reduction of features with Fisher Score, the results for K-Nearest Neighbor classifier are a bit better. The average (AVG) time for classification is nearly the same, because of the high number of features for classification.
• Applying feature reduction with info gain attribute select Table 8 shows the result for the use of "Info Gain Attribute Select" from Weka to reduce the number of features. The ranker selected 37 features for classification. The Attributes with bold letter were chosen by Fisher Score and "Info Gain Attribute Select".   Table 9, it is demonstrated that after ranking with "Info Gain Attribute Select", the results for Naïve Bayes and K-Nearest Neighbor classifier are a little worse than before. By the feature reduction, the classification time considerably decreases to AVG of <0.01 s for KNN. • ROC Analysis Figure 10 shows the ROC-Curve for KNN classifier. KNN can classify about 60% genuine correct and reject any impostor. About 93% TPR of all impostors were accepted.  Table 9, it is demonstrated that after ranking with "Info Gain Attribute Select", the results for Naïve Bayes and K-Nearest Neighbor classifier are a little worse than before. By the feature reduction, the classification time considerably decreases to AVG of <0.01 s for KNN. • ROC Analysis Figure 10 shows the ROC-Curve for KNN classifier. KNN can classify about 60% genuine correct and reject any impostor. About 93% TPR of all impostors were accepted.  Figure 11 on the next page shows the ROC-Curve for the Naïve Bayes classifier. The curve has a long linear interpolated part from origin (100% class probability) to the second threshold (99% class probability). About 60% of the impostors and about 80% of the genuine users have a class probability greater than 99%.  Figure 11 on the next page shows the ROC-Curve for the Naïve Bayes classifier. The curve has a long linear interpolated part from origin (100% class probability) to the second threshold (99% class probability). About 60% of the impostors and about 80% of the genuine users have a class probability greater than 99%.   Note that Naïve Bayes and Bayes Net never lead to low TPR, resulting in a long linearly interpolated section of the ROC-curve starting from the origin; it is quite difficult to compare the curves of them with KNN. However, at least we can say that none of the classifiers alone can classify the user and classify the genuine user and imposter with an acceptable trade-off. The results show that, possibly, it is a good idea to combine the classifiers to minimize the problem. For example: Naïve Bayes can reject nearly 40% of the impostors without rejecting too many genuine users. This would minimize the Problem and could maybe help to improve results for KNN and Bayes Net.

Results for Different Feature Sets in IAM Online Handwriting DB
Secondly, the authors show the results of the feature analysis of the handwritten passwords on a public dataset in order to validate the proposal. It is the well-known IAM online handwriting DB.
• Geometrical features Figure 11. ROC-Curve for Naïve Bayes classifier. Figure 12 shows the ROC-Curve for Bayes Net classifier. Authors need to decrease the step size for the threshold at a certain point to get a better resolution.   Note that Naïve Bayes and Bayes Net never lead to low TPR, resulting in a long linearly interpolated section of the ROC-curve starting from the origin; it is quite difficult to compare the curves of them with KNN. However, at least we can say that none of the classifiers alone can classify the user and classify the genuine user and imposter with an acceptable trade-off. The results show that, possibly, it is a good idea to combine the classifiers to minimize the problem. For example: Naïve Bayes can reject nearly 40% of the impostors without rejecting too many genuine users. This would minimize the Problem and could maybe help to improve results for KNN and Bayes Net.

Results for Different Feature Sets in IAM Online Handwriting DB
Secondly, the authors show the results of the feature analysis of the handwritten passwords on a public dataset in order to validate the proposal. It is the well-known IAM online handwriting DB.
• Geometrical features Note that Naïve Bayes and Bayes Net never lead to low TPR, resulting in a long linearly interpolated section of the ROC-curve starting from the origin; it is quite difficult to compare the curves of them with KNN. However, at least we can say that none of the classifiers alone can classify the user and classify the genuine user and imposter with an acceptable trade-off. The results show that, possibly, it is a good idea to combine the classifiers to minimize the problem. For example: Naïve Bayes can reject nearly 40% of the impostors without rejecting too many genuine users. This would minimize the Problem and could maybe help to improve results for KNN and Bayes Net.

Results for Different Feature Sets in IAM Online Handwriting DB
Secondly, the authors show the results of the feature analysis of the handwritten passwords on a public dataset in order to validate the proposal. It is the well-known IAM online handwriting DB.
• Geometrical features In Table 10, it is shown that the best result of 54.62% correct classification we achieved with the Naïve Bayes classifier. To sum it up: We reach only average results for the geometrical feature. • Statistical features In Table 11 it is shown that the results of statistical features are a little bit worse than the geometrical features' best result with 35.32% correct classification reached by the Naïve Bayes classifier for cross validation. The time for classification is nearly the same as the geometrical features. • Temporal features In Table 12 it is shown that the temporal features achieved the best result of 96.15% correctly classified for Naïve Bayes classifier with cross validation. The maximum time for classification is with 0.5 s, only a bit longer than the time for the other features. • Using all features All features were used for getting the results of Table 13. A small increase is found in comparison with the temporal features. The best classifier is Bayes Net with 98.65% correct classification. The time for the classification lies with 1.41 s. still within the scope of state-of-the-art methods. • Applying feature reduction with Fisher Score The ranked results of Fisher Score Reduction for IAM online handwriting DB are shown in Table 14. The ranker selects 59 features for classification. The best score is clearly achieved with time features, followed by geometrical and statistical features. The Fisher Score Reduction in Table 15 demonstrates that the results for Naïve Bayes and K-Nearest Neighbor are a bit better than before. By the reduction of the features, the AVG Time decreases to 1.3 s for Bayes Net classifier. • Applying feature reduction with info gain Attribute select Table 16 presents the ranked result of Info Gain Attribute Select and the ranker selected 22 features for the classification.  Table 17 shows that the result improved for Naïve Bayes and K-Nearest Neighbor classifier again in spite of the reduction on only 22 features. The time becomes considerably shorter with only 0.62 s for Bayes Net classifier for classification after ranking with Info gain Attribute Select. This dataset does not present impostors due to its building, different texts, paragraphs and syntax; therefore, this proposal could not compute a meaningful ROC-Curve. It is easy to understand that if it randomly takes other genuine samples as impostors, this method will not lead to a meaningful curve. Most of the impostors can be easily rejected and we get ROC-Curves with the exact 0% FPR for every threshold.
Finally, in order to show the robustness of this proposal, it is presented in a comparison table with the most representative references, which uses both dataset, the Secure Password DB 150 and the IAM online dataset (see Table 18). Table 18 is split into two parts, one part for private datasets and the second part for the IAM dataset. Some references apply different protocols, but with better conditions than this proposal. The idea is to show the value of accuracies vs. the proposal. Besides, the applied methods and their results are observed in order to check the behavior of each reference versus this proposal. In both cases, this work reaches a good answer for both datasets and it indicates its robustness. The most significant comparison is the type of level. The most of the papers, which work on IAM dataset, apply the writer identification for text line level and paragraph level. Our work can be applied to handwritten passwords or/and continuous writing, being an added value of this proposal and even, reaching good accuracies for both datasets.

Conclusions
This work presents a feature set for writer identification and verification by online handwriting data. The innovation of the work is on the concatenation and fusion of the set of 67 features and its reduction in order to improve the accuracy for verification and identification stages.
The feature set is applied on two different types of handwritten data. The proposal of the feature set is the innovation of the proposal, which can be applied with a good answer for different handwritten data, in particular, handwritten passwords composed on isolated characters and as well as handwritten cursive texts. After feature reduction, the results for KNN also improved. In summary, it can be stated that both the feature reduction and the classification method have a strong influence on the results for both data sets. Only there is to apply the adequate feature reduction method to each dataset, due to the nature of the both writing is different. IAM Online Handwriting DB is continuous writing and Secure Password DB 150 are isolated characters (a password). The best results delivered 95.38% correct classification for handwritten single character words using the Secure Password DB 150; about 93% TPR for real impostor test; and 98.34% for cursive texts using IAM Online Handwriting DB. The proposal is considered a strong and novel proposal, after to review the state-of-the-art.
The results of this work depend on the different parameter types and data types. However, the whole feature set for both datasets presents a similar and good answer and is very significant in comparison with the state-of-the-art due to the relation numbers of users and accuracy. In general, the feature reduction improves the accuracies of all features. In particular, the Fisher Score method reaches the best accuracy. Therefore, it is concluded that the proposal of this feature set shows a strong answer for the writer identification as can be observed in Table 18.
For future work, authors will generate an extended feature set in order to improve the actual proposal. Funding: This research is funded by "Ministerio de Ciencia, Innovación y Universidades" from The Spanish Government, under Grant "Estancias de profesores e investigadores senior en centros extranjeros, incluido en el Programa Salvador de Madariaga 2018" with the reference PRX18/00423. Too, this work was partially funded by the government of Spain, grant number TEC2016-77791-C4-1-R.