Classification Algorithm Optimization Based on Triple-GAN

: Generating an Adversarial network (GAN) has shown great development prospects in image generation and semi-supervised learning and has evolved into Triple-GAN. However, there are still two problems that need to be solved in Triple-GAN: based on the KL divergence distribution structure, gradients are easy to disappear and training instability occurs. Since Triple-GAN tags the samples manually, the manual marking workload is too large. Marked uneven and so on. This article builds on this improved Triple-GAN model (Improved Triple-GAN), which uses Random Forests to classify real samples, automate tagging of leaf nodes, and use Least Squares Generative Adversarial Networks (LSGAN) ideological structure loss function to avoid gradients disappear. Experiments were performed on the Improved Triple-GAN model and the Triple-GAN model using the MINIST, cifar10 and cifar100 datasets respectively, experiments show that the error rate of generated samples is greatly reduced. At the same time, the classification effect of the data set and the sharpness of the samples are greatly improved. And it has greatly improved the stability of model training and automation of labels.


Introduction
Image mining is an emerging field in data mining. Image classification is the basis of data mining. Faced with a large number of image data, image classification becomes more and more important. Image classification, as one of the important means to understand image content, has been successfully applied in the fields of finance, public safety, transportation, etc, and its importance is self-evident. Faced with a large number of image data sets, it is not easy to retrieve and classify the semantic attributes of the images by manual methods, which causes problems such as inaccurate image information analysis. Image classification currently has many classification techniques, such as decision tree [Liu and Fan (2014)], minimum distance method, neural network [Alec, Luke and Soumith (2015)], fuzzy classification, support vector machine [Jitender and Julka (2017)], k-means [Amorim and Mirkin (2012)], and so on. The GAN model [Wang, Xu, Yao et al. (2019)] proposed a new height in the field of image classification, and also promoted the development of image mining technology. Based on the GAN model research, it has become a research hotspot. The generation of confrontation networks (GAN) also shows great development prospects in the field of image generation [Ma, Yu, Liang et al. (2019)] and semisupervised learning [Leonardo and Alastair (1998)]. GAN has become an image and visual computing, speech and language processing, information security, chess competitions, especially artificial intelligence. Hot research directions in the field. Now GAN has evolved from a two-player game to a Triple-GAN game, and the Triple-GAN body includes a classifier (C), a generator (G), and a discriminator (D). Triple-GAN [Li, Xu, Zu et al. (2017)] ensures that classifiers and generators implement an optimal solution for classification from a game theory perspective and that generators can sample data in a particular class. Triple-GAN based on the KL divergence distribution structure of the target loss function, there is still when the distribution does not intersect, the gradient disappears. Therefore, it is proposed to use the least-squares generation countermeasure network (LSGAN) to use the least-squares loss function for the discriminator. Experiments show that the LSGAN objective function can minimize the 2 Pearson χ distribution. Compared with conventional GAN [Zhou, Liang, Song et al. (2019)], LSGAN has two advantages: LSGAN can generate higher quality images than conventional GAN; LSGAN behaves more stably during the learning process [Mao, Li and Xie (2017)]. However, Triple-GAN still has the following problems in the application process: (1) Due to the use of the KL divergence distribution, gradients disappear easily in the case where the distribution does not intersect, which results in the termination of training when the training does not reach the desired result. Make model training unstable [Brock, Donahue and Simonyan (2019)]. (2) There is a problem that a large amount of data labels has a large workload since manually labeling a sample causes uneven label labels [Zhang, Goodfellow, Metaxas et al. (2018)]. In response to the above problems, this paper will use Random Forests [Adele, Richard Cutler and John Stevens (2011)] to optimize the Triple-GAN classification algorithm. At the same time, using LSGAN to build loss distribution function based on minimizing the chi-square distribution, we plan to carry out the following work: (1) Improve the classifier and integrate the random forest into the classifier to construct a classifier with automatic tagging. (2) Using the inference process of the LSGAN loss function, combined with the Triple-GAN construction model, it is possible to inherit the stability of the LSGAN and inherit the advantages of the Triple-GAN.

Improved triple-GAN model 2.1 Improved triple-GAN build
Due to the existence of the Triple-GAN model, there is a problem that when the distribution does not intersect, the gradient disappear and the manual label becomes cumbersome. Based on this, this paper will improve Triple-GAN in two aspects [Salimans, Goodfellow, Zaremba et al. (2016)]. In the first aspect, it is proposed to refer to the random forest to implement the automatic label of real samples. In the second aspect, based on the LSGAN theoretical model, the Triple-GAN objective function is optimized and improved so that Triple-GAN inherits the LSGAN theoretical model to avoid the advantages of gradient disappearance [Sutherland, Tung, Strathmann et al. (2017)]. The Improved Triple-GAN model introduces three joint distribution networks, classifiers, discriminators, and generators [Arjovsky and Bottou (2017)]. The specific steps are as follows: (1) Randomly retrieving real samples, setting up a random forest decision tree, and automatically marking the leaves of decision trees. After the random forest decision tree is constructed, leaf node data with label and non-leaf node data with no label are input into the neural network (semi-supervised learning) for training [du Plessis, Niu and Sugiyama (2015)]. At the same time, the tagged leaf node is input to the discriminator. (2) The training sample is input into the generator so that the generator will obtain the joint distribution Pg(x,y) of the tag classy and the generated data x through training. (3) Pg(x,y) is input into the classifier, and the corresponding leaf node labels are found through the prediction analysis of the random forest decision tree to form the joint distribution Pc(x,y) of the leaf node labels and the data generated by the classifier. (4) Pg(x,y)and Pc(x,y)are input to a discriminator and judged by a target loss function. When the target loss rate is 0.5, it indicates that the generated tags and the classified tags are all optimal for the distribution, and the training is completed at this time. Otherwise iterate the above steps and repeat the training until the training reaches the optimal training. The block diagram of the model is shown in Fig. 1

Improved triple-GAN model theory analysis
Using the LSGAN objective function principle, the a-b encoding scheme is used for the discriminator, where a and b are the labels of the dummy data and the real data, respectively. In the formula, C represents a classifier, G represents a generator, and D represents a discriminator (the same below). For any fixed C and G, the game's optimal discriminator D is defined by the objective function U(C, G, D), as shown in Eq. (1): . Given classifier C and generator G, combined with the LSGAN objective function, where c denotes that G, C expect D to believe the value of the dummy data, the objective function can be rewritten as Eq. (2): When b-c=1 and b-a=2, according to the derivation of LSGAN, Eq. (3) can be derived as: , take the integral on both sides as shown in Eq. (5): According to Eq. (2), the objective function pair V(C, G) takes the minimum value, and D makes the maximum value judgment, and the objective function is obtained as shown in Eq. (6): For the Eq. (6) reasoning process description: represents the generated data, y represents the class label, z represents the leaf node corresponding to the generated data x; represents the generator (G) encodes the generated data, a ∈{-1,0,1}, b means the classifier (C) encodes the class label, b∈{-1,0,1}; c means the generator (G) uses the objective function to determine the optimal deceptive sample data for the generated data, c∈{-1,0,1}; in this formula: E(x, y)~p(x ,y) represents the joint distribution p (x,y) of the label pair (x,y) in the discriminator ( means that the label pair (x,y) is in the generator (G) Expected value of the joint distribution Pg (x, y); G (x,z) represents the mapping of the label pair (y,z) in the sample space in the generator, and D (G (y,z), y) represents the label in the discriminator Probability of generated samples for (y,z) versus real samples; Rc represents the equilibrium function; α is the adjustment parameter, α∈{0,1}. When b-c=1, b-a=2, and P(x,y)=Pa(x, y), V(G, C)=0, causing unbalanced training and making the results unoptimized. To solve this problem, construct Rc function, Rc uses 2 Pearson χ to arrange reliability control and distribution principle, and make discriminator and classification as the distribution equilibrium structure, namely: Combining the Eq. (6) and Eq. (7) to get the final objective function Eq. (8): According to the theoretical analysis, the final objective function inherits Triple-GAN's triple-antagonism model structure. At the same time, it refers to the construction of the LSGAN objective function and obeys the 2 Pearson χ distribution.

Experiment analysis 3.1 Model training results
In this paper, three data sets, MINIST [MNIST Dataset (2019) (1) The experimental results of the Improved Triple-GAN model MINIST data set in Tab. 1 analysis: The iterative error rate of the Improved Triple-GAN is 0.02720 at the first and 0.01150 at the 100th, indicating that the error rate gradually decreases after iteration. Improved Triple-GAN discriminant loss rate, generation loss rate, classification loss rate at the first iteration, the loss rate of the three values are larger (the discriminatory loss rate: 0.2077, generated loss rate:0.9897, classification loss rate: From the 0.3316) to the 100th iteration, the difference between the three-loss rates is not significant (discrimination loss rate: 0.3463, loss generation rate: 0.3471, classification loss rate: 0.3514), indicating that the Improved Triple-GAN model is trained iteratively. The loss rate is close to balance. The experimental results of Tab. 2 Triple-GAN model MINIST data set are analyzed: the iteration error rate of Triple-GAN is 0.05820 for the first time and 0.01360 for the 100th time, indicating that the error rate gradually decreases after iteration. At the same time, it can be seen that Triple-GAN discriminates the loss rate, the generation loss rate, and the classification loss rate. At the first iteration, the difference between the three values of the loss rate is relatively large. They are the discriminatory loss rate: 0.2021, and the generation loss rate: 0.9956. Classification loss rate: 0.3296. At the 100th iteration, there was no significant difference in the loss rates of the three categories, namely the discriminatory loss rate: 0.3462, the generated loss rate: 0.3473, and the classification loss rate: 0.3519, indicating that the Triple-GAN's loss rate through the iterative training approached equilibrium. Therefore, it can be judged that compared with Triple-GAN, the Improved Triple-GAN error rate is lower, and at the same time, it inherits the advantages of Triple-GAN balance. (2) The experimental results of Tab. 3 Improved Triple-GAN model cifar10 data set are analyzed: The iterative error rate of Improved Triple-GAN is 0.46810 for the first time and 0.57640 for the 40th time, which shows that the error rate gradually decreases after iteration.
(2) Improved Triple-GAN discriminative loss rate, generation loss rate, and classification loss rate at the first iteration, the three values of the loss rate are quite different. They are the discriminatory loss rate: 0.3173, the generated loss rate: 0.4077, the classification loss rate: 0.8093. By the 40th iteration, the three values of the loss rate are not much different. They are the discriminatory loss rate: 0.3422, the generated loss rate: 0.3543, and the classification loss rate: 0.3140, which indicates that the Triple-GAN through the iterative training loss rate is close to the three. balance. The experimental results of the Tab. 4 Triple-GAN model cifar10 data set are analyzed: the iteration error rate of Triple-GAN is 0.57640 for the first time and 0.26420 for the 40th time, indicating that the error rate gradually decreases after iteration. Triple-GAN discriminates loss rate, generation loss rate, and classification loss rate. At the first iteration, there is a large gap between the three-loss rates, which are the discriminatory loss rate: 0.3216, the generated loss rate: 0.3987, the classification loss rate: 0.8009, At the 40th iteration, the three values of the loss rate are not significantly different, they are the discriminatory loss rate: 0.3431, the generated loss rate: 0.3529, the classification loss rate: 0.3178), indicating that the loss rate of the Triple-GAN through iterative training is close to the three. balance. Therefore, compared to Triple-GAN, Improved Triple-GAN has a lower error rate, and at the same time, it inherits the balancing advantages of Triple-GAN. (3) The experimental results of Tab. 5 Improved Triple-GAN model cifar100 data set are analyzed: The iterative error rate of Improved Triple-GAN is 0.82530 for the first time and 0.35380 for the 100th iteration, indicating that the error rate gradually decreases after iteration. Improved Triple-GAN discriminant loss rate, generation loss rate, classification loss rate, the first iteration is that the loss rate of the three values are larger, they are the discriminatory loss rate: 0.383, the generated loss rate: 0.4447, the classification loss rate: 0.9969. At the 100th iteration, the loss rates of the three are not significant. They are the discriminatory loss rate: 0.3520, the generation loss rate: 0.3587, and the classification loss rate: 0.3556. This indicates that the Triple-GAN through the iterative training is nearly equal to the loss rate. The experimental results of Tab. 6 Triple-GAN model cifar100 data set are analyzed: the iteration error rate of Triple-GAN is 0.92000 for the first time and 0.42480 for the 100th time, indicating that the error rate gradually decreases after iteration. Triple-GAN discriminates the loss rate, the generation loss rate, and the classification loss rate. In the first iteration, the difference between the three values of the loss rate is relatively large. They are the discriminatory loss rate: 0.3095, the generation loss rate: 0.4202, and the classification loss rate: 0.9998. At the 100th iteration, there was no significant difference in the loss rates of the three categories, namely the discriminatory loss rate: 0.3020, the generated loss rate: 0.3187, and the classification loss rate: 0.3256, indicating that the Triple-GAN's loss rate was nearly balanced through iterative training. Therefore, it can be judged that the Improved Triple-GAN error rate is lower than that of Triple-GAN, and Triple-GAN inherits the balancing advantages.  Experiments were conducted on the Improved Triple-GAN model and the Triple-GAN model on the three data sets MINIST, cifar10, and cifar100, respectively. The experimental results show that compared with Triple-GAN, the error rate of Improved Triple-GAN is lower and inherits the balanced advantages of Triple-GAN. It can be concluded that Improved Triple-GAN can inherit the advantages of Triple-GAN and is better than Triple-GAN. Besides, through the comparison of cifar10 and cifar100 experiments, it can be seen that the more types of data, the error rate will be significantly improved. By comparing the experiment of MINIST with cifar10, we can get that the more complicated the data set, the higher the error rate.

Image effect
Three kinds of data sets, MINIST, cifar10, and cifar100, were analyzed on the images of the Triple-GAN model, the Triple-GAN model, and the LSGAN model. The following describes the experimental results from several aspects: (1) Analysis of image effects trained from iterative training process, as shown in Fig. 2 Improved Triple-GAN model MINIST iterative training image effect image sharpness analysis, Improved Triple-GAN model training MINIST data set from the iterative 10 times image output is more blurred After 200 iterations, the image output is relatively clear, indicating that after iterative training, the image becomes clearer and clearer. As shown in Fig (3) From the image classification effect analysis, from the comparative analysis of the MINIST dataset in Fig. 6, it can be seen that the effect of the MINIST data set on the Improved Triple-GAN model is better than that of the LSGAN model, and the image is clearer.

Results and Discussion
From the perspective of random forest algorithm and loss function obeying 2 Pearson χ distribution, the KL divergence is based on log (log) loss function, and 2 Pearson χ distribution is based on square (sqrt) form of distributed loss function. The distribution and random forest interface are used for effect analysis, as shown in Fig. 7. The results show that with the increase of the number of iterations, the error rate of random forest and 2 Pearson χ distribution is significantly lower than that of KL-divergence. In the Triple-GAN optimization model, the generator, discriminator, and classifier loss crossover curves of the loss function are analyzed. As shown in Fig. 8, the crossover loss is distributed between 2.5% and 5%. The rate remained relatively low, indicating that the Improved Triple-GAN model is relatively stable. From clarity and classification effect, the training data of the datasets of MINIST are more ideal. From the output data of the training process, the pre-training accuracy rate of different training numbers is extracted, as shown in Fig. 9. From the pre-training training accuracy curve in Fig. 4, it can be seen that as the number of pre-training increases, the accuracy of randomly arranged data and tag classification continues to decrease; as the number of training increases, the final accuracy tends to be one. The balance is stable at about 2.75%. From the perspective of classification performance, MINIST's simple hand-written digit set is taken as an example. Experiments show that MINIST running on the Improved Triple-GAN model can be quickly classified, and it can have a clear classification effect through iterative training. The Improved Triple-GAN model generates a sample effect for the generator, as shown in the randomly generated visual matching error rate graph of Fig. 10. It can be seen that with time, the generation error rate is significantly reduced, indicating that the random forest algorithm generates a similar direction for the generator and generates similar data sets in the data set class, and the classification is made smaller and smaller, making the generation the sample generation error rate is getting smaller and smaller. From the beginning 2.8% to 1.2%, which greatly reduces the error rate.

Conclusions
Based on the above experiments, it can be seen that the Improved Triple-GAN model classifies and sorts the same sort of insertions, which greatly reduces the error rate of generated samples. At the same time, the data set classification effect and sample definition are greatly improved. And there is a great improvement in model training stability and label tag automation.
Since there are few kinds of datasets, more datasets will be used to verify the practicality of the Improved Triple-GAN model and the reliability of the model will be evaluated in terms of speed, effect, and performance. At the same time, the Improved Triple-GAN model can also be combined with the principle of CNN to continuously supervise the learning classification. The generated sample data can be continuously learned by CNN, similar data can be generated, and duplicate generation or overfitting generation can be avoided, thereby greatly improving the speed of the algorithm.

Conflicts of Interest:
The authors declare that they have no conflicts of interest to report regarding the present study.