Artificial Neural Networks and Multilinear Least Squares to Model Physicochemical Properties of Organic Solvents

The mean molecular connectivity indices (MMCI) proposed and used in previous studies are used here in conjunction with the well-known molecular connectivity indices (MCI) to remodel six properties of organic solvents. The MMCI and MCI descriptors of the Multilinear relationships for the six properties, obtained with the multilinear least squares (MLS) procedure, were used to perform the artificial neural network (ANN) computations. The aim is to detect advantages and underline the limits of the ANN approach that, even if it improved the model, it is somewhat ‘fuzzy’ concerning the stability of the modeling. The MLS procedure replicates the obtained results as long as one wishes, a characteristic not shared by the ANN methodology, which, if on one side increases the quality of a description on the other increases also its overfitting. The present study reveals also how ANN methods prefer MCI relatively to MMCI descriptors. Four different types of ANN computations show that (i) MMCI descriptors are preferred with properties with poor number of points. MLS (ii) is to be preferred over ANN statistical results, with some exceptions, when the number of ANN weights is similar to the number of correlation coefficients of MLS. Furthermore, in (iii) some cases MLS modeling quality is quite similar to the modeling quality of ANN computations. __________________________________________________________________________________

shows the definitions of the MMCI (the first M stands for 'mean').
In Tables 1, and 2 i = 1-N denotes the atoms of a molecule, ij denotes directly σ-bonded atoms, and in Table 2, p = N. The Lehmer mean, L M, for p = 2, equals the symmetrical mean, S M.
Replacing in Table 1  with the valence delta,  v , allows to obtain the corresponding valence is the i th predicted value (network outputs) of the property, and P i is the target value. This function is the sum of differences between the prediction outputs and the target defined over the entire  Table 3) and test sets (TE = 20% of the values, i.e., the underlined bold values in this same Table) usually avoids overfitting as the network is repeatedly trained for a number of cycles so long as the test error is decreasing, otherwise the training is halted. This method, also known as 'early stopping' procedure 13 avoids the trap that the program will always choose the maximum number of hidden nodes. Actually each property shows an optimal number of nodes which rarely corresponds with its maximum possible.  For comparison purposes it was decided to maintain in the ANN calculations (see Tables 5-8) the same number of outliers excluded throughout the MLS procedure and given in Table 4,where the exclusion was done for residuals greater than 3s. Clearly, such restriction is no more valid throughout ANN tables 5 -8. In Table 5 are given the ANN results obtained with a single hidden neuron. Following Tables 6 and 7 display the multiple neuron cases: Table 6 with an externally imposed number of hidden neurons that was cycled from 2 to 12, and Table 7

Discussion:
For an easier lecture and interpretation the detailed and most important statistical results collected through Tables 4 -7 are summarized in Table 9. Table 8, illustrates a special case that will be discussed later on. While Tables 4 -7 collect the detailed information about the modeling of the six properties, and especially about the type of indices, valence deltas, and structure of the ANN computations, Table 9 gives direct information about of the various models.
N is the number of atoms, ij means corresponds to σ bond,  is the cyclomatic number.    (Table 4) and training + test sets {TR + TE} (Table 5).    Table 4 Table 8. ANN results for the set of descriptors of Table 4 with only one hidden neuron but where either one or   two indices have been left out, usually, those with lowest sensitivity values in Table 5. For the structure of this   table see Table 5. Only satisfactory results are shown here.  (Table 4) ANN 1HN (Table 5) ANN enHN (

Conclusions:
The first interesting result of the present ANN computations is that they prefer MCIs instead of MMCIs, especially with properties with relatively large number of points. In fact, only El, with minimal number of points is advantageously described by MMCI when ANN with more than one hidden neuron is used. The second result being that it is better to impose from outside the number of hidden neurons. The third result being that it is better to run ANNs using quite different numbers of networks to train, Ntr. The fourth and the more interesting result being that normally ANN improve over the MLS calculations, but also that in many cases this improvement is not striking.
It should be remembered that MLS is anyway used to derive the best set of descriptors that are passed over to the ANNs, and that its statistical results are definitive, i.e., no matter how many times you repeat the calculations with the same indices you will obtain always the same results at every statistical level. ANN results are instead unsystematic and non-reproducible as the weights of the ANN computations start from random values and the minimization procedure usually ends up with different values from run to run. This fuzzy character has nevertheless a positive side as if ANN computations are run over and over again the probability to end up with a quite good result is increasing. ANN results obtained with one hidden neuron either with the full set of descriptors (Table 5), or with a reduced set of descriptors, like in Table 8, if on one side they confirm the validity of the MLS calculations on the other side they leave open the possibility that somewhere there are ANN calculations that improve over them.
Anyway, (i) before throwing away a bad model for the training plus test compounds with ANN computations think it twice because they could hide a very good model for the evaluated compounds, and (ii) do not throw away the hydrogen atoms in calculations with MCIs or MMCIs as in many cases they are of good help.