Our improvement proposal for FCM consists of two parts: a new convergence indicator and an improved selection of the stop thresholds for the algorithm through the Pareto principle.
3.2. Method to Determine Threshold Values
It is known that the Pareto principle [
42,
43,
44,
45] allows for determining the optimal relation between effort and benefit when solving a dataset [
46,
47]. In this case, the effort is measured by the number of iterations or computer time, and the benefit is the value of the objective function when the algorithm stops. To apply the Pareto principle, it is necessary to solve at least one dataset with the same characteristics as those of the dataset that it is intended to be solved. Once we have the intermediate results of each iteration, the final value of the objective function and the number of iterations, a series of steps are applied to obtain the optimal relation between the number of iterations and the solution quality. In particular, the selection of the optimal relationship is guided by the minimum distance between the current solution and the ideal solution (0 iterations, 100% efficiency). From the determination of the optimal relation, the threshold values are established. A detailed description on the use of the Pareto principle can be found in [
46].
In order to illustrate the process we followed to calculate thresholds, the rest of this section shows the results from applying the Pareto principle to the results obtained from solving the real Abalone dataset, using FCM and 0.01 as the threshold. It is important to remark that we chose the value for ε = 0.01 because it is the least restrictive of the values proposed in
Table 1 and
Table 2. More restrictive threshold values cause the algorithm to generate more iterations.
Table 3 shows the relation that exists between computational effort and solution quality effort and solution quality. Column one indicates the iteration count; column two shows the number of iterations carried out
At (expressed as a percentage of the total number of iterations), where
At = 100 ∗ (
t/
T); column three presents the reduction of the objective function value in two consecutive iterations,
Bt, expressed as a percentage of the decrease of the objective function from the 1st iteration to the final one, where
Bt = 100 ∗ (
Jm(t−1) −
Jm(t))/(
Jm1 −
JmT); column four shows the value of the Euclidean distance
Dt between points (
At,
Ct) and (0, 100), where
; column five presents the cumulative value of the reduction percentages of the objective function values, where
Ct =
Ct−1 +
Bt; and column six shows the value of the convergence indicator Δ
newJm for each iteration.
Figure 1 shows the Pareto diagram for points (
At,
Ct); see columns two and five of
Table 3. The
A-axis indicates the percentages of the total number of iterations, and the
C-axis indicates the cumulative value of the reduction percentages of the objective function. Therefore, point G with coordinates (4.79, 96.92) means that, with 4.79% of iterations, 96.92% of the cumulative value of reductions of the objective function was attained. In case the algorithm is stopped at this point, there will only be a quality decrease of 3.08% of the objective function.
Notice that point G is the one with the smallest distance
Dt to point (0, 100), located in the upper left corner of the diagram. According to the Pareto principle, the aforesaid point is the one with an optimal relation between effort and benefit. As can be seen in row two of
Table 3, the coordinates of point G correspond to iteration 14 at the intersection with columns two and five. To determine the optimal value of the threshold, it is necessary to look at the crossing of row two and column six of the said table, and it is found that the value of Δ
newJm(t) is 2.71%. This means that, when the difference of the objective function in two consecutive iterations is less than 2.71% of the value of the objective function in the preceding iteration, the algorithm stops. It is worth mentioning that, if the algorithm stopped at this iteration, 95.21% of the iterations could be avoided, and 96.92% of the cumulative value of the reduction percentages of the objective function would be obtained (see the intersection of row two and column five). The quality decrease of the objective function would only be 3.08%.
Notice that points F, G, H, I, and J have cumulative values of the reduction percentages of the objective function equal to 96.25, 96.92, 97.48, 98.03, and 99.01, respectively, and they correspond to iterations 13, 14, 15, 17, and 29, respectively. As can be observed in
Figure 1, the relation between computational effort and solution quality is not linear. See also that, as one aims to increase the solution quality, the number of iterations required grows in a larger proportion.
Table 4 shows some intermediate results when solving the Abalone dataset using FCM in order to compare values of the solution quality and the computational effort required. The values in each row correspond to the iteration indicated in column one. Column one indicates the iteration number, column two shows the percentage of the total number of iterations that would be reduced if the algorithm stops in the current iteration, column three presents the cumulative value of the reduction percentages of the objective function up to this iteration, column four shows the percentage of quality decrease of the objective function in case the algorithm stops in this iteration, column five presents the value of the objective function calculated in the iteration, column six shows the value of the proposed convergence indicator for the iteration, and column seven presents the value of the traditional convergence indicator of FCM.
The rows of
Table 4 show the values for some iterations of interest. Notice that the values in column three are approximately 96, 97, 98, 99, and 100 percent and correspond to iterations 14, 15, 17, 29, and 292, respectively. As can be observed, for improving quality from 96 to 97 percent, only 1 iteration was needed; however, for improving from 99 to 100 percent, 263 iterations were required. The last case shows a possible inefficiency of the algorithm because a minimal improvement was obtained at the cost of a high computational cost. In contrast, according to the Pareto principle, the optimal relation between effort and benefit occurs in iteration 14. Therefore, the value 2.71 is suggested as the stop threshold using our proposed indicator (column six) instead of the value 0.44 when using the traditional convergence indicator of FCM (column seven). It is remarkable that the values of our indicator are more intuitive because they are defined as a percentage of the objective function value, which does not happen with the traditional indicator of FCM.
It is predictable that each application requires different degrees of solution quality and, therefore, different convergence thresholds. Thus, for example, if some user needs a solution quality of 97.5%, a value of 2.34% for the stop threshold is suggested for our indicator. With this value, there would be reductions of 94.86% in the number of iterations and only 2.52% in the solution quality.
Algorithm 2, shown next, integrates our proposed improvement to the FCM algorithm, which we call Pareto Fuzzy C-Means (P-FCM). In particular, we want to call attention to line 9, where the new convergence indicator is integrated.
Algorithm 2: P-FCM |
Input: Dataset X, V, c, m, ε |
Output: V, U |
1 | Initialization: |
2 | t: = 0; |
3 | U(t): = {u11, …, uij} is randomly generated; |
4 | Calculate centroids: |
5 | Calculate the centroids using Equation (6); |
6 | Calculate membership matrix: |
7 | Update and calculate the membership matrix using Equation (7); |
8 | Convergence: |
9 | If ΔnewJm(t) ≤ ε: |
10 | Stop the algorithm; |
11 | Otherwise: |
12 | U(t): = U(t + 1) and t: = t + 1; |
13 | Go to Classification |
14 | End of algorithm |
As a summary, this section shows the steps of the proposed method to obtain the thresholds for the FCM algorithm:
Figure 2 shows a diagram of the method to obtain the threshold.
Step 1: Solve a dataset with parameter values similar to those of the types of datasets that will be solved.
Step 2: Apply the Pareto principle to the solution results to determine the effort–quality relationship. It is necessary to store the intermediate and final data of the algorithm execution.
Step 3: Determine threshold values for dataset type. If necessary, select those thresholds that meet the required solution quality.
Step 4: Incorporate the convergence criterion in the FCM algorithm and solve the datasets of the selected dataset type.