FRCT: fuzzy-rough classification trees

Abstract

Using fuzzy-rough hybrids, we have proposed a measure to quantify the functional dependency of decision attribute(s) on condition attribute(s) within fuzzy data. We have shown that the proposed measure of dependency degree is a generalization of the measure proposed by Pawlak for crisp data. In this paper, this new measure of dependency degree has been encapsulated into the decision tree generation mechanism to produce fuzzy-rough classification trees (FRCT); efficient, top-down, multi-class decision tree structures geared to solving classification problems from feature-based learning examples. The developed FRCT generation algorithm has been applied to 16 real-world benchmark datasets. It is experimentally compared with the five fuzzy decision tree generation algorithms reported so far, and the rough decomposition tree algorithm. Comparison has been made in terms of number of rules, average training time, and classification accuracy. Experimental results show that the proposed algorithm to generate FRCT outperforms existing fuzzy decision tree generation techniques and rough decomposition tree induction algorithm.

Appendix

Consider a dataset [4] given in Table 7. Each pattern is described by four attributes and classified to either Don’t play or Play. Two attributes Temp and Humidity are real-valued and other two Outlook and Windy? are categorical. In the context of rough set theory, we may call categorical attributes as discrete attributes.

Table 7 Example dataset

Rough set theory requires a priori discretization of real-valued attributes. Real-valued attributes Temp and Humidity have been discretized using partition intervals given in Table 8, producing the dataset seen in Table 9. These intervals are crisp. For example, any value of Temp and Humidity lying in the intervals [68–81] and [74–87.5] respectively, will be treated as Med only, without considering their degree of belongingness to that interval.

Table 8 Discretization intervals

Table 9 Discretized dataset

For given dataset:

$$ U = {\left\{{1,2, \ldots, 14} \right\}}{{;}}y = {\left\{{Don\text{'}t\, play,Play} \right\}};n = 14. $$

Let x ₁ = Outlook, x ₂ = Temp, x ₃ = Humidity, and x ₄ = Windy?.

A set of patterns i ∈ U, for which attribute values are similar, partitions U into a set of equivalence classes. Partition of U by attribute x ₁ is

$$ {\left\{{{\left[ {Rain} \right]},{\left[ {Overcast} \right]},{\left[ {Sunny} \right]}} \right\}} = {\left\{\begin{aligned} &{\left\{{1,2,5,11,12} \right\}}, \\ &{\left\{{3,4,6,10} \right\}}, \\ &{\left\{{7,8,9,13,14} \right\}} \\ \end{aligned} \right\}}. $$

Partitions of U by class labels are:

$$ {\left[ {Don\text{'}t\, play} \right]} = {\left\{{1,2,8,9,12,13} \right\}}; [Play] = {\left\{{3,4,5,6,7,10,11,14} \right\}}. $$

The concept given by Pawlak is to approximate each class by a pair of exact sets, called the ‘lower’ and ‘upper’ approximations. Lower approximation is the set of patterns, which certainly are classified to a given class, while upper approximation is the set of patterns, which can be possibly classified to a given class.

Formally, if each attribute x _j takes a value from finite set of categorical values F _jk (1 ≤ k ≤ c _j ), classes ${\left[ {F_{jk}} \right]} = {\left\{{i \in U|x^{i}_{j} = F_{jk}} \right\}}; 1 \leq k \leq c_{j},$ are called the equivalence classes of U with respect to jth attribute x _j . By the same way, [ l ] = {i ∈ U|y ⁱ = l} is an equivalence class through lth classification label. Equivalence classes can also be generated by considering more than one attributes at a time. Given arbitrary class l and equivalence classes generated by attribute x _j , rough set is a tuple ${\left\langle {\underline{l}, \overline{l}} \right\rangle},$ where lower and upper approximations $\underline{l}$ and $\overline{l}$ respectively, are defined by Pawlak [21] as

$$ \begin{aligned} &\underline{l} = {\left\{{i \in U|{\left[ {F_{jk}} \right]} \subseteq {\left[ l \right]};k = 1, \ldots, c_{j}} \right\}} \\ &\overline{l} = {\left\{{i \in U|{\left[ {F_{jk}} \right]} \cap {\left[ l \right]} \ne \emptyset; k = 1, \ldots, c_{j}} \right\}} \end{aligned} $$

(10)

Lower and upper approximations for Don’t play and Play, through equivalence classes by x ₁ can be calculated by using Eq. (10).

$$ \begin{aligned} &\underline{{Don\text{'}t\, play}} = \emptyset; \underline{{Play}} = {\left\{{3,4,6,10} \right\}};\\ &\overline{{Don\text{'}t\, play}} = {\left\{{1,2,5,7,8,9,11,12,13,14} \right\}}; \overline{{Play}} = {\left\{{1,2,3,4,5,6,7,8,9,10,11,12,13,14} \right\}}. \end{aligned} $$

The positive region is the set of patterns which can be classified with certainty to a unique class; i.e.,

$$ \text{POS}_{{x_{j}}} {\left(y \right)} = {\bigcup\limits_{l = 1}^q {\underline{l}}} $$

(11)

For the considered example,

$$ \text{POS}_{{x_{1}}} {\left(y \right)} = \underline{{Don\text{'}t\, play}} \cup \underline{{Play}} = {\left\{{3,4,6,10} \right\}}. $$

It is clear that through attribute x ₁, patterns {1,2,5,7,8,9,10,11,12,13,14} can not be classified to unique decision class with certainty. For example, patterns 1 and 5 are classified in different decisions, even though they have same value Rain of attribute x ₁. But there is no classification ambiguity in patterns belonging to $\text{POS}_{{x_{1}}} {\left(y \right)}.$ If $\text{POS}_{{x_{j}}} {\left(y \right)} = U,$ then each pattern i ∈ U has been certainly classified to a unique class without any ambiguity. By this, Pawlak [21] defined the dependency degree $\gamma_{{x_{j}}} {\left(y \right)}$ of decision attribute y on condition attribute x _j :

$$ \gamma_{{x_{j}}} {\left(y \right)} = \frac{{{\left\| {\text{POS}_{{x_{j}}} {\left(y \right)}} \right\|}}}{n}, $$

(12)

where ${\left\| {\text{POS}_{{x_{j}}} {\left(y \right)}} \right\|}$ is the number of patterns belonging to $\text{POS}_{{x_{j}}} {\left(y \right)}.$

For considered example, The dependency degree can be calculated by Eq. (12), which is $\gamma_{{x_{1}}} {\left(y \right)} = \frac{{{\left\| {\text{POS}_{{x_{1}}} {\left(y \right)}} \right\|}}}{{14}} = 0.2857.$ By the same way we can obtain $\gamma_{{x_{2}}} (y) = 0,\gamma_{{x_{3}}} (y) = 0,{\hbox{and}}\; \gamma_{{x_{4}}} (y) = 0.$