Combining Bayesian and logical-probabilistic approaches for fuzzy inference systems implementation

In the paper, an original way to implement the fuzzy inference systems is considered. The proposed fuzzy inference model is based on usage of conjunction of probabilistic logic and Bayes’ formula at inference stage. For that preconditions of all the specified fuzzy production rules are transformed to the set of probabilistic functions. As the input probabilities, the values of membership functions of input linguistic variables terms are used. At that given set of production rules is reduced in the way that number of rules will be equal to number of terms of output linguistic variables. For that logic expressions of production preconditions are transformed to orthogonal DNFs. Calculated probability values are used as conditional probabilities in determining the posterior distributions on the set of hypotheses corresponding to values of the output linguistic variables. To determine the posterior probability distribution, we use the formula based on the Bayes’ formula. These posterior distributions are used at defuzzification stage for determining final output variable values, because they are calculated as expected values of corresponding random variables. The program implementation has allowed to estimate the proposed model effectiveness and to compare its results with results of traditionally used Mamdani and Sugeno fuzzy inference algorithms.


Introduction
In present days, researches in the field of fuzzy sets and fuzzy inferences, fuzzy models and fuzzy control systems are still relevant. The research area started by proceedings of L. A. Zadeh (see, for example, [1]) has begun to development by implementation of fuzzy controllers and its embedding to Japanese home appliances which now widespread. [2][3][4][5][6]. Developers of systems with artificial intelligence elements are continuously "retreating from precision in the face of overpowering complexity" [1, p. 201].
Authors of the papers had experience in using fuzzy logic inference and Bayesian approach for solving estimation tasks in conditions of incompleteness (inaccuracy) of information [7−9]. Both these approaches can use as source data for decision making the expert estimates that are inaccuracy (incompletely defined), this factor causes the methods similarity. Based on this assumption we consider new approach to fuzzy logic inference task. Main elements of the approach presented at XXII International conference on soft computing and measurements SCM'2019 (https://scm.etu.ru/2019/), then we have composed paper [10] containing joined and refined results.
Purpose of fuzzy logic inference, is determining values of output linguistic variables (LV) basing on information about crisp input variable values and taking into account set of specified fuzzy production rules (FPR).
We suggest that when the fuzzy inference is performed, there is possible [10]: 1. To consider set of values of each input LV as the source for set of Bayesian hypotheses about some output LV takes any value from its set of terms (term-set). 2. To interpret estimated "crisp" values at scales of input LVs, as evidence for all these Bayesian hypotheses. 3. To interpret values of membership function (MF) of fuzzy sets of corresponding terms of input LVs as arguments of some probability logic functions (PLF). 4. To transform set of FPRs to a set of PLFs, arguments of which are degrees of membership of the estimates-evidence to fuzzy sets, and calculated values are used as conditional probabilities (degrees of Bayesian hypotheses trueness, i.e. likelihood of them [11]) when determining the posterior distribution on the set of hypotheses corresponding to the values of the output LV. 5. To use obtained for each output LV the Bayesian probability distribution to determine defuzzified value of the variable as expected value of a random variable. As an argument for the possibility to use Bayesian logical-probabilistic approach in the fuzzy logic inference we take a remark by L. Zadeh himself about the analogy that can be drawn between the concepts of fuzziness and probability [1, p. 204]. In the example given by L. Zadeh, it is said about membership degree of a specific numerical value to a certain LV term. It is possible to mean this membership value as degree of correspondence of given evidence (estimated "crisp" value) to assumption about trueness of hypothesis that LV takes some value from its term-set, i.e. understand it as subjective or Bayesian probability, defined as degree of certainty that some assertion is true [12].
We should note that there is not only L. Zadeh had marked the similarity between fuzziness and probability: other researches also discussed their similarity (see, for example [13]); moreover, some of them also name fuzziness as "masked probability". Therefore, usage of the known apparatus of probabilistic logic in the Bayesian logical-probabilistic model of fuzzy inference seems understandable and justified.
In the paper, we present results were obtained by combining of Bayesian and logical-probabilistic approaches for implementation of the new fuzzy inference model.
This year, undergraduate student of the Department of Information and Computing Systems of our university G. А. Khamchichev, under the guidance of one of the authors, develops a software implementation of the model, which allowed us to compare our model with the traditionally used algorithms of Mamdani and Sugeno and confirmed its effectiveness.

Fuzzy logic inference stages
The fuzzy logic inference consists of the following stages [2,6] are calculated. 5. Proposed logical-probabilistic approach of fuzzy inference affects only the conclusions inference and defuzzification stages. We'll describe it without loss of generality for a case with single output LV Ỹ .

Preparatory definitions and transformations
Denote term-set of input LV i X as , and term-set of output LV Ỹ as . Each FPR t j R j , 1 , = , in a fuzzy inference system (FIS) can be denoted as follow: where In FIS, for each FPR j R is assigned a real number wj[0, 1], named rule weight. The weight is understood as measure of certainty that conclusion assertion in the FPR j R derived by precondition assertion.
If we denote k-th fuzzy logic assertion ( (1) Main proposed idea of the Bayesian logic-probability approach to fuzzy inference involves transition from BFs in preconditions of FPRs to PLFs, calculate PLF values and use them as conditional probabilities for determining posterior probability distribution on a set of Bayesian hypotheses corresponding to terms of output LV Ỹ .
A feature of the approach is the requirement that number of FPRs be equal to number of terms of output LV. So, if source FPR set contains several FPRs with the same conclusion part l T Y = , they replaced with one generalized FPR, containing assertion l T Y = as conclusion and containing at precondition part a BF constructed as disjunction of preconditions of generalized FPRs. After that, reduced FPR set will contain precisely N of FPRs N R R R,..., , 2 1 , where N is number of terms in ) (Y T , and, also, number of corresponding Bayesian hypotheses. When several FPRs replaced with one generalized FPR, weight w o of this FPR is determined as average value of weights of excluded FPRs. All the BFs that are FPR preconditions, in order to simplification transformation to PLFs, are preliminarily transformed to orthogonal DNF form (ODNF). This transformation is executed due to algorithm proposed in [14].
Formal rules of transformation BF described in ODNF form to the corresponding PLF the following: 1. Boolean variables n z z ,..., 1 are replaced with the corresponding probabilities n p p ,..., 1 .

Inversions
3. Conjunction and disjunction signs are replaced with arithmetic multiplication and addition correspondingly. In Table 1, examples of transformation elementary BFs to PLFs are presented. Note that probability logic operations presented in Table 1 are associative, commutative and monotonic, have segment [0, 1] as the codomain, moreover, they use either 1 or 0 as the neutral element. Therefore, they meet requirements for t-norm and t-conorm using for conjunction and disjunction operations implementation in the fuzzy logic [3].
So, the proposed model of fuzzy inference assumes that MF values of input LV terms are interpreted as subjective probabilities , presenting preconditions of these FPRs.

Conclusions inference stage
where N is number of hypotheses, m is number of evidence, and P(Hl) is prior probability of corresponding hypothesis Hl. In our case, number of hypotheses equals to number of FPRs (because, if necessary, FPR set is reduced, as it is described above), at that, for each hypothesis Hl there is only one evidence, i.e. m=1. Besides that, because prior probabilities are not used at fuzzy logic inference, let P(Hl) = 1/N. Also we must use FPR weight wl as the factor that decreases P(e | Hl) value as level of compliance of evidence e to assumption of trueness of Bayesian hypothesis Hl, N l , 1 = . Finally, we have the equation: Using equation (4), we make a posterior distribution of Bayesian probabilities on the set of hypotheses that the output LV Ỹ takes each value Tl from its term-set ) (Y T . This distribution we will use for output LV defuzzification.

Defuzzification stage
Purpose of the defuzzification stage is to obtain "crisp" value Y y   of final term TR is filled up in Figure 1. terms. Thus, defuzzified value * y must take into account MFs forms of output LV terms and given probability distribution.
In simple defuzzification case, it is enough to use boundary points of terms at scale of output LV, weigh them by given Bayesian probabilities At that, easy to spot that term-set   As an internal point of segment [Ti,min, Ti,max], there is possible to take the middle point: But equation (7) (7) and (8)

Defuzzification method requirements
It should be noted that defuzzification method used for fuzzy inference, must have such qualities as continuity, uniqueness and likelihood [4], at same time, this method must have low computational complexity because that is important for embedding to fuzzy regulators and microcontrollers.
Continuity assumes that low input values variation should perform low variation of defuzzified value * y , uniqueness means that for any input values combination the defuzzification method should return only single value * y , and likelihood means that the defuzzified value * y every time is "characterized value" of any output LV term. Finally, low computational complexity defines, how computationally intensive is process of finding defuzzified value * y basing on information about source terms of output LV and obtained final term. Note that with traditional fuzzy inference, defuzzified value can be obtained by five different ways, so uniqueness and likelihood of the result can be broken, therefore constructed FIS must be verified and maybe debugged [2][3][4]. Besides that, obtaining defuzzified value using centroid methods requires implementation of numerical integration methods, which often are computationally complex, especially if nonlinear MFs (Gaussian, sigmoid, etc.) there are used.
Proposed defuzzification method based on expected value E(y) calculation, meets continuity and likelihood conditions according to features of expected value: defuzzified value * y , depending on , shifts towards "characteristic value" of the most probable output LV term. Uniqueness of the method is not worse of traditional way, because it has less arbitrariness when "characterized values" of terms are chosen using equation (7) or (8). Moreover, the proposed approach does not require construction final term TR of output LV, this feature decreases computational complexity because in case using equation (7) to find "characterized values" of terms Ti, the obtained solution * y is weakly different from defuzzified value obtained by equation (8).

Problem "Dinner for two"
The well-known problem "Dinner for two" (included in MATLAB's guide [6]) is used as an illustrative example. The problem described by FIS containing FPRs, presented in Table 2. Weights of all these FPRs are equal to 1.00.

Fuzzy sets Membership degrees
Define PLFs according to three FPRs above: has "poor" value. Execute the fuzzy logic inference according to approach described above, and define tips amount for following cases: 1. Service is evaluated for 3, and food is evaluated for 8 scores.
2. Both service and food are evaluated for 5 scores. Inference results presented in Tables 5 and 6 correspondingly. To obtain "characterized values" of terms of output LV Tip, use the equation (7) because MFs of these terms are symmetric. Note that tip value in Table 5 almost same as the result of traditionally used fuzzy inference.

Problem "Road behavior"
The example above as it is very simple and demonstrative, did not require reduction of FPR set. Moreover, because MFs of output LV terms are symmetric, so it is possible to use segment middle points (7). At most common case, as noted above, can be required to use preliminary reduction of FPR set before conclusions inference and at defuzzification stage, weighed segment centers (8) should be used. Example of such FIS which emulates behavior of a car driver exceeds the car speed before road policeman, is descriped in details at [10]. Here we only compare result given by proposed approach with the traditionally used Mamdani algorithm results. Determined defuzzified output LV value is y * =29.57, and traditionally used fuzzy inference results (depending on specified way of defuzzification) are following for the same input data: Note that defuzzification results with the traditional fuzzy inference approach significantly depend on chosen defuzzification method, and the defuzzification of the Bayesian logical-probabilistic fuzzy inference result is closest to the defuzzification results given by center of gravity ( *

Software Implementation
In order to assess practical applicability of the proposed approach, as well as in order to assess complexity of its implementation, under the guidance of one of the authors, a software implementation of the above-considered Bayesian logical-probabilistic model of fuzzy inference was performed.
The program was implemented using the Java programming language. During the development process, class packages were implemented for the main elements of the proposed model. It is shown that due to the fact that the integration is used for simple term MFs, an acceptable accuracy of calculations is achieved using simple methods of numerical integration.
The developed program was verified by comparing the obtained results with the results of Mamdani and Sugeno algorithms implemented in MATLAB system. Comparison showed that software implementation of the proposed model gives results that are insignificantly differ from the results obtained using classical methods of fuzzy inference.

Conclusion
The article describes the Bayesian logical-probabilistic model of fuzzy inference, which differs from the traditionally used model by the content of the stages of fuzzy conclusions inference and defuzzification. It is based on the application of the apparatus of probabilistic logic and Bayes' formula. Here, set of FPRs is transformed into a set of PLFs, the arguments of which are the MF values of the input LVs terms, and the calculated values are used as conditional probabilities to determine the posterior Bayesian distribution on a set of hypotheses corresponding to the output LV terms. Resulting probability distribution is used to defuzzify the output LV value. Defuzzification is performed by calculating the expected value of a discrete random variable, the values of which are the average points of the intervals of the output LV terms, and the distribution law is the probability distribution obtained at the stage of inference of fuzzy conclusions on the set of hypotheses that the output variable will receive a term value from its term-set.
Effectiveness of the application of the proposed model is shown by examples and confirmed by software implementation, carried out under the guidance of one of the authors.