A New Belief Entropy to Measure Uncertainty of Basic Probability Assignments Based on Belief Function and Plausibility Function

How to measure the uncertainty of the basic probability assignment (BPA) function is an open issue in Dempster–Shafer (D–S) theory. The main work of this paper is to propose a new belief entropy, which is mainly used to measure the uncertainty of BPA. The proposed belief entropy is based on Deng entropy and probability interval consisting of lower and upper probabilities. In addition, under certain conditions, it can be transformed into Shannon entropy. Numerical examples are used to illustrate the efficiency of the new belief entropy in measurement uncertainty.

The D-S theory is used to combine belief functions [2,19,20]. However, in D-S theory, there is an open issue on how to measure the uncertainty of belief functions [2,5,21,22]. Uncertainty plays a significant role in some fields since it is the foundation and prerequisite to quantitatively study the questions [3,[23][24][25]. Shannon entropy has basically resolved the uncertainty of probability theory [26], which is widely used in many application systems [27][28][29]. Inspired by ideas, many scientists are devoted to studying uncertainty of belief function [30]. So far, there are some methods of uncertainties in belief function [31]. We classify these methods according to additivity [32,33]. Deng entropy [34] and Tsallis [35] entropy do not satisfy the additivity, which are non-extended entropy. In addition, Yager's specificity measure [31], Hartley entropy [36], Korner's specificity definition [37], Höhle confusion measure [38], discord measure [39] and conflict measure [40] satisfy additivity. Generally speaking, the measures can reduce to Shannon's entropy under certain conditions. However, in recent studies, there is an important discovery that belief function theory is not a successful generalization of probability theory [3,41]. The basic probability assignment (BPA) function is transformed into probability distribution through conversion, which results in the loss of information. Hence, it is unreasonable that uncertainty of belief functions was calculated by the evolution of Shannon entropy. Therefore, it is very desirable to define a new way of measuring uncertainty to avoid the loss of information. Based on that, many people have made some attempts in the field, Deng [34] has presented Deng entropy to simplify the calculation of uncertainty of BPAs by considering total non-specificity and discord simultaneously without the conversion from BPA to probability. Recently, the probability interval in BPA has aroused wide attention because it is also a key factor for uncertainty. Yang and Han [41] have defined a distance-based total uncertainty measure for BPA based on probability interval. Deng et al. [42] have improved this measure to avoid counter-intuitive results caused by it. They overcome some shortcomings of traditional measurement; however, the uncertainty of those methods is inconsistent with Shannon entropy when BPA is degenerated to probability distribution.
In this paper, we analyze the uncertainty of BPA based on intervals which contain more information than probability. We propose new belief entropy by combining probability interval and Deng Entropy's idea, which can degenerate Shannon entropy when there is probability distribution. Thus, our proposed method can effectively measure uncertainty in BPA and probability distribution. Since there is no switch between BPA and probability distribution, it can overcome these limitations in traditional measures. Thus, it is feasible to define an uncertainty measure for a BPA based on probability interval.
The paper is organized as follows. Basics of D-S evidence theory for BPA are briefly introduced in Section 2. Section 3 presents and existing uncertainty measures and new belief entropy of BPA. Some important examples are described in Section 4 in order to illustrate the efficiency of the new belief entropy. Finally, this paper is concluded in Section 5.

Preliminaries
In this section, some preliminaries are briefly introduced.

D-S Evidence Theory
Some basic definitions of D-S theory are briefly introduced [2,3]: A set of hypotheses Θ is the exhaustive hypotheses of variable θ [43]. The elements are mutually exclusive in Θ [44]. Then, Θ is called the frame of discernment, defined as follows [2,3]: The power set of Θ is denoted by 2 Θ [45], and where ∅ is an empty set [46]. A BPA function m is a mapping of 2 Θ to a probability interval [0, 1], formally defined by [2,3]: which satisfies the following conditions [47]: The mass m(A) represents how strongly the evidence supports A.
The belief function (Bel) is a mapping from set 2 θ to [0, 1] and satisfied: The Pl indicates the degree to which is not suspected. As can be seen from the above, ∀A ⊆ For the same evidence, the different BPAs come from the different evidence resources. The Dempster's combination rule can be used to obtain the combined evidence [2,48]: where K = ∑ B C=∅ m 1 (B)m 2 (C). It is remarkable that, if K > 1, the Dempster's rules can not apply to two BPAs.

Existing Uncertainty Measures for Belief Structures
There are many methods to handle uncertainty [49]. In 1948, Shannon pointed out: "Information is used to eliminate random uncertainty" and proposed the concept of "information entropy" (using the concept of entropy in thermodynamics) to solve the problem of information measurement [50]. The concept of entropy is derived from physics [50,51]; it has been a measure of uncertainty and disorder [52]. A system with higher uncertainty has greater entropy, which also contains more information [11].
The Shannon entropy H is derived as [26,53]: where N is the number of basic states in a system, and p i is the probability of state i appears satisfying Shannon entropy plays a key role in handling a basic probability problem, and there are some limitations of Shannon entropy [42]. The concept of entropy in the framework of D-S theory is an open issue. Many researchers have extended many measured functions based on it, such as: Dubois and Prade. Dubois and Prade weighted Hartley entropy of BPA was shown [54]: Höhle. One of the earlier confusion measures for D-S theory was due to Höhle [38]: Yager. Dissonance measure of BPA was defined by Yager, as follows [31]: Klir and Ramer. Another discord measure of BPA was defined by Klir and Ramer, as follows [39]: Klir and Parviz. Klir and Parviz defined entropy [40]: George and Pal. George and Pal suggested a definition of conflict measure [55]: It can clearly be seen that these methods are all based on the Shannon entropy. There are also some documents that give a detailed introduction to these functions [49,56,57], and these entropies have their own basic properties, such as consistency with D-S theory semantics, non-negativity, probability consistency, etc. and later Deng proposed the concept of Deng Entropy [34], which is a new function of measuring uncertainty. The Deng entropy is described as follows [34]: where |A| is the cardinality of A. As the above, Deng Entropy is very similar to Shannon Entropy, but Deng Entropy uses 2 |A| − 1 to deal with the BPA of multifocal elements, which is more advantageous than Shannon Entropy. In addition, additivity and boundary are expanded.

The New Belief Entropy
In D-S theory, the probability interval [Bel(A), Pl(A)] can be obtained more information based on the basic probability assigned to each focal element. In this article, we use the probability interval to extend new methods of measuring uncertainty, as follows: As mentioned, this probability interval whose lower and upper bounds are the Bel and the Pl, respectively [58,59]. For a probability distribution, there are some advantages, such as discord and non-specificity [60]. Moreover, central values of probability interval can be used to compare uncertainty. At length, we all know that cardinality of every BPA is very important for the measurement of uncertainty. Hence, the new belief entropy which considers Deng entropy and the interval probability can better measure the uncertainty of BPA. In addition, according to the the literature of Kirl and Lewis [32], Kirl [33], the basic properties of the new belief entropy are explored as follows: (P1) consistency with DS theory semantics: The new entropy is consistent with D-S theory semantics. Thus, it satisfies the consistency with D-S theory semantics property.
(P2) non-negativity: We know that 0 < {Ble(x)+Pl(x)} (P4) subadditivity: To check that new entropy does not verify the subadditivity property, we consider the following example: Let X × Y be the product space of the sets X = {x 1 , x 2 , x 3 } and Y = {y 1 , y 2 }. We have that the marginal BPAs on X × Y with masses m ({z 11 , z 12 , We have that the marginal BPAs on X × Y are the following ones: m 1 and m 2 , respectively Thus: Obviously,H bel (m) > H bel (m 1 ) + H bel (m 2 ), and the subadditivity property is not satisfied.
(P5) additivity properties: The new entropy is also non-additive. It is easy to check, in general, that 2 mn − 1 = (2 m − 1) × (2 n − 1). We can use the following counter example to prove it in a more direct way: Using the symbol of the previous example. Let X × Y be the product space of the sets X = {x 1 , x 2 , x 3 } and Y = {y 1 , y 2 }. We have that the marginal BPAs on X × Y are the following ones: m 1 and m 2 , respectively: Again, H bel m > H bel (m 1 ) + H bel (m 2 ), and the additivity property is not satisfied by the new belief entropy. Therefore, the new entropy satisfies the consistency with D-S theory semantics, non-negativity, probability, and does not satisfy additivity properties, sub-additives. Therefore, the basic properties of some current entropies are given in Table 1.
In addition, BPA reflects more information than probability distribution in D-S theory. There is a classic example as follows: Assume in a test that there are 32 students participating in a course examination. The teacher has scores of these students. A teacher is only allowed to answer "Yes" or "No" to any questions, in order to know who is (are) the top student who gets (get) the highest score(s). How many times do we need to ask at most? Assume that the time is t, and it is easy to answer the problem through calculating the information volume by using information entropy t = log 2 32 = 5 However, when we have been told that there are two students tied for first. The entropy is still 5? In this case, how many times do we need to ask at most to know who are the first ONES? In this case, obviously t ≥ 5.
It can be seen from this example that the uncertainty of BPA is greater than the probability distribution. Thus, the uncertain measure boundary of probability distribution should be extended.
On the other hand, it can be found from recent research that the application of Tsallis entropy as non-additive entropy is more and more extensive [61]. The additivity entropy is a special case of the non-additivity entropy. As a result, the two requirements above, namely boundary and additivity, should be improved.

Numerical Experimental
In this section, some numerical examples are used to illustrate the application of our approach.

Example 1
Assume that the frame of discernment is Θ = {A} and we are given a BPA from a sensor as m({A}) = 1. Thus, we can calculate the Bel and Pl by Equations (5) and (6): Moreover, their classical Shannon entropy and the new belief entropy was calculated as follows: From above, we can conclude that the new belief entropy will retrograde the Shannon entropy if the frame of discernment has a single element. Under these circumstances, there is no uncertainty:

Example 2
Given that the frame of discernment is Θ = {θ 1 , θ 2 , θ 3 , θ 4 }, for a mass function m(θ 1 ) = m(θ 2 ) = m(θ 3 ) = m(θ 4 ) = 1 4 , then: Obviously, the Shannon entropy and the new belief Entropy are the same when dealing with a mass function of a single element. It further demonstrates the feasibility of the new belief entropy.

Example 6
Given a frame of discernment Θ = {θ 1 , θ 2 , · · · , θ N }, there are three special cases of mass function as follows: Their associated new belief entropy accompanied by the change of N of m 1 , m 2 , m 3 was shown in Figure 1. It can be seen from Figure 1 that, with the increase of N, the mass function m 1 has the maximum uncertainty which grows very fast, while the Bayesian function m 3 has the minimal uncertainty. By comparison, we know that the m 1 represents more information than m 2 , m 3 .

Example 7
Given a frame with 15 elements identifying A, the elements are from 1 to 15, and the basic mass function is as follows:  Table 2 reflects the trend of the new belief entropy when A changes, which can be seen from Figure 2. The calculation results show that, as the elements in A continue to increase, the uncertainty of BPA also increases. It is rational that there is more uncertainty with more elements.   [31], Klir and Ramer's discord [39], Klir and Parviz's strife [40], and George and Pal's conflict measure [55]. The experimental results are shown in Figure 3. It is obvious that only the new belief entropy and Dubois and Prade's weighted Hartley entropy increase constantly with the rise of the size of A. On the contrary, it can be seen from the insert in Figure 3 that the uncertainty obtained by other methods are reducing or changing irregularly when the A increases, which is obviously unreasonable. Therefore, the uncertainty of the new entropy in BPA Measurements are effective. Moreover, there are some differences between the new belief entropy and Dubois and Prade's weighted Hartley entropy, and Dubois and Prade's weighted Hartley entropy is not degenerate into Shannon entropy when the mass function is defined as a probability distribution. Therefore, the new belief entropy is a reasonable measure among these given uncertainty measures, which combine probability interval and cardinality of multiple elements of the BPA, and it is also more flexible.

Conclusions
Shannon entropy can effectively measure uncertainty of probability distribution. For the BPA, although many methods have appeared to measure the uncertainty, there is an open issue. The main work of this paper is to propose a new belief entropy without the conversion from BPA to probability based on probability interval and cardinality of multiple elements of BPA. The new belief entropy would have more uncertainty than other entropies, and the boundary and additivity have been improved. The new belief entropy is a generalization of the Shannon entropy, which can degenerate into the Shannon entropy when the BPA is a probability distribution. Moreover, some numerical examples are used to show the efficiency of the proposed new belief entropy.