Ontology learning algorithm using weak functions

2.1 Multi-dividing ontology setting

Since most ontology graph structures are trees (no cycle graph), multi-dividing ontology learning trick is popular in recent years. In this special ontology learning setting, all the vertices in the ontology graph are divided into k parts for k levels. The values of different levels are determined by domain experts in the specific application. For ontology function f , what we want is that the real number corresponding to the vertex in level a is greater than the real number corresponding to the vertex in any level b, where 1 ≤ a < b ≤ k. That is to say, in the ideal case, f (v) > f (v^′) if the level of vertex v is smaller than the level of vertex v^′.

Formally, the learner is inferred to an ontology training set S = (S¹, S², · · · , S^k) ∈ Vⁿ¹ × Vⁿ² ×· ··× V^nk which consists of a sequence of ontology training samples S^a = (v1a,⋯,vnaa)∈Vna(1≤a≤k).By virtue of ontology sample S, a real-valued ontology function f : V → R is learned which allocates the future S ^a vertices larger value than S^b, where a < b. Set D_a as the conditional distributions for each rate 1 ≤ a ≤ k and n=∑i=1knias the total size of ontology sample set, where n_i = |S_i| for i ∈ {1, · · · , k}.

The expected multi-dividing ontology expected risk on the ontology graph for an ontology function f : V → R associated with the ontology function f : V → R is defined as

The other expression for expected ontology risk can be formulated by

A large class of learning algorithms is generated by regularization schemes. They penalize an empirical error which is chosen here to be the multi-dividing empirical error on the ontology graph defined for a f : V → R associated with the sample S as

Thus, the optimal ontology function can be obtained by f∗=argminer^T,l(f).We simply write er^S,l(f)aser^(f).

The aim of this paper is to provide a new ontology learning algorithm in terms of weak ontology functions and give the theoretical analysis of in the multi-dividing setting.

3.1 New ontology learning algorithm with weak ontology functions

Assume that the whole procedure can be separated by weak ontology functions in which it will be produced in each round. The new ontology learning algorithm keeps a distribution D_t over V×V which is passed on round t to the weak ontology function. In fact, it selects D_t to emphasize different parts of the ontology training samples, and the large weight assigned to a pair of ontology vertices implies that it is very important for the weak ontology functions to map their order correctly.

We assume that the weak ontology functions keep the form f_t : V → R, and it provides order information in the same fashion as final ontology function. In the normal ontology setting (S = {v₁, · · · , v_n} and not to be divided into k parts), the procedure can be stated as follows:

Step 1: Given initial distribution D over V × V and set D₁ = D;

Step 2: For t = 1,··· , T, do the following actions: train weak ontology function by means of distribution D_t; obtain weak ontology function f_t : V → R; select parameter α_t ∈ R; calculate

where Z _t is denoted as regularization parameter and thus D_t+1 will be determined;

Step 3: Return the final ontology function as the combination of weak ontology functions:

In the step 2 above, one problem is how to get parameter α_t. One method is to minimize Z _t, i.e.,

In the multi-dividing ontology setting, the above algorithm can be rewritten as follows.

Initialize: For each pair of (a, b) with 1 ≤ a < b ≤ k, and v∈Sa∪Sb,setρ1a,b(v)=1naifv∈Saandρa,b(v)=1nbif v ∈ S^b.

For t = 1, · · · , T:

• train the weak ontology function in terms of D_t(if v₁∈ S^a and v₂∈ S^b,then Dt(v1,v2)=ρta,b(v1)ρta,b(v2)) and obtain weak ontology function f_t : V → R;

• for each pair of (a, b) with 1 ≤ a < b ≤ k, select αta,b∈Rand update

if v ∈ S^a and

if v ∈ S^b.

Select balance parameter α_t for each weak ontology function, and return the final ontology function f(v)=∑t=1Tαtft(v).

3.2 Theoretical results

To ensure that the calculations for each iteration are valid, it must be confirmed that Dt(v1,v2)=ρta,b(v1)ρta,b(v2)is true at each step (here v₁∈ S^a and v₂∈ S^b). To show this, we use mathematical induction, assume that this is true for round t, and when it comes to round t + 1, according to the produces, we have

Note that our final ontology function has the form f(v)=∑t=1Tαtft(v),and we can set Θ : V × V → {−1, 0, 1} as

That is to say, if ∑t=1Tαtft(v1)−∑t=1Tαtft(v2)>0then Θ(v₁, v₂) = 1; if ∑t=1Tαtft(v1)=∑t=1Tαtft(v2),then Θ(v₁, v₂) = 0; and if∑t=1Tαtft(v1)−∑t=1Tαtft(v2)<0then Θ(v₁, v₂) = −1. For each pair of (a, b) with 1 ≤ a < b ≤ k, if function Θ(v^a, v^b) ≠1 where v^a ∈ S^a and v^b ∈ S^b, then it implies the error by the multi-dividing rule. Thus, the generalization error (expect risk) of Θ in multi-dividing ontology setting is denoted as

where I is the truth function, i.e., I(x) = 1 if x is true, otherwise I(x) = 0. Given the ontology training set S = (S¹, S², · · · , S^k) ∈ Vⁿ¹ × Vⁿ² × ··· × V^nk which consists of a sequence of ontology training samples Sa=(v1a,⋯,vnaa)∈Vna(1≤a≤k),the expected empirical error of Θ can be denoted as

The results presented in our paper aim to show that the difference betweenΔ^(Θ)andΔ(Θ)is small with large possibility. Setting Γ as the function space for functions Θ we have the following theorem.

Theorem 1 Suppose all the weak ontology functions belong to function space F^′ with a finite VC dimension K, the ontology functions f (as the weighted combinations of the weak ontology functions) belong to function space F. Let S = (S¹, S², · · · , S^k) ∈ Vⁿ¹ × Vⁿ² × ·· ·× V^nk be ontology training set which consists of a sequence of ontology training samples Sa=(v1a,⋯,vnaa)∈Vnaand S^a ∼ D_a (1 ≤ a ≤ k). We have with probability at least 1 − δ (0 < δ < 1), the following inequality holds for any f ∈ F:

where K^′ = 2(K + 1)(T + 1) log₂(e(T + 1)), T is the number of weak ontology functions in ontology algorithm and e is the base of the natural logarithm.

Proof of Theorem 1 First, we show that for each pair of (a, b) with 1 ≤ a < b ≤ k, and each δ > 0, there is a small number ε satisfying

where the value of ε will be determined later.

Define Ξ:V×V→{0,1}asΞ(va,vb)=I(Θ(va,vb)≠1). Clearly, Ξindicates whether Θ makes mistake or not for the ontology vertices pair (v^a, v^b) for v^a ∈ S^a and v^b ∈ S^b according to the multi-dividing rule. We infer

Obviously, it is enough to show that there exist ε₁ and ε₂ with ε₁ + ε₂ = ε such that (∃v^a ∈ V ^a in each pair of (a, b) for (1) and ∃v^b ∈ V^b in each pair of (a, b) for (2))

and

respectively, where Υ is the function space of Ξ.

Now, we only prove (2) in light of standard results, and (1) can be yielded in the same fashion. Let Υ_vb be the set of all such functions Ξ for a given v^b, then the selection of Ξ in (2) is from function space ∪_vbΥ_vb . In view of theorem of Vapnik [32] which provides a selection of ε₂ in (2) relying on the size n_a of S^a for each pair of (a, b), complexity K^′ of ∪_vbΥ_vb (considered as VC Dimension), and the possibility δ. Specifically, for any δ > 0, set

we have

Next, we need to determine the VC Dimension of ∪_vbΥ_vb : K^′. For a given v^b ∈ V^b, we obtain

where c=∑t=1Tαtft(vb)is a constant since v^b is given. It reveals that the functions in the space ∪_vbΥ_vb are the subset of all potential thresholds of all the linear combination of T ontologyweak functions. Using the standard result on VC Dimension of weak functions, we yield that if the ontology weak function space has VC Dimension K bigger than two, then K^′ can’t exceed to 2(K + 1)(T + 1) log₂(e(T + 1)).

Therefore, we get the desired conclusion.

According to Theorem 1 above, the generalization bound converges to zero at a rate of O(∑a=1k−1∑b=a+1kmax{log⁡(na)na,log⁡(nb)nb}).

For each pair of (a, b) with 1 ≤ a < b ≤ k, the shatter coefficient is denoted as r^a,b(F, n_a, n_b) (see Gao and Wang [33] for more details). Then we deduce the following result. Theorem 2 Let F be the real valued ontology function space on V, then with probability at least 1 − δ (0 < δ < 1), for any f ∈ F, we have

Theorem 2 implies that if the ontology function is a linear function in the one-dimensional function space, then for each pair of (a, b) with 1 ≤ a < b ≤ k, r^a,b(F, n_a, n_b) are constants, regardless of the values of n_a and n_b, and thus the bound converges to zero at a rate of O(∑a=1k−1∑b=a+1kmax{1na,1nb}).It further reveals that the bound yield in Theorem 2 is sharper than bound obtained in Theorem 1. However, if the ontology function is a linear function in the d-dimensional function space (where d ≥ 2), then r^a,b(F, n_a, n_b) with order O((n_an_b)^d), and in this case the bound in Theorem 2 has convergence rate relying on VC dimension, i.e., still O(∑a=1k−1∑b=a+1kmax{log⁡(na)na,log⁡(nb)nb}).