Inverse Result of Approximation for the Max-Product Neural Network Operators of the Kantorovich Type and Their Saturation Order

Abstract

In this paper, we consider the max-product neural network operators of the Kantorovich type based on certain linear combinations of sigmoidal and ReLU activation functions. In general, it is well-known that max-product type operators have applications in problems related to probability and fuzzy theory, involving both real and interval/set valued functions. In particular, here we face inverse approximation problems for the above family of sub-linear operators. We first establish their saturation order for a certain class of functions; i.e., we show that if a continuous and non-decreasing function f can be approximated by a rate of convergence higher than $1/n$, as n goes to $+\infty$, then f must be a constant. Furthermore, we prove a local inverse theorem of approximation; i.e., assuming that f can be approximated with a rate of convergence of $1/n$, then f turns out to be a Lipschitz continuous function.

1. Introduction

The introduction of the max-product version of families of linear approximation operators is due to Bede, Coroianu and Gal (see, e.g., [1,2]) and it led to a new branch of approximation theory. The new theory of max-product operators has been deeply studied, and recently the above-mentioned authors summarized their results in a complete monograph [3].

In general, the max-product version of a sequence/net of linear operators is a family of nonlinear (more precisely sub-linear) operators with better approximation properties of their original version: in many cases, the order of convergence is faster than their linear counterparts [4,5,6]. Further, the above operators can also be useful, for instance, in the applications of probability and fuzzy theory involving both real and interval/set valued functions (see, e.g., [7,8]).

In the present paper, we study the max-product form of the neural network (NN) operators of the Kantorovich type $K_n^{\left(M\right)}$, first introduced in [9] and here recalled in Definition 3 of Section 2. In general, the NN operators (see [10]) are strictly related to the theory of artificial neural networks, which has been introduced in order to provide a very simple model for the human brain, which is able to reproduce all its main abilities [11,12,13,14].

Each basic element composing a neural network is called an artificial neuron; its behavior is regulated by suitable activation functions, which must represent the two possible states of the biological neuron: the activation and the quiet phases [15]. From the mathematical point of view, functions which better represent the latter fact are those with sigmoidal shape (see, e.g., [16]).

For the above reasons, in the present paper, we mainly consider the operators $K_n^{\left(M\right)}$ activated by suitable sigmoidal functions. Very useful examples of sigmoidal functions (in view of their importance in learning algorithms, [17]) are, e.g., the logistic and the hyperbolic tangent functions [18].

However, independently of its biological meaning, in some recent papers, a new unbounded activation function has also been introduced and deeply investigated. This function is the so-called rectified linear unit (ReLU) function (see, e.g., [19]), and it is simply defined by the positive part of x, for every $x\in \mathbb{R}$. The ReLU activation is revealed to be very suitable for training deep (i.e., multi-layer) neural networks, in view of the very simple form that is assumed by its derivative (whenever it exists).

Here, we show that the above operators can also be based on a certain finite linear combination of ReLU activation function, and in this case, the approximation properties of $K_n^{\left(M\right)}$ are also preserved.

Problems of interpolation, or more in general, of approximation, are related to the topic of training a neural network by sample values belonging to a certain training set: this explains the interest in studying approximation results by means of NN operators in various contexts [15,20,21,22,23,24].

Indeed, as can be seen from the references therein, results in this sense have been studied deeply in terms of various aspects, such as the convergence and the order of approximation.

In this paper, we deal with the problem of the saturation order and of inverse results of approximation.

In general, the problem of establishing the saturation order for a family of operators $L_n$ (see [25,26,27,28]), $n\in \mathbb{N}$, consists in determining a class of functions $\mathcal{D}$, a certain subclass $\mathcal{E}$ of trivial functions of $\mathcal{D}$, and a positive non-increasing function $\phi \left(n\right)$, $n\in \mathbb{N}$, such that there exists $g\in \mathcal{D}\backslash \mathcal{E}$ with $\parallel L_{n}g-g\parallel =\mathcal{O}\left(\phi \left(n\right)\right)$, as $n\to +\infty$, and with the property that, for any $f\in \mathcal{D}$ with:

$$\parallel L_{n}f-f\parallel =o\left(\phi \left(n\right)\right),n\to +\infty ,$$

it turns out that $f\in \mathcal{E}$, and vice versa. Here, $\parallel ·\parallel$ denotes any suitable norm on $\mathcal{D}$. In this case, $\phi \left(n\right)$ is said to be the saturation order of the approximation process $L_n$, and it represents the best possible order of approximation that can be achieved on $\mathcal{D}$ by the above approximation operators.

In case of the max-product NN operators of the Kantorovich type, according to the studies given in [9,29], we expect that for a certain subclass $\mathcal{D}$ of $C\left(\right[0,1\left]\right)$ (endowed with the usual max-norm), with $\phi \left(n\right)=1/n$, $n\in \mathbb{N}$, then f is constant over $[0,1]$. Hence, we also have that the trivial class of functions $\mathcal{E}$ is given by constant functions. Indeed, one of the main results that we establish in the present paper is exactly the proof of the above claim.

Further, since in [9] it has been proved that, in the space $\mathrm{Lip}\left(\right[0,1\left]\right)$, the order of approximation is exactly $1/n$, as n goes to $+\infty$, it is natural to ask if also the converse implication holds.

In this paper, we proved exactly a local version of such an inverse approximation theorem, i.e., if the relation:

$${∥K_n^{\left(M\right)}\left(f,·\right)-f\left(·\right)∥}_{\infty }=O\left(1/n\right),n\to +\infty ,$$

olds, with f continuous and non-decreasing, then f belongs to $\mathrm{Lip}\left(\left[a,b\right]\right)$, for every sub-interval $[a,b]\subset [0,1]$.

2. Preliminaries

By $C\left(\left[0,1\right]\right)$, we will denote the space of continuous functions $f:\left[0,1\right]\to \mathbb{R}$, while by $C_{+}\left(\left[0,1\right]\right)$, we will indicate the subspace of $C\left(\left[0,1\right]\right)$ of the non-negative valued functions. Furthermore, we will denote by $\mathrm{Lip}\left(\left[0,1\right]\right)$ the subspace of $C\left(\left[0,1\right]\right)$ of the Lipschitz continuous functions on $\left[0,1\right]$, i.e., the space of functions f for which there exists a positive constant L such that

$$\left|f\left(x\right)-f\left(y\right)\right|\le L\left|x-y\right|,x,y\in [0,1].$$

Finally, we also denote by ${∥·∥}_{\infty }$ the classical max-norm. Obviously, all the above notations can be given by replacing the interval $[0,1]$ with any bounded or unbounded interval $I\subset \mathbb{R}$.

We now recall the definition of a sigmoidal function introduced by Cybenko [30].

Definition 1.

Let $\sigma :\mathbb{R}\to \mathbb{R}$ be a measurable function. We call σ a sigmoidal function if

$$\underset{x\to -\infty }{lim}\sigma \left(x\right)=0,\mathrm{and}\underset{x\to +\infty }{lim}\sigma \left(x\right)=1.$$

In what follows, we consider non-decreasing sigmoidal functions $\sigma$ that satisfy the following conditions:

• ($\mathrm{\Sigma }$1) $\sigma \left(x\right)-1/2$ is an odd function;
• ($\mathrm{\Sigma }$2) $\sigma \in C^2\left(\mathbb{R}\right)$ is concave for $x\ge 0$;
• ($\mathrm{\Sigma }$3) $\sigma \left(x\right)=O\left({\left|x\right|}^{-\alpha -1}\right)$ as $x\to -\infty$ for some $\alpha >0$.

Notice that assumptions $\left(\mathrm{\Sigma }i\right),i=1,2,3$ are satisfied by the main examples of sigmoidal functions known in the literature.

For instance, examples of sigmoidal activation functions are given by ${\sigma }_{\ell }\left(x\right)$: $={\left(1+e^{-x}\right)}^{-1},x\in \mathbb{R}$ (i.e., the so-called logistic function), and by ${\sigma }_h\left(x\right):=\left(tanh\left(x\right)+1\right)/2$, $x\in \mathbb{R}$ (i.e., the so-called hyperbolic tangent activation function). Note that both ${\sigma }_{\ell }\left(x\right)$ and ${\sigma }_h\left(x\right)$ satisfy $\left(\mathrm{\Sigma }3\right)$ for all $\alpha >0$ in view of their exponential decay at $x\to -\infty$.

Further, we can also recall the definition of the sigmoidal functions that can be generated by the well-known central B-splines of order $n\in {\mathbb{N}}^{+}$

$$M_n\left(x\right):=\frac{1}{\left(n-1\right)!}\sum _{i=0}^n{\left(-1\right)}^i\left({\displaystyle \genfrac{}{}{0pt}{}{n}{i}}\right){\left(\frac{n}{2}+x-i\right)}_{+}^{n-1},x\in \mathbb{R},$$

where the function ${\left(x\right)}_{+}:=max\left\{x,0\right\}$. Hence, we can define the sigmoidal function ${\sigma }_{M_n}$ generated by $M_n$ as follows:

$${\sigma }_{M_n}\left(x\right):={\int }_{-\infty }^x{M}_n\left(t\right)dt,x\in \mathbb{R}.$$

Obviously, ${\sigma }_{M_n}$ satisfies assumption $\left(\mathrm{\Sigma }1\right)$ for every $n\ge 1$. Further, since $M_n$ have compact supports, and the supports are contained in the intervals ${\displaystyle \left[-\frac{n}{2},\frac{n}{2}\right]}$, it turns out that ${\sigma }_{M_n}$ also satisfies $\left(\mathrm{\Sigma }3\right)$ for every $\alpha >0$. Further, assumption $\left(\mathrm{\Sigma }2\right)$ is satisfied for $n\ge 1$.

Now, considering (from now on) a function $\sigma$ that satisfies the above assumptions, we can recall the definition of the density (kernel) function ${\varphi }_{\sigma }$, that is:

$${\varphi }_{\sigma }\left(x\right):=\frac{1}{2}\left[\sigma \left(x+1\right)-\sigma \left(x-1\right)\right],x\in \mathbb{R}.$$

For the function ${\varphi }_{\sigma }$, the following lemma can be proved.

Lemma 1.

$\left(i\right)$${\varphi }_{\sigma }\left(x\right)\ge 0$ for every $x\in \mathbb{R}$, with ${\varphi }_{\sigma }\left(2\right)>0$, and ${lim}_{x\to \pm \infty }{\varphi }_{\sigma }\left(x\right)=0$;

$\left(ii\right)$ ${\varphi }_{\sigma }\left(x\right)$ is an even function;

$\left(iii\right)$ ${\varphi }_{\sigma }\left(x\right)$ is non-decreasing for $x<0$ and non-increasing for $x\ge 0$;

$\left(iv\right)$ ${\varphi }_{\sigma }\left(x\right)=O\left({\left|x\right|}^{-\alpha -1}\right)$ as $x\to \pm \infty$ where α is the positive constant of condition $\left(\mathrm{\Sigma }3\right)$.

For a proof of conditions $\left(i\right)$–$\left(iv\right)$, see [31].

Now, we introduce the following notation used in the literature (see, e.g., [3]) in order to define the so-called max-product type operators.

Definition 2.

Let $K_1,K_2$ be two integers with $K_1\le K_2$ and let $A_k\in \mathbb{R},k=K_1,\cdots ,K_2$. Then we define

$$\underset{k=K_1}{\overset{K_2}{\bigvee }}{A}_k:=max\left\{A_k,k=K_1,\cdots ,K_2\right\}.$$

Now, we recall the following lemma that will be useful in order to show that the family of operators investigated in this paper are well-defined.

Lemma 2.

([29])${\displaystyle \underset{k=0}{\overset{n-1}{\bigvee }}}{\varphi }_{\sigma }\left(nx-k\right)\ge {\varphi }_{\sigma }\left(2\right)>0$ for every $x\in \left[0,1\right]$.

The definition of the max-product NN operators of the Kantorovich type can now be recalled.

Definition 3.

Let $f:\left[0,1\right]\to \mathbb{R}$ be a bounded and locally integrable function and let $n\in {\mathbb{N}}^{+}$. The max-product NN operators of the Kantorovich type activated by σ are defined by:

$$K_n^{\left(M\right)}\left(f,x\right):=\frac{{\displaystyle \underset{k=0}{\overset{n-1}{\bigvee }}{\varphi }_{\sigma }\left(nx-k\right)n{\int }_{k/n}^{\left(k+1\right)/n}f\left(u\right)du}}{{\displaystyle \underset{k=0}{\overset{n-1}{\bigvee }}}{\varphi }_{\sigma }\left(nx-k\right)},x\in \left[0,1\right].$$

Clearly, in view of the properties established in Lemma 2, it turns out that $K_n^{\left(M\right)}\left(f,x\right)$ are well-defined, and, moreover, it is quite simple to observe that ${∥K_n^{\left(M\right)}\left(f,·\right)∥}_{\infty }\le {∥f∥}_{\infty }<+\infty .$

Concerning the assumptions $\left(\mathrm{\Sigma }1\right)$, $\left(\mathrm{\Sigma }2\right)$, and $\left(\mathrm{\Sigma }3\right)$ assumed above, we can observe that condition $\left(\mathrm{\Sigma }2\right)$ could be avoided, requiring that the sigmoidal function $\sigma$ is such that $\sigma \left(3\right)>\sigma \left(1\right)$ and condition $\left(iii\right)$ of Lemma 1 is fulfilled by ${\varphi }_{\sigma }$. The main advantage that can be achieved by the latter fact is that one could apply all the approximation results established below also to discontinuous and non-smooth sigmoidal functions.

An example of continuous but non-smooth sigmoidal function (given according to the above remark) is the so-called ramp function ${\sigma }_R\left(x\right)$ (see [12]) defined as follows:

$${\sigma }_R\left(x\right):=\left\{\begin{array}{cc}0,\hfill & x<-\frac{3}{2}\hfill \\ \frac{x}{3}+\frac{1}{2},\hfill & -\frac{3}{2}\le x\le \frac{3}{2}\hfill \\ 1,\hfill & x>1.\hfill \end{array}\right.$$

In particular, ${\sigma }_R$ satisfies condition $\left(\mathrm{\Sigma }3\right)$ for all $\alpha >0$, and ${\varphi }_{{\sigma }_R}$ turns out to be a function with compact support; moreover, ${\sigma }_R\left(3\right)>{\sigma }_R\left(1\right)$.

Note that (see [27]) the sigmoidal function ${\sigma }_{M_1}(3·)$ coincides with the ramp function ${\sigma }_R$; now recalling the definition of the well-known rectified linear unit (ReLU) activation function (see, e.g., [19,32]):

$${\psi }_{ReLU}\left(x\right):={\left(x\right)}_{+},x\in \mathbb{R},$$

it turns out that:

$${\sigma }_{M_1}\left(3x\right):={\psi }_{ReLU}(3x+1/2)-{\psi }_{ReLU}(3x-1/2),x\in \mathbb{R}.$$

Thus, the density function ${\varphi }_{{\sigma }_{M_1}(3·)}$ can be expressed in terms of ReLU activation function as follows:

$${\varphi }_{{\sigma }_R}\left(x\right)={\varphi }_{{\sigma }_{M_1}(3·)}\left(x\right)={\psi }_{ReLU}(3x+1)-2{\psi }_{ReLU}\left(3x\right)+{\psi }_{ReLU}(3x-1),x\in \mathbb{R}.$$

As a consequence of the latter relation, the NN operators $K_n^{\left(M\right)}$ activated by ${\sigma }_{M_1}(3·)$ can be considered as an NN activated by the above linear combination of ReLU activation functions. Recently, it has been proved that ${\psi }_{ReLU}$ is very suitable in order to train deep (i.e., multi-layer) neural networks; see, e.g., [33,34]. For more details concerning ${\psi }_{ReLU}$, see also [19,35].

3. The Saturation Order

It is well known that, if $f\in C_{+}\left(\left[0,1\right]\right)$ the family $\left(K_n^{\left(M\right)}\left(f,·\right)\right)$ converges uniformly to f (see [9]). Moreover, we also know that the following quantitative estimates:

$${∥K_n^{\left(M\right)}\left(f,·\right)-f\left(·\right)∥}_{\infty }\le M\omega \left(f,\frac{1}{n}\right),$$

(1)

as $n\to +\infty ,$ there holds if condition $\left(\mathrm{\Sigma }3\right)$ is satisfied for $\alpha \ge 1$, $M>0$ and where:

$$\omega \left(f,\frac{1}{n}\right):=sup\left\{\left|f\left(x\right)-f\left(y\right)\right|:x,y\in \left[0,1\right],\mathrm{with}\left|x-y\right|\le 1/n\right\}$$

denotes the usual modulus of continuity of the function $f\in C_{+}\left(\left[0,1\right]\right)$ (see, e.g., [36]). From the latter result, it turns out that, if the function f belongs to $\mathrm{Lip}\left(\left[0,1\right]\right)$, then the order of uniform approximation is $1/n$, as $n\to +\infty$.

In this section, we study the saturation order for the NN operator of the Kantorovich type activated by sigmoidal functions; i.e., we show that $1/n$ is the best possible order of approximation that can be achieved for non-decreasing functions that belong to $C_{+}\left([0,1]\right)$.

In order to reach our main purpose, we need some preliminary lemmas.

Lemma 3.

For any $j\in \left\{0,\cdots ,n-1\right\}$, $n\in {\mathbb{N}}^{+}$, we have:

$$\underset{k=0}{\overset{n-1}{\bigvee }}{\varphi }_{\sigma }\left(nx-k\right)={\varphi }_{\sigma }\left(nx-j\right)$$

for every $x\in \left[\frac{j}{n}-\frac{1}{2n},\frac{j}{n}+\frac{1}{2n}\right]\cap \left[0,1\right]$. Further, if $x\in \left(1-\frac{1}{2n},1\right]$ we have:

$$\underset{k=0}{\overset{n-1}{\bigvee }}{\varphi }_{\sigma }\left(nx-k\right)={\varphi }_{\sigma }\left(nx-n+1\right).$$

Proof. 

Let $j\in \left\{0,\cdots ,n-1\right\}$ be fixed and $x\in \left[\frac{j}{n}-\frac{1}{2n},\frac{j}{n}+\frac{1}{2n}\right]\cap [0,1]$. Observing that

$$\left|nx-j\right|=n\left|x-j/n\right|\le 1/2$$

(2)

and, since ${\varphi }_{\sigma }\left(x\right)$ is even and non-increasing for $x\ge 0$, we get:

$${\varphi }_{\sigma }\left(nx-j\right)={\varphi }_{\sigma }\left(\left|nx-j\right|\right)\ge {\varphi }_{\sigma }\left(1/2\right)\ge {\varphi }_{\sigma }\left(2\right)>0.$$

Similarly, if $k\in \left\{0,\cdots ,n-1\right\}$ and $k\ne j$, we have

$$\left|nx-k\right|=n\left|x-k/n\right|\ge 1/2$$

(3)

and so, using the previous properties of ${\varphi }_{\sigma }\left(x\right)$ we have:

$${\varphi }_{\sigma }\left(nx-k\right)={\varphi }_{\sigma }\left(\left|nx-k\right|\right)\le {\varphi }_{\sigma }\left(1/2\right).$$

If $x\in \left(1-\frac{1}{2n},1\right]$ we note that

$$\left|nx-n+1\right|=n\left|x-\left(n-1\right)/n\right|\le 1$$

and, if $k\ne n-1$

$$\left|nx-k\right|=n\left|x-k/n\right|>1$$

and the claim follows, arguing as in (2) and (3), respectively. □

Lemma 4.

Let $I\subseteq \mathbb{R}$ be a bounded or unbounded interval and $f\in C\left(I\right)$. Suppose in addition that there exists an absolute positive constant C with the property that for every $\epsilon >0$ there exists $n\left(\epsilon \right)\in {\mathbb{N}}^{+}$ such that for any $n\in \mathbb{N}$, $n\ge n\left(\epsilon \right)$ and $j\in \mathbb{Z}$, with $\frac{j}{n},\frac{j+1}{n}\in I$, we have

$$\left|f\left(\frac{j+1}{n}\right)-f\left(\frac{j}{n}\right)\right|\le \frac{C\epsilon }{n}.$$

(4)

Then f is a constant function.

Proof. 

Let us choose arbitrary $x_0,y_0\in I$, $x_0<y_0$ and $\epsilon >0$. The continuity of f implies the existence of $n_0\left(\epsilon \right)\in {\mathbb{N}}^{+}$, such that for any $x,y\in I$, $\left|x-x_0\right|\le \frac{1}{n_0\left(\epsilon \right)}$, $\left|y-y_0\right|\le \frac{1}{n_0\left(\epsilon \right)}$, we have

$$\left|f\left(x\right)-f\left(x_0\right)\right|\le \epsilon ,\left|f\left(y\right)-f\left(y_0\right)\right|\le \epsilon .$$

We now fix $n_1=max\left\{n\left(\epsilon \right),n_0\left(\epsilon \right),\frac{1}{y_0-x_0}\right\}$, where $n\left(\epsilon \right)$ is the constant arising from the assumptions, and let us choose arbitrary $n\in \mathbb{N}$ such that $n\ge n_1$. Since $\frac{1}{n_1}\le y_0-x_0$, it follows that there exists $k\in \mathbb{Z}$ and $l\in \mathbb{N}$ such that

$$\frac{k-1}{n}\le x_0\le \frac{k}{n}\le \frac{k+1}{n}\le \cdots \le \frac{k+l}{n}\le y_0\le \frac{k+l+1}{n}.$$

Applying successively the triangle inequality we get

$$\begin{array}{ccc}\hfill \left|f\left(x_0\right)-f\left(y_0\right)\right|& \le & \left|f\left(x_0\right)-f\left(\frac{k}{n}\right)\right|+\left|f\left(\frac{k}{n}\right)-f\left(\frac{k+1}{n}\right)\right|\hfill \\ & & +\cdots +\left|f\left(\frac{k+l-1}{n}\right)-f\left(\frac{k+l}{n}\right)\right|\hfill \\ & & +\left|f\left(\frac{k+l}{n}\right)-f\left(y_0\right)\right|.\hfill \end{array}$$

By relation (4), we have

$$\left|f\left(\frac{k+p}{n}\right)-f\left(\frac{k+p+1}{n}\right)\right|\le \frac{C\epsilon }{n},p=0,1,\dots ,l-1,$$

and since $max\left\{\left|x_0-\frac{k}{n}\right|,\left|y_0-\frac{k+l}{n}\right|\right\}\le \frac{1}{n}\le \frac{1}{n_0\left(\epsilon \right)}$, we get

$$\left|f\left(x_0\right)-f\left(y_0\right)\right|\le \frac{lC\epsilon }{n}+2\epsilon .$$

On the other hand, we observe that $\frac{k+l}{n}-\frac{k}{n}\le y_0-x_0$, which implies that $l\le n\left(y_0-x_0\right)$. Thus, we obtain

$$\left|f\left(x_0\right)-f\left(y_0\right)\right|\le C\epsilon \left(y_0-x_0\right)+2\epsilon .$$

Now, since $\epsilon >0$ has been chosen arbitrarily, passing to the infimum for $\epsilon >0$ in the previous inequality, we deduce that $f\left(x_0\right)=f\left(y_0\right)$. By the arbitrariness of $x_0$ and $y_0$, it turns out that f is a constant function on the whole I. □

Lemma 5.

Let $I\subseteq \mathbb{R}$ be a bounded or unbounded interval and $f\in C\left(I\right)$ be a non-decreasing function with the property that for any couple $a,b$, $a<b$, of inner points of I, and for every $\epsilon >0$ there exists $n(a,b,\epsilon )\in {\mathbb{N}}^{+}$ such that for any $n\in \mathbb{N}$, $n\ge n(a,b,\epsilon )$ and $j\in \mathbb{Z}$, such that $\frac{j+1}{n},\frac{2j+1}{2n}\in [a,b]$, we have

$$f\left(\frac{j+1}{n}\right)-f\left(\frac{2j+1}{2n}\right)\le \frac{\epsilon }{n}.$$

(5)

Then f is a constant function over I.

Proof. 

Let $n\in \mathbb{N}$, $n\ge n(a,b,\epsilon )$, such that $\frac{1}{n}\le b-a$, and let us choose arbitrary $j\in \mathbb{Z}$ such that $a\le \frac{j}{n}\le \frac{j+1}{n}\le b$. We observe that for any $k\in {\mathbb{N}}^{+}$, we have

$$\frac{j}{n}<\frac{2^{k}j+1}{2^{k}n}<\frac{j+1}{n}.$$

Therefore, applying successively relation (5) we obtain

$$\begin{array}{ccc}\hfill f\left(\frac{j+1}{n}\right)-f\left(\frac{2j+1}{2n}\right)& \le & \frac{\epsilon }{n},\hfill \\ \hfill f\left(\frac{2j+1}{2n}\right)-f\left(\frac{4j+1}{4n}\right)& \le & \frac{\epsilon }{2n},\left(j:=2j,n:=2n\right)\hfill \\ \hfill f\left(\frac{4j+1}{4n}\right)-f\left(\frac{8j+1}{8n}\right)& \le & \frac{\epsilon }{4n},\left(j:=4j,n:=4n\right)\hfill \\ & & ·\hfill \\ & & ·\hfill \\ \hfill f\left(\frac{2^{k}j+1}{2^{k}n}\right)-f\left(\frac{2^{k+1}j+1}{2^{k+1}n}\right)& \le & \frac{\epsilon }{2^{k}n},\left(j:=2^{k}j,n:=2^{k}n\right).\hfill \end{array}$$

Taking, respectively, the sums of all the terms in the first and second parts of the previous inequalities, we obtain

$$f\left(\frac{j+1}{n}\right)-f\left(\frac{2^{k+1}j+1}{2^{k+1}n}\right)\le \frac{\epsilon }{n}·\left(1+\frac{1}{2}+\cdots +\frac{1}{2^k}\right)\le \frac{2\epsilon }{n}$$

Since

$$\underset{k\to \infty }{lim}f\left(\frac{2^{k+1}j+1}{2^{k+1}n}\right)=f\left(\frac{j}{n}\right),$$

it follows that

$$f\left(\frac{j+1}{n}\right)-f\left(\frac{j}{n}\right)\le \frac{2\epsilon }{n}$$

and since f is non-decreasing, we get

$$\left|f\left(\frac{j+1}{n}\right)-f\left(\frac{j}{n}\right)\right|\le \frac{2\epsilon }{n}.$$

By Lemma 4 it follows that f is constant in $\left[a,b\right]$. Since a and b are two arbitrary inner points of I and f is continuous, it easily results that f is constant in I. □

Note that Lemma 4 and Lemma 5 can also be extrapolated from the monograph [3].

Now we can prove the main theorem of this section.

Theorem 1.

Let $f\in C_{+}\left([0,1]\right)$ be a non-decreasing function such that

$${∥K_n^{\left(M\right)}(f,·)-f(·)∥}_{\infty }=o\left(n^{-1}\right)$$

as $n\to \infty$. Then f is constant over $\left[0,1\right]$.

Proof. 

Let us choose arbitrary $a,b\in (0,1)$, $a<b$. Further, let $n\in {\mathbb{N}}^{+}$ be sufficiently large such that $\frac{1}{n}<b-a$ and $b<1-\frac{1}{2n}$. Now, we fix $j\in \{0,1,\cdots ,n-2\}$, such that $\frac{j+1}{n},\frac{j}{n}+\frac{1}{2n}\in [a,b]$. We have

$$K_n^{\left(M\right)}\left(f,\frac{j}{n}+\frac{1}{2n}\right)=K_n^{\left(M\right)}\left(f,\frac{2j+1}{2n}\right)=\frac{{\displaystyle \underset{k=0}{\overset{n-1}{\bigvee }}}{\varphi }_{\sigma }\left(n·\frac{2j+1}{2n}-k\right)n{\displaystyle \underset{k/n}{\overset{(k+1)/n}{\int }}}f\left(u\right)du}{{\displaystyle \underset{k=0}{\overset{n-1}{\bigvee }}}{\varphi }_{\sigma }\left(n·\frac{2j+1}{2n}-k\right)}.$$

By Lemma 3, it follows that

$$\begin{array}{ccc}\hfill \underset{k=0}{\overset{n-1}{\bigvee }}{\varphi }_{\sigma }\left(n·\frac{2j+1}{2n}-k\right)& =& {\varphi }_{\sigma }\left(n·\frac{2j+1}{2n}-j\right)\hfill \\ & =& {\varphi }_{\sigma }\left(n·\frac{2j+1}{2n}-(j+1)\right)\hfill \end{array}$$

which easily implies that

$$K_n^{\left(M\right)}\left(f,\frac{2j+1}{2n}\right)\ge n\underset{(j+1)/n}{\overset{(j+2)/n}{\int }}f\left(u\right)du.$$

Moreover, recalling that f is non-decreasing, it follows that $K_n^{\left(M\right)}\left(f,\frac{2j+1}{2n}\right)\ge f\left(\frac{j+1}{n}\right)$, and this implies

$$0\le f\left(\frac{j+1}{n}\right)-f\left(\frac{2j+1}{2n}\right)\le {∥K_n^{\left(M\right)}(f,·)-f∥}_{\infty }=o\left(n^{-1}\right),asn\to +\infty .$$

Then, we can prove that for every $\epsilon >0$, there exists $n\left(\epsilon \right)\in {\mathbb{N}}^{+}$ such that for any $n\in {\mathbb{N}}^{+}$, with $n\ge n\left(\epsilon \right)$ and $j\in \mathbb{Z}$, such that $\frac{j+1}{n},\frac{2j+1}{2n}\in [a,b]$, we have

$$f\left(\frac{j+1}{n}\right)-f\left(\frac{2j+1}{2n}\right)\le \frac{\epsilon }{n},$$

and hence the proof follows by Lemma 5. □

Remark 1.

The previous theorem can be easily generalized in the case of functions defined on arbitrary intervals $\left[a,b\right]$, $a,b\in \mathbb{R}$, instead of $\left[0,1\right]$. This is possible by defining the operators $K_n^{\left(M\right)}$ on generic intervals $[a,b]$ (as made in [9]) and then working with continuous and non-decreasing $f:[a,b]\to {\mathbb{R}}^{+}$.

4. Local Inverse Result

The main aim of this section is to prove an inverse theorem of approximation. We will use a strategy similar to that presented in the previous section.

Lemma 6.

Let $I\subseteq \mathbb{R}$ be a bounded or unbounded interval and $f\in C\left(I\right)$. Suppose in addition that there exists an absolute positive constant C such that for any sufficiently large $n\in \mathbb{N}$ and every $j\in \mathbb{Z}$, with $\frac{j}{n},\frac{j+1}{n}\in I$, we have

$$\left|f\left(\frac{j+1}{n}\right)-f\left(\frac{j}{n}\right)\right|\le \frac{C}{n}.$$

(6)

Then $f\in \mathit{Lip}\left(I\right).$

Proof. 

Let us choose arbitrary $x_0,y_0\in I$, $x_0<y_0$. Moreover, let $\epsilon >0$ with $\epsilon <y_0-x_0.$ The continuity of f implies the existence of $n_0\left(\epsilon \right)\in {\mathbb{N}}^{+}$, such that for any $x,y\in I$, $\left|x-x_0\right|\le \frac{1}{n_0\left(\epsilon \right)}$, $\left|y-y_0\right|\le \frac{1}{n_0\left(\epsilon \right)}$, we have

$$\left|f\left(x\right)-f\left(x_0\right)\right|\le \epsilon ,\left|f\left(y\right)-f\left(y_0\right)\right|\le \epsilon .$$

We now fix $n_1=max\left\{n_0\left(\epsilon \right),\frac{1}{y_0-x_0}\right\}$, and let us choose arbitrary $n\in \mathbb{N}$ with $n\ge n_1$. Proceeding as in the proof in Lemma 4, we get

$$\left|f\left(x_0\right)-f\left(y_0\right)\right|\le \frac{lC}{n}+2\epsilon .$$

and since $\epsilon <y_0-x_0$, we get

$$\left|f\left(x_0\right)-f\left(y_0\right)\right|\le (C+2)\left(y_0-x_0\right).$$

Then the thesis follows by the arbitrariness of $x_0$ and $y_0$. □

Lemma 7.

Let $I\subseteq \mathbb{R}$ be a bounded or unbounded interval and $f\in C\left(I\right)$ be a non-decreasing function with the property that, for any couple $a,b$, $a<b$, of inner points of I, there exists a $C>0$ such that for every sufficiently large $n\in \mathbb{N}$ and $j\in \mathbb{Z}$, with $\frac{j+1}{n},\frac{2j+1}{2n}\in [a,b]$, we have

$$f\left(\frac{j+1}{n}\right)-f\left(\frac{2j+1}{2n}\right)\le \frac{C}{n}.$$

(7)

Then $f\in \mathit{Lip}\left(\right[a,b\left]\right)$.

Proof. 

Let $n\in \mathbb{N}$ be sufficiently large such that $\frac{1}{n}\le b-a$, and let us choose arbitrary $j\in \mathbb{Z}$ such that $a\le \frac{j}{n}\le \frac{j+1}{n}\le b$. Arguing as in Lemma 5, we get

$$f\left(\frac{j+1}{n}\right)-f\left(\frac{2^{k+1}j+1}{2^{k+1}n}\right)\le \frac{C}{n}·\left(1+\frac{1}{2}+···+\frac{1}{2^k}\right)\le \frac{2C}{n},k\in {\mathbb{N}}^{+}.$$

Taking the limit as $k\to +\infty$, we obtain

$$f\left(\frac{j+1}{n}\right)-f\left(\frac{j}{n}\right)\le \frac{2C}{n}$$

and, since f is non-decreasing,

$$\left|f\left(\frac{j+1}{n}\right)-f\left(\frac{j}{n}\right)\right|\le \frac{2C}{n}.$$

Hence, by Lemma 6, it follows that $f\in \mathrm{Lip}\left(\right[a,b\left]\right)$. □

For results similar to that ones of Lemma 6 and Lemma 7 (only in the case of bounded intervals), one can see, e.g., [3].

Now we can finally prove a (local) inverse theorem of approximation.

Theorem 2.

Let $f\in C_{+}\left([0,1]\right)$ be a non-decreasing function such that

$${∥K_n^{\left(M\right)}(f,·)-f(·)∥}_{\infty }=O\left(n^{-1}\right)$$

as $n\to \infty$. Then, for every $[a,b]\subset (0,1)$, it turns out that$f\in \mathit{Lip}\left(\right[a,b\left]\right)$.

Proof. 

As in Theorem 1, we choose arbitrary $a,b\in (0,1)$, $a<b$. Let now $n\in {\mathbb{N}}^{+}$ be sufficiently large such that $\frac{1}{n}<b-a$ and $b<1-\frac{1}{2n}$. Then, let $j\in \{0,1,\cdots ,n-2\}$, such that $\frac{j+1}{n},\frac{2j+1}{2n}\in [a,b]$. Proceeding as in the proof of the above-mentioned theorem, we immediately get

$$0\le f\left(\frac{j+1}{n}\right)-f\left(\frac{2j+1}{2n}\right)\le {∥K_n^{\left(M\right)}(f,·)-f(·)∥}_{\infty }=O\left(n^{-1}\right),asn\to +\infty ,$$

and, by Lemma 7, this implies that $f\in \mathrm{Lip}\left(\right[a,b\left]\right)$. □