Automatic differentiation of uncertainties: an interval computational differentiation for first and higher derivatives with implementation

Acquiring reliable knowledge amidst uncertainty is a topical issue of modern science. Interval mathematics has proved to be of central importance in coping with uncertainty and imprecision. Algorithmic differentiation, being superior to both numeric and symbolic differentiation, is nowadays one of the most celebrated techniques in the field of computational mathematics. In this connexion, laying out a concrete theory of interval differentiation arithmetic, combining subtlety of ordinary algorithmic differentiation with power and reliability of interval mathematics, can extend real differentiation arithmetic so markedly both in method and objective, and can so far surpass it in power as well as applicability. This article is intended to lay out a systematic theory of dyadic interval differentiation numbers that wholly addresses first and higher order automatic derivatives under uncertainty. We begin by axiomatizing a differential interval algebra and then we present the notion of an interval extension of a family of real functions, together with some analytic notions of interval functions. Next, we put forward an axiomatic theory of interval differentiation arithmetic, as a two-sorted extension of the theory of a differential interval algebra, and provide the proofs for its categoricity and consistency. Thereupon, we investigate the ensuing structure and show that it constitutes a multiplicatively non-associative S-semiring in which multiplication is subalternative and flexible. Finally, we show how to computationally realize interval automatic differentiation. Many examples are given, illustrating automatic differentiation of interval functions and families of real functions.


INTRODUCTION
Uncertainty arises in all fields of modern science. It is a state of limited knowledge where ''To know'' means ''To be uncertain of''. Acquiring reliable knowledge amidst uncertainty is the raison d'être of the present work. Motivated by an ever-increasing indeterminacy and complexity in physics and engineering and fueled by developments in computational and uncertainty mathematics, this work puts forward a categorical system of interval differentiation arithmetic that wholly addresses the computation of first and higher order automatic derivatives under uncertainty. Although scientists are fond of determinism, contemporary physical sciences have shown clearly that complete certainty is not reachable. The description of processes and states of physical systems discloses increasingly growing manipulations of uncertain quantifiable properties. Many features of the object world are rendered as numerical values that can either be measured or estimated by experts. Due to imperfection of our measuring methods, finiteness of our computations and lack of information, measured or estimated quantities can only be represented by finite approximations and thus are merely imprecise abstractions of reality (Dawood & Dawood, 2019a;Dawood &Dawood, 2020, andDawood &Dawood, 2022).
In the effort to deal with the challenge of uncertainty, the subject of uncertainty mathematics has been developed in an extensive manner and many theoretical approaches have been introduced including fuzzy, probabilistic, and interval methods. A hot and fundamental topic of research that shades of into all approaches of uncertainty mathematics is interval analysis (see, e.g., Dawood, 2014;Dawood &Dawood, 2019a, andDawood &Dawood, 2020). The key advantage of the interval methods is that they provide ''guaranteed interval enclosures'' of the exact values of quantifiable uncertainties. In practice, when modelling physical systems, we have two distinct approaches: getting guaranteed bounds of an uncertain quantity and computing a numerical approximation thereof. The two approaches are not equivalent: the former includes the latter, but the latter does not imply the former. For example, to guarantee stability under uncertainty in control systems and robotics, it is crucial to compute guaranteed enclosures of the quantifiable features of the system under consideration (Dawood, 2014 andDawood &Dawood, 2020). Interval arithmetic brings forth a reliable way to cope with such problems. An interval number (a closed and bounded interval of real numbers) is a guaranteed enclosure of an imprecisely measured real-valued quantity, and an interval-valued function is consequently a guaranteed enclosure of a real-valued function under imprecision or uncertainty (or more generally, as we will see in this article, a reliable enclosure of the image of a family of real-valued functions). Historically speaking, the terms ''interval arithmetic'', ''interval analysis'', and ''interval computations'' are reasonably recent: they date from the fifties of the twentieth century. But the idea has been known since the third century BC, when Archimedes (287-212 BC) used lower and upper error bounds in the course of his computation of the constant π (Heath, 2009). In the dawning of the twentieth century, the first rigorous developments of the theory of intervals appeared in the works of Norbert Wiener, John Charles Burkill, Rosalind Cecily Young, andMihailo Petrovic (see Wiener, 1921, Burkill, 1924;Young, 1931;Petrovic, 1932, andPetkovic, 2020). Later, several distinguished developments of interval arithmetic appeared in the works of Paul S. Dwyer, Mieczyslaw Jan Warmus, Teruo Sunaga, and others (see, e.g., Dwyer, 1951;Warmus, 1956, andSunaga, 1958). However, it was not until 1959 that ''interval analysis'' in its modern sense was presented by the American mathematician and computer scientist, Ramon Edgar Moore (1929Moore ( -2015, who was the first to recognize the power of interval arithmetic as a viable computational apparatus for coping with uncertainty and imprecision (Moore, 1959). Nowadays, interval mathematics is a bold enterprise that comprises many different kinds of problem and has many fruitful applications in diverse areas of science and engineering (see, e.g., Allahviranloo, Pedrycz & Esfandiari, 2022;Beutner, Ong & Zaiser, 2022, Dawood, 2019Dawood & Dawood, 2020;Dawood & Dawood, 2022, IEEE 1788Committee, 2018Jiang, Han & Xie, 2021;Kearfott, 2021, Mahato, Rout & Chakraverty, 2020Matanga, Sun & Wang, 2022, Shary & Moradi, 2021, and Zheng et al., 2020.
Two strands of research have led to the birth of the present work. The first strand starts from research in interval mathematics. The other strand stems from ordinary (real) automatic differentiation. Derivatives play an indispensable role in scientific computing. The expressions 'automatic differentiation', 'auto-differentiation', 'computational differentiation', 'algorithmic differentiation', and 'differentiation arithmetic' are in the just acceptation synonyms. They refer to a subtle and central tool to automatize the simultaneous computation of the numerical values of arbitrarily complex functions and their derivatives with no need for the symbolic representation of the derivative, only the function rule or an algorithm thereof is required (Dawood & Megahed, 2019). Auto-differentiation is thus neither numeric nor symbolic, nor is it a combination of both. It is also preferable to ordinary numerical methods: In contrast to the more traditional numerical methods based on finite differences, auto-differentiation is 'in theory' exact, and in comparison to symbolic algorithms, it is computationally inexpensive (Dawood & Megahed, 2019). The literature on algorithmic differentiation is immense and very diversified. For further reading, (see, e.g., Corliss & Rall, 1996;Dawood, 2014, Dawood & Megahed, 2019Griewank & Walther, 2008;Moore, 1979, Neidinger, 2010, and Mitchell, 1991. Currently, for its efficiency and accuracy in computing first and higher order derivatives, auto-differentiation is a celebrated technique with diverse applications in scientific computing and mathematics. It should therefore come as no surprise that there are numerous computational implementations of auto-differentiation. Among these, we mention, without pretension to be complete, INTLAB, Sollya, and InCLosure (see, e.g., Rump, 1999, Chevillard, Joldes & Lauter, 2010, and Dawood, 2020. In practice, there are two types (modes) of algorithmic differentiation: a forward-type and a reversed-type (Dawood & Megahed, 2019). Presently, the two types are highly correlated and complementary and both have a wide variety of applications in, e.g., non-linear optimization, sensitivity analysis, robotics, machine learning, computer graphics, and computer vision (see, e.g., Abdelhafez, Schuster & Koch, 2019;Dawood, 2014, Dawood & Megahed, 2019Fries, 2019, Sommer, Pradalier & Furgale, 2016, and Tingelstad & Egeland, 2017.
The use of ordinary auto-differentiation in the description and modeling of real world physical systems faces the problem of uncertainty. With the aid of interval mathematics, auto-differentiation can be intervalized to handle uncertainty in quantifiable properties of real world physical systems and accordingly provide the computational methods that suffice to deal with the important problem of ''getting guaranteed bounds''. Interval differentiation arithmetic combines subtlety of ordinary algorithmic differentiation with power and reliability of interval mathematics. By integrating the complementary perspectives of both fields, interval differentiation arithmetic extends real differentiation arithmetic so markedly both in method and objective, and so far surpasses it in power and applicability. Real differentiation arithmetic, on the one hand, is concerned with the simultaneous calculation of the values of real functions and their derivatives with no requirement of the symbolic representation of the derivative. On the other hand, the subject matter of interval differentiation arithmetic is ''interval functions'' and its objective is the concurrent computation of guaranteed enclosures of images of real functions and their derivatives. This integration of interval and differentiation arithmetic is readily applicable to modelling and predicting the behaviour of real-world systems under uncertainty. Also, it has proved accuracy and efficiency in many scientific computations. As examples, we can mention enclosures of Taylor's coefficients, gradients, integrals, bounding boxes in ray tracing, and solutions of ordinary differential equations.
Three main problems have motivated the research conducted in this article. In the first place, despite its major importance in both basic research and practical applications, to the best of our knowledge, the algebraic aspects of interval differentiation arithmetic are not in-depth investigated. In the second place, almost no attempt has been made so far to explicitly axiomatize the theory of interval differentiation arithmetic in terms of clear and distinct elementary logical notions. In the third place, although an interval function is naturally an extension of a family of real functions, to the best of our knowledge, in all interval literature, the notion of a family of functions is not considered, and an interval function is assumed to extend a single real function. This presumption introduces an unnecessary restriction to the semantic of an interval function in the general sense. Families of functions arise naturally in many real-life and physical applications. In economics, the Cobb-Douglas family of production functions is an example; in physics, electron models, dynamical systems, quantum models, Camassa-Holm and Novikov wave-breaking models, and many other physical phenomena are described by families of functions (see, e.g., Cobb & Douglas, 1928;Silberberg & Suen, 2001, Anco, da Silva & Freire, 2015, and Engesser, Gabbay & Lehmann, 2011. Providing the mathematical tools to get guaranteed enclosures of the images of families of real functions and their derivatives would provide an efficient way of predicting and controlling such physical systems and, thus, could have a substantial impact not only on theoretical research but also on many areas of applications. By the pursuit of this, formalizing the notion of a family of functions within the context of interval mathematics and interval differentiation arithmetic is one of the main motivations of this research. Throughout the present text, we will understand by ''interval differentiation arithmetic'' (''interval differentiation algebra'', '' J -algebra'', or '' J -arithmetic'') the fundamental algebraic structure underlying interval auto-differentiation as it is currently practised and implemented. It is our object, in this article, to present a consistent and categorical formalization of a theory of dyadic interval differentiation numbers ( J -numbers) that fully addresses first and higher order auto-derivatives of families of real functions. The fundamental significance of categoricity is that if an axiomatization of J -numbers is categorical, then it correctly accounts, up to isomorphism, for every structure of 1 InCLosure(Interval enCLosure) is an environment and a language for reliable scientific computing. Latest version of the software package InCLosure is freely accessible from CERN's archive via https://doi.org/10.5281/zenodo.2702404. This is clearly described by John Corcoran in Corcoran (1980) and best reiterated in the words of Stewart Shapiro, ''a categorical axiomatization is the best one can do'' (Shapiro, 1985). In accordance with this categorical sense, the present article attempts to provide this ''best'' characterization. For this goal to be accomplished, we need to take a closer look at and formalize several fundamental analytic and algebraic concepts in the language of the theory to be axiomatized, so that one can establish the metatheoretic assertions of consistency and categoricity. This reformalization is mainly done in 'On theories and structures: some metatheoretical fundamentals' and 'A differential interval algebra'. In 'On theories and structures: some metatheoretical fundamentals', we set the stage by establishing the mathematical terminology, notions, and definitions that will be used throughout the rest of this article. 'Real differentiation arithmetic' is devoted to describing briefly the basic elements of the theory T R of real differentiation arithmetic ( R -arithmetic). In 'A differential interval algebra', we lay out an axiom system for the theory T δJ of a differential interval algebra and then we present the notion of an interval extension of a family of real functions, together with some analytic notions of interval functions. In 'A categorical axiomatization of interval differentiation arithmetic', we axiomatize a theory T J of interval differentiation numbers ( J -numbers) as a two-sorted extension of the theory T δJ of a differential interval algebra, and then we prove its consistency and categoricity. In order for the theory T J to fully address and compute higher order and partial auto-derivatives using only dyadic J -numbers, in 'Differentiation extension of interval functions and higher-order auto-differentiability', we introduce the notion of a differentiation extension of interval functions, characterize differentiability for J -numbers, and establish their differentiability conditions. In 'The algebraic structure of interval differentiation arithmetic', we investigate the algebraic structure of J -arithmetic, establish its fundamental algebraic properties, and show that it forms a multiplicatively non-associative S-semiring in which multiplication is subalternative and flexible. Then, in 'Monotonicity and isomorphism theorems for interval differentiation numbers', we establish some monotonicity and isomorphism theorems for J -numbers and prove a result concerning the structure of R -numbers. Finally, in 'Machine implementation of interval auto-differentiation', we demonstrate the computational implementation of interval auto-differentiation and illustrate, by many numerical examples, how to concurrently compute guaranteed enclosures of images of both families of real functions and their first and higher order derivatives. The algorithms discussed in 'Machine implementation of interval auto-differentiation' are coded into reliable Common Lisp as a part of the software package, InCLosure 1 (Dawood, 2020). The InCLosure commands to calculate the results of the numerical examples are described and InCLosure input and output files are accessible as a supplementary material to the article (see Dawood, 2020 andDawood, 2023).
The attempted contribution of this article is therefore both a ''logico-algebraic formalization'' and an ''extension'' of interval differentiation arithmetic. The article gives an axiomatization of a comprehensive algebraic theory of interval differentiation arithmetic. Being based on clear and distinct elementary ideas of real and interval algebras, this formalized theory places the diverse approaches of interval auto-differentiation on 2 Grassmann algebras (or exterior algebras) were introduced by Hermann Grassmann in Grassmann (1844). Clifford's algebras, introduced in Clifford (1873) by William Clifford, are Grassmann algebras in one dimension. Exterior algebras have nowadays many applications (see, e.g., Colombaro, Font-Segura & Martinez, 2020;Dawood &Megahed, 2019, andTrindade, Pinto &Floquet, 2019). a firm and unified mathematical basis. We extend this theory in two directions. On the one hand, to the best of our knowledge, in almost all computational differentiation literature, researchers tend to 'borrow' or 'reinvent' Clifford's and Grassmann algebras 2 as proposed algebraic characterizations respectively for first and higher-order algorithmic differentiation. Without resorting to defining any sort of Grassmann structures, our axiomatization of dyadic interval differentiation numbers extends to fully address interval auto-derivatives of first and higher order. On the other hand, from the very beginning, our axiomatic system includes the notion of an interval extension of a family of real functions and the differentiability criteria thereof. By virtue of introducing this notion, the theory is extended to provide the mathematical tools to get guaranteed enclosures of the images of families of real functions and their derivatives. Also noteworthy here is that with a few basic modifications, the categorical system T J axiomatized in this text can be extended analogously to compute fuzzy auto-derivatives.

ON THEORIES AND STRUCTURES: SOME METATHEORETICAL FUNDAMENTALS
To achieve a rigorous formalization of the mathematical theory of this work, a specific formalized language and a particular logical apparatus are therefore required to attain all the results from obvious and distinct elementary mathematical concepts. So before we begin our axiomatization of interval auto-differentiation, we need to take a closer look at and formalize several preliminary mathematical concepts. To this aim, this section establishes the mathematical terminology, notions, and definitions that will be used throughout the rest of this article.
To make this article self-contained, we start by rehearsing some set-theoretical definitions. Let A be a set and let A n be its n-th Cartesian power. A set is an nary (finitary) relation on A iff ⊆ A n and is a binary relation from A n−1 to A. Thus, for s = (s 1 ,...,s n−1 ) ∈ A n−1 and t ∈ A, an n-ary relation is characterized to be ⊆ A n = {(s,t )|s ∈ A n−1 ∧ t ∈ A}. Accordingly, a finitary relation is a binary relation whose domain, range, field, and converse are characterized to be, Obviously t s ⇔ s t and = (Dawood & Dawood, 2019a andDawood &Dawood, 2020).

Definition 2.1 (Images of Finitary Relations Dawood & Dawood, 2020):
For 1 ≤ k ≤ n − 1, let be an n-ary relation on A, and for (s,t ) ∈ , let s = (s 1 ,...,s n−1 ), with each s k is restricted to vary on S k ⊂ A, that is, s is restricted to vary on S ⊂ A n−1 . Then, the image of S (or the image of the sets S k ) with respect to , in symbols I , is characterized to be T = I (S) = I (S 1 ,...,S n−1 ) = {t ∈ A|(∃s ∈ S)(s t )} = {t ∈ A| ∃ n−1 k=1 s k ∈ S k ((s 1 ,...,s n−1 ) t )}.
The preimage S of T is characterized to be the image of T with respect to the converse of . In other words In consequence of the equivalence t s ⇔ s t , apparently T = I (S) ⇔ S = I (T ).
In this sense, a general characterization of an n-ary (finitary) function can be introduced (Dawood & Dawood, 2019a). A set q is an n-ary function (a function of n variables) on a set A iff q is an (n + 1)-ary relation on A, and (∀s ∈ A n )(∀t ,w ∈ A) sqt ∧ sqw ⇒ t = w . That is, an n-ary (finitary) function is an (n + 1)-ary relation. Restricting ourselves to the particular case of functions, we can pass up the set-theoretical notation sqt in favor of the common notation t = q(s). In accord with this formulation, the preceding definitions of domain, range, field, and converse also apply to finitary functions. We say that a function q is invertible or has an inverse q −1 iff the converse relation q is also a function, in which case q −1 = q (Dawood & Dawood, 2020). Hereon, functions will be denoted by the letters q, u, and v. With a few exceptions, from now onwards, we will usually consider only unary functions.
In order to achieve the overarching objective of this work, it is necessary first to take a closer look at several metamathematical 3 concepts. A metalinguistic characterization of a formalized theory (an axiomatic theory) can be given. An axiomatic theory T is characterized by an object formal language L and a finite set of axioms T (see Dawood & Dawood, 2020). Given an object formal language L and a finite set T of axioms (L-sentences), let ϕ denote an L-sentence and let L denote the semantic consequence relation. The axiomatic L-theory T of the set T is the closure of T under L , that is T = {ϕ ∈ L| T L ϕ} (Dawood & Dawood, 2020). Next, the metatheoretical notions of a model, categoricity, and consistency of an axiomatic L-theory are characterized (see Dawood & Megahed, 2019).

Definition 2.2 (Model of a Theory): Let A be a mathematical structure (interpretation).
A is said to be a model of an axiomatic L-theory T, in symbols A | T, iff every formula ϕ of T is satisfied by A. That is A model of an axiomatic L-theory T is a mathematical structure A = A,σ A that makes the L-sentences of T true. Particular mathematical structures are indispensable for the objective of this work. These are defined next (Dawood &Dawood, 2019a andDawood &Dawood, 2020). Definition 2.5 (Ringoid Dawood & Dawood, 2019a): A ring-like structure (or a ringoid) is an algebraic structure A = A;+ A ,× A with + A and × A are total binary operations on the universe set A. The operations + A and × A are called respectively the addition and multiplication operations of the ringoid A.
Definition 2.6 (S-Ringoid Dawood & Dawood, 2020): A subdistributive ringoid (or an S-ringoid) is a ringoid A = A;+ A ,× A that satisfies at least one of the following subdistributive properties.
Properties (i) and (ii) in the previous definition are called respectively left and right S-distributivity (or subdistributivity) (Dawood & Dawood, 2020).

Definition 2.7 (Semiring Dawood & Dawood, 2019a): A ringoid
If × A is commutative, then A is said to be a commutative semiring.
It is important here to point out that an S-semiring generalizes the notion of a nearsemiring; a near-semiring is a structure satisfying the axioms of a semiring except that it is either left or right distributive (For further details on near-semirings and related concepts, the reader can refer to Clay, 1992;Pilz, 1983, andvan Hoorn &van Rootselaar, 1967).
Lastly, we define two new algebraic structures. Definition 2.9 (NA Semiring): A ringoid A = A;+ A ,× A is said to be an additively non-associative semiring (in short, +-NA semiring) iff A satisfies (ii), (iii), and (iv) in definition 2.7, and A;+ A is a non-associative commutative monoid with identity element 0 A . Similarly, A is said to be a multiplicatively non-associative semiring (in short, ×-NA semiring) iff A satisfies (i), (iii), and (iv) in definition 2.7, and A;× A is a non-associative monoid with identity element 1 A . Definition 2.10 (NA S-Semiring): An S-ringoid A = A;+ A ,× A is said to be an additively non-associative S-semiring (in short, +-NA S-semiring) iff A satisfies (ii) and (iv) in definition 2.7, and A;+ A is a non-associative commutative monoid with identity element 0 A . Similarly, A is said to be a multiplicatively non-associative S-semiring (in short, ×-NA S-semiring) iff A satisfies (i) and (iv) in definition 2.7, and A;× A is a non-associative monoid with identity element 1 A .
It is clear that if multiplication is commutative in a NA S-semiring, then it is both left and right subdistributive.

REAL DIFFERENTIATION ARITHMETIC
Before setting forth the assertions of an axiomatic theory of interval differentiation arithmetic in the succeeding sections, we need to describe briefly the basic elements of the theory T R of real differentiation arithmetic (henceforth R -arithmetic). For further details and other constructions of R -arithmetic, the reader may consult, e.g., Dawood, 2014;Dawood & Megahed, 2019, Beda et al., 1959Wengert, 1964;Moore, 1979, Rall, 1981, and Corliss & Rall, 1996 We hereon use the letters q, u, and v as function symbols, and the letters s, t , and w as real variable symbols. Given a class σ = {+,×;−, −1 ;0,1,≤} of descriptive (non-logical) signs, let R = R;σ R be the field of real numbers, R 1 be the set of unary real functions, and δ be the differential operator for elements of R 1 . For a q in R 1 , we use the predicate diff q,s 0 to mean that q is differentiable at some s 0 ∈ R. We understand by a differential real field a structure R δ = R;σ R ;δ constructed by equipping R with the operator δ and its basic axioms. It is natural to begin with the definition of a real differentiation number ( R -number).

Definition 3.1 (Real Differentiation Numbers):
The set of all real differentiation numbers ( R -numbers, or R -pairs), with respect to a constant s 0 ∈ R, is defined to be That is, a R -number is an ordered pair of real numbers. Let the letters q, u, and v, or equivalently the pairs q,δq s 0 , (u,δu) s 0 , and (v,δv) s 0 , be variable symbols ranging over the elements of U R . Also, let a, b, and c, or equivalently (a,0 R ) s 0 , (b,0 R ) s 0 , and (c,0 R ) s 0 , designate constants of U R . In particular, we use The theory T R of a real differentiation algebra (or a R -algebra) can then be axiomatized as follows (Dawood & Megahed, 2019). Definition 3.2 (Theory of Real Differentiation Algebra): Let q,δq s 0 and (u,δu) s 0 be in U R . A differentiation algebra over a differential real field R δ = R;σ R ;δ , or a R -algebra, is a two-sorted structure U R = U R ;R;σ UR . The theory T R of U R is the deductive closure of the axioms of R δ together with the following sentences.
Subtraction and division are defined as usual in terms of the four basic R -operations. For an economic exposition, we assert statements (DA2)-(DA5) as axioms but it should be mentioned that they are derivable from simpler statements. Hereafter, where no confusion is likely, the subscripts '' U'', '' R'', and '' s 0 '' will be omitted. Also, we will usually write the structure U R as U R ;+ U R ,× U R ;0 U R ,1 U R , omitting the universe set R.
Differentiable real functions can be extended to R -numbers via an extension principle (Dawood & Megahed, 2019). Let u be a real function differentiable at s 0 ∈ R, that is there is u = (u,δu) s 0 ∈ U R , and let Q be a function rule.
We will not discuss the algebraic properties of R -numbers further in the present section, for these will be considered later in 'Monotonicity and isomorphism theorems for interval differentiation numbers', in the general framework of the theory T J of interval differentiation arithmetic.

A DIFFERENTIAL INTERVAL ALGEBRA
In order to axiomatize a categorical system of interval differentiation arithmetic ( Jarithmetic) in the next sections, we need to lay out an axiom system for the theory T δJ of a differential interval algebra. The intended model of the axiomatic system T δJ is the differential S-semiring J R ,R;+ J ,× J ;δ , where J R is the set of real closed intervals (interval numbers, or J -numbers) and δ is the differential operator for unary interval functions (J -functions).
To be able to prove categoricity and consistency of J -arithmetic, the first step towards axiomatizing the theory T δJ necessitates dealing first with the notion of differentiability in a continuously ordered field in a purely syntactic way (leaving out any references to mathematical analysis or possible interpretations). For further details on the syntactic approaches to these notions, see, e.g., Dawood, 2012;Dawood & Dawood, 2020, Montague, Kalish & Mar, 1980Robinson, 1951, andTarski, 1983. The theory T F of continuously ordered fields (cofields) is characterized in the following definition Dawood & Megahed, 2019. Definition 4.1 (Theory of a Cofield): Let F = F;+ F ,× F ;0 F ,1 F ;≤ F be a totally ordered field. The theory T F of a cofield (or a continuously ordered field) is the theory of F together with the following axiom of continuity We designate by ≥ F the converse of the non-strict total order ≤ F , and by '' − F '' and '' −1 F '' the unary F-operations of negation and reciprocal, respectively. Subtraction and division are defined as customary. From now onwards, when the context is clear, we may drop the subscript '' F''. 4 Many mathematicians use the term ''complete ordered field'' as a synonymous substitute for ''continuously ordered field''. Following Alfred Tarski (see, e.g., Tarski, 1994), we pass up the adjective 'complete' in favor of the word 'continuously'. We reserve the word 'complete' for different logical uses.
For n ≥ 1, let K n designate the class of all n-ary F-functions. Hereon, the letters q, u, and v are used as variable symbols ranging over the elements of K 1 (unary F-functions).
The intended interpretation (model) of the theory T F corresponds the structure F to the continuously 4 (complete) ordered field R;+ R ,× R ;0 R ,1 R ;≤ R of real numbers and F 1 is interpreted by the set R 1 of unary R-functions.
Toward formalizing a differential interval algebra, we first need to extend the theory T F of a cofield by axiomatizing some analytic concepts. Let q ∈ F 1 , and let s and l be, respectively, an F-variable symbol and an F-constant symbol. The 'limit' operator of the function q(s) with respect to m, denoted lim s→m q(s), is defined thus (Dawood & Megahed, 2019): where the one-place operation symbol |·|, called an F-absolute value (or F-modulus), is defined by If there is no such M ∈ F, then the limit of q at m is said to be nonexistent in F. For an F-constant symbol s 0 ∈ dom q , the 'continuity' predicate is a binary predicate, cont q,s 0 , defined by If cont q,s 0 is true, then q is said to be continuous at s 0 . We also say that q is continuous on S 0 ⊆ F iff it is continuous at all s 0 ∈ S 0 , that is cont q,S 0 ⇔ (∀s 0 ∈ S 0 ) cont q,s 0 .

Definition 4.2 (n-Differential F -Operator Dawood & Megahed, 2019):
Let n ≥ 0, and let s and β be F-variable symbols. The n-differential F-operator of a function q(s) ∈ F 1 , denoted δ n q(s), is characterized recursively by the following equations.
Evidently if the limit in definition 4.2 exists, then the n-differential δ n q(s) of q is consequently a unary F-function. Henceforth, we will usually write δ n q and δq for δ n q(s) and δ 1 q(s), respectively.
Closely related to the differential operator is the n-differentiability predicate, which is characterized as follows.

Definition 4.3 (n-Differentiability F -Predicate Dawood & Megahed, 2019):
Let n ≥ 0, let q(s) ∈ F 1 , and let s 0 ∈ dom q be an F-constant symbol. The ternary ndifferentiability F-predicate for the function q, denoted diff n q,s 0 , is defined by If diff n q,s 0 is true, then q is said to be n-differentiable at s 0 . 5 There are so many systems of interval algebras (see, e.g., Hansen, 1975, Kulisch, 2013Gardenyes, Mielgo & Trepat, 1985;Markov, 1995, Kaucher, 1980Shary, 2002;Piegat & Dobryakova, 2020, Dawood, 2011Dawood, 2012;Dawood &Dawood, 2019b, andDawood &Dawood, 2020). Here we axiomatize classical (naive) interval algebra as introduced in, e.g., Alefeld & Mayer, 2000;Moore, 1979, Dawood, 2014, andDawood, 2019. An axiom system for an interval differentiation algebra over a different theory of intervals will be fundamentally the same as the axiom system presented in this text, but it might differ in the resulting algebraic structure. 6 For two sets A and B, Since for s 0 ∈ dom q , δ 0 q(s 0 ) = q(s 0 ) ∈ F, the predicate diff 0 q,s 0 is always true and accordingly every q ∈ F 1 is 0-differentiable at s 0 ∈ dom q . Apparently, if diff n q,s 0 is true, then for 0 ≤ m < n, δ m q(s 0 ) ∈ F. Definition 4.4 (Continuous Differentiability F-Predicate ): Let n ≥ 0, let q(s) ∈ F 1 , and let s 0 ∈ dom q be an F-constant symbol. The continuous n-differentiability Fpredicate for the function q, denoted cdiff n q,s 0 , is characterized recursively by the following statements.
If cdiff n q,s 0 is true, then q is said to be continuously n-differentiable at s 0 .
In a manner analogous to the differential operator, if cdiff n q,s 0 is true, then for 0 ≤ m < n, cdiff m q,s 0 is true as well.
A theory T J of an interval algebra or a classical 5 interval algebra (henceforth a J -algebra) over a cofield can then be characterized as follows (Dawood &Dawood, 2020 andDawood &Dawood, 2022). ;0,1} be a class of descriptive (non-logical) signs, and let F = F;σ F be a cofield. The theory T J of an interval algebra (a J -algebra) over F is the theory of a many-sorted algebraic structure Axiom (I1) of the above definition characterizes what a J -number (an interval number, or an F-interval) is. Axioms (I2) and (I3) prescribe, respectively the binary operations of J -addition ('' + J '') and J -multiplication ('' × J ''), and the unary operations of Jnegation ('' − J '') and J -reciprocal ('' −1 J ''). The intended model of T J corresponds the sets ''F'' and '' J F '' to the sets '' R'' and '' J R '' (of real numbers and real closed intervals), respectively, and the symbols '' • F '', and '' F '' to the binary and unary R-operations.
In the sequel, the upper-case letters S, T , and W , or equivalently s,s , t ,t , and w,w , will be used as variable symbols ranging over the domain J F of J -numbers. A point (singleton, or degenerate) J -numbers {s} will be denoted by [s]. Also, the letters A, B, and C, or equivalently a,a , b,b , and c,c , will be used to designate constants of J F . In particular, we will use 1 J and 0 J to designate, respectively, the singleton J -numbers It is convenient here to single out the set J [s] of point J -numbers. This is defined thus: Equality of J -numbers is an immediate consequence of the axiom of extensionality 6 of set theory plus the fact that a J -number is a totally ≤-ordered subset of F. Precisely, The categoricity of the theory T J of J -algebra is established by the following theorem. Theorem 4.1 (Categoricity of the Interval Theory): The theory T J of J -algebra is categorical. That is, Cat T J .
Proof Let σ = {+,×;−, −1 ;0,1} be a class of descriptive (non-logical) signs of L, and let J 1 = J 1 ;F 1 ;σ J 1 and J 2 = J 2 ;F 2 ;σ J 2 be two structures such that J 1 | T J ∧ J 2 | T J . Accordingly, F 1 ;σ F 1 and F 2 ;σ F 2 are two cofields. A theory of cofields is categorical, that is, there is one and up to isomorphism only one cofield. The structure R;σ R is characterized, up to isomorphism, as the only cofield.
Let i : F 1 → F 2 be the isomorphism from F 1 onto F 2 . We can then define I : J 1 → J 2 by for all S = s,s in J 1 where s,s ∈ F 1 . By definition 4.5, It is straightforward to show that I is an isomorphism from J 1 onto J 2 . This proves that T J is categorical.
That is, the theory T J uniquely characterizes the algebra of J -numbers, and the structure J R ;R;+ J ,× J ;0 J ,1 J is, up to isomorphism, the only possible model of T J . Accordingly, in establishing our assertions about J -numbers, the properties of real numbers are assumed in advance.
By means of definition 4.5 and from the fact that J -numbers are ordered sets of R, the following theorem is derivable (Dawood, 2012 andDawood &Dawood, 2020).
and min and max are respectively the ≤ R -minimal and ≤ R -maximal.
If no confusion is likely, we will often omit the subscripts J and R. It is clear that interval addition, multiplication, and negation are total J -operations, while interval reciprocal is a partial J -operation. As customary, interval subtraction and division are defined respectively as The set-theoretic characterization of interval arithmetic brings to the fore a peculiar feature that seems strange at first. Definition 4.5 entails that a J -operation considers all occurrences of variables as independent (Dawood & Dawood, 2020). Let two J -variables S and T be assigned the same J Provided that images of R-functions are inclusion monotonic (see, e.g., Dawood, 2012 andDawood &Dawood, 2019b), we have the nice enclosure I dep ⊆ I ind , and therefore the result of a J -operation S× J T is a guaranteed interval enclosure of the image of the corresponding R-function. Although this is typically appraised as one of the strengths of interval analysis, in many practical situations, interval enclosures might be too wide to be beneficial. The name of this crucial phenomenon is the interval dependency problem, a concept that we make precise in the next theorem (see Dawood &Dawood, 2019a andDawood &Dawood, 2020).
..,s n ) be a continuous R-function with s i ∈ S i , and let q J (S 1 ,...,S n ) be a Jfunction defined by the same rule as q R . The result of computing the image of the intervals S i under q R , denoted I q (S 1 ,...,S n ), using classical J -operations (definition 4.2), cannot be generally exact if some s i are functionally dependent. That is, What this theorem shows is that the result obtained by the J -function q J is usually overestimated due to the presence of functional dependence. Interval dependency is a 'deep-rooted' problem, dating back to the early works on interval arithmetic. A recent investigation of the logical underpinnings and some ways out of the problem can be found in Dawood & Dawood (2019a) and Dawood & Dawood (2020). A plausible definition and a graphical representation (dependency diagrams) of the dependence of interval variables were also proposed in Shary & Moradi (2021). Plenty of effort has been made to administer feasible remedies. With convenient refined techniques, the interval enclosure q J can be made arbitrarily close to the image I q . By noting regions of monotonicity, one technique is defining the elementary interval functions as the exact images of their corresponding real counterparts. Let n be a nonnegative integer and S = s,s be a J -number. We can define as instances s n ,s n iff s > 0 or n is odd, s n ,s n iff s < 0 and n is even, 0,|S| n iff 0 ∈ S and n is even; where |S| = max{ s ,|s|} is the J -absolute value(or J -modulus) of S.
Performing naive J -arithmetic (theorem 4.2) on these exact images we can get sharper enclosures of their algebraic combinations. Moreover, a diversity of interval methods has been devised to compute narrower interval enclosures. Without pretension to be complete, we can mention the subdivision method, centered forms, circular complex centered forms, generalized centered forms, Hansen's method, remainder forms (see, e.g., Dawood & Dawood, 2019a;Moore, 1979, Rokne & Ratschek, 1984, Kulisch, 2013and Alefeld & Mayer, 2000. For instance, the subdivision method presented by Moore in Moore, 1966 andMoore, 1979 is a celebrated method that can be described as follows. Let S = s,s be a J -number. First, subdivision of S into n subintervals S i is applied such that where (S) = s − s and (S i ) = (S)/n are respectively the widths (lengths) of S and S i . Consequently S = ∪ n i=1 S i . Then, evaluating a J -function q J for each subinterval S i yields the enclosure (Dawood, 2014) As the number n of subintervals gets larger, ∪ n i=1 q J (S i ) gets arbitrarily close to the exact image I q (S). The subdivision method thus gives sharper enclosures than the naive evaluation q J (S). In 'Machine implementation of interval auto-differentiation', we will deploy the subdivision method in order to compute reliable and realistic enclosures of families of real functions and their derivatives.
The characterization of the interval algebraic operations implies a number of familiar algebraic properties. However, being a particular kind of set arithmetic, interval arithmetic (J -arithmetic) has certain peculiar properties involving set inclusion. The singleton intervals 0 J and 1 J are identities for J -addition and J -multiplication, respectively; J -addition and J -multiplication are both associative and commutative; J -addition is cancellative; J -multiplication is cancellative only for all S ⊇ 0 J ; a J -number is invertible for J -addition (respectively, J -multiplication) if and only if it is a singleton J -number (respectively, a nonzero singleton J -number); and J -multiplication left and right Sdistributes over J -addition (see definition 2.6 of 'On theories and structures: some metatheoretical fundamentals'). In other words, in accordance with definition 2.8, the structure J R ;+ J ,× J ;0 J ,1 J of classical J -numbers can be shown to be a commutative S-semiring (Dawood &Dawood, 2019a andDawood &Dawood, 2020).
Throughout this text we will make use of the following theorems (see Dawood, 2014 andDawood &Dawood, 2020). Theorem 4.4 (Inclusion Monotonicity for J -Numbers): Let S 1 , S 2 , T 1 , and T 2 be J -numbers such that S 1 ⊆ T 1 and S 2 ⊆ T 2 . Let • J ∈ {+,×} be a binary J -operation and From inclusion monotonicity, plus the fact that s ∈ S ⇔ [s,s] ⊆ S, if s ∈ S and t ∈ T , then for • J ∈ {+,×} and J ∈ {−, −1 }, we obviously have s• R t ∈ S• J T and R s ∈ J S. At this point, let us introduce an abbreviation that we will make use of. Let s = (s 1 ,...,s i ,...,s n ) be an ordered real n-tuple, and let S = (S 1 ,...,S i ,...,S n ) and T = (T 1 ,...,T i ,...,T n ) be two ordered n-tuples of J -numbers, then

In the following theorem, let [s]≤
(ii) The product S × T is a point J -number iff each of S and T is a point J -number, or at least one of S and T is 0 J , that is

Theorem 4.7 (Zero Divisors in J -Numbers): Nonzero zero divisors do not exist in
Before turning to the axioms of the theory T δJ of a differential J -algebra, it is necessary for our purpose to formalize some analytic concepts within the framework of the theory T J of J -numbers.
Before proceeding any further, let us agree on some basic notation. By an n-ary real function (in short, R-function) we will always mean a function q R : D R ⊆ R n → R, and by an n-ary interval function (in short, J -function) we will always mean a function q J : D J ⊆ J n → J . The R-subscripted symbols q R ,u R ,v R will designate R-functions, while the J -subscripted symbols q J ,u J ,v J will designate J -functions. For simplicity of notation, if the function type is apparent from the type of its variables(arguments), the subscripts '' R'' and '' J '' will usually be dropped. For instance, whenever unambiguous, we use the notations q(s 1 ,...,s n ) and q(S 1 ,...,S n ) for, respectively, an R-function and a J -function, which are both defined by the same rule. For 1 ≤ i ≤ n and 1 ≤ j ≤ k , let S i and A j be respectively J -variable symbols and J -constant symbols. We denote by q J S i:n ;A j:k an n-ary (or multivariate) J -function in the interval variables S i and the interval constants A j . Similarly, we understand by q R s i:n ;a j:k an n-ary R-function in the real variables s i and the real constants a j . For instance, is a binary J -function whose variable arguments are S 1 and S 2 , and whose constants are A 1 and A 2 . With a few exceptions, without loss of generality, the present discussion will be confined to unary functions only. For brevity, therefore, we will often adopt the standard notations q(s) and q(S) respectively for the unary functions q s;a j:k and q S;A j:k . The sets of unary real and interval functions will be denoted by R 1 and J 1 respectively. 7 From the fact that the converse relation q is always definable, the preimage of a function q is always definable, regardless of the definability of the inverse function q −1 .
Next, we define the interval enclosure of a bounded set of real numbers.  [1,3]. An important notion we will need is that of the image set of bounded subsets of R, under an n-ary real-valued function. This notion is a special case of that of the corresponding (n + 1)-ary relation on R. More precisely, we have the following definition. Definition 4.7 (Image of Bounded Real Sets): Let q be an n-ary function on R, and for (s,t ) ∈ q, let s = (s 1 ,...,s n ), with each s i is restricted to vary on a bounded set S i ⊂ R, that is, s is restricted to vary on a set S ⊂ R n . Then, the image of S (or the image of the sets S k ) with respect to q, in symbols I q , is characterized to be The preimage 7 S of T is characterized to be the image of T with respect to the converse q of q. In other words S = I q (T ) = {s ∈ R n |(∃t ∈ T ) t qs }.
Two notions essential for the investigation conducted in this article are those of a family of real functions and its image. Definition 4.8 (Real Family): For 1 ≤ i ≤ n and 1 ≤ j ≤ k , an n-ary real family (a family of n-ary real functions, or in short, an R-family), denoted Q R s i:n ;a j:k , is a set of real functions q R s i:n ;a j:k subject to the following conditions (i) q is a function rule, (ii) s i are variable symbols varying on bounded subsets S i of R, (iii) a j are constant symbols (coefficients) from bounded subsets A j of R, and (iv) for each a j ∈ A j , q R s i:n ;a j:k is continuous on the sets S i . We understand by the converse of Q R , denoted Q R , the set of the converse relations q.
Note that a real family is generated by one function rule, that is, the functions q R s i:n ;a j:k in Q R all have the same rule q but different constant arguments. If the sets A j are singletons, then the family Q R reduces to exactly one n-ary real function. To clarify the matters, we give some examples. (i) Let Q R be the family generated by the function rule q R (s i:2 ;a) = s 2 1 + as 2 , with the variables s 1 and s 2 vary respectively on the bounded sets [2,4[ and [5,6] and the constant a is from the bounded set {3,7}. The family Q R has exactly the two binary functions q R (s i:2 ;3) = s 2 1 + 3s 2 and q R (s i:2 ;7) = s 2 1 + 7s 2 .
(ii) Let U R be the family generated by the function rule u R (s;a) = as 4 , with the variable s varies on the bounded set [2,4[ and the constant a is from the bounded set [1,2]. The family U R has an infinite number of unary functions. Among these are, for example u R (s;1) = s 4 ,u R s; 3 2 = 3s 4 2 ,...,etc.
We characterize the image of a real family as follows. Definition 4.9 (Image of a Real Family): Let Q R be a real family generated by a function rule t = q s i:n ;a j:k , with s i ∈ S i and a j ∈ A j . Then, the image of the family Q R (or the image of the sets S i with respect to Q R ), denoted I Q , is the union of the images of S = (S 1 ,...,S n ) ⊂ R n with respect to each q in Q R for all a j ∈ A j . That is Obviously, for each q in Q R , I q ⊆ I Q . An immediate consequence of definition 4.9 and the well-known extreme value theorem (see Dawood, 2012) is the following important property. Theorem 4.8 (Main Theorem of Image Evaluation): Let Q R be a real family generated by a function rule q s i:n ;a j:k , with s i ∈ S i and a j ∈ A j . If S i and A j are real closed intervals, then the image I Q (S 1 ,...,S n ) of S i , with respect to the family Q R , is in turn a real closed interval such that I Q (S 1 ,...,S n ) =    min s i ∈S i a j ∈A j q s i:n ;a j:k , max s i ∈S i a j ∈A j q s i:n ;a j:k    .
If the sets A j of coefficients are singletons, then the family is in turn a singleton and the image of I Q reduces to the usual image I q of a real function q over real closed intervals By referring to definition 4.6, we can characterize the important notion of the interval extension of a real family. Definition 4.10 (Interval Extension of a Real Family): Let Q R be an n-ary real family generated by a function rule q R s i:n ;a j:k , with s i ∈ S i and a j ∈ A j . We understand by an interval extension of Q R an n-ary interval function q J S i:n ;A j:k of the same rule as q R , and whose arguments are S i = E J (S i ) and A j = E J A j .
Clearly, if S i and A j are real closed intervals, then S i = S i and A j = A j . We will henceforth deploy the predicate Ext q J ,Q R to mean that an interval function q J is the interval extension of the real family Q R , or equivalently, the family Q R is the real intension of the interval function q J . If A j are singletons, then the family Q R is a singleton and we call q J a simple extension of Q R . If S i and A j are singletons, then we call the point-valued interval function q J a point extension of Q R .
The following example will illustrate this point.

The previous discussion faces us with the reasonable question: does every interval function have a real intension?
In order to answer this, we next define what a proper interval function is.

iff it is set-theoretically definable in terms of a real function of the same rule. That is
By definitions 4.11 and 4.5, the following result is derivable.
In accordance with definition 4.11 and its previous consequence, we have then the following important result.

Theorem 4.10 (Intensionality of an Interval Function): An interval function is intensionable iff it is proper. In other words
For example all elementary interval functions are intensionable. On the contrary, degenerate functions such as the midpoint or radius of an interval are not proper and accordingly not intensionable. Definition 4.11 and the deductions from it can be easily generalized to proper J m -valued functions, in which case their intensions will be families of R m -valued functions.
Toward axiomatizing a theory of a differential interval algebra, it remains to formalize the notions of differentiability of a real family and of an interval function. Henceforth, we will consider only families of unary real functions and their interval extensions. Accordingly, when there is no potential for ambiguity, we will write Q(s), or simply Q, for the unary real family Q s;a j:k .
Next, we extend the differential operator to families of unary real functions. Definition 4.12 (Differential Operator for a Real Family): Let Q R s;a j:k be a unary real family in the real variable s and the real constants a j . For a nonnegative integer n, the n-differential operator of Q R s;a j:k , denoted δ n Q R s;a j:k , is defined to be the set of all real functions δ n q s;a j:k for every q ∈ Q R and every constant a j .
We have yet nothing to tell us if a real family is differentiable. The following two definitions introduce, respectively, the notions of differentiability and continuous differentiability of a unary real family Q s;a j:k . Definition 4.13 (Differentiability of a Real Family): A unary real family Q R s;a j:k is n-differentiable at a real constant s 0 , in symbols diff n (Q,s 0 ), iff for every q in Q R , s 0 ∈ dom q , and q is n-differentiable at s 0 . That is diff n (Q,s 0 ) ⇔ ∀q ∈ Q s 0 ∈ dom q ∧ diff n q,s 0 .

Definition 4.14 (Continuous Differentiability of a Real Family):
A unary real family Q R s;a j:k is continuously n-differentiable at a real constant s 0 , in symbols cdiff n (Q,s 0 ), iff for every q in Q R , s 0 ∈ dom q , and q is continuously n-differentiable at s 0 . That is cdiff n (Q,s 0 ) ⇔ ∀q ∈ Q s 0 ∈ dom q ∧ cdiff n q,s 0 .
In accordance with the above concepts, the differential operator for interval functions is then definable. Definition 4.15 (Interval Differential Operator): Let n ≥ 0, and let q(S) be a unary interval function that has a real intension the family Q(s). The n-differential J -operator of q(S), denoted δ n q(S), is characterized to be the interval extension of δ n Q(s). In other words, let δ n Q(s) = U(s), then δ n q(S) = u(S).
In a manner analogous to differentiability in R, the interval differentiability predicate is definable as follows. Definition 4.16 (Interval Differentiability Predicate): Let n ≥ 0, let q ∈ J 1 , and let S 0 ∈ dom q be a J -constant symbol. The ternary n-differentiability J -predicate, denoted diff n q,S 0 , is defined by If diff n q,S 0 is true, then the interval function q is said to be n-differentiable at the closed interval S 0 .
By means of definitions 4.15 and 4.16 plus a simple continuity argument, we have the following theorem that establishes the criteria for interval differentiability. Theorem 4.11 (Interval Differentiability Criteria): An interval function q J ∈ J 1 is n-differentiable at a J -number S 0 if and only if (i) q J (S) is proper with a real intension Q(s), and (ii) Q(s) is continuously n-differentiable at every s 0 ∈ S 0 .
From the fact that images of R-functions are inclusion isotonic (Dawood, 2012), we have the next key result concerning interval enclosures of R-families. Theorem 4.12 (Image Enclosure of a Real Family): Let q(s) be a real function in a family Q(s), with s is restricted to vary on a real closed interval S 0 , and let q(S 0 ) be the interval extension of Q(s) at S 0 . The following two sentences are true.
Moreover, finer enclosures of real families can be obtained via the subdivision method. The following corollary is implied by theorem 4.12. Corollary 4.1 (Subdivision Enclosure of a Real Family): Recall the notation used in theorem 4.12, and let S 0 be subdivided into n ≥ 1 subintervals. Then To the best of our knowledge, in all interval literature, an interval-valued function is assumed to have singleton (real) constants and accordingly an interval function might be only an extension of a single real function. An interesting and important observation from the above discussion is that this presumption introduces an unnecessary restriction to the semantic of an interval function in the general sense. As above characterized, a proper interval function q J S i:n ;A j:k is an extension of a whole family of real functions and this family is a singleton if, and only if, the interval constants A j are singletons.
With the aid of the notions now at hand, we can then axiomatize the theory T δJ of a differential interval algebra (henceforth a differential J -algebra). ;0,1} be a class of non-logical signs, and let T J be the theory of an interval S-semiring J = J ;σ J . The theory T δJ of a differential J -algebra J δ = J ;σ J ;δ is the deductive closure of T J together with the following two axioms.

A CATEGORICAL AXIOMATIZATION OF INTERVAL DIFFERENTIATION ARITHMETIC
Building on the system T δJ of a differential J -algebra axiomatized in the previous section, the present section provides a rigorous mathematical foundation for interval differentiation arithmetic (henceforth J -arithmetic). We are almost ready to lay out an axiom system for the theory T J of interval differentiation numbers (henceforth J -numbers) as a two-sorted extension of T δJ . By virtue of the mathematical underpinnings presented in 'A differential interval algebra', we axiomatize, in the present section, the basic operations of T J and prove some of their fundamental properties. Moreover, we prove categoricity and consistency of J -arithmetic.
An obvious starting point is to define interval differentiation n-tuples. Definition 5.1 (Interval Differentiation n-Tuples): Let J δ = J ;σ J ;δ be a differential J -algebra, let q be a unary J -function, and for an integer n ≥ 0, let J n be the n-th Cartesian power of J . The set of all interval differentiation n-tuples over J , with respect to an individual J -constant S 0 ∈ J , is characterized to be An interval differentiation n-tuple is thus an ordered n-tuple of J -constants. Hereafter, we will usually write q, δq,..., δ n q for δ 0 q(S 0 ), δ 1 q(S 0 ), . . . , δ n q(S 0 ), respectively. The present article is concerned with dyadic interval differentiation tuples, that is n-tuples with n = 1; and we will hereon adopt the name ''interval differentiation numbers'' ('' J -numbers'', or '' J -pairs '') for dyadic interval differentiation tuples. Let U J designate the set of Jnumbers at some J -constant S 0 , and let the letters Q, U, and V , or equivalently the pairs q,δq S 0 , (u,δu) S 0 , and (v,δv) S 0 , be variable symbols varying on the set U J of J -pairs. Also, let the letters A, B, and C, or equivalently (a,0 J ) S 0 , (b,0 J ) S 0 , and (c,0 J ) S 0 , designate constants of U J . In particular, we use 1 U J to designate the J -number (1 J ,0 J ) S 0 and 0 U J to designate the J -number (0 J ,0 J ) S 0 . Moreover, it is convenient for our purpose to define a proper subset of U J as We are now ready to axiomatize the theory T J of an interval differentiation algebra (or a J -algebra) over an interval S-semiring. ;0,1} be a class of non-logical signs, and let q,δq S 0 , (u,δu) S 0 , and (v,δv) S 0 be in U J . An interval differentiation algebra (or, in short, a J -algebra) over a differential J -algebra J d = J ;σ J ;δ is a two-sorted structure U J = U J ;J ;σ U J . The theory T J of U J is the deductive closure of the system T δJ of J d and the following set of axioms.
8 Categoricity is a bedrock of mathematics. For further details on the key role of categoricity in logic and mathematics, see, e.g., Corcoran, 1980, Dawood & Megahed, 2019, and Shapiro, 1985 Proof Since the theory T J of a J -algebra has the model J R ;σ J of J -numbers, it follows that the theory T J has a model U J ;J ;σ U J , and thus existence of a J -algebra is proved.
Theorem 5.4 (Categoricity of J -Arithmetic): The theory T J of J -numbers is categorical.
Proof The theorem follows from the categoricity of the theory T J of interval algebra by an argument analogous to the one used in theorem 4.1.
That is, the theory T J uniquely characterizes the algebra of J -numbers, and the structure U J ;+ U J ,× U J ;0 U J ,1 U J is, up to isomorphism, the only possible model of T J . To reiterate, in accord to theorem 5.4, the system T J , axiomatized in definition 5.2, is the ''best'' axiomatization of J -numbers, in the sense that it rightly accounts, up to isomorphism, for every structure of J -arithmetic. 8 The next theorem establishes the consistency of the theory T J of J -numbers. Theorem 5.5 (Consistency of J -Arithmetic): The theory T J of J -numbers is consistent.
Proof In accord to definition 2.4, the proof is immediate from theorem 5.3. The theory T J is satisfiable by the model U J ;+ U J ,× U J ;0 U J ,1 U J and thus is consistent.
Owing to the categoricity theorem for T J , the algebraic properties of J -numbers are naturally assumed priori. Therefore, whenever unambiguous, hereon we will use these properties without further mention.
Noteworthy, by virtue of the theory developed so far, we have the profound results that each J -number represents a guaranteed interval enclosure of the image of a whole family of R-functions and their derivatives and accordingly that a J -number is an interval extension of every R -number that corresponds to each function in the real family (See 'Machine implementation of interval auto-differentiation' for clarifying numerical examples). In consequence of theorem 4.12, these important results are made precise in the following immediate theorem and its corollary. Theorem 5.6 (Differential Enclosure of a Real Family): Let Q be a unary real family continuously differentiable on a real closed interval S 0 and let q J be its interval extension. Then, for every q R in Q

Dawood and Megahed
Finally, let us note that we can get sharper enclosures of the pair I Q (S 0 ),I δQ (S 0 ) with the aid of the subdivision method. In consequence of theorems 5.2 and 5.6 we are led to the following theorem. Theorem 5.7 (Subdivision Theorem for J -Numbers): Recall the notation used in theorem 5.6, and let S 0 be subdivided into n ≥ 1 subintervals. Then

DIFFERENTIATION EXTENSION OF INTERVAL FUNCTIONS AND HIGHER-ORDER AUTO-DIFFERENTIABILITY
We aim to fully address and compute higher order and partial auto-derivatives using only dyadic J -numbers ( J -pairs), and without resorting to defining any sort of n-dimensional Grassmann algebras. Towards this end, we are to extend the theory T J , by introducing the notion of a differentiation extension of J -functions, characterizing differentiability for J -numbers, and establishing their differentiability conditions.
In view of our definition of J -numbers, the following alternate characterization of interval differentiability is at our disposal.
In order to have J -functions beyond the rational functions defined in 'A categorical axiomatization of interval differentiation arithmetic', an extension principle should be introduced. Thus we require to extend J -functions to J -functions. In accord to the above characterization, we have the next definition. Definition 6.1 (Differentiation Extensions of J -Functions): For k ∈ {1,...,n}, let u k ∈ J 1 be differentiable at S 0 ∈ dom(u k ), that is for each u k there is U k = (u k (S 0 ),δu k (S 0 )) ∈ U J . Let Q J (u 1 ,...,u n ) be an n-place J -function of u 1 ,...,u n which is differentiable at S 0 . A differentiation extension of Q J is an n-place J -function Q U J U 1 ,...,U n defined to be Q U J U 1 ,...,U n = (Q J (u 1 ,...,u n ),δQ J (u 1 ,...,u n )), and obtained from Q J by replacing, in Q J , each occurrence of a J -function symbol u k by the corresponding J -variable symbol U k .
The definition is so framed that since diff 1 (Q J ,S 0 ) is true, the differentiation extension Q U J of Q J is in U J . Thus, Q J and Q U J are both defined by the same symbolic rule but with different types of arguments (variables); Q J is a J -function whereas Q U J is a J -function. By analogy with rational J -functions, a rational J -function is a (multivariate) J -function obtained by the application of a finite number of the binary and unary algebraic J -operations • U J ∈ {+ U J ,× U J } and U J ∈ {− U J , −1 U J }. Hereon, if the function type is apparent from the context, the subscripts J and U J will be omitted. For instance, whenever unambiguous, we use the notations Q(u 1 ,...,u n ) and Q U 1 ,...,U n for, respectively, a J -function and its differentiation extension.
Here it will suffice to give an example. Let the J -functions u 1 (S) = cosS and u 2 (S) = S 3 be both differentiable at some S 0 , and let Q J (u 1 ,u 2 ) be differentiable at S 0 such that The differentiation extension of Q J is then By virtue of our definition of the extension principle for J -functions (definition 6.1), we are able to define fundamental J -functions. For example, replacing Q by the ''cos'' function, one obtains the trigonometric J -function cos(u,δu) S 0 = (cos(u),δcos(u)) S 0 . In 'Machine implementation of interval auto-differentiation', we will give further discussion on differentiation extensions of J -functions as well as more illustrative numerical examples.
Here, let us stress that restricting our discussion to single-variable J -functions is not a loss of generality, since an n-variable J -function can be viewed as a class of n single-variable J -functions. What is noteworthy in addition is that higher-order interval auto-derivatives can be computed in the framework of our system T J of dyadic J -numbers ( J -pairs). With the aid of definition 6.1, we next characterize the n-differential operator and the n-differentiability predicate for J -pairs. Definition 6.2 (n-Differential Operator of a J -Number): For an integer n ≥ 0, the ndifferential operator of a J -pair U = (u,δu) ∈ U J , in symbols δ n U, can be characterized recursively by (i) δ 0 U = U, (ii) δ 1 U = (δu,δ(δu)) = δu,δ 2 u = δ 1 δ 0 U, (iii) n ≥ 1 ⇒ δ n U = δ n u,δ n+1 u = δ 1 δ n−1 U. readily available Leibniz's rules for J -numbers. By definitions 6.2 and 4.17, plus theorem 5.2, the following theorem is derivable. Theorem 23 (Leibniz's Rules for J -Numbers): Let Q and U be J -numbers. Then By virtue of this theorem, and applying induction, the general Leibniz rule for Jnumbers can be easily established. Let Q and U be n-times differentiable J -numbers. Then A nice consequence that we wish to point out is that with the general Leibniz rule for Jnumbers at our disposal, and once we have in our machine implementation differentiation kernels (seeds) for the higher order dyads (u,δu), δu,δ 2 u , . . . , δ n u,δ n+1 u , it is readily possible to compute higher order auto-derivatives by doing only dyadic J -arithmetic. In other words, within the framework of the theory T J of dyadic J -numbers, higher order auto-differentiation is directly realizable without resorting to defining a Grassmann algebra for n-ary vectors of the form (u,δu,...,δ n u). This, along with some illustrative examples, will be discussed further in 'Machine implementation of interval auto-differentiation'.
Note also that considering only single-variable J -functions is not a loss of generality, since an n-variable J -function can be viewed as a class of n single-variable J -functions. Accordingly, partial auto-derivatives, gradients and Hessians are readily computable.
Finally, let us conclude this section with a few additional comments. In accord to definition 5.5, in the theory T J of J -numbers, a singleton J -number defines a Rnumber. That is, all the results of this section apply to R -arithmetic as well. Moreover, computing the J -number q,δq S 0 is very useful in practice. In engineering and physical sciences, a recurring problem is to compute the derivatives under parametric uncertainty (For further details, the reader may consult, e.g., Dawood, 2014, Dawood & Dawood, 2022Kulisch, 2013;Moore, 1966, Neidinger, 2010Sommer, Pradalier &Furgale, 2016, andTingelstad &Egeland, 2017). Also noteworthy here is that with a few basic modifications, the categorical system T J axiomatized in this text can be extended analogously to compute fuzzy auto-derivatives (For further details on fuzzy analysis, see, e.g., Goetschel &Voxman, 1986 andPuri &Ralescu, 1983).

THE ALGEBRAIC STRUCTURE OF INTERVAL DIFFERENTI-ATION ARITHMETIC
Building on the parts of the theory established in 'A differential interval algebra' and 'A categorical axiomatization of interval differentiation arithmetic', this section provides a detailed investigation of the algebraic structure of J -numbers. By virtue of the categoricity of the theory T J (theorem 5.4), the properties of J -numbers are assumed priori.
We commence this section by establishing the algebraic properties of J -addition and J -multiplication. Theorem 7.1 (Algebraic Properties of J -Addition): The following algebraic properties hold for J -addition.
Proof The proof for (i) follows from theorem 5.2. (ii) follows from theorems 4.6 and 5.2. By cancellativity, commutativity and associativity of J -addition, (iii), (iv) and (v) are easily provable by theorem 5.2.

Theorem 7.2 (Algebraic Properties of J -Multiplication):
The following algebraic properties hold for J -multiplication.
Proof The proof for (i) and (ii) follows immediately from theorem 5.2. For (iii), assume that Q × U = 1 U J = ([1],[0]), which yields, by theorems 5.2, 4.6, and the invertibility properties of J -arithmetic, that Q ∈ U [q] ∧ U = Q −1 ∧ 0 ∈ q(S 0 ). The converse direction is easily derivable by assuming the right hand side. By the cancellative properties of Jarithmetic, (iv) follows from theorems 5.2 and 4.2. By commutativity of J -multiplication, (v) is easily derivable from theorem 5.2.
Thus, not all elements of U J are invertible for addition or multiplication. A J -number Q is invertible for addition if, and only if, it is a point J -number and is invertible for multiplication if, and only if, it is a point J -number with 0 ∈ q(S 0 ). Also, unlike interval arithmetic, J -arithmetic has nonzero zero divisors, since [0], α,α × [0], β,β = 0 U J . Moreover, J -multiplication is not associative, which is figured in the following theorem. Theorem 7.3 (Associativity of J -Multiplication): In general, J -multiplication is not associative. That is

MONOTONICITY AND ISOMORPHISM THEOREMS FOR INTERVAL DIFFERENTIATION NUMBERS
In this section, some monotonicity and isomorphism theorems for J -numbers are established, and finally a corollary concerning the structure of R -numbers is entailed. A first key result we will next prove is the inclusion monotonicity theorem for J -arithmetic, which establishes that the inclusion relation is compatible with the algebraic J -operations. Theorem 8.1 (Inclusion Monotonicity for J -Numbers): Let Q 1 , Q 2 , U 1 , and U 2 be J -numbers such that Q 1 ⊆ U 1 and Q 2 ⊆ U 2 . Let • ∈ {+,×} be a binary J -operation and ∈ −, −1 be a definable unary J -operation. Then Proof By hypothesis, we have Q 1 ⊆ U 1 and Q 2 ⊆ U 2 . Then, according to definition 5.3 and theorem 4.4, we have Analogously, Q 1 ×Q 2 ⊆ U 1 ×U 2 and Q 1 ⊆ U 1 . This completes the proof.
In consequence of this theorem, from the fact that q(s 0 ) , δq(s 0 ) ⊆ Q ⇔ q ∈ Q, we have the following important special case. Corollary 8.1 (Membership Monotonicity for J -Numbers): Let Q and U be J -numbers with q ∈ Q and u ∈ U . Let • ∈ {+,×} be a binary J -operation and ∈ −, −1 be a definable unary J -operation. Then Two important results, concerning isomorphism theorems for J -arithmetic, are figured in the following theorems. Theorem 8.2 (Isomorphicity to R -Numbers): Proof Let ι : U R →U [q] be the mapping from U R to U [q] given by ι(q) = q = q , δq .
By means of the fact that point intervals are isomorphic to real numbers (theorem 4.5), it is straightforward to show that ι is an isomorphism from U R onto U [q] .
Accordingly, up to isomorphism, the sets J R and U (J ,0) are equivalent, and therefore the structure U (J ,0) is a commutative S-semiring.
In consequence of theorems 7.9 and 8.2, we have the following corollary concerning the structure of R -numbers. Corollary 8.2 (Commutative Ring of R -Numbers): The algebra U R = U R ;+ U R ,× U R ; 0 U R ,1 U R of R -numbers is a commutative unital ring in which every element whose first component is nonzero has an inverse for × U R .

MACHINE IMPLEMENTATION OF INTERVAL AUTO-DIFFERENTIATION
In this last section, we consider some aspects of the computational implementation of interval auto-differentiation in the framework of our theory T J of J -numbers ( J -pairs). The algorithm of the theory is coded in Common Lisp as a part of the software package InCLosure (InCL) (Dawood, 2020 andDawood, 2023). After providing a mathematical flavor of the algorithm, 9 we offer insights of the theory by giving some simple examples that illustrate how to concurrently compute guaranteed enclosures of images of families of real functions and their derivatives, then we deal with a more sophisticated problem whose result values will be calculated to an arbitrary precision using InCL commands, and finally, we give a brief account of how to calculate higher order interval auto-derivatives using the theory T J of dyadic J -arithmetic.
In a way analogous to that of R -arithmetic (see, e.g., Dawood, 2014 andDawood &Megahed, 2019), J -arithmetic can be machine realized. Toward calculating the J -pair of a differentiable J -function at S 0 ∈ J , we begin with a minimal class of symbolic rules of differentiable J -functions and their derivatives which acts as J -seeds ( J -kernels) for carrying out the computation. As examples of J -kernels, one can start with the following elementary J -pairs. AS b ,AbS b−1 , (ln(S),1/S), e S ,e S , (sin(S),cos(S)), (cos(S),−sin(S)), and so forth.
The class of unary J -kernels will be denoted by P 1 , and we will understand by J -kernels a class K 1 = q ∈ J 1 | q,δq ∈ P 1 . Accordingly, the first-order interval auto-derivative of a J -function q(S), at S 0 ∈ J , can be viewed as To further illustrate, we next give some examples that can be worked by hand. Computing the J -pair q,δq [1,4] yields q,δq [1,4]    The J -pair q,δq [−1,2]  The previous example clearly embodies that the theory T J presented in this text is powerful and reliable for simultaneously providing guaranteed enclosures of families of real functions and their derivatives. For instance, the following real functions and their derivatives are members of the families Q R and δQ R of example 9.2 respectively. q 1 (s) = s 3 + s 2 + 2,q 2 (s) = −s 3 δq 1 (s) = 3s 2 + 2s, δq 2 (s) = −3s 2 .
The exact images of these functions are included in the result of example 9.2 as follows. The overestimation of the exact image of δQ R in example 9.2 naturally arises from the interval dependency problem. One noteworthy virtue of the subdivision theorem for J -numbers (theorem 5.7) is that one can deploy the subdivision method to decrease the overestimation and hence obtain arbitrarily sharper intervals that get closer to the exact image. The next example clarifies the matters. We desire to compute the J -pair q,δq [−1,2]  In the previous example, the subdivision technique with only three subintervals yields the exact images. In some problems, when the result is far away from the exact images, increasing the number of subintervals could give arbitrarily better enclosures of the images but with the disadvantage of increased computational time. Now, we move on to compute,  [−74958.36726177567900543373,71542.87733701227696496573] to an arbitrarily sharper intervals, the result of a more sophisticated example using the software package InCLosure. The package InCLosure guarantees arbitrarily sharper intervals which are restricted only by the machine's computational capabilities. The J -pair q,δq [−1,2] for the J -function q at [−1,2] can be computed using the following InCL command.
Example 9.4 shows that computing auto-derivatives under interval uncertainty, with the arbitrarily sharper and guaranteed results of InCLosure, is competitive and obviously preferable to the ordinary numerical approximation methods.
In closing, let us get a grip on how to compute higher-order interval auto-derivatives in the framework of our theory T J of dyadic J -numbers ( J -pairs). By virtue of Leibniz's rules for J -numbers (theorem 6.2), and once we have extended the class P 1 of J -kernels by including the higher order dyads (u,δu), δu,δ 2 u , . . . , δ n u,δ n+1 u , for an arbitrary n, we can compute higher order interval auto-derivatives by doing only dyadic J -arithmetic. Consequently, within our own development, one can implement higher order interval auto-differentiation without resorting to defining any sort of n-dimensional Grassmann algebra for n-ary vectors of the form (u,δu,...,δ n u). With this in mind, let us consider the J -function q(S) = cos S 2 + ln(S).

CONCLUSION
As we detailed in the introduction and elsewhere, combining subtlety of ordinary automatic differentiation with reliability of interval mathematics results in an intervalized version of algorithmic differentiation, namely ''interval differentiation arithmetic'', which so markedly surpasses its ordinary counterpart in power and reliability. With the aid of interval mathematics, automatic differentiation can be intervalized to handle uncertainty in quantifiable properties of real world physical systems and accordingly provide the computational methods that suffice to deal with the important problem of ''getting guaranteed bounds'' of images of real functions and their derivatives; and so, this article has been devoted to recasting interval differentiation arithmetic in a formalized theory, by putting into a systematic form its fundamental notions, and thus attaining the advantage of a concrete algebraic foundation that has then enabled us to extend the theory in such a manner that adds to its power, reliability, and applicability.
In the first place, after formalizing some set-theoretical and logical notions of particular importance for our purpose, we gave an axiomatization of a theory of a differential interval algebra and then we presented the notion of an interval extension of a family of real functions, together with some analytic notions of interval functions. Secondly, we set up an axiomatic theory of interval differentiation arithmetic, as a two-sorted extension of the theory of a differential interval algebra, and then we gave the proofs for its categoricity and consistency. We consequently constructed the algebraic system of interval differentiation arithmetic, deduced its fundamental properties, and showed that it constitutes a multiplicatively non-associative S-semiring in which multiplication is subalternative and flexible. Then, we established some monotonicity and isomorphism theorems for interval differentiation numbers and proved a result concerning the structure of real differentiation numbers. And, lastly, we gave a brief account of the computational implementation of interval differentiation arithmetic and showed how to concurrently compute guaranteed enclosures of images of both families of real functions and their first and higher order derivatives.
From the very beginning, our axiomatic system included the notion of an interval extension of a family of real functions and the differentiability criteria thereof. Also, our construction differs in that we did not make use of Clifford's dual numbers or Grassmann numbers that are repeatedly 'borrowed' or 'reinvented' in the literature as proposed algebraic characterizations respectively for first and higher-order algorithmic differentiation. Moreover, a well-known fact of logic is that a ''categorical'' formalization of a theory is the ''best'' characterization thereof. By dint of being categorical, the axiomatic theory presented in this work lays serious claim to being ''best'' in the sense that it rightly accounts, up to isomorphism, for all structures of interval differentiation arithmetic. Furthermore, along the course to the main business of this study, a number of useful notions have been introduced and formalized within the context of the proposed theory. Among these, we can mention interval enclosure of a bounded set, interval extension of a real family, proper interval functions and the criteria thereof, differentiability and continuous differentiability of a real family, interval differentiability criteria, differential enclosure of a real family, differentiation extension of an interval function, and differentiability criterion for interval differentiation numbers.
We would also remark that, on the strength of our axiomatization, many nice consequences come for free: categoricity and consistency of both the theory of interval algebra and the theory of interval differentiation algebra follow immediately, criteria for differentiability of families of real functions and their interval extensions are easily established, and the algebras of intervals and real differentiation numbers are both isomorphically embedded in the algebra of interval differentiation numbers.
The main contribution is therefore both a ''logico-algebraic formalization'' and an ''extension'' of interval differentiation arithmetic. The article provides an axiomatization of a comprehensive algebraic theory of interval differentiation arithmetic based on clear and distinct elementary ideas of real and interval algebras. We extend this formalized theory in two directions. On the one hand, although we made use of neither Clifford's dual numbers nor Grassmann hyper-dual numbers, our new formalization of dyadic interval differentiation numbers fully addresses interval auto-derivatives of first and higher order. On the other hand, by virtue of introducing the notion of an interval extension of a family of real functions, the theory is extended to provide the mathematical tools to get guaranteed enclosures of the images of families of real functions and their derivatives. Noteworthy also is that with a few basic modifications, the categorical system axiomatized in this text can be extended analogously to compute fuzzy auto-derivatives. Nevertheless, despite all the aforementioned advantages, guaranteed interval enclosures come at a price: the interval subdivision method could be computationally inefficient when manipulating problems involving thousands of uncertain quantities. Fortunately, there are many ways out of this problems. Among these, we mention, without pretension to be complete, Hansen's centered forms, remainder forms, Kulisch's complete intervals, Kaucher intervals, and Dawood's universal intervals (For further details, see, e.g., Dawood & Dawood, 2019a;Dawood & Dawood, 2020;Dawood &Dawood, 2022, andShary &Moradi, 2021).
In conclusion, the ''self-validating'' feature of interval automatic differentiation makes it useful and applicable in a wide range of scientific fields. In engineering and physical sciences, a recurring problem is to compute the derivatives under parametric uncertainty. In this regard, an intervalized theory of algorithmic differentiation is believed to be very useful for manipulating problems involving quantifiable uncertainties. The system proposed in this article provides a rigorous and extended mathematical foundation for both real and interval automatic differentiation. Being both a formalization and an extension of differentiation arithmetic as it is currently practised, the authors believe that such a formalization, hopefully, might have a worthwhile impact on both theoretical research and real world applications, with computational advantages for the solutions of new types of practical problems which can be expressed in terms of the mathematical machinery presented in the body of this article.