Computational modeling of consistent observation of asynchronous distributed computation on N – manifold

: The present-day distributed computing systems are asynchronous in nature, and they cover heterogeneous nodes as well as networks having geographic distribution scale. These distributed systems are prone to unpredictable network partitioning, communication delay, and node failures. The realizations of consistent observation in such systems are challenging. The traditional models of distributed computation are not fully adequate to incorporate characteristics arising in the new paradigm. Alternatively, the homology and topology based distributed computing models are formulated to gain a new perspective. This paper proposes a computational model of consistent observation of asynchronous distributed computation on N -manifold. The proposed model offers control of granularity of fineness of observation of computation to varying degree. The discrete geometric simulations of computational structures on 3-manifold offer different analytical insights. The extracted lattice chains of distributed computation on manifold illustrate execution dynamics of the system under consideration

scale distributed computing systems are difficult to deploy maintaining stability and consistency without employing any monitor. The observation of distributed computation is an important factor in ensuring stability of computation (Fidge, 1996;Parlangeli & Notarstefano, 2012). Furthermore, correctness of distributed computation can be determined by incorporating consistency detection mechanism in a sequence of observations of computation (Babaoglu & Marzullo, 1993;Goldberg, Gopal, Lowry, & Strom, 1991). In large-scale mobile distributed systems involving wireless sensor networks, the network partitioning is unpredictable, and it results in random failures of distributed computation on aggregated data. Thus, the applications of monitor to enhance observability and consistency of distributed computation are required in order to generate stable output. The traditional models of distributed computations are constructed by using elements of graph theory and computational logic based formalisms (Garg, Agarwal, & Ogale, 2014;Schwarz & Mattern, 1994). However, a relatively new approach to model distributed computing systems is to employ concepts of algebraic topology and homology (Bauer, Kerber, & Reininghaus, 2014;Conde & Rajsbaum, 2012;Goubault, 2003). The topological models of distributed computing offer analytical insight to complex systems in new perspectives. The topological and homological models help in analyzing distributed systems having large event space, which results in formation of very large space of combinatorial executions. The manifold is a multidimensional space, which can be represented in geometric forms having certain characteristics. The construction of models of consistent observability of asynchronous distributed systems on a manifold would offer analytical insight to the sequences of executions represented as snapshots. Moreover, granularity of fineness of observation can be controlled in such asynchronous systems appropriately.

Motivation
The consistent observation of sequence of computation in a distributed system is critical to determine correctness of execution (Babaoglu & Marzullo, 1993). The determination of ordering of events in a distributed system is difficult due to enlarged combinatorial execution space of computation. As a result, the observability of distributed systems becomes difficult (Fidge, 1996). The recovery of distributed computation is often required if a fault is detected in the execution sequence. The mechanism of restoring consistent states in a large-scale distributed system involves persistent logging of multiple checkpoints for recovery (Goldberg et al., 1991). However, this mechanism enhances space complexity to a large extent if the combinatorial execution state space is large. Furthermore, determining correct combinatorial execution checkpoint is computationally expensive in a distributed system having a large state space. The realization of monitor of distributed computation is not trivial due to the existence of causal relations between any two pair of events (Schwarz & Mattern, 1994).
The formal modeling of computing systems enables to gain an analytical insight to the complex systems, which facilitates robust designs. For example, a formal specification of dependability of computation in large-scale pervasive systems is constructed (Ayara & Najjar, 2008). The stable predicate detection in distributed computation having infinite run is formulated by restricting the run within finite graph model (Garg et al., 2014). However, constraining the infinite computation on finite structure may invite oscillatory execution pattern incorporating indeterminate convergence. A comparatively new approach to model distributed computing systems involves elements of algebraic topology, lattice theory, and homological algebra offering new insights Bauer et al., 2014;Conde & Rajsbaum, 2012;Herlihy & Rajsbaum, 1999). This paper proposes a formal model of consistent observation of a distributed computation on N-manifold. The proposed model does not impose any condition of finiteness of computation. The variable filter function can control the granularity of observation of states in a sequence of computation while maintaining consistency conditions. The main contributions of this paper are as follows.
• Construction of a formal model of observation of asynchronous distributed computing on N-manifold structure • Integrating filtering method to generate consistent observation of distributed computation • Incorporation of variability of filter to prepare consistent observation of computation with varying granularity • Evaluation of resulting manifold structure of distributed computation in 3-D and lattice chain embedding on it The rest of the paper is organized as follows. Section 2 presents related work. Section 3 describes preliminary concepts. Sections 4 and 5 present construction of distributed computing on N-manifold structure and formulation of consistent observation model, respectively. Section 6 presents analytical properties. Section 7 and 8 describe computational evaluation results and application aspects, respectively. Finally, section 9 concludes the paper.

Related work
In general, the formulation of analytical model of distributed computing is performed by using modular graph structures and relational algebraic operators (Rhode, Presicce, Simeoni, & Taentzer, 1999;Sitohang, 2002). The observations of a distributed computation in multiple phases are required in order to determine the controllability of the system. In theory of distributed computing, consistent observation of sequences of computation is a critical factor (Babaoglu & Marzullo, 1993). In general, observability of a distributed system is hard to realize due to combinatorial arrival order of events (Fidge, 1996). A distributed computation can be modeled as a cyclic graph, where nodes represent distributed processes and edges represent networks between processes. The graph structures can be dynamic, and hybrid distributed systems are modeled by using dynamic graphs structures (Kuhn, Lynch, & Oshman, 2010). It is proposed that number theoretic approaches can be successfully applied to determine observability and reachability of paths as well as cycles in an arbitrary graph (Parlangeli & Notarstefano, 2012). Furthermore, the unsolvable systems can be recognized by Eigen analysis. The main challenge of this model is that, detection of a prime number is necessary to realize observability of cycles in the graph, which is computationally expensive. This is because the prime number detection is computationally hard.
The debugging of distributed computation requires rollback recovery in the presence of faults. Researchers have proposed a rollback and reply mechanism to realize restoration of consistent global states in a distributed system (Goldberg et al., 1991). The design requires records of persistent logs and stable checkpoints, which enhances space complexity to a large extent if the combinatorial execution state space is large. Moreover, the determination of causal relationship between events in the event space of a distributed computing is necessary in order to implement computation monitor (Schwarz & Mattern, 1994). The topological models of distributed computing offer an analytical insight to the complex systems. In recent times, distributed computing systems are modeled by employing combinatorial topological structures (Armstrong, 1983;Edelsbrunner & Harer, 2010;Herlihy & Rajsbaum, 1999). The shared memory based distributed computing model is formulated by employing algebraic topology (Conde & Rajsbaum, 2012). The conceptual framework of persistent homology is utilized in modeling distributed computation (Bauer et al., 2014;Zomorodian & Carlsson, 2005). The homotopy theory and topological spaces are employed in modeling the mutual exclusion as well as its complexity in concurrent computing systems (Carson & Reynolds, 1987;Fajstrup, Rauben, & Goubault, 2006;Goubault, 2003;Gunawardena, 1994). However, the homotopy theory needs adaptation while formulating distributed systems due to directional property of homotopy theory.

Preliminary concepts
Let X be a point set and τ x ΩðXÞ where ΩðXÞ is a power set of X. If the 1-D space represented by X is Euclidean, then the space can be metrized to generate a metric space ðX; d X Þ by equipping it with a real-valued function, d X : X 2 ! ½0; þ1Þ. The resulting space ðX; d X Þ is a 1-D finite metric space having size n ¼ X j j if following axioms are maintained: n 2 ð1; þ1Þ; "a; b; c 2 X; d X ða; bÞ ¼ d X ðb; aÞ; If τ x represents a topology on X, then the resulting topological space induced in the metric space is ðX; d X ; τ x Þ, where "A i 2 τ x ; ðA i ; d X Þ constructs metric subspace. The metric space having topological structure ðX; d X ; τ x Þ maintains following properties: The properties of generalized topological space state that the space is closed under arbitrary union as well as intersection operations. However, the intersection operation should be finite. Let, N 2 Z þ n 1 f g; f i : B i ! < N be the homeomorphic and invertible function operated on the topological space B i X. If X is Hausdorff, then the resulting structure ðB i ; f i Þ is called a chart on N-manifold (Lee, 2013). If ðB i & X; f i Þ and ðB k & X; f k Þ are two charts on N-manifold, then a transition map can be constructed as, It can be possible that, an N-manifold is smooth to a degree v< þ 1. Thus, different classes of manifolds can be formed depending on respective natures.

Distributed computing on N-manifold
The set of events local to a distributed process p i 2 P is denoted by E i , where ϕ 2 E i . Let E P be N-dimensional given by If "p i 2 P; h i : E i ! C P ðE P Þ is invertible, then a N-manifold M N is given as, In the proposed model of distributed computation on N-manifold, the underlying manifold structure is not differentiable due to discrete nature of computing space represented by T E .

Hausdorff property of computing space
The distributed computing space can be metrized to form metric space by equipping it with a suitable function . Let a discrete metric d S : T 2 E ! ½0; þ1Þ be defined on distributed computing space T E as, The resulting computing space ðT E ; d S Þ is Hausdorff because the following axiom is maintained by it, 9ε 2 ð0; 1Þ; aÞb : This validates that the distributed computing space T E is metrizable Hausdorff space, which can be mapped on an N-manifold.

Locally Euclidean space of computation
The set of events local to individual distributed processes can be counted by employing locally applicable monotone logical clock function defined over respective intervals as, The logical clock function Cð:Þ is a local bijection. Hence, the local event spaces of distributed processes can be converted into respective metric spaces þ1Þ. This leads to the restriction to local formulation as, The resulting metrization function is defined as, This indicates that ðE i ; d E Þ is a Euclidean metric space of the asynchronous distributed computation under consideration.

Computation observation on manifold
The observability of distributed computation is an important parameter in order to determine stability of systems and to measure control dynamics. The observability of a distributed system enables implementation of consistency detection in execution sequence as well as fault recognition. If a distributed system is designed with possibility of faults other than Fail-Stop mode, then consistent observation of distributed computation is required to ensure error-free execution. This section presents the model of consistent observation of distributed computation on N-manifold having varying granularity of observations.

The ε k -fine cut of computation
Given a distributed computation on M N , the computation is not consistent everywhere on M N . The consistency of a computation can be verified by considering computational cuts on M N preserving snapshot of execution status of distributed processes. A set of N-dimensional cuts on M N is given The ε k -cut of Λ C is a restriction on taking snapshots of execution on M N preserving consistency of observation of distributed computation. The ε k -cut (Λ k ) is defined by following axioms for ε k ; k 2 Z þ , Evidently, "ε k >0, the Λ k may not provide meaningful information about execution states of computation. This is due to the fact that, a widely dispersed cut on M N for a sufficiently large ε k would violate consistency of observation of computation considering varying execution states of a group of processes in asynchronous distributed computing systems. If S ε ¼ ε k : k & Z þ f grepresents a set of choices to generate ε k -cut on M N , then an acceptable value to perform consistent observation in finite form is given by Thus, the finest possible observation Λ d of an asynchronous distributed computation can be performed by maintaining the following axioms, The coarse grained observation can be performed by considering ε e ¼ supðS ε Þ and d E ða i ; a k 2 E k Þ ε e . Hence, the variation of fineness of observation of asynchronous distributed computation can be performed on N-manifold.

Consistency and filter model
The manifold M N generated by distributed computing is not consistent everywhere within the space. Thus, an arbitrary function sequence on N-manifold of asynchronous distributed computation, given by F N ¼ ðh i Þ i¼N i¼1 , may not be computationally consistent everywhere. The consistency can be maintained by employing a filter function on the N-manifold. Let all possible cuts of execution on N-manifold be given by Λ Λ k . A predicate ΓðβÞ 2 0; 1 f g determines consistency of cut β 2 E P by evaluating the predicate in corresponding event space given as, Any generalized and arbitrary cuts of distributed computation are not observably consistent. Hence, a filter is required to generate consistent cuts out of a set of cuts of computation. Let us assume that ΓðβÞ ¼ 1 signifies consistent cut on N-manifold. Let a function be given as, The function gð:Þ is a filter function on cuts on execution manifold if it satisfies the following axioms, A Λ M ; "β 2 E Γ : ½ΓðβÞ ¼ 1 ) ½ðg C P ÞðβÞ 2 B; The filter function gð:Þ checks the validity of stable predicate while filtering out inconsistent cuts on manifold. The granularity of observable consistent cuts is determined by restricting the variations of clocks within cuts having finite limits. This indicates the non-commutative composition, "h i 2 H P ; g h i ð:Þ, can be further restricted to generate a set of consistent observation on N-manifold at appropriate granularity having ε k -fineness by maintaining following conditions, In another view, it can be considered as an additional refinement to control the granularity of observation of computation.

Lattice embedding on N-manifold
Let the ε k -fine and consistent snapshot of observation of a distributed computation on M N be B k .
Let a partial ordering relation < & B 2 k be constructed on M N maintaining lattice properties. If L B k , then a lattice chain ðL; <Þ can be formed by maintaining the following condition, "l x ; l y 2 L : ½ðl x ; l y Þ 2 < È ½ðl y ; l x Þ 2 < It is important to note that ðL; <Þ is an execution lattice chain in N-manifold. Evidently, an execution lattice chain represents a consistent sequence of observations of asynchronous distributed computation on the respective manifold.

Theorem: The space (X, d X ) is not Hausdorff
However, there is no globally consistent clock in any distributed systems and CðT E Þ & Z þ 0 . Thus, the following axioms can be satisfied by a distributed computing system, DÞϕ; ½CðaÞ 2 D^CðbÞ 2 D , ½d X ðCðaÞ; CðbÞÞ ¼ 0 Hence, 9ε > 0 such that, if B ε ðCðaÞÞ ¼ fx : d X ðCðaÞ; xÞ<εg and B ε ðCðbÞÞ ¼ fy : d X ðCðbÞ; yÞ<εg, then the following axiom is satisfied, Hence, the space ðX; d X Þ is not Hausdorff under local logical clock.
6.2 Theorem: If C E i ð Þ I i and I i \ I k ¼ ϕ for iÞk, then (X, d X ) is Hausdorff.
Proof: As before, let be X ¼ [ CðE i Þ in a distributed computing system. However, for any two intervals I i \ I k ¼ ϕ in the system if iÞk. Moreover, any distributed computing system maintains following axiom due to monotone property of logical clock, "p i 2 P : Hence, the global computation in the respective distributed system will maintain the following property, "x; "y 2 X; d X ðx; yÞ>0 Thus, ðX; d X Þ is a metric space. As every metric space is Hausdorff, hence ðX; d X Þ is Hausdorff.

Computational evaluation
The computational evaluation is performed through simulation considering 3-manifold (M 3 ), which is made up of execution event spaces of three distributed processes. The simulation is performed through Discrete Computational Geometry (DCG) modeling technique considering 3-D shapes. The shapes are constructed based on sets containing discrete data points using mesh topology. The data grid lines are spaced in equal distribution (50:50) to maintain topological mesh without any skew. The metric (norm) is fixed to constant positive integer value that equals to 02 while constructing the grid lines. The distributed computational structures represented as manifold shapes are simulated by considering four different network communication models. The simulation considers a closed group of three processes representing a distributed system on 3-manifold. The group communication models between processes considered in experimentation are: (1) unreliable cyclic unicast in monotone (UCU), (2) reliable cyclic unicast (RCU), (3) unreliable cyclic broadcast (UCB), and (4) reliable cyclic broadcast (RCB). The execution spaces of distributed processes are consisting of local event spaces of individual processes mapped under monotone logical clock function in positive integer range including origin (Z þ [ 0 f g). The M 3 manifold structure is constructed by following the consistent execution sequences. The sets of inconsistent sequences of distributed computation are discarded by applying filter function. The snapshots of distribution profiles of points in sets in 3-D in UCU mode and RCU mode are illustrated in Figures 1 and 2, respectively.
The corresponding snapshots of distribution profiles of points in sets in UCB and RCB modes are presented in Figures 3 and 4, respectively. The detailed description about resulting manifold structures and embedded lattice chains are presented below considering reliable and unreliable communication models. The process execution spaces on 3-D axes in manifold structures represent execution events-spaces of distributed processes, which are dynamic in nature having combinatorial forms.

Experiment I: unreliable network communication
The 3-manifold structure of distributed computation for UCU communication model is presented in Figure 5. In this case, the group of processes synchronizes computation involving a shared variable by using unreliable unicasts in repeated cycles between a pair of processes in the group. The 3-manifold structure illustrates existence of multiple local supremum and infimum points. However, a global supremum and infimum can be computed representing global state of computation. The majority of surface areas are relatively smoother during computation indicating finite bound in computation. In this model of computation, the overall message complexity is minimum in stable network condition because the communication is cyclic unicast and no acknowledgment is transacted between processes. The corresponding embedded lattice chain on 3-manifold representing a consistent distributed computing sequence is presented in Figure 6.

Distribution of points in event-sets of processes in 3-D (UCU)
The lattice chain illustrates the presence of an apparently periodic structure, which is inline to the cyclic nature of synchronization between processes in the closed group. In the next experiment step, the unicast communication model is converted to broadcast network communication model (UCB) in the closed group. The resulting 3-manifold structure generated by distributed computation is presented in Figure 7.
In this case, the processes in a closed group perform broadcasts in cycles without considering reliability of messages. It is considered in simulation that no two messages can be simultaneously sent or received by any process at a single time instant. The 3-manifold structure illustrates that distance between different local supremum and infimum points are reduced if the network is relatively stable. The overall smoothness of the surface is maintained in the majority of places with localized convex and concave deformations depending on the nature of process states. The corresponding embedded lattice chain of consistent distributed computing sequence is presented in Figure 8.   The lattice chain embedded on 3-manifold of UCB model represents that apparent periodicity of computation exists with relative dispersion. The dispersion effect is due to the time dilation required for completing a broadcast in a closed group.

Experiment II: reliable network communication
In this experimental set, the unreliable network communication models are changed by incorporating reliability in communication. The simulation model implements First-In-First-Out (FIFO) model of network communication. In the first experimentation, the UCU model is converted into RCU by incorporating reliability in the FIFO network communication. The resulting 3-manifold structure generated by distributed computation is presented in Figure 9.
In this case, the number of local supremum and infimum points are reduced, and surface appears to be relatively uniform locally (but not strictly uniform globally). The overall smoothness of the surface is moderately enhanced as compared to unreliable systems. The reason is that transitions are highly synchronized in reliable systems having bidirectional communication. The overall message complexity is higher in this 3-manifold as compared to unreliable unicast model. The corresponding embedded lattice chain of consistent distributed computing is presented in Figure 10. Evidently, the apparent presence of any periodicity is reduced, and the computation proceeds in phases. There are several discrete transitions between phases of distributed computation. The reason is that the cyclic synchronization forms several closed-loop graphs in RCU model of distributed computation. The 3-manifold structure generated by RCB model of network communication is presented in Figure 11.
In this case, the processes in the closed group perform synchronization by using reliable broadcast in repeated cycles. However, no two messages can be simultaneously sent or received by any process at a single time instant, which is inline to real-life implementation of data communication. The global smoothness of the surface is further enhanced due to complete synchronization between every process in the system with reliable communications. The message complexity is highest in this case. The surface of RCB 3-manifold appears to be locally uniform covering a larger section indicating communication reliability and complete covering of the group of processes in each cycle. The corresponding lattice chain embedded on RCB 3-manifold is presented in Figure 12.
The lattice chain of RCB communication based distributed computation illustrates reduced frequency of discrete transitions in the sequence of consistent computation. This signifies the existence of highest reliability of data and distributed synchronization in a process group. The Figure 11. Distributed computing manifold using cyclic reliable broadcast. Figure 12. Execution lattice chain on 3-manifold for reliable broadcast based computing.
dynamics of controlled filter values for generating consistent execution sequence of distributed computation is presented in Figure 13.
The infimum of filter is kept fixed at lowest value (unity), and the supremum is varied while maintaining consistency of observation of distributed computation.

Application aspects-faults detection in computation
The modern approaches to analyze distributed computation employ various geometric analysis techniques. These techniques effectively result in shape analysis in order to detect consistency of computation in structural forms. For example, the simplicial complexes and topological models of distributed computation form a set of dynamic graph like shapes, which helps in determining consistency of computation (Conde & Rajsbaum, 2012;Herlihy & Rajsbaum, 1999). The manifold is a geometric structure, and its application helps in determining consistency of observation of distributed computation in N-dimensional space. The manifold structure employed in this paper facilitates computational shape analysis in order to detect inconsistency in observable distributed computation. The two main faults that occur in a distributed computation are: Fail-Stop fault and Byzantine fault. The faults are simulated by distribution of points in events space of processes with skews. In case of Fail-Stop fault, the points are clustered within event space of a faulty process. However, in case of Byzantine fault, the points are distributed arbitrarily within a sub-space of topological execution space of faulty process having nonconvergent nature. The application of manifold structure to detect fail-stop fault in distributed computing (UCU mode) is illustrated in Figure 14.
The first (top) surface of Figure 14 represents distributed computing having fail-stop fault within execution space of process 2. The local deformations are indicated on the surfaces representing fail-stop and regular computations in UCU mode. The manifold of a faulty distributed computation appears to be smoother with respect to regular manifold structure due to partial blocking of global execution space eliminating transitions. The deformation on manifold due to Byzantine fault in process 2 execution space in RCU mode of computation is illustrated in Figure 15.
The Byzantine fault induces a highly localized distortion in a bounded region. The distortion is severe due to Byzantine nature of fault in computation enhancing unpredictability. The manifold outside this region of deformation appears to be relatively smoother than regular distributed computation. This is due to the partial blocking of other processes due to existence of a Byzantine (faulty) process in the system. These examples illustrate the application of manifold model of distributed computation to monitor a system by detecting the existence of different types of failures of computation in a deterministic system. It would help the distributed systems designers to formulate and analyze possible fault models a priori along with detection of locality of faults in a system.

Conclusions
The observation of an asynchronous distributed computation is required in order to maintain stability in an execution sequence. The consistency of computation is evaluated for generating correct output of computation and to detect faults. The consistent observation of large-scale asynchronous distributed systems is challenging due to enlarged events space. The construction of computational model on manifold structure helps in gaining analytical insights to complex systems. The formulation proposed in this paper offers a model and analysis of consistent observation of asynchronous distributed computation on N-manifold. The fineness of consistent observation can be controlled by employing filter. The simulations of resulting structures offer better understanding of the dynamics of the computation in view of analysis.

Funding
The author received no direct funding for this research.

Citation information
Cite this article as: Computational modeling of consistent observation of asynchronous distributed computation on N-manifold, Susmit Bagchi, Cogent Engineering (2018), 5: 1528029.