Fault Tolerant Gamma-Minus Network

In large-scale supercomputers thousands of processors are connected together to their respective memory modules which are controlled by several control units connected in parallel. Large data streams have to be communicated between these processors and memories through interconnection networks. Multistage interconnection network (MIN) is an efficient way to provide these communications at a very reasonable cost. For such large systems MIN employed should be highly reliable and fast to meet the desired specifications of these high speed switch fabrics. In this paper two new architectures of MIN are proposed which are named as fault-tolerant Gamma-Minus (FTGM-1 & FTGM-2) networks. These proposed architectures are multipath MIN with totally disjoint paths and are highly reliable and fault tolerant. The routing algorithm proposed for these structures is simple distance tag routing with non-backtracking which overcomes rerouting overheads and hence improve communication delays. The proposed MIN ensures multiple fault tolerance at each stage including input and output stage with minimal horizontal distance between source node to destination node. For validating the results, a comparison has been presented between existing MINs with these proposed designs. The proposed MINs outperform the existing MIN in terms of reliability, fault tolerance and up to some extent cost as well.

1. Links are assumed to be more reliable than SEs (SE). 2. Reliability of all SE are assumed to be same. 3. SEs at input stage and output stage are assumed to be fault free. 4. SE Failure is non repairable.
In second assumption reliability of all SE are assumed to be same and all of the research of reliability evaluation of MINs in past is centric around this assumption for Gamma interconnection network family. 7,12,16,20 Although in some of the papers on a irregular network it has been clearly mentioned that the reliability of any SE depends upon the number of cross points used in that SE. For example if reliability of 3 × 3 SE is assumed to be "r" or "r 9\9 " then reliability of 1 × 3 or 3 × 1 SE is "r 3\9 " and reliability of other switch element can also be calculated in same manner.
In third assumption it has been is assumed that SEs at input and output stages are fault free and if fault occurs at these stages then whole network will get failed. 20,[23][24][25][26][27][28][29]32,35,37 When SE at intermediate stage can fail then how can it be assumed that switch element at input and output stages are fault free although the terminology used for making these SEs are same. If the assumed reliability of all SE in these papers are also same then failure rate of these SE should also be same and any SE can fail at anytime. So the acclaimed disjoint paths in these topologies are partially disjoint assuming input and output node as critical notes or fault free nodes. Further in these topologies number of SE per stage vary for different stages which complexes the connection pattern for these networks. Due to these Complex structures the routing algorithm also becomes Complex. The dynamic routing algorithm used in these networks is useful only in case when a prior knowledge of fault is there in the network. 2,3,[23][24][25][26][27][28] But if fault in network occurs suddenly without any prior knowledge packet will either get dropped or backtracking will has to be used and no algorithm for such a case has been specified in these papers. Another network called reliable interconnection network (RIN) has been proposed in 2015 33 which possess two totally disjoint paths and four partially disjoint paths from each source destination node pair. This network is at least one fault-tolerant at input and output stage and multiple fault-tolerant at intermediate stages with moderate Reliability. This network is robust at intermediate stages against fault but is not much robust at input and output stages in presence of faults. The dynamic routing used in this network is also Complex and no backtracking algorithm for this network is specified.
Some of the other network proposed in last two decades are centered around increasing either number of stages or switch element size such as extra stage Gamma network (EGIN), 12,16 incomplete z E-mail: shilpa1_goyal@rediffmail.com Gamma network(IGIN), 37 incomplete cyclic Gamma network (ICGIN), 37 balanced Gamma network (BGIN), 24 enhanced improved augmented data manipulators (EIADM) 2 etc or by reshuffling the connection pattern of Gamma network such as mono Gama network (MGIN), 1 cyclic Gama network (CGIN), 20 modified balanced Gama network (MBGIN) 32 etc All these topologies provide redundant paths for each source-destination node pair with the basic assumptions same as discussed earlier. Some other networks such as partially chain Gamma interconnection network (PCGIN), 25 fully chain Gamma interconnection network (FCGIN) 25 use chaining in the corresponding stages to provide redundancy in the network. These network provides variable vertical distance from each source to destination and it is difficult to analyze reliability of these networks.
In this paper two new designs have been proposed to enhance the reliability and fault tolerance capabilities of network which mitigates all disadvantages of the previous topologies and provide totally disjoint path with the minimal vertical distance between source and destination which has not been considered as a performance metrics ever. The motivation behind proposing new topologies is as follow:

Motivation
1. All networks in Literature provide partially disjoint paths assuming input and output nodes as critical nodes and are fault free. This motivates to propose a new design to tolerate multiple faults at each stage.
2. RIN is one fault tolerant at input and output stage and is multiple fault tolerant at intermediate stages but if multiple faults occur at input and output stage then the network will get failed. so network should be multiple fault tolerant at each stage including input and output stages. 3. In supercomputer scenario MIN find huge applications because of their low cost. So reduction in cost of MIN is the main performance metrics although this metric has dealt in Literature with great effort but some Delta networks possess low cost then Gamma network family and no network from this family has low cost comparative to Delta network family this also motivates to introduce a new design from Gamma network family with lowest cost comparative to Delta networks. 4. No network in Literature comments on vertical distance from source to destination although in big data communications, transmission time delay is very important performance metrics. While proposing new designs this issue has been considered with great effort and hence newly proposed architectures have minimal vertical distance or minimal path set from each source to destination. 5. Supercomputers are the machineries which connect thousands of processes in cascade. In MIMD number of processors connected varies from 2 6 to 2 16 in cascade. So it means interconnection network used for these machines should be of configuration much higher than 8 × 8 network size. No research has been done to analyze reliability of such a big interconnection networks. 6. Reliability of all SEs in a given network has been assumed to be same as discussed earlier which motivates to recalculate reliability of all networks by assuming different reliability for each SE of different configuration.
To achieve highest reliability better than the previously proposed networks is always a goal for today's fast growing scenario and to meet challenging requirement of daily improving super processors. 34,35 The designs proposed in this paper are cost comparative with large number of alternative paths available providing lowest path length between source-destination node pair. These highly reliable multiple fault tolerant networks process multiple totally disjoint paths between source and destination. All MIN proposed in literature belongs Gamma network family are compared with these newly proposed topological designs and hence proved acclaimed highest reliability ever in same category.
Rest of the paper is organized as follows: In section 2 topologies of proposed networks are given. In section 3 routing algorithms used for these networks have been stated. In section 4 reliability analysis has been done for these two networks and compared with others MIN. In section 5 Conclusion and discussion is given followed by future scope and references. In FTGM-2 as 1 × 2 SE is used with 2 output links at this stage so jth SE at stage 0 will be connected to (j)th and (j+1)th SE at stage 1.
At stage 1: In FTGM-1 4 × 2 SE is used with the two output links so the connection pattern at this stage to the next stage is all even SE are cross-connected and all odd SE are cross-connected with in a same group.
In FTGM-2 group of 8 SE of configuration 2 × 4 is divided into which all even SE are cross-connected and all odd SE are crossconnected with 4 output links with in a same group.
At stage 2 onwards: In both networks two consecutive groups are cross connected through SE of configuration 2 × 2 in FTGM −1 for stage 2 onwards and 4 × 2 at stage 2 and 2 × 2 from stage 3 onwards to make one group. Topology of FTGM-1 and FTGM-2 are shown in Fig. 1  Routing.-Gamma class of networks uses distance tag routing and destination tag routing 6,12,16,20 where the rerouting of alternative path or backtracking algorithms are required. For rerouting of packet through alternative paths dynamic routing techniques have been used but this routing technique is useful only when there is knowledge of fault in the next stage prior to the transmission [16-17, 36 26-30]. If all SE connected at output link of current stage are faulty then packet will have to be back-tracked to the previous stage but no technique have been suggested for this kind of situation. 3 The techniques used here in this paper activate all alternative links at same time and no rerouting of packets will be required. In this scheme routing distance tag has to be calculated by taking difference between source and destination tag modulo N. The algorithm is as follows: Step 1: distance tag has to be calculated as discussed.
Step 2: In FTGM-1, from stage 0 to stage 1(j)th and ( j±2)th connections will be followed if tag is even and if tag is odd then (j ±1)th and ( j±3)th connections will be followed to the next stage.
In FTGM-2 from stage 0 to stage 1 if tag is even then this straight connection will be followed and if it is odd then (j+1)th connection will be followed.
Step 3: In both networks from stage 1 to 2 all output links will be activated.
Step 4: For further stage (stage 2 to 3, stage 3 to 4 and so on) in higher network sizes if destination belongs to same group of networks then straight connection will be activated otherwise other (upward or downward whatever applicable) connection will be activated. Lemma 1. No rerouting or back tracking is required according to this algorithm.
Proof 1: as shown in Fig. 2 all alternative paths are selected for routing of any packet so no other alternative path is there to be selected through rerouting of packet so it is not required. Proof 2: In FTGM-1 packet has to be routed from source "2" to destination "5" with distance tag t = "3" which is odd then from stage 0 to stage 1 SE "3" and "5" are selected. From stage 1 to stage 2 all output links will be selected and packet would reach two different destinations as shown in Fig. 2a. So it can tolerate one fault at each stage.
Proof 3: In FTGM-2 according to the above example as tag is odd so (j+1) connection will be followed and all odd SE will be selected as shown in Fig. 2b. From stage 1 to stage 2 all output links will be selected and packet would reach four different destinations as shown in Fig. 2a

Results and Discussion
Reliability analysis.-Reliability of any system is an important performance metrics upon which figure of merit of any system can be judged. In designing of MIN it is an important issue to be taken in consideration. 5,7,9,13,24,33,34,43 Reliability of any system may be defined as normal or desired functioning of that system in normal as well as critical conditions. There are three measures of Reliability which can be taken in consideration these are: Terminal reliability (TR): TR of a network is defined as the probability of existence of at least one fault free path between each source-destination node pair through one of its network configuration. It is also known as source to Terminal reliability (STR).
Broadcast reliability (BR): BR of a network is defined as the probability of existence of at least one fault free broadcast path between each shows to all destinations through one of its network configuration. it is also known as sources to all terminal reliability (SAT).
Network reliability (NR): It is defined as the probability that all nodes of a given network can communicate with each other through any fault free network configuration. it is also known as all terminal reliability (ATR).
To evaluate reliability of proposed network there are some basic assumptions which have been is used these are: (i) All SE are non repairable and can have only two States these are working or failed. (ii) Links are more reliable than SEs. (iii) Reliability of 3 × 3 SE is considered as "r" or "r 9/9 " and reliability of other SEs can be calculated by number of cross points of respective SE divide by "9". Where "r" may be represented as: r e where is system failure rate assumed as 10 (iv) All SE failure a random and statically independent in nature.
(v) Any SE in networks can fail at any time no SE is assumed as critical SE or fault free element.
For reliability evaluation of network reliability block diagram techniques have been used which have been used and suggested by many researchers in past. 7,12 Reliability block diagram for TR, BR and NR of these networks are shown in Fig. 3. Table II shows the reliability equations of both the networks. All recently proposed MINs 1,20,23-29,32,35,37 are evaluated for their reliability by assuming different "R SE " for different configuration of SE (using Inclusion-Exclusion method) and comparison of these reliability values and previously calculated values 1.,12.,16.,20.,23.-29.,32. has been done with the proposed apologies for "N"=8. Tables III-V shows the result of reliability evaluation of these networks. From Tables III-V it is clear that two newly proposed FTGM networks outperforms all previous MIN in terms of cost and reliability with uniform minimal path distance from each source to destination.
Cost of any given network is also very important parameter as system's cost mainly rely on the cost of MIN employed. One common method of calculating cost of MIN which has been used in literature is by calculating the switch complexity with the assumption that the cost of the switch is directly proportional to its complexity, where complexity of SE is defined as a number of gates or the number of total cross-points employed in that SE. for example, a MIN comprising of 2 × 2 switch has a complexity of "2 × 2 = 4 units" i.e. it associates total hardware cost of "4" units. Cost of each and every element used in MIN can be calculated as shown by the Eq. 2.  Table III. It is evident from Table III that cost of FTGM-1 is the lowest among different Gamma MIN, whereas FTGM-2 shows slightly higher cost for size "N" = 8. FTGM-2 has "176" units of cost, whereas, 4-Disjoint GIN and FTIN has "160" units of cost for "N = 8" but it can be proved through cost modeling using Eq. 2 that as the size of the network increases to N = 1024 or higher, FTGM-2 possesses low cost than RIN-1, 4-DGIN and FTIN.

Conclusion and Discussion
In this paper 2 new topologies of fault-tolerant gamma Minus network have been proposed which provide two and four disjoint paths for each source-destination node pair. Few networks in Literature also provide two to four partially disjoint path but new topologies proposed in this paper are better from the others in terms of reliability and fault tolerance capability.     (iii) FTGM-1 provides same number of totally disjoint paths as provided by RIN but FTGM-2 provides more number of totally disjoint path then RIN and hence is more reliable. (iv) FTGM-2 has slightly higher cost then 4-Disjoin GIN and FTIN but FTGM-1 causes lowest cost in Gamma class of networks. (v) Same number of paths is there for each Tag Value in FTGM-1 and FTGM-2 and is independent of Tag and hence the reliability calculations are not cumbersome. (vi) In all previously proposed networks number of paths increases with increasing network size "N" but reliability (TR, BR, NR) of these networks is decreasing a lot. But in the proposed topologies in this paper although number of paths are same for any network size but reliability is not decreasing much. So it can be concluded that the topologies proposed for FTGM-1 and FTGM-2 are best topologies in Gamma class of networks and increasing the number of paths of network does not ensures the increase in reliability. (vii) Proposed networks provide same number of paths for any network size "N" and hence reduce the complexity of reliability evaluation for higher network sizes. No other network has this ability. (viii) The proposed new topological designs provide minimal path set for each source destination node pair which has not been considered anywhere in the previous literature. (ix) Simple routing algorithm is used with no re routing overheads as compared to all other MIN.
With the above discussion it can be concluded that fault-tolerant Gamma Minus network surpasses all other networks of this class in all performance indices and hence may be considered as a best choice for MIMD machines or supercomputers. For future scope these MIN can be simulated using MIN simulate to comment upon other performance parameters such as latency power dissipation efficiency bandwidth etc to get deep inside of these MIN.