Daniel Barbara
Hector Garcia-Molina
Abstract
Voting is used commonly to enforce mutual exclusion in distributed systems. Each node is assigned a number of votes, and only the group with a majority of votes is allowed to perform a restricted operation. This paper describes techniques for dynamically reassigning votes upon node or link failure, in an attempt to make the system more resilient to future failures. We focus on autonomous methods for achieving this, that is, methods that allow the nodes to make independent choices about changing their votes and picking new vote values, rather than group consensus techniques that require tight coordination among the remaining nodes. Protocols are given which allow nodes to install new vote values while still maintaining mutual exclusion requirements. The lemmas and theorems to validate the protocols are presented. A simple example shows how to apply the method to a database object-locking scheme; the protocols, however, are versatile and general purpose, and can be used for any application requiring mutual exclusion. In addition, policies are presented that allow nodes to autonomously select their new vote values. Simulation results are presented comparing the autonomous methods to static vote assignments and to group consensus strategies. These results demonstrate that under high failure rates, dynamic vote reassignment shows great improvement over a static assignment of votes in terms of availability. In addition, many autonomous methods for determining a new vote assignment yield almost as much availability as a group consensus method and at the same time are faster and more flexible.
- 1 EL ABBADI, A., AND TOUEO, S. Availability in partitioned replicated databases. In Proceedings of the Fifth ACM SIGACT-SIGMOD Symposium on Principles of Database Systems (Mar. 1986). ACM, New York, 1986, pp. 240-251. Google Scholar
- 2 BARBARA, D., AND GARCIA-MOLINA, H. Optimizing the reliability provided by voting mechanisms. In Proceedings of the Fourth International Conference on Distributed Computing Systems (San Francisco, Oct. 1984). IEEE, New York, 1984, pp. 340-346.Google Scholar
- 3 BERNSTEIN, P., AND GOODMAN, N. Concurrency control in distributed database systems. ACM Comput. Surv. 13, 2 (June 1981), 185-221. Google Scholar
- 4 CRISTIAN, F., AGHILI, H., STRONG, R., AND DOLEV, D. Atomic broadcast: From simple message diffusion to Byzantine agreement. In Proceedings of the Fifteenth International Symposium on Fault-Tolerant Computing (Ann Arbor, Mich., June 1985). IEEE, New York, 1985.Google Scholar
- 5 DAVCEV, D., AND BURKHARD, W. Consistency and recovery control for replicated files. In Proceedings of the Tenth ACM Symposium on Operating Systems Principles (Dec. 1985). ACM, New York, 1985, pp. 85-96. Google Scholar
- 6 DAVIDSON, S., GARCIA-MOLINA, H., AND SKEEN, D. Consistency in partitioned networks. ACM Comput. Surv. 17, 3 (Sept. 1985), 341-370. Google Scholar
- 7 FISCHER, M. J., LYNCH, N. A., AND PATERSON, M.S. Impossibility of distributed consensus with one faulty process. J. ACM 32, 2 (Apr. 1985), 374-382. Google Scholar
- 8 GARCIA-MOLINA, H. Elections in a distributed computing system. IEEE Trans. Comput. C-31, 1 (Jan. 1982), 48-59.Google Scholar
- 9 GARCIA-MOLINA, H. Reliability issues for fully replicated distributed databases. IEEE Comput. 15, 9 (Sept. 1982), 34-42.Google Scholar
- 10 GIFFORD, D.K. Weighted voting for replicated data. In Proceedings of the Seventh Symposium on Operating Systems Principles (Pacific Grove, Calif., Dec. 1979). ACM, New York, 1979, pp. 150-162. Google Scholar
- 11 GRAY, J.N. Notes on database operating sytems. In Operating Systems: An Advanced Course, Springer-Verlag, New York, 1978, pp. 393-481. Google Scholar
- 12 HERLIHY, M. Dynamic quorum adjustment for partitioned data. A CM Trans. Database Syst. 12, 2 (June 1987), 170-194. Google Scholar
- 13 JAJODIA, S., AND MUTCHLER, D. Dynamic voting. In Proceedings of the ACM SIGMOD International Conference on Management of Data (May 1987). ACM, New York, 1987, pp. 227-238. Google Scholar
- 14 JAJODIA, S., AND MUTCHLER, D. Enhancements to the voting algorithm. In Proceedings of the Thirteenth International Conference on Very Large Data Bases (Brighton, Eng., Sept. 1987). IEEE, New York, 1987. Google Scholar
- 15 JAJODIA, S., AND MUTCHLER, D. Integrating static and dynamic voting protocols to enhance file availability. In Proceedings of the Fourth International Conference on Data Engineering (Los Angeles, Feb. 1988). IEEE, New York, 1988, pp. 144-153. Google Scholar
- 16 PARIS, J.F. Voting with witnesses: A consistency scheme for replicated files. In Proceedings of the Sixth International Conference on Distributed Computing Systems (Cambridge, Mass., May 1986). IEEE, New York, 1986, pp. 606-612.Google Scholar
- 17 PARIS, J. F., AND LON6, D. D.E. Efficient dynamic voting algorithms. In Proceedings of the Fourth International Conference on Data Engineering (Los Angeles, Feb. 1988). IEEE, New York, 1988, pp. 268-275. Google Scholar
- 18 SCHLICHTING, R. D., AND SCHNEIDER, F.B. Fail-stop processors: An approach to designing fault-tolerant computing systems. ACM Trans. Comput. Syst. 1, 3 (1983), 222-238. Google Scholar
- 19 THOMAS, R.H. A majority consensus approach to concurrency control. ACM Trans. Database Syst. 4, 2 (June 1979), 180-209. Google Scholar
Index Terms
- Increasing availability under mutual exclusion constraints with dynamic vote reassignment
Recommendations
Fair group mutual exclusion
PODC '03: Proceedings of the twenty-second annual symposium on Principles of distributed computingIn the group mutual exclusion problem [6], which generalizes mutual exclusion [2], a process chooses a session when it requests entry to the Critical Section. A group mutual exclusion algorithm must ensure that the mutual exclusion property holds: If ...
A survey of permission-based distributed mutual exclusion algorithms
The problem of mutual exclusion in distributed systems has attracted considerable attention over the last two decades. The mutual exclusion problem requires that, at a time, only one of the contending processes be allowed to enter its critical section (...
Distributed mutual exclusion on hypercubes
Hypercube have been commercially available in the past few years to their high degree of connectivity, symmetry, and low degree of diameter. In this paper, we analyse the performance in number of messages on d-dimensional hypercube, for two groups of ...
Reviews
Greg Minshall
One method of enforcing mutual exclusion in a distributed system is to assign a vote to each node in the system and, before a node enters a critical section, require that node to acquire a majority of votes from the nodes in the distributed system. The node informs the other nodes when it exits the critical section. A node keeps track of whether a given critical section has been entered by any node in the system, and votes “no” on any subsequent attempt to enter that section until notified that the section has been exited. A timer-based mechanism is required to protect against lost “exited” messages. Under static assignment of votes, if two elections each garner a majority of votes, it is easy to see that at least one node has voted “yes” in both elections. This is sufficient to provide for mutual exclusion, since a proper implementation would prevent a node from voting “yes” in two elections for the same critical section with no intervening “exited” message. A static assignment of votes produces a risk that multiple partitions will cause the system to enter a state in which no single partition can achieve a majority. A system that has reached this state is said to be halted. By dynamically reassigning the votes of the nodes after each partition (and possibly after each join), it is possible to reduce the risk of the system halting in this fashion, thus increasing its reliability. Two approaches to dynamically reassigning votes are possible. Group consensus causes all the nodes in a partition to conspire together to determine an optimal vote for each node in that partition. Because this strategy uses global knowledge of the state of the partition, it can achieve an optimal result, that is, reassign the votes in a way that best protects the system in the face of subsequent partitions. Group consensus requires a complex and possibly expensive series of interactions among the nodes in the partition, however. Autonomous reassignment lets each node, following some predetermined policy, determine its new vote without regard to the actions of the other nodes in the partition, while requiring that a node's new vote be installed only if that node can collect a majority of votes for the vote change. Since each node's view of the network may be imperfect, autonomous reassignment does not always achieve an optimal reassignment of votes, but it is less expensive than group consensus. In addition to providing a general introduction to the problem of using distributed election protocols for mutual exclusion, this paper examines whether a dynamic reassignment of votes can safely occur concurrently with other activities, such as other vote reassignments or voting for mutual exclusion. The authors present an example using dynamic reassignment of votes within a database system, with lock and unlock as the equivalents of enter critical section and exit critical section. The paper also studies whether autonomous reassignment can achieve roughly the same level of availability in the face of multiple partitions as global consensus can. The section on the safety of dynamic reassignment of votes first considers a system in which votes can only increase in value. The authors present a protocol for collecting votes and deciding the outcome of the election and a protocol for increasing the vote of a node. Following the protocols and some definitions, they give a series of lemmas and a theorem. The theorem is important—it states that for any pair of possibly concurrent successful vote increases, at least one of the pair knows of the other's existence. Next, they present a similar set of protocols, definitions, lemmas, and theorems for a system in which votes may increase or decrease. Unfortunately, the two collections of lemmas, theorems, and proofs are not correct for the vote-collecting protocols presented. (Most of the lemmas are true for the presented protocols, but the key lemma, 2.1.2, is not.) They are correct for a slightly altered set of protocols, in which a vote-collecting node i sets its concept of node j 's vote to the maximum of each voting node's concept of j 's vote, even if node j votes. The authors compare system availability under global consensus and under autonomous reassignment using a simulation. The results show that system availability is almost as high for certain policies of autonomous reassignment as for global consensus. Unfortunately, it is not clear whether the simulation used the protocols as presented in the paper (which are not proven to be safe) or the safe altered protocols referred to above. Thus, while a set of protocols that leads to a safe system for dynamically reassigning votes exists, we cannot say that within such a system policies exist for autonomously reassigning votes that approach the level of availability of a global consensus policy . Despite these faults, the paper is quite good. Anyone with a serious interest in mutual exclusion in a distributed system, or in understanding some of the subtleties of distributed systems, should spend some time understanding at least part of this paper. It should also be considered for educational use.
William W. Oblitey
The techniques for reassigning votes dynamically in a distributed system upon node or link failure discussed in this paper use mutual exclusion constraints and attempt to accomplish higher availability of the distributed system. The authors discuss two basic methodologies: group consensus and autonomous reassignment. They present examples of these two methodologies. The paper then concentrates on the autonomous method of dynamic vote reassignment. The paper examines protocols that permit nodes to install new vote values and still maintain mutual exclusion. The authors present and prove theorems to validate these protocols. They also discuss simulation results for the several policies that they considered for the implementation of the dynamic vote reassignment. The paper should interest people who deal with shared or distributed resources or replicated data items.
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments