Yanhong A. Liu
Scott D. Stoller
Skip Abstract Section
Abstract
This article describes a very high-level language for clear description of distributed algorithms and optimizations necessary for generating efficient implementations. The language supports high-level control flows in which complex synchronization conditions can be expressed using high-level queries, especially logic quantifications, over message history sequences. Unfortunately, the programs would be extremely inefficient, including consuming unbounded memory, if executed straightforwardly.
We present new optimizations that automatically transform complex synchronization conditions into incremental updates of necessary auxiliary values as messages are sent and received. The core of the optimizations is the first general method for efficient implementation of logic quantifications. We have developed an operational semantics of the language, implemented a prototype of the compiler and the optimizations, and successfully used the language and implementation on a variety of important distributed algorithms.
- Umut A. Acar, Guy E. Blelloch, and Robert Harper. 2006. Adaptive functional programming. ACM Transactions on Programming Languages and Systems 28, 6, 990--1034. Google Scholar
Digital Library
- Gul Agha. 1986. Actors: A Model of Concurrent Computation in Distributed Systems. MIT Press, Cambridge, MA. Google ScholarCross Ref
- Frances E. Allen, John Cocke, and Ken Kennedy. 1981. Reduction of operator strength. In Program Flow Analysis, Steven S. Muchnick and Neil D. Jones (Eds.). Prentice-Hall, Upper Saddle River, NJ. 79--101.Google Scholar
- P. Alvaro, T. Condie, N. Conway, J. M. Hellerstein, and R. Sears. 2010. I do declare: Consensus in a logic language. ACM SIGOPS Operating Systems Review 43, 4, 25--30. Google ScholarDigital Library
- Gregory R. Andrews and Ronald A. Olsson. 1993. The SR Programming Language: Concurrency in Practice. Benjamin Cummings. Google ScholarDigital Library
- Michael P. Ashley-Rollman, Peter Lee, Seth Copen Goldstein, Padmanabhan Pillai, and Jason D. Campbell. 2009. A language for large ensembles of independently executing nodes. In Proceedings of the 25th International Conference on Logic Programming. Springer, 265--280. Google ScholarDigital Library
- Hagit Attiya and Jennifer Welch. 2004. Distributed Computing: Fundamentals, Simulations, and Advanced Topics (2nd ed.). Wiley, Hoboken, NJ. Google ScholarDigital Library
- Joshua S. Auerbach, Arthur P. Goldberg, Germán S. Goldszmidt, Ajei S. Gopal, Mark T. Kennedy, Josyula R. Rao, and James R. Russell. 1994. Concert/C: A language for distributed programming. In Proceedings of the USENIX Winter 1994 Technical Conference. USENIX Association, Article 8, 18 pages. Google ScholarDigital Library
- Antonio Badia. 2007. Question answering and database querying: Bridging the gap with generalized quantification. Journal of Applied Logic 5, 1, 3--19.Google ScholarCross Ref
- Antonio Badia, Brandon Debes, and Bin Cao. 2008. An implementation of a query language with generalized quantifiers. In Proceedings of the 27th International Conference on Conceptual Modeling. Springer, 547--548. Google ScholarDigital Library
- Antonio Badia, Dirk Van Gucht, and Marc Gyssens. 1995. Query languages with generalized quantifiers. In Applications of Logic in Databases, R. Ramakrishnan (Ed.). Kluwer Academic, Dordrecht, The Netherlands.Google Scholar
- Jason Baker, Chris Bond, James C. Corbett, J. J. Furman, Andrey Khorlin, James Larson, Jean-Michel Léon, Yawei Li, Alexander Lloyd, and Vadim Yushprakh. 2011. Megastore: Providing scalable, highly available storage for interactive services. In Proceedings of the Conference on Innovative Database Research. 223--234.Google Scholar
- Berkeley Orders of Magnitude 2013. Bloom Programming Language. Retrieved April 8, 2017 from http://www.bloom-lang.net.Google Scholar
- Mark Bickford. 2009. Component specification using event classes. In Proceedings of the 12th International Symposium on Component-Based Software Engineering. Springer, 140--155. Google ScholarDigital Library
- Andrew P. Black, Norman C. Hutchinson, Eric Jul, and Henry M. Levy. 2007. The development of the Emerald programming language. In Proceedings of the 3rd ACM SIGPLAN Conference on History of Programming Languages. 11--1--11--51. Google ScholarDigital Library
- Mike Burrows. 2006. The chubby lock service for loosely-coupled distributed systems. In Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation. 335--350. Google ScholarDigital Library
- Jiazhen Cai, Philippe Facon, Fritz Henglein, Robert Paige, and Edmond Schonberg. 1991. Type analysis and data structure selection. In Constructing Programs from Specifications, Bernhard Möller (Ed.). North-Holland, Amsterdam, The Netherlands, 126--164.Google Scholar
- Saksham Chand, Yanhong A. Liu, and Scott D. Stoller. 2016. Formal verification of multi-Paxos for distributed consensus. In Proceedings of the 21st International Symposium on Formal Methods. Springer, 119--136.Google Scholar
- Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber. 2008. Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems 26, 2 (2008), 4. Google ScholarDigital Library
- Jens Claußen, Alfons Kemper, Guido Moerkotte, and Klaus Peithner. 1997. Optimizing queries with universal quantification in object-oriented and object-relational databases. In Proceedings of the 23rd International Conference on Very Large Data Bases. Morgan Kaufman, San Francisco, CA, 286--295. Google ScholarDigital Library
- Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms (3rd ed.). MIT Press, Cambridge, MA. Google ScholarDigital Library
- Flavio Cruz, Ricardo Rocha, Seth Copen Goldstein, and Frank Pfenning. 2014. A linear logic programming language for concurrent programming over graph structures. Theory and Practice of Logic Programming 14, 7, 493--507.Google ScholarCross Ref
- Jeffrey Dean and Sanjay Ghemawat. 2008. MapReduce: Simplified data processing on large clusters. Commun. ACM 51, 1, 107--113. Google ScholarDigital Library
- Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall, and Werner Vogels. 2007. Dynamo: Amazon’s highly available key-value store. ACM SIGOPS Operating Systems Review 41, 6, 205--220. Google ScholarDigital Library
- DistAlgo. 2016. DistAlgo: A Language for Distributed Algorithms. Retrieved April 8, 2017 from http://github.com/DistAlgo.Google Scholar
- Erlang. 2015. Erlang Programming Language. Retrieved April 8, 2017 from http://www.erlang.org/.Google Scholar
- Colin J. Fidge. 1988. Timestamps in message-passing systems that preserve the partial ordering. In Proceedings of the 11th Australian Computer Science Conference. 56--66.Google Scholar
- F. Fioravanti, A. Pettorossi, M. Proietti, and V. Senni. 2011. Program transformation for development, verification, and synthesis of programs. Intelligenza Artificiale 5, 1, 119--125.Google ScholarCross Ref
- Vijay K. Garg. 2002. Elements of Distributed Computing. Wiley. Google ScholarDigital Library
- Gautam and S. Rajopadhye. 2006. Simplifying reductions. In Conference Record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. 30--41. Google ScholarDigital Library
- Chryssis Georgiou, Nancy A. Lynch, and Panayiotis Mavrommatis and Joshua A. Tauber. 2009. Automated implementation of complex distributed algorithms specified in the IOA language. International Journal on Software Tools for Technology Transfer 11, 2, 153--171.Google ScholarDigital Library
- Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. 2003. The Google file system. ACM SIGOPS Operating Systems Review 37, 5, 29--43. Google ScholarDigital Library
- Michael Gorbovitski, Yanhong A. Liu, Scott D. Stoller, Tom Rothamel, and Tuncay Tekle. 2010. Alias analysis for optimization of dynamic languages. In Proceedings of the 6th Symposium on Dynamic Languages. ACM Press, 27--42. Google ScholarDigital Library
- Deepak Goyal. 2000. A Language Theoretic Approach to Algorithms. Ph.D. Dissertation. Department of Computer Science, New York University, New York, NY. Google ScholarDigital Library
- Adam Granicz, Daniel M. Zimmerman, and Jason Hickey. 2003. Rewriting UNITY. In Proceedings of the 14th International Conference on Rewriting Techniques and Applications. Springer, 138--147. Google ScholarDigital Library
- Ashish Gupta, Inderpal Singh Mumick, and V. S. Subrahmanian. 1993. Maintaining views incrementally. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data. 157--166. Google ScholarDigital Library
- David Hansel, Rance Cleaveland, and Scott A. Smolka. 2004. Distributed prototyping from validated specifications. Journal of Systems and Software 70, 3, 275--298. Google ScholarDigital Library
- Patrick Hunt, Mahadev Konar, Flavio Paiva Junqueira, and Benjamin Reed. 2010. ZooKeeper: Wait-free coordination for Internet-scale Systems. In USENIX Annual Technical Conference. 9. Google ScholarDigital Library
- Dilsun Kaynar, Nancy Lynch, Roberto Segala, and Frits Vaandrager. 2010. The Theory of Timed I/O Automata. 2nd ed.) Morgan 8 Claypool, San Francisco, CA. Google ScholarDigital Library
- Ingolf Heiko Krüger. 1996. An experiment in compiler design for a concurrent object-based programming language. Master’s thesis. The University of Texas at Austin, Austin, TX.Google Scholar
- A. D. Kshemkalyani and M. Singhal. 2008. Distributed Computing: Principles, Algorithms, and Systems. Cambridge University Press, New York, NY. Google ScholarDigital Library
- Avinash Lakshman and Prashant Malik. 2010. Cassandra: A decentralized structured storage system. ACM SIGOPS Operating Systems Review 44, 2, 35--40. Google ScholarDigital Library
- Leslie Lamport. 1978. Time, clocks, and the ordering of events in a distributed system. Communications of the ACM 21, 7, 558--565. Google ScholarDigital Library
- Leslie Lamport. 2002. Specifying Systems: The TLA+ Language and Tools for Hardware and Software Engineers. Addison-Wesley, New York, NY. Google ScholarDigital Library
- Leslie Lamport. 2009. The pluscal algorithm language. In Proceedings of the 6th International Colloquium on Theoretical Aspects of Computing. Springer, 36--60. Google ScholarDigital Library
- Jim Larson. 2009. Erlang for concurrent programming. Communications of the ACM 52, 3, 48--56. Google ScholarDigital Library
- Barbara Liskov. 1988. Distributed programming in Argus. Communications of the ACM 31, 3, 300--312. Google ScholarDigital Library
- Yanhong Annie Liu. 2013. Systematic Program Design: From Clarity to Efficiency. Cambridge University Press. New York, NY. Google ScholarDigital Library
- Yanhong A. Liu, Jon Brandvein, Scott D. Stoller, and Bo Lin. 2016. Demand-driven incremental object queries. In Proceedings of the 18th International Symposium on Principles and Practice of Declarative Programming. ACM Press, 228--241. Google ScholarDigital Library
- Yanhong A. Liu, Michael Gorbovitski, and Scott D. Stoller. 2009. A language and framework for invariant-driven transformations. In Proceedings of the 8th International Conference on Generative Programming and Component Engineering. ACM Press, 55--64. Google ScholarDigital Library
- Yanhong A. Liu and Scott D. Stoller. 2003. Dynamic programming via static incrementalization. Higher-Order and Symbolic Computation 16, 1--2, 37--62. Google ScholarDigital Library
- Yanhong A. Liu and Scott D. Stoller. 2009. From datalog rules to efficient programs with time and space guarantees. ACM Transactions on Programming Languages and Systems 31, 6, 1--38. Google ScholarDigital Library
- Yanhong A. Liu, Scott D. Stoller, Michael Gorbovitski, Tom Rothamel, and Yanni E. Liu. 2005a. Incrementalization across object abstraction. In Proceedings of the 20th ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications. 473--486. Google ScholarDigital Library
- Yanhong A. Liu, Scott D. Stoller, Ning Li, and Tom Rothamel. 2005b. Optimizing aggregate array computations in loops. ACM Transactions on Programming Languages and Systems 27, 1, 91--125. Google ScholarDigital Library
- Yanhong A. Liu, Scott D. Stoller, and Bo Lin. 2012. High-level executable specifications of distributed algorithms. In Proceedings of the 14th International Symposium on Stabilization, Safety, and Security of Distributed Systems. Springer, 95--110. Google ScholarDigital Library
- Yanhong A. Liu, Scott D. Stoller, Bo Lin, and Michael Gorbovitski. 2012. From clarity to efficiency for distributed algorithms. In Proceedings of the 27th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages and Applications. 395--410. Google ScholarDigital Library
- Yanhong A. Liu, Chen Wang, Michael Gorbovitski, Tom Rothamel, Yongxi Cheng, Yingchao Zhao, and Jing Zhang. 2006. Core role-based access control: Efficient implementations by transformations. In Proceedings of the ACM SIGPLAN 2006 Workshop on Partial Evaluation and Program Manipulation. 112--120. Google ScholarDigital Library
- Nuno P. Lopes, Juan A. Navarro, Andrey Rybalchenko, and Atul Singh. 2010. Applying Prolog to develop distributed systems. Theory and Practice of Logic Programming 10, 4--6, 691--707. Google ScholarDigital Library
- Nancy A. Lynch. 1996. Distributed Algorithms. Morgan Kaufman, San Francisco, CA. Google ScholarDigital Library
- Friedemann Mattern. 1989. Virtual time and global states of distributed systems. In Proceedings of the International Workshop on Parallel and Distributed Algorithms. North-Holland, Amsterdam, The Netherlands, 120--131.Google Scholar
- Petar Maymounkov and David Mazières. 2002. Kademlia: A peer-to-peer information system based on the XOR metric. In Peer-to-Peer Systems. 53--65. Google ScholarDigital Library
- MPI. 2015. Message Passing Interface Forum. Retrieved April 8, 2017 from http://www.mpi-forum.org/.Google Scholar
- Hiroaki Nakamura. 2001. Incremental computation of complex object queries. In Proceedings of the 16th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications. 156--165. Google ScholarDigital Library
- Robert Paige. 1989. Real-time simulation of a set machine on a RAM. In Proceedings of the International Conference on Computing and Information. Canadian Scholars Press, Toronto, ON, 69--73.Google Scholar
- Robert Paige and Shaye Koenig. 1982. Finite differencing of computable expressions. ACM Transactions on Programming Languages and Systems 4, 3, 402--454. Google ScholarDigital Library
- Vyacheslav Petukhin. 1997. Programs with universally quantified embedded implications. In Proceedings of the 4th International Conference on Logic Programming and Nonmonotonic Reasoning. Springer, 310--324. Google ScholarDigital Library
- PRL Project. 2013. EventML. Retrieved April 8, 2017 from http://www.nuprl.org/software/#WhatisEventML.Google Scholar
- William Pugh and Tim Teitelbaum. 1989. Incremental computation via function caching. In Conference Record of the 16th Annual ACM Symposium on Principles of Programming Languages. 315--328. Google ScholarDigital Library
- G. Ramalingam and Thomas Reps. 1993. A categorized bibliography on incremental computation. In Conference Record of the 20th Annual ACM Symposium on Principles of Programming Languages. 502--510. Google ScholarDigital Library
- Michel Raynal. 1988. Distributed Algorithms and Protocols. Wiley, Hoboken, NJ. Google ScholarDigital Library
- Michel Raynal. 2010. Communication and Agreement Abstractions for Fault-Tolerant Asynchronous Distributed Systems. Morgan 8 Claypool, San Francisco, CA. Google ScholarDigital Library
- Michel Raynal. 2013. Distributed Algorithms for Message-Passing Systems. Springer. Google ScholarDigital Library
- Tom Rothamel and Yanhong A. Liu. 2008. Generating incremental implementations of object-set queries. In Proceedings of the 7th International Conference on Generative Programming and Component Engineering. ACM Press, 55--66. Google ScholarDigital Library
- Antony Rowstron and Peter Druschel. 2001. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms (Middleware 2001). Springer, 329--350. Google ScholarDigital Library
- Diptikalyan Saha and C. R. Ramakrishnan. 2003. Incremental evaluation of tabled logic programs. In Proceedings of the 19th International Conference on Logic Programming. Springer, 392--406.Google Scholar
- Michael L. Scott. 1991. The Lynx distributed programming language: Motivation, design, and experience. Computer Languages 16, 3, 209--233. Google ScholarDigital Library
- Traian Florin Serbanuta, Grigore Rosu, and Jose Meseguer. 2009. A rewriting logic approach to operational semantics. Information and Computation 207, 2, 305--340. Google ScholarDigital Library
- Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. 2010. The Hadoop distributed file system. In Proceedings of the IEEE 26th Symposium on Mass Storage Systems and Technologies. IEEE CS Press, 1--10. Google ScholarDigital Library
- I. Stoica, R. Morris, D. Liben-Nowell, D. R. Karger, M. F. Kaashoek, F. Dabek, and H. Balakrishnan. 2003. Chord: A scalable peer-to-peer lookup protocol for Internet applications. IEEE/ACM Transactions on Networking 11, 1, 17--32. Google ScholarDigital Library
- Theresa Swift, David S. Warren, and others. 2016. The XSB System Version 3.7,x. Retrieved April 8, 2017 from http://xsb.sourceforge.net.Google Scholar
- Gerard Tel. 2000. Introduction to Distributed Algorithms (2nd ed.) Cambridge University Press, New York, NY. Google ScholarDigital Library
- Robbert van Renesse and Deniz Altinbuken. 2015. Paxos made moderately complex. Computer Surveys 47, 3, 42:1--42:36. Google ScholarDigital Library
- Robbert van Renesse and Fred B. Schneider. 2004. Chain replication for supporting high throughput and availability. In Proceedings of the 6th USENIX Symposium on Operating Systems Design and Implementation. USENIX Association, 91--104. Google ScholarDigital Library
- Dan E. Willard. 1984. Efficient processing of relational calculus expressions using range query theory. In Proceedings of the 1984 ACM SIGMOD International Conference on Management of Data. 164--175. Google ScholarDigital Library
- Dan E. Willard. 2002. An algorithm for handling many relational calculus queries efficiently. Journal of Computer and System Sciences 65, 295--331.Google ScholarCross Ref
- Andrew K. Wright and Matthias Felleisen. 1994. A syntactic approach to type soundness. Information and Computation 115, 38--94. Google ScholarDigital Library
- Ben Y. Zhao, Ling Huang, Jeremy Stribling, Sean C. Rhea, Anthony D. Joseph, and John D Kubiatowicz. 2004. Tapestry: A resilient global-scale overlay for service deployment. IEEE Journal on Selected Areas in Communications 22, 1, 41--53. Google ScholarDigital Library
Index Terms
- From Clarity to Efficiency for Distributed Algorithms
Recommendations
From clarity to efficiency for distributed algorithms
OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applicationsThis paper describes a very high-level language for clear description of distributed algorithms and optimizations necessary for generating efficient implementations. The language supports high-level control flows where complex synchronization conditions ...
From clarity to efficiency for distributed algorithms
OOPSLA '12This paper describes a very high-level language for clear description of distributed algorithms and optimizations necessary for generating efficient implementations. The language supports high-level control flows where complex synchronization conditions ...
Type-directed automatic incrementalization
PLDI '12: Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and ImplementationApplication data often changes slowly or incrementally over time. Since incremental changes to input often result in only small changes in output, it is often feasible to respond to such changes asymptotically more efficiently than by re-running the ...
Comments