Abstract
This paper describes a system called Trends for managing IT resources in a production server environment. The objective of Trends is to reduce operational costs associated with unplanned outages, unbalanced utilization of resources, and inconsistent service delivery. The Trends resource manager balances utilization of multiple resources such as processor and disk space, manages growth to extend resource lifetimes, and factors in variability to improve temporal stability of balancing solutions. The methodology applies to systems in which workload has a strong affinity to databases, files, or applications that can be selectively placed on one or more nodes in a distributed system. Studies in a production environment demonstrate that balancing solutions remain stable for as long as the 9–12 months covered by our data. This work takes place in the context of the Lotus Notes distributed database system, and is based on analysis and data from a production server farm hosting over 20,000 databases.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
L. Kawell Jr., S. Beckhardt, T. Halvorsen, R. Ozzie, and I. Grief, Replicated document management in a group communication system, in D. Marca, and G. Bock, (eds.), Groupware: Software For Computer-Supported Cooperative Work, IEEE Computer Society Press, Los Alamitos, CA, pp. 226–235, 1992.
J. Lamb and P. Lew, Lotus Notes & Domino 5 Scalable Network Design, McGraw-Hill, 1999.
A. Demers, D. Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, H. Sturgis, D. Swinehart, and D. Terry, Epidemic algorithms for replicated database maintenance, Proceedings of the Sixth Symposium on Principles of Distributed Computing, Vancouver, B.C., Canada, pp. 10–12, August 1987.
W. G. Pope, Planning domino e-mail servers using notes transactions, Proceedings of the 24th International Conference for the Management and Performance Evaluation of Enterprise Computing Systems, Anaheim CA, December 1998.
N. Bobroff, K. Beaty, and G. Bozman, A Load Balancing and Resource Management Strategy for a Distributed Database System, IBM Research report RC-21933, October 2000
B. Gavish and O. Sheng, Dynamic file migration in distributed computer systems, Communications of the ACM, Volume 33, No. 2, February 1990.
Lawrence W. Dowdy and Darrell V. Foster, Comparative Models of the File Assignment Problem, Computing Surveys, Volume 14, No. 2, June 1982.
Joel L. Wolf, The placement optimization program: A practical solution to the disk file assignment problem, Proceedings of the 1989 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, Oakland, California, United States, pp. 1–10, 1989.
Reed A. Hill, System for Managing Data Storage Based on Vector-Summed Size-Frequency Vectors for Data Sets, Devices, and Residual Storage on Devices, U.S. Patent No. 5,345,584, September 6, 1994.
Pope, William G and Mummert, Lily B. Using capacity space methodology for balancing server utilization: description and case studies, 26th International Computer Measurement Group Conference, December 10–15, 2000, Orlando, FL, USA, Proceedings. Computer Measurement Group 2000.
D. Ferrari and S. Zhou, An empirical investigation of load indices for load balancing applications, Proceedings of Performance ‘87, the 12th Annual International Symposium on Computer Performance Modeling, Measurement, and Evaluation, pp. 515–528, 1987.
C. G. Rommel, The Probability of Load Balancing Success in a Homoegenous Network, IEEE Transactions on Software Engineering, September 1991, pp 922–933.
L. Lee, P. Scheuermann, and R. Vingralek, File Assignment in Parallel I/O Systems With Minimal Variance of Service Time, IEEE Transactions on Computers, Vol. 49, No. 2, pp. 127–140, February 2000.
G. Bozman, A File Allocation Study in a Large Time-sharing System, MS thesis, New Jersey Institute of Technology, 1980.
R. Flynn and W. Tetzlaff, Disk striping and block replication algorithms for video file servers, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems, 1996.
A. Dan and D. Sitaram, An online video placement policy based on Bandwidth to Space Ratio (BSR), Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California, May 22–25, 1995.
M. Crovella, M. Harchol-Balter, and C. Murta, Task assignment in a distributed system: Improving performance by unbalancing load, Proceedings of ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, June 1998, Madison, WI.
Author information
Authors and Affiliations
Corresponding author
Additional information
Norman Bobroff has been at IBM T.J. Watson Research Center since receiving his PhD in Physics in 1983. After working in optical lithography and laser metrology he joined computer science. His current interest is storage area networks and server performance.
Lily Mummert is a researcher at IBM T.J. Watson Research Center in Hawthorne, NY. She received a PhD in Computer Science from Carnegie Mellon University in 1996. Her interests include distributed and mobile systems, self-managing systems, and distributed systems management.
Rights and permissions
About this article
Cite this article
Bobroff, N., Mummert, L. Design and Implementation of a Resource Manager in a Distributed Database System. J Netw Syst Manage 13, 151–174 (2005). https://doi.org/10.1007/s10922-005-4439-4
Issue Date:
DOI: https://doi.org/10.1007/s10922-005-4439-4