Abstract
We investigate how different replication policies ranging from least aggressive to most aggressive affect the level of preservation achieved by autonomic processes used by web objects (WOs). Based on simulations of small-world graphs of WOs created by the Unsupervised Small-World algorithm, we report quantitative and qualitative results for graphs ranging in order from 10 to 5000 WOs. Our results show that a moderately aggressive replication policy makes the best use of distributed host resources by not causing spikes in CPU resources nor spikes in network activity while meeting preservation goals. We examine different approaches that WOs can communicate with each other and determine the how long it would take for a message from one WO to reach a specific WO, or all WOs.
Similar content being viewed by others
References
Alam, S.: HTTP mailbox-asynchronous RESTful communication. Master’s thesis, Old Dominion University, Norfolk, VA (2013)
Alam, S., Cartledge, C.L., Nelson, M.L.: HTTP mailbox-asynchronous RESTful communication. Technical report. arXiv:1305.1992 (2013)
Albert, R., Jeong, H., Barabási, A.-L.: Error and attack tolerance of complex networks. Nature 406(6794), 378–382 (2000)
Barabási, A.-L., Albert, R., Jeong, H.: Scale-free characteristics of random networks: the topology of the world wide web. Physica A 281(1), 69–77 (2000)
Beck, M., Moore, T., Plank, J.S.: An end-to-end approach to globally scalable network storage. In: Proceedings of the 2002 conference on applications, technologies, architectures, and protocols for computer communications, pp. 339–346 (2002)
Birman, K.P., Hayden, M., Ozkasap, O., Xiao, Z., Budiu, M., Minsky, Y.: Bimodal multicast. ACM Trans. Comput. Syst. 17(2), 41–88 (1999)
Bollobás, B.: Modern Graph Theory. Springer, New York (1998)
Bollobás, B., Riordan, O., Spencer, J., Tusnády, G.: The degree sequence of a scale-free random graph process. Random Struct Algorithms 18(3), 279–290 (2001)
Carriero, N., Gelernter, D.: Linda in context. Commun. ACM 32(4), 444–458 (1989)
Cartledge, C.: Preserve Me! (... if you can, using Unsupervised Small-World graphs.). http://ws-dl.blogspot.com/2013/10/2013-10-23-preserve-me-if-you-can-using.html/ (2013)
Cartledge, C.L.: A Framework for Web Object Self-Preservation. PhD thesis, Old Dominion University, Norfolk, VA 23529, August (2014)
Cartledge, C.L., Nelson, M.L.: Self-arranging preservation networks. In: Proceedings of the 8th ACM/IEEE-CS joint conference on digital libraries, pp. 445–445 (2008)
Cartledge, C.L., Nelson, M.L.: Unsupervised creation of small world networks for the preservation of digital objects. In: Proceedings of the 9th ACM/IEEE-CS joint conference on digital libraries, pp. 349–352 (2009)
Cartledge, C.L., Nelson, M.L.: Analysis of graphs for digital preservation suitability. In: Proceedings of the 21st ACM conference on hypertext and hypermedia, pp. 109–118. ACM (2010)
Cartledge, C.L., Nelson, M.L.: Connectivity damage to a graph by the removal of an edge or vertex. Technical report. Old Dominion University, Computer Science Department, Norfolk, VA. arXiv:1103.3075 (2011)
Ciancarini, P., Gorrieri, R., Zavattaro, G.: Towards a calculus for generative communication. Formal Methods Open Object-Based Distrib. Syst. 1, 283 (1997)
Cooper, B., Crespo, A., Garcia-Molina, H.: Implementing a reliable digital object archive. In: Proceedings of the 4th European conference on research and advanced technology for digital libraries, pp. 128–143 (2000)
Cooper, B.F., Garcia-Molina, H.: Peer-to-peer data trading to preserve information. ACM Trans. Inf. Syst. 20(2), 133–170 (2002)
Dabek, F., Kaashoek, M.F., Karger, D., Morris, R., Stoica, I.: Wide-area cooperative storage with CFS. In: Proceedings of the 18th annual ACM symposium on operating systems principles, October (2001)
de la Rosa, J.L., Del Acebo, E., Trias, A., Aciar, S., Quisbert, H.: Crew intelligence systems for digital objects preservation. In: The 2nd swarm intelligence algorithms and applications symposium-SIAAS, vol. 9 (2009)
de la Rosa, J.L., Olvera, J.A.: First studies on self-preserving digital objects. In: Artificial intelligence research and development: Proceedings of the 15th international conference of the Catalan Association for Artificial Intelligence and Applications, pp. 213–222 (2012)
Duchon, P., Hanusse, N., Lebhar, E., Schabanel, N.: Could any graph be turned into a small-world? Theor. Comput. Sci. 355(1), 96–103 (2006)
Duchon, P., Hanusse, N., Lebhar, E., Schabanel, N.: Towards small world emergence. In: ACM symposium on parallelism in algorithms and architectures, pp. 225–232 (2006)
Gantz, J., Reinsel, D.: The Digital Universe in 2020: big data, bigger digital shadows, and biggest growth in the far east. IDC iView IDC Anal. Futur. 2007, 1–16 (2012)
Gaume, B., Mathieu, F.: From random graph to small world by wandering. Technical report 6489, Unité de recherche INRIA Rocquencourt (2008)
Gelernter, D., Carriero, N.: Coordination languages and their significance. Commun. ACM 35(2), 97–107 (1992)
Goh, K.I., Kahng, B., Kim, D.: Universal behavior of load distribution in scale-free networks. Phys. Rev. Lett. 87(27), 278701 (2001)
Hunter, J., Choudhury, S.: A semi-automated digital preservation system based on semantic web services. In: Proceedings of the 4th ACM/IEEE-CS joint conference on digital libraries, pp. 269–278 (2004)
Ikeda, S., Kubo, I., Yamashita, M.: The hitting and cover times of random walks on finite graphs using local degree information. Theor. Comput. Sci. 410(1), 94–100 (2009)
Kahn, R., Wilensky, R.: A framework for distributed digital object services. Int. J. Digit. Libr. 6(2), 115–123 (2006)
Kleinberg, J.: The small-world phenomenon: an algorithmic perspective. In: Proceedings of the 32nd ACM symposium on theory of computing 32, 163–170 (2000)
Klemm, K., Eguíluz, V.M.: Growing scale-free networks with small-world behavior. Phys. Rev. E. 65(5), 26107 (2002)
Maniatis, P., Roussopoulos, M., Giuli, T.J., Rosenthal, D.S.H., Baker, M.: The LOCKSS peer-to-peer digital preservation system. ACM Trans. Comput. Syst. 23(1), 2–50 (2005)
McCown, F., Nelson, M.L.: What happens when facebook is gone? In: Proceedings of the 9th ACM/IEEE-CS joint conference on digital libraries, pp. 251–254 (2009)
Milian, M.: GeoCities’ Time has Expired. Yahoo Closing the Site Today. Los Angeles Times, Los Angeles, USA (2009)
Miller, I., Freund, J.E.: Probability and Statistics for Engineers. Prentice-Hall, Englewood Cliffs, NJ (1977)
Nelson, M.L., Van de Sompel, H.: IJDL special issue on complex digital objects: Guest editors’ introduction. Int. J. Digit. Libr. 6(2), 113–114 (2006)
Newman, M.E.J.: Models of the small world: a review. J. Stat. Phys. 101, 819 (2000)
Nguyen, V., Martel, C.: Analyzing and characterizing small-world graphs. In: ACM-SIAM symposium on discrete algorithms, pp. 311–320 (2005)
Payette, S., Staples, T.: The Mellon Fedora Project. In: Proceedings of the 6th European conference on research and advanced technology for digital libraries, pp. 406–421 (2002)
Rajasekar, A., Wan, M., Moore, R.: MySRB and SRB: components of a data grid. In: Proceedings of the 11th IEEE international symposium on high performance distributed computing, pp. 301–310 (2002)
Rajasekar, A., Wan, M., Moore, R., Schroeder, W.: A prototype rule-based distributed data management system. In: HPDC workshop on next generation distributed sata management (2006)
Ratnasamy, S., Francis, P., Handley, M., Karp, R., Schenker, S.: A Scalable Content-Addressable Network. In: Proceedings of the 2001 conference on applications, technologies, architectures, and protocols for computer communications, pp. 161–172 (2001)
Reich, V.: CLOCKSS—it takes a community. Ser. Libr. 54(1–2), 135–139 (2008)
Reynolds, C.W.: Flocks, herds and schools: a distributed behavioral model. SIGGRAPH Comput. Graph. 21(4), 25–34 (1987)
Rhea, S., Wells, C., Eaton, P., Geels, D., Zhao, B., Weatherspoon, H., Kubiatowicz, J.: Maintenance-free global data storage. IEEE Internet Comput. 5(5), 40–49 (2001)
Rosenthal, D.S.H., Rosenthal, D.C., Miller, E.L., Adams, I.F., Storer, M.W., Zadok, E.: The economics of long-term digital storage. Paper presented at the Memory of the World in the Digital Age, Vancouver, BC (2012)
Rosenthal, D.S.H.: Estimating storage costs. http://blog.dshr.org/2013/11/estimating-storage-costs.html (2013)
Rosenthal, D.S.H., Robertson, T.S., Lipkis, T., Reich, V., Morabito, S.: Requirements for digital preservation systems: a bottom-up approach. Dlib Mag. 11 (2005)
Rothenberg, J.: Avoiding Technological Quicksand: Finding a Viable Technical Foundation for Digital Preservation. A Report to the Council on Library and Information Resources. Council on Library and Information Resources, Washington, DC (1999)
Salaheldeen, H.M., Nelson, M.L.: Resurrecting My Revolution: Using Social Link Neighborhood in Bringing Context to the Disappearing Web. In: Proceedings of theory and practice of digital libraries, pp. 333–345 (2013)
Smith, M.: DSpace: an institutional repository from the MIT libraries and Hewlett Packard Laboratories. In: Proceedings of the 6th European conference on research and advanced technology for digital libraries, pp. 543–549 (2002)
Spector, L., Klein, J., Perry, C., Feinstein, M.: Emergence of collective behavior in evolving populations of flying agents. In: Genetic and Evolutionary Computation Conference, pp. 61–73. Springer (2003)
Van de Sompel, H., Bekaert, J., Liu, X., Balakireva, L., Schwander, T.: aDORe: a modular, standards-based digital object repository. Comput. J. 48(5), 514–535 (2005)
Walker, R.: Cyberspace When You’re Dead. The New York Times, NY, New York (2011)
Waters, D., Garrett, J.: Preserving Digital Information. Report of the Task Force on Archiving of Digital Information. The Commission on Preservation and Access, Washington, DC (1996)
Watts, D.J., Strogatz, S.H.: Collective dynamics of ‘small world’ networks. Nature 393, 440–442 (1998)
Wohlsen, M.: Digital Data that Never Dies. Associated Press, NY, New York (2011)
Yin, S.: Flickr Permanently Deletes User’s Account, 4,000 Photos by Accident. PC Magazine, February (2011)
Acknowledgments
This work supported in part by the National Science Foundation (NSF), Project 370161.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Cartledge, C., Nelson, M.L. When should I make preservation copies of myself?. Int J Digit Libr 16, 183–205 (2015). https://doi.org/10.1007/s00799-015-0155-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00799-015-0155-1