Abstract
File systems are often used to store persistent application data, but manipulating file systems using standard APIs can be difficult for programmers. Forest is a domain-specific language that bridges the gap between the on-disk and in-memory representations of file system data. Given a high-level specification of the structure, contents, and properties of a collection of directories, files, and symbolic links, the Forest compiler generates tools for loading, storing, and validating that data. Unfortunately, the initial implementation of Forest offered few mechanisms for controlling cost-e.g., the run-time system could load gigabytes of data, even if only a few bytes were needed. This paper introduces Incremental Forest (iForest), an extension to Forest with an explicit delay construct that programmers can use to precisely control costs. We describe the design of iForest using a series of running examples, present a formal semantics in a core calculus, and define a simple cost model that accurately characterizes the resources needed to use a given specification. We propose skins, which allow programmers to modify the delay structure of a specification in a compositional way, and develop a static type system for ensuring compatibility between specifications and skins. We prove the soundness and completeness of the type system and a variety of algebraic properties of skins. We describe an OCaml implementation and evaluate its performance on applications developed in collaboration with watershed hydrologists.
- S.-C. Buraga. An XML-based semantic description of distributed file systems. In RoEduNet, 2003.Google Scholar
- K. Fisher and R. Gruber. PADS: A domain specific language for processing ad hoc data. In PLDI, June 2005. Google ScholarDigital Library
- K. Fisher, Y. Mandelbaum, and D. Walker. The next 700 data description languages. J. ACM, 57, February 2010. Google ScholarDigital Library
- K. Fisher, N. Foster, D. Walker, and K. Q. Zhu. Forest: A language and toolkit for programming with filestores. In ICFP, 2011. Google ScholarDigital Library
- M. Gabriel, C. Knightes, E. Cooter, and R. Dennis. Evaluating relative sensitivity of SWAT-simulated nitrogen discharge to projected climate and land cover changes for two watersheds in North Carolina, USA. Hydrological Processes, 2015.Google Scholar
- K. K. Garg, L. Bharati, A. Gaur, B. George, S. Acharya, K. Jella, and B. Narasimhan. Spatial mapping of agricultural water productivity using the SWAT model in Upper Bhima Catchment, India. Irrigation and Drainage, 61(1), 2012.Google ScholarCross Ref
- B. Guse, M. Pfannerstill, and N. Fohrer. Dynamic modelling of land use change impacts on nitrate loads in rivers. Environmental Processes, 2(4), 2015.Google Scholar
- T. Harris, S. Marlow, S. Peyton Jones, and M. Herlihy. Composable memory transactions. In PPoPP, 2005. Google ScholarDigital Library
- LINQ: .NET Language-Integrated Query. LINQ: .NET language-integrated query. http://msdn.microsoft.com/ library/bb308959.aspx, Feb. 2007.Google Scholar
- D. N. Moriasi, J. G. Arnold, M. W. Van Liew, R. L. Bingner, R. D. Harmel, and T. L. Veith. Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Transactions of the ASABE, 50(3), 2007.Google ScholarCross Ref
- J. Nash and J. Sutcliffe. River flow forecasting through conceptual models part I — A discussion of principles. Journal of Hydrology, 10(3), 1970.Google Scholar
- T. S. Ngo, D. B. Nguyen, and P. S. Rajendra. Effect of land use change on runoff and sediment yield in Da River Basin of Hoa Binh province, Northwest Vietnam. Journal of Mountain Science, 12(4), 2015.Google ScholarCross Ref
- SWAT. Soil and Water Assessment Tool. http://swat.tamu. edu.Google Scholar
- SWAT IO Documentation. SWAT Input/Output Documentation. http://swat.tamu.edu/documentation/ 2012-io/, 2012.Google Scholar
- D. Syme. Looking ahead with F#: Taming the data deluge. Presentation at the Workshop on F# in Education, Nov. 2010.Google Scholar
- C. B. Zou, L. Qiao, and B. P. Wilcox. Woodland expansion in central Oklahoma will significantly reduce streamflows – a modelling analysis. Ecohydrology, 2015.Google Scholar
Recommendations
Incremental forest: a DSL for efficiently managing filestores
OOPSLA 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and ApplicationsFile systems are often used to store persistent application data, but manipulating file systems using standard APIs can be difficult for programmers. Forest is a domain-specific language that bridges the gap between the on-disk and in-memory ...
Forest: a language and toolkit for programming with filestores
ICFP '11: Proceedings of the 16th ACM SIGPLAN international conference on Functional programmingA filestore is a structured collection of data files housed in a conventional hierarchical file system. Many applications use filestores as a poor-man's database, and the correct execution of these applications requires that the collection of files, ...
Forest: a language and toolkit for programming with filestores
ICFP '11A filestore is a structured collection of data files housed in a conventional hierarchical file system. Many applications use filestores as a poor-man's database, and the correct execution of these applications requires that the collection of files, ...
Comments