A holistic framework to assess the sustainability of irrigated agricultural systems

Irrigated agriculture is a key activity for the long-term survival of humanenvironmental systems and the assessment of agricultural sustainability has been gaining increasing relevance. In spite of several proposals developed, there is not a holistic approach that can be generally applied to assess sustainability of irrigated agricultural areas. In this paper we present a framework and associated indicators for the assessment of sustainability of irrigated agricultural systems in different contexts and locations. The framework covers four main sustainability dimensions: environmental integrity, economic resilience and profitability, social wellbeing and good governance. This approach was tested in 10 agricultural areas in eight different countries that represent a wide variety of situations in terms of agricultural development, environmental conditions, socio-economic settings and political contexts, but that share the fact that water use is a critical aspect for agricultural development. The obtained results illustrate the usefulness of the proposed framework to obtain a holistic picture of sustainability in irrigated agriculture areas, even in situations *Corresponding author: Paula Antunes, Faculdade de Ciências e Tecnologia, CENSE – Centre for Environmental and Sustainability Research, Universidade Nova de Lisboa, Caparica 2829-516, Portugal E-mail: mpa@fct.unl.pt Reviewing editor: Fatih Yildiz, Middle East Technical University, Turkey Additional information is available at the end of the article

Paula Antunes is professor at Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa and a researcher in CENSE -Center for Environmental and Sustainability Research. Her research interests focus in sustainability assessment, mapping and assessment of ecosystem services, environmental management and methods for participatory decision support in sustainability issues .
This work has been developed by a large team of co-authors in the scope of the EU FP7 SIRIUS project (Sustainable Irrigation water management and River-basin governance: Implementing User-driven Services). Rui Santos and Inês Cosme are researchers in the same institute. They have developed the sustainability assessment framework presented herein and coordinated the data collection process with the first author. Anna Osann and Alfonso Calera from Universidad de Castilla-La Mancha are the coordinators of SIRIUS. All authors share a common interest in the development of EO based products and services to support of sustainable irrigated agriculture.

PUBLIC INTEREST STATEMENT
Parallel to the acknowledgement of the critical role that irrigated agriculture plays in meeting the demand for food and fiber of a growing world population, there is significant evidence about its environmental and social impacts. The promotion of a more sustainable agriculture is pointed as a solution, but generally applicable tools to support the measurement of agricultural sustainability are still lacking.
We developed a framework to assess sustainability of irrigated agricultural systems in different contexts covering four main dimensions: environmental integrity, economic resilience and profitability, social wellbeing and good governance. This approach was tested in 10 areas in eight countries that represent a wide variety of situations in terms of agricultural development, environmental conditions, socio-economic settings and political contexts. The results illustrate its usefulness to get an overall picture of sustainability in irrigated agriculture areas and as a starting point for the construction of roadmaps of actions towards sustainability.
In this context, agricultural sustainability has gained increasing relevance as a way to overcome these challenges and its assessment has become a principal endeavor for agricultural research, policy and practice (Van Cauwenbergh et al., 2007). Most definitions of sustainable agriculture are focused on the practices and technologies adopted (Pretty, 2008), referring to practices that are environmentally sound, economically profitable and socially just (Dale, Kline, Kaffka, & Langeveld, 2013). Sustainable agriculture must be capable of maintaining its productivity and usefulness to society indefinitely (Ikerd, 1993;Lewandowski, Härdtlein, & Kaltschmitt, 1999). Such an agriculture should use farming systems that conserve resources, protect the environment, produce efficiently, compete commercially, and enhance the quality of life for farmers and society overall.
The goal of agricultural sustainability is particularly difficult to achieve in intensive agricultural systems that are dependent on the input of one of the key resources for agricultural production: water. Irrigation has always played a central role in the development of agricultural systems in many parts of the world. Civilizations have risen and fallen with the growth and decline of their irrigation systems, while others have maintained sustainable irrigation for thousands of years (van Schilfgaarde, 1994). Parallel to the acknowledgement of the critical role that irrigated agriculture plays in meeting the demand for food and fiber of a growing world population, there is significant evidence about the environmental and social impacts of irrigated agriculture (Matson, 1997;Tilman et al., 2002). Even the economic sustainability of agricultural infrastructure projects is questioned in many cases.
In this paper we present a framework for the assessment of sustainability of irrigated agricultural systems that was developed in the scope of the EU FP7 SIRIUS project. SIRIUS (Sustainable Irrigation water management and River-basin governance: Implementing User-driven Services) aimed to stimulate the development of satellite assisted services for irrigation water management in agricultural regions under conditions of water scarcity and drought and for supporting water governance issues (e.g. trans-boundary water resources management, dealing with illegal water abstractions, multi-users and sectors). The application of SIRIUS was tested in 10 pilot areas in eight different countries (Figure 1) that represent a wide variety of situations in terms of agricultural development, environmental conditions, socio-economic settings and political contexts, but that share the fact that water use is a critical aspect for the sustainability of agriculture.
In the following sections, we review different approaches and frameworks that have been developed to assess the sustainability of agricultural systems and describe the SIRIUS framework, stressing the main contributions in relation to existing approaches. We then present the results that we obtained from the application of this framework in the pilot areas and derive lessons in terms of pathways for sustainable agriculture and suitability of the suggested framework.

The challenge of measuring agricultural sustainability
Agricultural sustainability is not only a difficult concept to define (Hayati, Ranjbar, & Karami, 2010), but also is difficult to implement, measure and monitor (Binder, Feola, & Steinberger, 2010;Farshad & Zinck, 1993;Hayati et al., 2010;Rao & Rogers, 2006). Several approaches to guide the assessment of agricultural sustainability have been proposed in the literature, relying usually on the definition of a set of sustainability criteria and associated indicators, organized around a conceptual or operational framework, generally structured following a hierarchy (de Olde, Oudshoorn, Sørensen, Bokkers, & de Boer, 2016). These methods differ in terms of their purpose, scope for assessment, underlying sustainability concepts, dimensions and indicators covered and development approach. In Table 1 we present an overview of some of these proposals, distinguishing their focus, coverage of sustainability dimensions and methodological approach followed for assessment and integration.
Many of the frameworks and approaches identified have been guided by local priorities and practices and only limited attempts at developing systematic, widely applicable frameworks are reported, as has already been noted in Rao and Rogers (2006). Most of the approaches that have been   (Paracchini et al., 2015) Farm level assessment of sustainability of management practices considering environmental and economic pillars 22 + 6 9 In environmental we have considered the indicators included in the agronomic dimension (22) and in the ecological dimension (6) To be used by farmers, technical advisors and policy makers Aggregation of indicators using a compensatory approach within each dimension and non-compensatory among the three dimensions suggested focus on the assessment of agricultural practices at farm level (e.g. farming practices and farming systems), cropping systems, agricultural organizations (e.g. agri-businesses), or the food value chain. Methods aiming to assess sustainability at the landscape/region/watershed scale are not so common (Dale et al., 2013).
Some of the proposals are more targeted to environmental concerns and agricultural management practices (Castoldi & Bechini, 2010;Rigby, Woodhouse, Young, & Burton, 2001), while others (FAO, 2013a;Zahm, Viaux, Girardin, Vilain, & Mouchet, 2006) try to cover more sustainability dimensions combining environmental with economic and/or social aspects. Governance aspects are absent in most of the reviewed methods, with the exception of the SAFA (FAO, 2013b) and Sustainable Rural Livelihoods (Scoones, 1998;Woodhouse, Howlett, & Rigby, 2000) frameworks. The last one acknowledges explicitly the linkage between the wider economic, institutional and policy context as part of the analysis of strategies of natural resource management.
Some authors use sustainability polygons/webs to represent graphically the sustainability of systems along the dimensions considered, while others rely on the calculation of an aggregate indicator/composite index. Displaying individual indicator scores separately avoids having to aggregate across different incommensurable scales/dimensions and is more compatible with a strong sustainability approach (Gómez-Limón & Sanchez-Fernandez, 2010;Martinez-Alier, Munda, & O'Neill, 1998). On the other hand, it does not allow the direct comparison of cases/farming systems, or the establishment of sustainability rankings and the results are more difficult to interpret and communicate. In the end, the approach to aggregation of indicators depends largely on the objectives of the sustainability evaluation exercise.
In the scope of SIRIUS there was a need to assess the sustainability of irrigated agricultural areas in a holistic perspective, in order to support the development of roadmaps for action at different levels towards a sustainable future. We also wanted to identify key entry points to stimulate the uptake of SIRIUS products and services in farming systems. Although the different approaches reviewed contain elements that are useful for these purposes, there was no single approach that could be directly transferred. As stated, most of the existing frameworks do not cover all sustainability dimensions in a comprehensive and balanced way and/or are not focused on the irrigation perimeter/watershed scale. Water management, which is a central issue in irrigated agriculture, is fully addressed only in the Water Vulnerability Index (Sullivan & Meigh, 2005), which, on the other hand does not cover other sustainability aspects. These constraints have motivated the proposal for a new framework for assessing the sustainability of irrigated agricultural systems at the scale of an agricultural area/irrigation perimeter.

Purpose and framework development process
The main purpose of the SIRIUS sustainability assessment framework is to provide a holistic understanding and represent the "big picture" regarding sustainability of irrigated agricultural areas in order to support the formulation of sustainability strategies. The main objective is to understand the major issues related to the environmental, economic and social dimensions of sustainability of agricultural systems and associated governance requirements. This framework is to be used to support the identification of strong points and opportunities for improvement, as well as the development of roadmaps of actions to promote sustainability of agricultural systems.
The SIRIUS framework was designed to be of quite general application, meaning that it may be used to assess sustainability of irrigated agricultural systems in different parts of the world and in very different contexts, ranging from highly sophisticated, intensive and technology-based irrigation systems, to more traditional and family-based agro-systems. We adopted a transdisciplinary approach (Binder et al., 2010) to the development of the framework in which we combined a theory-based definition of sustainability with the input from relevant stakeholders. We took as a starting point the definition of J. Ehrenfeld that sustainability is the "possibility that humans and other life will flourish on the Earth forever" (Ehrenfeld, 2008) and we have worked towards the identification of the basic conditions necessary to ensure this possibility in irrigated agricultural systems. In line with what is suggested by Pretty (2008), our approach to agricultural sustainability incorporates concepts of both resilience (the capacity of systems to buffer shocks and stresses) and persistence (the capacity of systems to continue over long periods), and addresses wider environmental, economic and social conditions as well as the associated governance systems to achieve these two aims. We have used some elements of other approaches, such as the framework for socio-economic assessment of irrigated agricultural systems developed by Santos, Antunes, and Beça (2008), the lessons from the multi-criteria analysis of irrigated agricultural systems presented in Antunes, Karadzic, Santos, Beça, and Osann (2011) and the Climate Vulnerability Index and Water Vulnerability Index (Sullivan, 2011;Sullivan & Meigh, 2005). The dimensions and core components considered in the SAFA framework (FAO, 2013a) were also used as a basis to define some elements of this framework.
In order to identify important aspects to be included in the framework, we organized a participatory workshop with members of SIRIUS regional teams that included researchers, water users and water managers (e.g. members of regional agricultural/water bodies, water users associations) from the 10 pilot areas, in a total of 29 people. We organized the participants in groups of 2-4 elements according to their area of origin and asked them to discuss whether they considered their area, and the agricultural activity developed therein, to be sustainable. Each group had to identify the arguments pro and against the claim for sustainability in their pilot area. The groups had to report back to the plenary and there was a joint discussion around the main issues that were identified. From the inputs of the different areas, we extracted a list of key sustainability aspects to be included in the framework. Given the diversity of geographical, cultural and agricultural contexts covered by SIRIUS pilot areas, we ensured this way the coverage of concerns from different perspectives. This process was fundamental to adjust and complement the list of issues that had been previously identified based on literature. In this way, we combined a top-down and a bottom-up approach in the framework development process.
The SIRIUS framework is to be applied to assess sustainability of irrigated agricultural systems in very different countries and contexts, at the irrigation perimeter/watershed scale, and therefore one important concern was its suitability to be applied in contexts of reduced availability of baseline information. Many of the indicators used to measure sustainability "are not particularly useful to farmers/water managers or are too time-consuming to measure on a regular basis" (de Olde et al., 2016;Norman, Janke, Freyenberger, Schurle, & Kok, 1997). Taking this into account, the associated burden of data collection was considered in the selection the indicators to be included.
The SIRIUS framework was conceived and tested for application in irrigation perimeters. However, it may also be applied at other spatial levels, such as the farm level, landscape level, the hydrological region (river basin level), or even at the national level.

Main elements of the framework
The SIRIUS framework considers four basic dimensions in sustainability assessment of irrigated agricultural areas: (1) environmental integrity; (2) economic profitability and resilience; (3) social wellbeing and (4) good governance. For each dimension a set of core issues were identified, to which aspects to be evaluated and corresponding indicators are associated. Figure 2 summarizes the four dimensions and core issues of the SIRIUS framework. Table 2 details the aspects to be evaluated in each core issue and suggested indicators to be used in the assessment. Not all indicators listed in Table 2 are primary indicators (i.e. that can be directly measured). Some of them are indices or composite indicators that result from the combination of several indicators/variables. We use here the term indicator to refer to all measurable variables that are used to evaluate the sustainability performance of an irrigation perimeter in each core issue, as used in the SAFA system (FAO, 2013a). Some of the indicators refer to variables for which quantitative data are available or that can be measured in an objective way (e.g. water productivity in agriculture), while others refer to the assessment of subjective aspects and perceptions (e.g. sense of community). In these cases, and in the absence of data, we recommend the collection of information from expert opinion or stakeholder consultation (see Section 3.3). Note that the indicators list that is provided herein is illustrative and may be adjusted for application in different contexts.

Environmental integrity
The environmental integrity dimension encompasses the evaluation of the integrity of the natural capital of irrigated agricultural areas, and its capacity to ensure the fulfillment of the basic ecological functions underpinning the delivery of a sustainable flow of ecosystem services.
Since the framework is particularly targeted towards irrigated agricultural systems, the sustainability of water use patterns is a major aspect to consider in this dimension. The definition of aspects and indicators used in this core issue was based on the Water Vulnerability Index (Sullivan, 2011).
Water balance is evaluated considering the availability of water resources, their variability and the patterns of water consumption and use in the area. Water productivity is a key aspect for determining sustainability of irrigated agricultural systems. Water productivity is generally defined as the ratio of the net benefits obtained from agricultural system to the amount of water required to produce those benefits (Molden et al., 2007). In this dimension, we are particularly concerned with the physical water productivity that relates the mass of agricultural output to the amount of water used. The results obtained for this indicator are compared with the reference values considered in the Water Footprint Network (WFN) database for the main crops in the corresponding region (Mekonnen & Hoekstra, 2011). The water footprint (Hoekstra & Chapagain, 2007) of the cultivated crops and the virtual water balance (Allan, 1998;Hoekstra & Hung, 2005) for the area are very useful and powerful indicators in this dimension, but they may be difficult to operationalize in many cases.
Water productivity in agriculture can be improved through different pathways that include smallscale water management practices, deficit irrigation, modern irrigation technologies and soil water conservation measures (Ali & Talukder, 2008;Molden et al., 2007). The identification of the most suited strategy (or combination of strategies) in each case is very context dependent and can only be realized taking an integrated and locally adjusted perspective (Molden et al., 2010).
The quality of water resources is measured by the ratio of water bodies classified as having poor or bad quality, according to the local water quality standards, and also by the evaluation of existing cases of constraints to agricultural water use due to poor quality.  The contribution of agricultural areas to biodiversity conservation is assessed by the percentage of cultivated land that is managed adopting biodiversity enhancement practices, such as EU Common Agricultural Policy (CAP) agro-environmental measures (Burton & Paragahawewa, 2011;Kleijn et al., 2006), biodiversity certification systems (Tscharntke et al., 2015) or payments for ecosystem services schemes (Engel, Pagiola, & Wunder, 2008;FAO, 2011;Wunder, 2015).
The maintenance of soil quality is critical to environmental sustainability (Arshad & Martin, 2002), but pressures such as poor fertilizer and water management, soil erosion and shortened fallow periods have led to a growing human-induced soil degradation and loss of productivity (Tilman et al., 2002). Land degradation and soil quality are measured in SIRIUS by the share of soils with poor structure (physical soil quality), degree of soil salinization, share of soils with contamination (chemical quality) and areas under high erosion risks.
Agriculture depends directly on climatic conditions and will be influenced by climate change. On the other hand, agricultural activities use natural resources, in particular energy, and have an impact on climate change (Underwood et al., 2013). Climate change and energy issues are accounted

Core issues Aspects Indicators
Water pricing Water tariffs and taxes Tariff structure (volume/area based)

Value of water tariffs
Other taxes on water

Social well-being
Working environment Wages Average salary in agriculture compared with regional average  considering two broad concerns: (1) vulnerability to climate change and corresponding adaptation plans; and (2) energy intensity of agriculture in the area, with the associated carbon footprint.
Climate change may affect crop productivity through changes in the growing season of crops, the timing of the crop cycle, water availability and irrigation requirements and the increasing frequency and unpredictability of extreme events such as floods and droughts (Calzadilla et al., 2013;Nelson et al., 2009). The direction and magnitude of the expected impacts of these changes in agricultural production in different parts of the world is variable and uncertain. Vulnerability to climate change is an important aspect for the sustainability of agriculture and there has been a growing recognition of the need to develop adaptation measures in face of the expected climate change scenarios (Howden et al., 2007). This is measured in SIRIUS using a 5-point Likert scale, from very low to very high vulnerability. Users are asked to classify the vulnerability of their area in this scale and document the sources of information (e.g. climate change impact assessments, adaptation plans) that ground their score.
Agriculture is an important source of greenhouse gas (GHG) emissions worldwide (Vermeulen, Campbell, & Ingram, 2012). The use of fossil fuels in agricultural machinery and irrigation systems can be an important source of GHG emissions in irrigated agriculture. The indicator energy intensity of agriculture refers mainly to direct energy consumption, comprising the use of energy in field and farm operations (e.g. water pumping). Indirect energy consumption, i.e. energy consumed to produce inputs (e.g. fertilizers) should also be considered, but it is more difficult to assess. Ideally the carbon footprint (Pandey, Agrawal, & Pandey, 2011;Wiedmann & Minx, 2007) of agricultural activities in the study area taking a life-cycle perspective should be calculated, but this indicator may be difficult to calculate in practice and therefore is not considered in the assessment in these cases.

Economic resilience and profitability
The economic dimension of sustainability is assessed considering three main issues: economic efficiency, economic vulnerability and water pricing. Economic efficiency deals mainly with the production capacity of the agro-system and the value it generates for farmers. Besides the standard economic indicators used in this domain, a particular focus is placed in the assessment of water and energy costs and in the calculation of the economic water productivity in agriculture, i.e. the economic benefits obtained per unit of water used (Molden et al., 2007(Molden et al., , 2010. Economic vulnerability encompasses the evaluation of key aspects that reflect how vulnerable the agricultural system is to macroeconomic fluctuations, food market conditions, market power of key actors, financial markets crisis, significant environmental changes or to social and political instability (Briguglio, Cordina, Farrugia, & Vella, 2009;Varela-Ortega, Esteve, Downing, & Bharwani, 2009). Economic vulnerability is related with the capacity to accommodate the impact of these factors in farm income and with the risk that the minimum income that will allow farmers to continue operating will not be achieved. Vulnerability is measured in SIRIUS by the share of economically vulnerable households in the community, the level of subsidy dependence in agriculture, the water reliance of main cultures, the capacity to pay for water and the capacity to use diversification as a strategy to reduce risks (e.g. crops, seeds, financial resources).
Since all SIRIUS pilot areas share the fact that water use is a critical aspect for agricultural development, a particular attention is focused on water pricing as a key economic instrument for water management (Johansson, 2002;Molle, 2009). This includes an analysis of tariffs applied on water services (water tariffs), including their structure and unit values, as well as of other water taxes (e.g. taxes on water abstraction for agriculture) that provide relevant incentives for decisions on water allocation and use.

Social well-being
Similarly to the approach followed in the Sustainable Rural Livelihoods (SRL) framework (Scoones, 1998) and in the SAFA system (FAO, 2013b), the evaluation of social sustainability is centered in the assessment of the assets and conditions to fulfill basic human needs, such as means of living, health and safety, working conditions, as well as the means to satisfy aspirations for a better life, such as education and non-discrimination. Strong social relations and population dynamics are also essential elements of social sustainability of agricultural systems.
The living and working conditions of farmers are an essential element of social sustainability (Källström & Ljung, 2005). In the SIRIUS framework, this accounted by the comparison of wages in agriculture with those in other activity sectors, by the assessment of working conditions for agricultural laborers and by the amount of working hours in agriculture. Non-discrimination and equity are also important issues that are evaluated in terms of gender equity and of the distribution of opportunities and income in society. Issues such as reliance on child labor and other aspects that may be particularly relevant for some areas can be added to the framework in this dimension.
Education and qualification of agricultural workforce, access to medical care and degree of coverage with social security, unemployment and retirement pensions represent basic assets to ensure sustainable livelihoods in agricultural areas.
Population dynamics is assessed by the analysis of the main demographic trends in the area and the age structure of rural populations. The degree of social interaction in the population is measured by perceptions regarding sense of community and the engagement of local population in civil society organizations.

Good governance
Environmental governance refers to the set of regulatory processes, mechanisms and organizations through which political actors influence environmental actions and outcomes (Lemos & Agrawal, 2006). Governance is measured considering four main aspects: accountability, institutions for water management, capacity and participation, that are directly linked with the Good Governance Principles established by the United Nations Development Program (1997,2014).
Sustainable development will require that public officials account for actions taken in the public's name and with public resources (UNDP, 2014). Accountability measures the degree of commitment of responsible authorities to respond to and balance the needs of stakeholders in policy and other decision-making processes, as well as the strength of the existing formal and informal mechanisms to monitor the implementation of policy processes. Transparency and responsibility are also important elements in this regard (Lockwood, Davidson, Curtis, Stratford, & Griffith, 2010).
Effective rule of law is also a basic principle in governance for sustainable development (Rogers, Hall, Van de Meene, Brown, & Farrelly, 2003). The degree of adherence to rules and the law enforcement by resource managers are evaluated in the institutions for water management aspect. Here we also consider the perceived fairness of water allocation rules and the existence of (formal or informal) institutions to handle conflicts over the allocation of resources.
Capability refers to the systems, plans, resources, skills, leadership, knowledge, and experiences that enable organizations, and the individuals who work for them, to effectively deliver on their responsibilities (Lockwood et al., 2010). Sustainable agriculture requires technical capacity from both authorities and farmers/landowners to ensure sustainable management of resources. On the other hand, it also requires financial capacity of water management organizations to ensure basic activities such as monitoring, auditing, information management and compliance control.
Participation is a key element in governance for sustainable development contributing to improve the quality and legitimacy of decisions and to build capacity of all involved in the processes (National Research Council, 2008;Renn, 2006;Renn & Schweizer, 2009). The use of participatory approaches has been advocated on grounds of justice and democracy in procedure and an appreciation that complex, multidimensional issues, such as the ones involved in sustainable agriculture and water management cannot be effectively handled on a purely technocratic basis (Antunes, Kallis, Videira, & Santos, 2009;De Marchi & Ravetz, 1999;Pahl-Wostl, Mostert, & Tàbara, 2008). In the SIRIUS framework the degree of participation in decision-making is evaluated by placing existing practices along the spectrum for public participation proposed by the International Association of Public Participation (IAP2) 2 that includes 5 levels of involvement: information, consultation, involvement, collaboration and empowerment, inspired by the public participation ladder (Arnstein, 1969).

Reference values and data collection
The concept of agricultural sustainability is relative and case dependent (Pretty, 2008), as it depends on the particular environmental, cultural and social context of each case/area. Furthermore, sustainability is not an absolute condition of an agricultural system, but more seen as an evolving concept. What is considered "sustainable" or "unsustainable" depends very much on local conditions and recent trends (Zhen & Routray, 2003). For this reason, the SIRIUS framework does not have underlying sustainability target values and does not explicitly include absolute reference values for the suggested indicators. Relative reference values, which compare indicators values with an initial value, a regional average or a desirable trend for that particular context (de Olde et al., 2016), are used instead.
For those indicators where there exists a sound basis for establishing locally adjusted reference values, they should be used in the assessment. For example, we suggest the comparison of obtained water productivity values for the main crops with the reference values used by the Water Footprint Network (Mekonnen & Hoekstra, 2011) to support water footprint calculations. Also, for instance, local water quality standards should be used to assess the share of water bodies with poor quality.
Being so, the results obtained through the application of this framework should not be used to derive a single measure of sustainability of an area, to rank different areas or to establish sustainability ratings of agricultural systems. The idea is to support the development of change processes leading to agricultural sustainability (Rigby & Cáceres, 2001) rather than to prescribe a set of practices or absolute targets. Information used to support the application of the SIRIUS framework in a given area should be obtained from officially published data (e.g. official statistics) or other verifiable sources (e.g. scientific literature) whenever possible. This should be complemented with tailored data collection and qualitative assessments when necessary. When needed, data should be collected at the scale that is most relevant to understand the issue at hand (for example data on costs and revenues should be collected at the farm/field level). The focus of analysis of the SIRIUS framework is at the irrigation perimeter/regional level, and therefore indicators should be reported at this scale.
Many of the indicators used in the social wellbeing and good governance dimensions reflect stakeholders' perceptions regarding each issue (e.g. perceived fairness of water allocation rules). In these cases, we use a 5-point Likert scale to express the assessment (e.g. very good, good, fair, poor, very poor) and recommend that information should be collected with a wider involvement of relevant actors, either through the organization of public opinion polls, targeted surveys or the organization of participatory events such as focus groups.

Application: sustainability assessment in SIRIUS pilot areas
To test the usefulness of the framework and its capacity to be applied in different contexts we used it to assess the sustainability of 10 irrigated agricultural areas in 8 different countries, corresponding to the pilot areas in the SIRIUS project.
A questionnaire was developed to support the data gathering process in the pilot areas (available as supplementary material to this paper). This questionnaire was distributed to leaders of regional teams that took the responsibility of collecting the relevant information in appropriate sources in each area. Meetings and interviews with relevant stakeholders in each pilot area played a very relevant role to support data collection and to validate the information. Table 3 synthesizes the main results obtained for the 10 pilot areas. A more detailed version of this Table is presented as supplementary material and a full account of the obtained results is provided in (Antunes, Santos, & Cosme, 2013). The interpretation of the results obtained should be performed with caution and take into consideration the difficulties in obtaining the underlying data that were present in all areas.
For each pilot area, a "sustainability profile" was prepared (Figure 3). This profile depicts the score in a 0-5 scale that was attributed to each of the core issues of the framework, based on the values obtained for the different indicators. Since there are no absolute reference values, we did not use a mathematical aggregation rule to calculate this score, but performed a classification of the values obtained, considering local standards and reference values as explained in Section 3.3.
The radar plots are very useful to summarize results, to understand the main sustainability challenges and to identify opportunities for improvement in each pilot area. However, they should not be used to support a comparison across areas, since there are no absolute reference values and many of the indicators used in the framework (in particular social and governance issues) are very context dependent. Their interpretation should be accompanied by the narrative description provided in Table 3 and by the analysis of the results obtained for the different indicators. Nevertheless, it is possible to derive some common lessons.
Environmental integrity is an underlying basic pillar for sustainability. Even in areas with an overall good performance across all dimensions, poor performance in one aspect of environmental integrity (e.g. water availability or soil quality) can be a limiting factor for sustainability. In irrigated agricultural areas, it is particularly relevant to develop plans to ensure sustainable provision of water with good quality in the future and to balance current and future needs of the different uses within the carrying capacity of the environmental system.
Water scarcity and water quality are the most critical issues in this dimension. There is still a large room for improvement of agricultural water productivity in all studied areas, showing that the development of technologies and capabilities to enhance water (and nutrients) management is an essential element of sustainability roadmaps in very different contexts, ranging from the more sophisticated intensive systems (e.g. Spanish and Italian pilot areas) to less developed (e.g. India pilot area) agricultural systems.
Vulnerability to climate change is a key aspect in all pilot areas and it is important to develop and implement adaptation plans to ensure resilience of irrigated agriculture areas under climate change scenarios.
Agriculture is very relevant for the regional economy in the studied areas, but profitability is generally reported as low and dependent on a limited number of cultures and on the availability of water. Tailored water pricing schemes are an essential element of the policy toolbox to provide incentives for water saving, increasing water productivity and managing water scarcity in an efficient way. This aspect is still poorly developed in the areas considered in our study: in some of them (e.g. Brazil and Egypt) water is not paid; others (e.g. Italy, Turkey) have implemented area-based tariffs, which provide no incentive for water saving and finally, those that have volume-based schemes in place (e.g. Spain and Mexico) refer that these need to be adjusted to provide adjusted and effective incentives. In all cases, it is important to consider affordability as a criterion in the design and implementation of water pricing mechanisms, balancing water costs and farmers' capacity to pay for water.
The dependence upon public subsidies is common to almost all studied areas, and in many agricultural areas around the world. In this context, conditioning the granting of public subsidies to the The area is highly dependent on agriculture (particularly maize), which is highly dependent on irrigation. Management of water resources is key for sustainable development.
Water quality is the most worrisome issue. The adoption of measures to control pressures on water quality is a priority, alongside with improvement in water use efficiency Agriculture is relevant for the regional economy, although it is not very profitable and depends on a limited number of cultures. It is market oriented although highly dependent on public subsidies. The economic productivity of water is very high for some crops but much lower for others. The existing area-based water pricing scheme does not provide a strong incentive to improve water productivity Social wellbeing is overall perceived as good. The most relevant concern is the perceived lack of opportunities for farmers, which leads to abandonment of agricultural activities, emigration and shrinking population Increase in accountability and financial capacity of water management institutions seem to be the main governance concerns. Direct involvement of farmers in decision-making processes should increase, since participation now takes place mainly through representative delegation The farms are characterized by the prevalence of the micro property and dispersion of parcels. Agriculture is highly dependent on irrigation from surface water. This source is reliable and water scarcity is not a constraint. However, poor water quality is a serious concern. Energy intensity of agriculture is high, since all water is pumped from the river Agriculture is very relevant for the regional economy and is profitable, although with variations. Crops cultivated for seeds are more profitable. Investment in agriculture is increasing, but agriculture is highly dependent on public subsidies. The economic productivity of water is much higher for maize seeds than for other crops. Water energy costs are relevant in the cost structure. The existing volume and area based water pricing scheme needs to be adjusted to promote water efficiency, considering farmers' capacity to pay The working conditions and distribution of opportunities and income are perceived as fair.
There is a high rate of adult literacy and agricultural workers are qualified, but access to medical care and social security is still poor.
adoption of sustainable management practices (eco-conditionality) may play an important role in leveraging transitions towards more sustainable agriculture.
It is important to ensure a fair distribution of opportunities and income in society-this means fairness in access to resources between small and large farmers, balance in salaries between agricultural workers and other economic sectors and fairness of opportunities across genders. Even in areas where social well-being is perceived as good (e.g. Spain and Italy) the still unequal distribution of opportunities and income between agricultural and non-agricultural workers leads to abandonment of agricultural activities, emigration and a shrinking population. In some areas (e.g. Brazil, Turkey, India) working conditions in agriculture are still unsatisfactory and access to education and   medical care is still poor, pointing directly to a high priority of actions in these domains to improve sustainability performance of these agricultural systems.
Clear and accepted water allocation rules, coupled with strong institutions and technical capacity are fundamental governance elements. Without these it is impossible to promote sustainable water use, enforce decisions and handle water allocation conflicts in an equitable manner. In some of the studied areas, (e.g. Italy, Romania) accountability and capacity of water management authorities are issues of concern regarding governance. In all areas, there is a perception that a more direct involvement of farmers and a higher transparency in decision-making processes regarding water management could contribute to obtain better solutions, building a sense of ownership towards the decisions taken, fostering social learning and strengthening community relations.
Technologies and farm advisory services, such as the ones developed in the SIRIUS project, can be used as an entry point to promote improved water governance and increase water use efficiency in irrigated agriculture. They can play a very important role and contribute to the development of a shared understanding and a new mind-set regarding water management problems and solutions among the different stakeholders.

Conclusions
The proposed framework for sustainability assessment of irrigated agricultural systems brings a new perspective in this domain by adopting an encompassing understanding of sustainability, incorporating concepts of both resilience and persistence. It combines the evaluation of the conditions of the natural system that supports agricultural activities with an assessment of central elements required to ensure a flourishing livelihood for rural populations. The emphasis on the irrigation perimeter/watershed scale is also an important aspect of this framework, that enables the understanding of the links between the ecological system, agricultural competitiveness and the living conditions of the population.
The regional teams involved in the application of the framework in the pilot areas acknowledged that this exercise allowed them to gain a wider perspective and a deeper understanding of the main issues of concern. It also allowed them to grasp the main inter-linkages between the different sustainability dimensions. Even in those areas where data available were scarce, such as the case of the India pilot area, the framework proved to be useful and applicable, although it was not possible to obtain quantitative values for many indicators.
Another important feature of this framework is the process that led to its development, which combined a theory-based proposal with inputs from stakeholders engaged in agricultural research and practice from different parts of the world, representing very different contexts. This has contributed to enrich the framework, while making it simple and robust enough to be of general applicability.
The application of the SIRIUS framework proved the usefulness of this approach to obtain a holistic picture of sustainability in irrigated agriculture areas in different contexts. It allowed users to understand "where they are", so that they could then define a strategy to move towards "where they want to go" (the sustainability vision). Therefore, the results of the application of the SIRIUS framework are particularly meaningful for the social actors in each area. Their discussion can be a starting point for the organization of participatory processes aimed at developing a shared vision for sustainability in an irrigated agricultural area and the roadmap to get there. In this process, stakeholders are encouraged to think about their area and their agricultural activity, identifying what are the main bottlenecks and opportunities for a transition towards a sustainable future.
The adoption of such a framework to measure agricultural sustainability can also be useful to serve as a guiding tool to support the design and implementation of long term monitoring programs of irrigated agricultural systems and associated practices and policies aiming at a more sustainable farming sector. Notes 1. For example, organic farming focuses mainly in the environmental dimension (although recently its scope has been enlarged to account also for social aspects), while sustainable intensification relates the environmental and economic dimensions, by focusing on the goal of enhancing agricultural production while reducing environmental impacts.