Where do Qualitative Assessments Fit in an Era of Increasingly Quantitative Monitoring? Perspectives from Interpreting Indicators of Rangeland Health

On the Ground Interpreting Indicators of Rangeland Health and other well-designed qualitative assessments are useful for understanding ecological function and can be used to prioritize areas for monitoring, restoration, or management changes. When completed by experienced, trained multidisciplinary teams, qualitative assessments provide reliable information about ecological processes and are repeatable across time and geographic locations. Consistency and repeatability of qualitative assessments are maximized when the assessments are supported by appropriate quantitative indicator data. Likewise, quantitative datasets may be complemented by qualitative assessments, which can provide insight into indicators that are difficult to measure, providing a more complete view of vegetation, soils, and underlying ecological processes. Qualitative assessments can be used as a communication tool for developing a common understanding of resource issues and a shared vision for, and commitment to future stewardship. New opportunities are emerging to enable further integration of qualitative and quantitative field protocols, ecological models, and remotely sensed products to benefit rangeland assessment, monitoring, and management.


Introduction
Rangeland managers are challenged with making management decisions across large landscapes while having limited time and financial resources to understand rangeland conditions. Increasingly, these decisions involve neighbors, partners, and stakeholders with diverse interests, experiences, and educational backgrounds. Thus, protocols for assessing rangeland conditions and ecosystem processes that are time-efficient and help build a common understanding across broad user groups and support decision-making are needed.
Rangeland assessment protocols can generally be categorized as qualitative or quantitative in nature. Qualitative protocols use observations to estimate or judge conditions, often categorizing indicators based on systematic observations. In contrast, quantitative protocols rely on recording measurements and calculating numerical indicator values. Regardless of whether an assessment is based primarily on qualitative observations or quantitative measurements, this information must be compared with some type of objective reference or description of desired conditions to make a judgement or reach a conclusion.
Qualitative and descriptive rangeland assessments and surveys have been used for decades, 1 and federal agencies in the United States have widely used two qualitative assessment protocols since the 1990s: the Proper Functioning Condition (PFC) protocol to assess the functionality of riparian areas and wetlands 2-3 and the Interpreting Indicators of Rangeland Health (IIRH) protocol to assess upland rangeland health. 4 Here we focus on application of these assessments within the context of federal land management in the western United States. However, these protocols are also used on public and private lands throughout the Unites States as well as in other countries. Geographic differences in t ypes and availabilit y of information such as soil maps may necessitate somewhat different approaches than those we describe. However, the over-2022 arching principles and ideas are broadly applicable and may contribute to the development and use of other qualitative assessment protocols.
Due in part to the difficulty of achieving consistent application of qualitative assessment protocols across observers and locations, standardized quantitative methods have been emphasized with the goal of greater objectivity and repeatability. Over the past decade, the collection and management of quantitative data about rangeland conditions have benefited from increased adoption of standardized field methods and electronic data management tools. For example, federal agency initiatives, including the Bureau of Land Management's Assessment, Inventory and Monitoring (AIM) strategy, and the Natural Resources Conservation Service's National Resources Inventory (NRI), have resulted in large quantitative datasets in rangelands, especially across the western United States. These data are aggregated so that they can be accessed electronically and used at many scales (e.g., grazing allotment, field office, or state). In addition, advances in remote sensing and associated products, such as the Rangeland Analysis Platform 5 and National Land Cover Database 6 enable scientists and managers to map aspects of rangeland condition across large landscapes. These products have benefited from the availability of standardized, quantitative data, which improve product training and validation. Considering the increasing focus on collecting and anal y zing quantitative data, what is the role of qualitative assessment protocols in rangeland management?
Here,we consider the role of qualitative assessments within the context of applying and teaching IIRH. First, we provide some background of qualitative assessments for rangeland management, including IIRH. Next, we suggest opportunities for gaining insights about rangelands from qualitative assessments that may not be apparent from quantitative indicators alone. Finally, we recommend best practices for the use of qualitative assessments in rangeland management. We conclude that using qualitative data in conjunction with complementary quantitative data provides the most comprehensive understanding of rangeland health to support management decisions. Lastly, we offer suggestions for further integrating qualitative assessments with quantitative information including models and remotely-sensed data.

Development of Interpreting Indicators of Rangeland Health
Over time, scientists and managers have wrestled with how best to characterize and keep track of changes occurring in rangelands through quantitative measurements and qualitative approaches, often concluding that a combination of the two was needed to fully describe resource conditions. For example, early assessment protocols, such as the range survey methods developed by the U.S. Department of Agriculture Forest Service (USDA FS) in the first half of the 20th century, initially relied on estimates and observations, but as they were further developed more quantitative measurements were incorporated. 1 In 1950 K.W. Parker provided this justification for collecting a combination of qualitative and quantitative data for the Step Method: Some of the factors for judging trend on these sites, such as density and floristic composition, can be measured with a reasonable degree of accuracy. Other factors, such as plant vigor and erosion rates, must be recorded largely in descriptive terms. Both types of information are essential for a complete picture of trend in condition. 7 Over time, federal agencies such as the Bureau of Land Management (BLM) and Natural Resources Conservation Service (NRCS) began to adopt more quantitative and semiquantitative methods to collect information about rangelands. However, methods differed among and within agencies and resulted in limited and sometimes conflicting conclusions about rangeland condition. This was partly due to differences in observers' disciplinary backgrounds and agency objectives and use of methods and protocols that were not designed to assess rangeland biophysical processes and functions. For example, objectives for maintaining forage for livestock grazing might differ from objectives developed for wildlife habitat. Field protocols to assess conditions related to these objectives would likely measure different indicators. When an area is meeting an objective for forage production, but is not meeting wildlife habitat objectives, conclusions of whether the area is in "good"or "poor"condition could be made subjectively, depending upon how the respective resources are valued. In contrast, protocols that are designed to assess ecological function are more capable of informing managers about the capacity for rangelands to sustain multiple resource values, rather than a single use.
To promote ecological process-based approaches to rangeland assessment, in 1994 the National Research Council (NRC) recommended using a suite of indicators to look holistically at rangeland health attributes. 1 The NRC recommendation came shortly after publication of the first technical reference describing the PFC protocol for assessing streams, rivers, and their riparian areas with a focus on ecological function. 8 Federal agencies, including the BLM and NRCS, responded to the NRC recommendation by coordinating an interagency effort to develop a protocol similar to PFC to evaluate the ecological function of upland areas, which resulted in the IIRH protocol.
The IIRH protocol was developed to assess the health of upland areas in rangelands and has been used broadly on private and public rangelands in the United States, as well as other countries for the past 20 years. 9 , 10 Two versions of the protocol were developed and tested in the late 1990s by the BLM, NRCS, United States Geologic Survey (USGS), and Agricultural Research Service (ARS) with input from rangeland researchers and consultants. The interagency technical reference, "Interpreting Indicators of Rangeland Health Version 3," which assessed 17 indicators using a largely qualitative approach, combined these two versions and was released and adopted by the BLM, NRCS, USGS, and ARS in 2000. 11 Table 1 The three attributes of rangeland health and their associated indicators The protocol has been further refined and periodically updated with Version 4 in 2005, and Version 5, which was also adopted by the USDA FS, being published in 2020. The 17 indicators of rangeland health focus on soils, plants, and hydrologic features and are used to rate three attributes of rangeland health-soil and site stability, hydrological function, and biotic integrity ( Table 1 ). IIRH assessments are completed at evaluation areas within a known ecological site or other land unit with a known potential. This potential is derived from the interactions of landscape position, geology, soil, climate, and hydrology of the site. These interactions result in plant communities and dynamic soil properties that occur within the reference state, in which ecological processes are functioning at a sustainable and resilient level under the natural disturbance regime. 12 Each indicator is rated by comparing indicator characteristics observed in an evaluation area to the expected condition for the indicator in the reference state, which is described in a reference sheet. Five categories are used to rate the degree of departure from conditions described in the reference sheet for each indicator ( Table 2 ). Once the 17 indicators are rated, nine or ten of the indicators are used to rate the degree of departure for each attribute using a preponderance of evidence approach.
Using a reference sheet that provides an accurate description of site potential for the evaluation area is critical for a meaningful assessment of rangeland health. For instance, departure for the indicator of annual production is rated based on the extent to which production at the evaluation area is reduced relative to the expected production described in the reference sheet for an ecological site. Expected annual pro-duction for an ecological site with shallow soil may be 226.8 kilograms (500 pounds) and expected annual production for an ecological site on a deep soil in the same area could be 408.2 kilograms (900 pounds). In this example, the departure rating for the indicator of annual production will differ significantly if the incorrect reference sheet is used.

Gaining insights from qualitative assessments
The rationale for assessing rangeland health qualitatively is similar to qualitative approaches used in the medical field, in which patient case studies are used to recognize patterns and understand overall patient health. 13 Likewise, qualitative rangeland assessment protocols, including IIRH and PFC, provide a holistic snapshot of ecosystem function that can account for ecological indicators that are not easily measured and may detect patterns of land health degradation that would not be captured by a strictly quantitative assessment.
IIRH assessments can be done relatively quickly, allowing an initial understanding of an evaluation area's functional status, while still in the field. The ability to rapidly detect issues can help managers make decisions about where additional monitoring may be necessary or where management changes or restoration treatments are likely to be effective. The ability to understand functional status in the field, rather than requiring additional quantitative analysis, also makes IIRH a useful tool for discussing resource issues. Often, managers focus attention on vegetation changes without fully considering changes in underlying ecological processes. Rating the IIRH indicators and attributes and considering patterns of Common. Lengths and/or widths slightly to moderately higher than expected. Minor erosional and/or depositional areas. Infrequently connected.
Scarce. Length and width nearly match expected. Some minor erosional and/or depositional areas. Rarely connected.
Reference sheet narrative inserted here.
Note. "None to Slight" is used for an indicator or attribute that resembles the natural range of variability for the ecological site being evaluated and is provided separately in a reference sheet.
departure from reference descriptions guides field observers toward assessing interrelated ecological processes in a repeatable way. However, as we discuss below, these assessments are most repeatable when conducted by an experienced and welltrained multidisciplinary team. It is often challenging to assemble and maintain such a team to conduct large numbers of assessments.

A framework for applying ecological concepts
The global scientific community is constantly developing and refining ecological concepts and unifying principles that can be applied to local science and management questions. 14 However, these concepts and principles often need to be translated and synthesized for practical application to local management. For example, the concept of ecological sites is used to classify areas with soil and physical characteristics that differentiate them from other areas in their potential to produce specific kinds and amounts of vegetation. This ecological site concept unifies multiple scientific principles and packages them in a way that provides practical applications to science and management. Ecological sites also serve as a reference to inform IIRH assessments. IIRH and other well-designed qualitative assessment protocols in turn provide frameworks for integrating scientific concepts about rangeland dynamics to understand and provide consistent interpretations of observations in the field.
IIRH uses scientific concepts, such as landscape context, natural range of variability, resistance and resilience, and alternate state theory, that are applied with scientific, regional, and local knowledge when conducting an IIRH assessment. 4 IIRH rates indicators at an evaluation area against the reference sheet descriptions for an ecological site, but the protocol recognizes and accounts for both spatial and temporal variability of indicators within the reference sheet descriptions. Observers are instructed to consider the influences of recent weather, disturbances, and biophysical characteristics (e.g., slope and aspect) to refine their understanding of site potential and then rate each indicator accordingly. For example, in many systems, bare ground amount and connectivity are expected to increase for 1 or more years following a wildfire. When an assessment is conducted in an area that is recently burned, it is important for the observers to understand the effects of wildfire and recovery patterns for bare ground and other indicators under the natural range of variability, and account for these expected temporary changes when conducting an assessment. Although IIRH assessments are intended to be done within the context of local knowledge for the ecological site being evaluated, and with an understanding of indicators interactions at the site scale, the 17 indicators should be applied consistently at all evaluation areas to examine degradation patterns occurring across all rangelands.

Quantitative and qualitative data are complementary
When used together, quantitative and qualitative data can provide a more complete picture of rangeland health than either one alone. Quantitative measurements and indicators have less interobserver variability and are therefore preferred for direct comparisons between evaluation locations and over time. However, quantitative measures are often more time consuming to collect and focus on vegetation-based indicators, such as annual production and foliar and litter cover, which provide direct measurements of biotic function and only indirect measurements of soil and hydrologic function. In contrast, qualitative assessments allow rapid observation of multiple factors related to a broad set of soil and vegetation indicators that are used to assess biotic integrity, soil and site stability, and hydrologic function. Additionally, IIRH provides an interpretive framework for considering both measurable and nonmeasurable indicators collectively, and for developing a narrative about the status of the three attributes of rangeland health in an evaluation area.
Quantitative measurements are recommended to support IIRH qualitative assessments because they can reduce interobserver variability for many indicators when applied systematically and with appropriate training and calibration. To complete an IIRH assessment, bare ground cover, litter cover, soil surface stability, and total annual production must be quantified. Other quantitative indicators, particularly those related to plant communities, can be derived from data collected using the line-point intercept, belt transects, and production methods ( Table 3 ). Note that evaluators must train and calibrate before making ocular estimates of some quantitative indicators, including percent cover and annual production. Some quantitative measurements relate directly to criteria that are considered when rating an indicator, and other quantitative measurements are not as directly related to these criteria but may still provide supporting information that can be helpful when rating the indicator. For example, criteria that should be considered when rating the bare ground indicator in the IIRH protocol include the total amount of bare ground and bare patch size and connectivity. Collecting cover data with the line-point intercept method provides a reliable estimate of the total amount of bare ground ( Fig. 1 ); however, it does not provide a picture of bare patch size and connectivity, which are harder to measure but also influence ecosystem function. Canopy gap intercept is a quantitative method that provides an indication of the continuity of plant cover, which can be helpful for understanding the extent of bare ground patches. However, if the gaps between plant canopies are protected by rocks, litter, and biological soil crusts, they would not be considered bare ground patches. Therefore, observers must consider how much protective soil cover is present in canopy gaps when using gap size measurements to support qualitative observations of bare patch size and connectivity in rating the bare ground indicator.
The BLM AIM and NRCS NRI programs frequently complete IIRH assessments along with quantitative data collection, enabling analysis of how both qualitative and quantitative indicators are changing over time. The quantitative indicators included in these datasets inform and support the associated IIRH assessments, which are combined to report land condition at multiple spatial scales-from ecoregionally to nationally (e.g., BLM Rangeland Resource Assessment 15 ; National Resources Inventory Rangeland Resource Assessment 16 ).
Recognizing the benefits of integrating qualitative assessments with quantitative indicators and measurements, Version 5 of the IIRH technical reference provides more specific ties between the 17 qualitative indicators and quantitative measurements based on core field methods used in the BLM AIM and NRCS NRI programs ( Table 3 ). 4 This linkage of IIRH indicators to core quantitative indicators will enable our understanding of the relationships between qualitative and quantitative indicators to be refined over time.

Assessing indicators that are not easily measured
Some indicators relevant to land degradation and management cannot be effectively quantified in the field or can only be quantified with intensive and expensive measurements. For example, quantifying soil health and function has been a longstanding, difficult challenge. Soil function is a complex interaction of numerous chemical, biological, and physical proper ties. 17 Soil proper ties, such as texture and organic matter content, can be quantified through laboratory analysis, but these analyses are costly and time-consuming. The interactions among soil properties that affect a site's potential and function are dynamic and often unique to the site being evaluated, so it is preferable to use rapid, field-based approaches that can be integrated with other field data collection methods and protocols.
Several of the 17 indicators used in the IIRH assessment are most feasible to assess qualitatively. Two examples are the indicators of plant vigor and soil surface loss and degradation. Plant vigor can be more easily assessed qualitatively, as it is highly contextual, with expected plant appearance varying with season, weather, recent grazing, species, and location being used together to support assessment of the bare ground indicator. Both the quantitative measurement of total bare ground, and a qualitative assessment of the size and connectivity of bare patches are considered when determining the appropriate departure rating for this indicator in an evaluation area. ( Fig. 2 ). For example, when evaluating bunchgrass vigor, measurements of crown diameter or plant height may be useful, but factors including seedhead production, length and width of leaf blades, and density of growing plant parts can all contribute to a comprehensive assessment of plant vigor. The indicator of soil surface loss and degradation is influenced by loss of soil material, depleted organic matter, reduced porosity, and altered soil structure. These changes signify a decline in the soil's capacity to function and that a threshold has been crossed, reducing a site's ability to recover ecological functions. 18 Quantitative field protocols that might detect these changes are cost and time prohibitive and may not capture their combined effects on soil function. Thus, a qualitative field assessment based on soil surface horizon thickness, color, and structure is often the most feasible way to rate this indicator.
There are other soil and site stability, as well as hydrologic function, indicators that are difficult to measure quantitatively. For example, when evaluating erosion pedestals, the frequency of pedestals in a defined area, whether plant roots are exposed, and the prominence of the pedestals above the soil surface are all considered ( Fig. 3A ). It might be possible, but time-consuming, to sample and quantify each of these characteristics. Additionally, overland water flow patterns are difficult to quantify consistently; instead, the IIRH protocol provides guidance for rating this indicator based on how extensive, long, wide, and connected water flow patterns are and whether there are distinct depositional or erosional areas ( Table 2 , Fig. 3B ).
Although these indicators are difficult to measure, they can be important for detecting signs of degradation that might be missed by strictly quantitative approaches. For example, Jablonski et al. 19 conducted IIRH assessments on grazing allotments in Dinosaur National Monument in Colorado and Utah. For evaluation areas where the soil and site stability attribute were rated as a slight-to-moderate departure from reference, the authors noted that although sample sizes did not support statistical analysis, the water flow patterns and soil surface resistance to erosion indicators tended to show departure from reference conditions and thus had the greatest influence in detecting departure for this attribute. In contrast, the authors noted that departures from reference for the biotic integrity attribute were more often influenced by departures in the functional-structural groups, invasive plants, and annual production indicators, which are more easily quantified using common rangeland monitoring methods. In this instance, the qualitative indicators assisted with detection of potential losses of soil and site stability that would not be easily detectable with quantitative measurements, and it may have been possible to rely largely on quantitative information to detect reductions in biotic integrity in the study area.

Qualitative assessments provide a platform for communication and collaboration
Effective r angeland management relies on the recognition of problems and buy-in to solutions from managers, partners, and stakeholders. Participatory processes and social learning  are useful for building understanding of resource issues and developing shared stewardship goals. 20 A shared understanding of resource issues and vision for the future can lead to identifying more effective management solutions and inspire commitment to implementing those solutions over the long term. 21 Because PFC and IIRH assessments are relatively rapid and can be completed with a group of diverse participants, they can be used to focus collaborative communications and identify priorities for additional quantitative data collection or other management efforts. The Creeks and Communities strategy, which is led by the BLM and USDA FS in partnership with the NRCS, uses the riparian PFC assessment as a standard part of its place-based problem-solving, specifically to facilitate communication and shared understanding of riparian functionality. Focusing communication on maintaining or restoring riparian function promotes communication and collaboration, as compared with focusing on resource values, which may vary, leading to conflict or disagreement among collaborators. 22 Similarly, partner and stakeholder participation in IIRH assessments can facilitate discussions about the relationships of indicators and ecological processes to build shared understanding of ecological function in upland areas. The IIRH attribute ratings for soil and site stability, hydrologic function, and biotic integrity can be used to identify changes in ecological function associated with transitions away from the reference state. 23 Recognizing when an area is at risk of crossing, or has crossed a threshold to an alternate state, is important for identifying management options and setting realistic expectations for management outcomes. 4 State and transition models and descriptions of ecological site dynamics can be useful to help identify where departures for indicators or attributes of rangeland health may be indicative of an at-risk community phase or a state change.

Rangelands
Qualitative indicators are effective tools for communication among land managers, partners, and/or stakeholders and also provide a mechanism for those partners and stakeholders to participate in adaptive management by identifying and communicating changes they see in the field. Livestock operators and others who make repeated visits to specific areas can make frequent observations and potentially notice both positive and negative changes in indicators. Recognition of the qualitative indicators themselves can provide early warning signs of impending problems that might be missed by a land manager who is unable to make frequent visits to the same field locations. Once these early warning signs are communicated, land managers can consider reprioritizing monitoring or management as needed to adapt to the changing conditions. For example, changes in plant vigor may be one of the first noticeable signs of changing ecological function resulting in cascading effects. More specifically, as bunchgrass vigor decreases, the root mass, basal and foliar cover, and litter also decline, thereby increasing bare ground, reducing soil organic matter, and altering soil structure. Therefore, early recognition of losses in plant vigor may enable effective adaptive management to prevent more serious (and less reversible) changes from occurring.

Recommendations for using and interpreting qualitative data
Reflecting on years of applying and teaching the IIRH protocol, we recommend several best practices for using and interpreting qualitative data. Following these practices can address many of the limitations of qualitative assessments, including poor repeatability among different observers. In addition, consistently implemented qualitative assessments can generate a reliable set of information across large areas with limited time and financial resources to understand rangeland health. The intended purpose of the assessments should be considered when determining the needed range of expertise and amount of supporting quantitative data. Maximizing consistency among assessments should be prioritized when results will be aggregated for broader analysis purposes, or the assessments will be used as part of a long-term decisionmaking process. When reference information or the required breadth of expertise is not available, qualitative descriptions of difficult-to-measure indicators also can add value to quantitative datasets or be used to communicate observations. In these situations, a protocol such as Describing Indicators of Rangeland Health, which does not require a reference, but describes and classifies the 17 indicators, rather than rating them, could be used.

Ensure observers are qualified
Collecting quantitative data related to rangeland health primarily requires plant identification skills and knowing the data collection methods. In contrast, qualitative assessment protocols such as IIRH require observers to interpret observations in the field, so observers need to know the procedures and definitions for indicators and also have the skills to use that information in context and recognize the indicators in a variety of settings. Therefore, qualifications for conducting qualitative assessments include a sound understanding of relevant ecological concepts, ability to determine the appropriate reference such as the ecological site being evaluated and local knowledge of the historical soil and plant communities and disturbance regimes. Robust qualitative assessments require observers to connect management, on-site and off-site influences, and disturbance dynamics, which often interact at different spatial and temporal scales. Linking this knowledge is best done by attending formal training.
Over two decades of experience teaching the IIRH protocol to natural resources professionals, students, scientists, and landowners throughout the United States and abroad has demonstrated that indicator and attribute ratings become more consistent with increased learning and practice. Additional training and calibration exercises help to further improve consistency of both teams and individuals. Miller 24 examined results of over 500 IIRH assessments in Utah and found that soils expertise is often in short supply, which may reduce the sensitivity in rating changes in soils indicators. Individual observers tend to focus on certain indicators, often based on their background and training. 24 Therefore, multidisciplinary assessment teams usually provide the most consistent and thorough overall assessments, while reducing individual bias. 25 Qualitative assessments are not intended to detect trends Although experienced, trained individuals have demonstrated relative consistency in rating the attributes of rangeland health, the protocol is not intended to be used to infer trend by comparing assessments completed at two or more moments in time. Departure rating categories are often too broad to detect trends with sufficient precision. This is also the reason that redundancy is built into the attribute ratings through the use of related indicators. However, quantitative data collected to support a qualitative assessment can be used as a baseline for monitoring specific indicators to detect change over time. We recommend using the results of qualitative assessments to help identify quantitative indicators that are likely to best detect changes in land health over

Rangelands
Understand site reference conditions IIRH requires the use of a reference sheet describing expected conditions for the ecological site or equivalent land unit being assessed. This is the "benchmark" against which the indicators are compared. If the benchmark descriptions are too broad, inaccurate, or do not provide the necessary information, it becomes impossible to generate a good qualitative assessment. Because IIRH is a moment in time assessment, it is not possible to revisit an assessment with a new reference sheet and revise ratings. Instead, an entirely new moment in time assessment would be needed. Therefore, it is important to obtain or develop reference sheets that describe the natural spatial and temporal variability for an ecological site and address the characteristics that are considered when rating each indicator (e.g., for the litter cover and depth indicator, the reference sheet should describe the expected percent litter cover, average litter depth and effect of disturbance, weather, and natural herbivory on litter accumulation). For assessments that do not require a reference sheet or similar set of benchmarks, observers must be knowledgeable about the ecosystem being addressed and thoroughly understand each assessment site's potential in effectively recognizing degradation. 2 Understand differences in qualitative and quantitative data As addressed above, qualitative and quantitative data serve different, but complementary, functions. However, there are instances where these data have not complemented each other and instead have provided conflicting information. In some cases, this might be due to different people with varying levels of training collecting the data, errors in data transcription, or other errors in field data collection.
When differences cannot be attributed to these types of errors, an effort should be made to determine the root cause of conflicting information. Other causes of conflicting information to investigate include, but are not limited to, 1) differences in site selection procedures (randomly vs. subjectively placed plots or evaluation areas), 2) using qualitative indicators that do not directly relate to quantitative indicators, 3) differences in the timing of assessments, either within a year or across years, 4) disparate sample sizes, and/or 5) the reference sheet(s) or observer's understanding of site potential for the qualitative assessments was not fully accurate. For example, Miller 24 observed a large degree of variability in quantitative indicator values among IIRH evaluation areas within a single ecological site that were assigned the same attribute ratings. The author concluded that the attribute ratings were less variable because they represent a synthesis of both qualitative indicator ratings and quantitative indicator values. 23 However, if patterns of disagreement between certain qualitative and quantitative indicators emerge over time, these incongruences may warrant further investigation, and possible reconsideration of how the qualitative indicator, quantitative indicator, or both, are assessed and interpreted. The increase in consistent quantitative indicator data will improve our ability to examine these relationships, especially where qualitative assessments accompany the quantitative data, allowing direct comparisons within and across sites.

Conclusions
Given the complexity of assessing ecological processes on rangelands over space and time, and the need for communication among stakeholders for effective management, qualitative assessments provide significant value and are complementary to quantitative assessment approaches. Just as rangeland management is often referred to as both an art and science, we conclude that both qualitative and quantitative approaches are needed to best understand rangeland ecological processes and dynamics. Qualitative assessments conducted by qualified observers using appropriate site reference information can provide reliable information that can be used for a variety of applications at multiple scales.
In coming years, we hope that advances in data availability, modeling, and remote-sensing products for rangelands will be leveraged to continue to refine our understanding of qualitative and quantitative indicators and their relationships and strengthen frameworks that support robust qualitative assessments. Understanding and defining appropriate references is a continuing challenge that currently limits the application of IIRH in many areas. Analysis of quantitative indicator data across the range of variability of an ecological site can assist with more accurately describing the expected variability for certain indicators. These data also can be used to develop and validate ecological site concepts and state and transition models; when accompanied by complementary qualitative data, more value can be extracted from these datasets to refine our understanding of reference conditions, as well as conditions indicative of at-risk community phases, thresholds, and state changes. As remotely sensed data are collected at more frequent timeframes relevant to growing season conditions, it also may become possible to leverage these datasets to better understand how both qualitative and quantitative field assessment results might be extrapolated to similar sites across a management area or landscape.
Although field-based qualitative assessment protocols are expected to benefit from expanded availability of quantitative datasets, modeling and remote sensing-based products also can benefit from the results of qualitative assessments. Consistent documentation of qualitative indicators, especially at known spatial locations, can help to validate and refine models and train remotely sensed products. This may be especially important for the soil erosion indicators that cannot be directly measured in the field and are instead predicted based on factors such as slope, precipitation, wind, and vegetation cover. For example, the relationships between remotely sensed bare ground estimates and qualitative ratings of difficult to measure indicators, such as rills or water flow patterns, could be investigated. Defining these relationships could help refine water erosion models by quantifying important ecological thresholds and benchmarks, such as bare ground amounts above which soil loss accelerates due to formation of these concentrated water erosion features.
These are just a few ideas for possible areas to explore for future research and development. Additional approaches to increasing the utility of qualitative assessments directly for rangeland management and leveraging their results to assist with interpretation of various quantitative datasets will undoubtably emerge. As these ideas are further developed and implemented, we believe that they will provide reciprocal benefits for modeling, remote sensing, and qualitative and quantitative field assessment approaches, ultimately improving our understanding of rangeland conditions and ability to manage their resource values into the future.

Declaration of Competing Interest
The authors have no known conflicting or competing interests relating to the review or publication of the manuscript. The content of sponsored issues of Rangelands is handled with the same editorial independence and single-blind peer review as that of regular issues.