Invited Perspective: The Promise of Fit-for-Purpose Systematic Evidence Maps for Supporting Regulatory Health Assessment

The U.S. Environmental Protection Agency (U.S. EPA) is charged with the Herculean task of critically assessing the safety of tens of thousands of chemicals. New methods that support improved efficiency and effectiveness of risk assessments, including the systematic evidence map described by Carlson et al. in this issue, are greatly needed. The application of systematic review (SR) methods to the field of environmental health began in earnest nearly a decade ago with the development and publication of applicable methods.2–4 Today the approach has evolved to include systematic evidence maps (SEMs), which provide access to study data extracted from a large body of evidence to inform SR, risk assessment, and other chemical management workflows. SEMs are extremely useful for assessing large chemical classes, such as perand polyfluoroalkyl substances (PFAS), for which the scientific evidence base is poorly characterized. PFAS are widely used in consumer and industrial products; they are persistent and mobile, and thus ubiquitous in the environment; and several have been demonstrated to be harmful to humans and wildlife.6–11 PFAS have been detected in the bodies of nearly every person tested, in the United States and worldwide. With more than 12,000 PFAS identified to date, the time it would take to assess them individually would lead to unnecessary delays in regulating these chemicals when so many people are already at risk. We and others have called for management of PFAS as a single class. Until that happens, the U.S. EPA can be commended for its efforts to assess large groups of PFAS such as those evaluated by Carlson et al. in their SEM of approximately 150 PFAS (PFAS-150). PFAS-150 follows many of the best practices for conducting SR and SEM, including a clear statement of objectives, a comprehensive literature search that interrogates diverse data repositories (including gray literature), and a structured format for organizing extracted study details. Therefore, we believe the data extracted for display in this SEM, which is a very resourceintensive endeavor, should be used in future health assessments of the included PFAS. Carlson et al. used machine learning tools to greatly improve the efficiency of the SEM workflow, including SWIFT-Active Screener, which iteratively prioritizes titles and abstracts for manual screening. The use of SWIFT-Active Screener for this purpose has been externally validated. The authors also used evidence stream filters in the related tool, SWIFT-Review, to prioritize studies most applicable to human health risk assessment. Although this tool appears valuable for reducing the screening burden, the validity and reliability of the tagging against manual review needs to be assessed. In the meantime, other tools such as DistillerSR’s “Check for Screening Errors” are available to confirm there were no “false excludes.” As new machine learning and artificial intelligence tools are developed to support the emerging field of SR and SEM, we believe it is imperative they be evaluated for accuracy and consistency. The scope of an SEM is determined and defined by the Population, Exposure, Comparison, and Outcome (PECO) statement.We recently released the PFAS-ToxDatabase, an SEMof 29 PFAS available at https://pfastoxdatabase.org/. Our SEM covers a similar time period, but our PECO statement differs significantly from the one used for PFAS-150. This resulted in different results for the eight PFAS in common between the two SEMs (Table 1). For example, whereas we identified 54 animal studies for PFUnDA, Carlson et al. identified only 2. The PECO statement guiding development of the PFAS-ToxDatabasewas intentionally very broad, because our goal was to present the entirety of the peer reviewed health and toxicological literature for the included PFAS. In comparison, Carlson et al. used a narrower PECO statement to guide their work, with their goal being a fit-forpurpose SEM that informs the type of human health assessment work that the U.S. EPA routinely conducts.

organizing extracted study details. 5,17 Therefore, we believe the data extracted for display in this SEM, which is a very resourceintensive endeavor, should be used in future health assessments of the included PFAS.
Carlson et al. 1 used machine learning tools to greatly improve the efficiency of the SEM workflow, including SWIFT-Active Screener, which iteratively prioritizes titles and abstracts for manual screening. The use of SWIFT-Active Screener for this purpose has been externally validated. 18 The authors also used evidence stream filters in the related tool, SWIFT-Review, to prioritize studies most applicable to human health risk assessment.
Although this tool appears valuable for reducing the screening burden, the validity and reliability of the tagging against manual review needs to be assessed. 19 In the meantime, other tools such as DistillerSR's "Check for Screening Errors" are available to confirm there were no "false excludes." As new machine learning and artificial intelligence tools are developed to support the emerging field of SR and SEM, we believe it is imperative they be evaluated for accuracy and consistency.
The scope of an SEM is determined and defined by the Population, Exposure, Comparison, and Outcome (PECO) statement. We recently released the PFAS-Tox Database, an SEM of 29 PFAS available at https://pfastoxdatabase.org/. 20,21 Our SEM covers a similar time period, but our PECO statement differs significantly from the one used for PFAS-150. This resulted in different results for the eight PFAS in common between the two SEMs (Table 1). For example, whereas we identified 54 animal studies for PFUnDA, Carlson et al. identified only 2. The PECO statement guiding development of the PFAS-Tox Database was intentionally very broad, because our goal was to present the entirety of the peer reviewed health and toxicological literature for the included PFAS. In comparison, Carlson et al. used a narrower PECO statement to guide their work, with their goal being a fit-forpurpose SEM that informs the type of human health assessment work that the U.S. EPA routinely conducts.  Table 1 shows the number of studies for the eight PFAS included in both SEMs. Differences were primarily due to the inclusion in our SEM of wildlife observational studies, experimental studies in nonrodent species, and all routes of exposure. Additionally, PFAS-150 included nonpeer-reviewed gray literature not in our SEM, such as industry studies available in the U.S. EPA's Health and Environmental Research Online database or the European Chemicals Agency's Registration, Evaluation, Authorization and Restriction of Chemicals database.
A conclusion of our SEM was that there were more studies available than we anticipated for several PFAS. Carlson et al. concluded that many of the PFAS they assessed were data poor. 1 Considering both SEMs, this comparison makes clear that there are many available studies on some PFAS that would not be used by the U.S. EPA in a regulatory context. However, it is our hope that such studies would be included in a meaningful way in the body of evidence that the U.S. EPA uses to inform health assessments.
Though Carlson et al. concluded that many of the PFAS-150 chemicals were data poor, it is important to remember that lack of research does not mean lack of biological effect. SEMs such as PFAS-150 and the PFAS-Tox Database can help inform future read-across efforts in which conclusions about data-poor chemicals are derived from evidence from data-rich chemicals. To this point, it is noteworthy that several PFAS recently reviewed or currently under review at the U.S. EPA were not included in the PFAS-150 SEM (GenX, PFBA, PFHxA, PFOA, PFNA, PFDA, PFBS, PFHxS, PFOS). Swift efforts should be made to add these PFAS to the SEM to better inform read across for PFAS and support evaluation of the entire class.
A primary goal of SEMs in environmental health is to gather large amounts of data in one place to support deeper scientific analyses and decision-making that protects public health and the environment. Keeping SEMs up to date is challenging, especially for PFAS, because the body of literature is growing rapidly. We encourage the U.S. EPA to continue populating PFAS-150 with current evidence. We applaud the U.S. EPA authors for their efforts to support the field in this way and look forward to future SEMs and appropriate regulatory action on other chemicals of concern.