Improving the Human Hazard Characterization of Chemicals: A Tox21 Update

Background: In 2008, the National Institute of Environmental Health Sciences/National Toxicology Program, the U.S. Environmental Protection Agency’s National Center for Computational Toxicology, and the National Human Genome Research Institute/National Institutes of Health Chemical Genomics Center entered into an agreement on “high throughput screening, toxicity pathway profiling, and biological interpretation of findings.” In 2010, the U.S. Food and Drug Administration (FDA) joined the collaboration, known informally as Tox21. Objectives: The Tox21 partners agreed to develop a vision and devise an implementation strategy to shift the assessment of chemical hazards away from traditional experimental animal toxicology studies to one based on target-specific, mechanism-based, biological observations largely obtained using in vitro assays. Discussion: Here we outline the efforts of the Tox21 partners up to the time the FDA joined the collaboration, describe the approaches taken to develop the science and technologies that are currently being used, assess the current status, and identify problems that could impede further progress as well as suggest approaches to address those problems. Conclusion: Tox21 faces some very difficult issues. However, we are making progress in integrating data from diverse technologies and end points into what is effectively a systems-biology approach to toxicology. This can be accomplished only when comprehensive knowledge is obtained with broad coverage of chemical and biological/toxicological space. The efforts thus far reflect the initial stage of an exceedingly complicated program, one that will likely take decades to fully achieve its goals. However, even at this stage, the information obtained has attracted the attention of the international scientific community, and we believe these efforts foretell the future of toxicology.

Thousands of chemicals to which humans are exposed have inadequate data on which to predict their potential for toxicological effects. However, dramatic technological advances in molecular and systems biology, computational toxicology, and bioinformatics have provided researchers and regulators with powerful new public health tools [National Research Council (NRC) 2006. High content screening (HCS) and high throughput screen ing (HTS) techniques are now routinely used in conjunction with computational methods and information technology to probe how chemicals interact with biological systems, both in vitro and in vivo. Progress is being made in recognizing the patterns of response in genes and pathways induced by certain chemicals or chemical classes that might be predictive of adverse health outcomes in humans. However, as with any new technol ogy, both the reliability and the relevance of the approach need to be demonstrated in the context of current knowledge and practice.
In 2008, in response to the National Academy of Sciences' (NAS) report Toxicity Testing in the 21st Century, a Vision and a Strategy (NRC 2007), Collins et al. (2008) outlined a collaboration between the National Institute of Environmental Health Sciences (NIEHS)/National Toxicology Program (NTP), the U.S. Environmental Protection Agency's (EPA) National Center for Computational Toxicology (NCCT), and the National Human Genome Research Institute (NHGRI)/National Institutes of Health (NIH) Chemical Genomics Center (NCGC) (now located within the National Center for Advancing Translational Sciences) to develop a vision and devise an implementation strat egy to shift the assessment of chemical hazards from traditional experimental animal toxicol ogy studies to targetspecific, mechanismbased, biological observations largely obtained using in vitro assays. In mid2010, the U.S. Food and Drug Administration (FDA) joined the collabo ration, which is known informally as Tox21.
The Tox21 partner agencies (Collins et al. 2008) agree to collaborate to • Research, develop, validate, and translate innovative compound testing methods to characterize toxicity pathways. • Identify compounds, assays, informatic analy ses, and targeted testing needed to support development of the new methods. • Identify patterns of compoundinduced bio logical response in order to characterize toxicity pathways, facilitate cross species extrapolation, and model lowdose extrapolation.
• Prioritize compounds for more extensive toxicological evaluation. • Develop predictive models for biological response in humans. • Make all data publicly available.
The purpose of this review is to outline the efforts of the U.S. EPA, the NCGC, and the NTP up to the time the FDA joined the collaboration; to describe the approaches taken to develop the science and technolo gies currently being used; to assess the current status; and to identify problems that could impede further progress as well as approaches to address those problems.
To support the goals of Tox21, four work ing groups-Compound Selection, Assays and Pathways, Informatics, and Targeted Testing-were established; a representative of each Tox21 partner serves as a cochair on each working group. The working groups reflect the different components of the NAS vision (NRC 2007) and cooperatively address the four major focus areas necessary to bring about this paradigm shift [see NIEHS (2012) for additional information on the approaches and components of Tox21].

Chemical selection and lessons learned.
Developing a comprehensive set of sub stances (i.e., a compound library) of toxico logic concern is critical to the ultimate ability of Tox21 to develop relevant prioritization schemes and prediction models. Ideally, any library should be populated with sub stances of known identity and purity that are compatible with the solvent of choice for the assay platforms being used. In 2006, the NTP and the U.S. EPA established at the NCGC "proof of principle" libraries of 1,408 and 1,462 compounds, respectively, with each compound dissolved and stored in dimethyl sulfoxide (DMSO), to be evaluated for activity in 1,536well plate quantitative high throughput screens (qHTS), as described by Inglese et al. (2006). In traditional HTS, compounds are tested at a single concentra tion and the results are therefore burdened by frequent false negatives or positives. In contrast, in qHTS, many thousands of com pounds are screened in a single experiment across a broad concentration range in order to generate concentration-response curves. The method identifies compounds with a wide range of activities with a much lower false positive or falsenegative rate. The resulting concentration-response curves can be classi fied to rapidly identify actives and inactives with a variety of potencies and efficacies, pro ducing rich data sets that can be mined for reliable biological activities.
The identities and structures of the com pounds in these libraries are available in PubChem (http://pubchem.ncbi.nlm.nih. gov/) using the assay identification num ber (AID). To evaluate withinassay repro ducibility, each library included a number of compounds in duplicate (based on struc ture: 66 for the NTP; 77 for the U.S. EPA) although the two libraries contained 411 compounds in common, generally represent ing different suppliers and/or lots when from the same supplier. During examination of the data generated using these libraries, issues were identified (e.g., inaccurate information on certificates of analysis accompanying pur chased chemicals, lack of compound stabil ity under the conditions of storage and use) that informed efforts during the development of the much larger compound library to be screened as part of Tox21 Phase II using a recently established, dedicated Tox21 robotics facility at the NCGC (NIEHS 2011a).
The Tox21 Phase II compound library includes structurally defined compounds intended to broadly capture chemical and toxicological "space." The libraries include compounds with extensive to no toxicologi cal information and with use, production, chemical class identity, and/or environmental exposure patterns that make them of poten tial concern to regulatory agencies. To pro duce this compound library, an initial list of approximately 120,000 compounds was culled to approximately 11,000 unique com pounds with known structures. The physi cal property cutoffs for the Phase II library were a molecular weight range of 100-1,000, a vapor pressure of < 10 Pa, and a calculated log pvalue of -2 to 6. The desired solubil ity in DMSO was 20 mM, but some com pounds of special interest, soluble only at a lower concentration, have been included. The library contains the compounds in the U.S. EPA's ToxCast™ Phases I and II, including approximately 150 pharmaceuticals that failed in clinical trials (U.S. EPA 2013f). These failed pharmaceuticals were provided to the U.S. EPA by Pfizer (New York, NY), Merck (Summit, NJ), GlaxoSmithKline (Research Triangle Park, NC), Sanofi (Bridgewater, NJ), Roche (Indianapolis, IN), and Astella (Northbrook, IL), with the understanding that the identity, structure, and toxicity data would be made public. The NCGC contribu tion to the library is the recently developed NCGC Pharmaceutical Collection (NPC), a comprehensive, publicly accessible collec tion of approved and investigational drugs for HTS (Huang et al. 2011b). The NPC contains approximately 3,500 small mol ecules that have been approved for clinical use by the United States (FDA), European Union (European Medicines Agency), Japan (Minister of Health, Labor, and Welfare), and Canada (Health Canada) and that are amen able to HTS screening.
The completion of the Tox21 Phase II library was announced in December 2011 (NIEHS 2011b). This library contains > 10,000 (10K) compounds [8,193 of which are unique; see U.S. EPA (2013e) for the com plete list]; the compounds fall into classes that include, among others, industrial chemicals, sunscreen additives, flame retardants, pesticides and selected metabolites, plasticizers, solvents, food additives, natural product components, drinking water disinfection byproducts, pre servatives, therapeutic agents, and chemical synthesis byproducts. Although the focus of the 10K library is on individual compounds with known structures, a few hundred formu lations prepared from sets of 8-62 compounds with selection based on estrogen receptor activ ity, androgen receptor activity, and in vitro cytotoxicity profiles have been included, as well as each individual constituent, to explore how a mixtures library could be established and the resulting HTS data evaluated as part of Tox21. Another future plan is to establish a library with water as a solvent for hydrophilic com pounds that are relatively insoluble in DMSO.
To evaluate withinrun reproducibility, a set of 88 broadly bioactive compounds is included in duplicate on each 1,536well assay plate. The library also includes multi ple sam ples of many compounds, providing another measure of compound and assay variability. The 10K library is being screened three times in each qHTS assay at the NCGC, with com pounds in a different well location during each run, to better evaluate assay reliability and to increase the ability to distinguish between weak active and inactive compounds.
To address compound identity and purity, to confirm the stock solution concentration (generally 20 mM), and to determine com pound stability in DMSO under the storage conditions used, quality control analysis of the entire library is being conducted using a tiered approach. First, a high throughput, HPLC system with multiple detectors [mass spec trometry, ultraviolet diode array, evaporative light scattering detection (ELSD), and chemi luminescent nitrogen detection (CLND)] is being used for identity characterization and purity estimation. Identity confirmation is performed by a matching molecular ion in the mass spectrum with the desired com pound; purity analysis is conducted with the ELSD. For compounds containing nitro gen, the CLND provides quantitation of the compound concentration. This system does not work well for some of the more volatile compounds or for those that will not properly ionize in the mass spectrometer. As needed, followup analyses are being conducted by gas chromatography with mass spectrometry and other analytical techniques. The stability of the compounds under the conditions of use will be determined also. Chromatographic and quality control data for all components of the Tox21 Library will be linked to the master chemical list and the qHTS data and made publicly available.
Quantitation of the compounds in DMSO is complex given the diversity of their chemical properties. To date, we have not identified a costeffective approach for confirming the concen tra tion of each compound under test conditions in the 1,536well format. Accomplishing this requires additional sensi tivity because the plates are assayed at far lower concentrations than the "source" plate, and the analytical system has to be compatible with water and buffers.
Assay selection and lessons learned. At the NCGC, using qHTS, tens of thousands of compounds can be rapidly screened at mul tiple concentrations (typically 15 concentra tions, from ~ 0.5 nM to ~ 92 µM) to yield concentration-response curves defining com pound activity. Assay selection during Phase I was constrained by the availability of suitable assays, both from a technological and a biologi cal perspective. Essentially, Phase I screening at the NCGC was a pilot study to evaluate assay performance, methods of assay protocol opti mization, and the extent to which protocols could be varied without compromising results. The qHTS data generated were also used to develop appropriate statistical analysis proce dures to allow automated evaluation of thou sands of qHTS concentration curves to identify actives and inactives in different kinds of assays.
volume 121 | number 7 | July 2013 • Environmental Health Perspectives In addition, a number of strategies for orthogo nal (i.e., the same biological outcome on a dif ferent assay platform) or followup screens to confirm and extend the results obtained were explored .
Assay selection was accomplished via sev eral mechanisms. Initially, four commercially available cellbased assays were selected to evaluate the suitability of the qHTS approach in the 1,536well format for screening a nondrug-like compound library. These assays were the Promega CellTiter Glo® cell viability assay, which measures intracellular ATP levels, and the Promega Caspase Glo® 3/7, 8, and 9 assays, which measure apoptosis (Promega Corporation, Madison, WI). The CellTiter Glo® cell viability assay was used first by screening the NTP 1,408com pound library for cytotoxicity in 13 cell types (9 human, 2 rat, 2 mouse) ). The cell types originated from different tis sues and included cell lines, cell strains, and primary cell populations. As anticipated, over the concentration range tested, there were compounds that were cytotoxic in all cell types. However, there were also compounds that were uniquely cytotoxic to only one or a few cell types. Similar results were obtained when the NTP 1,408 compounds were screened for apoptosis in the same cell types ). These results indicated that no single cell type would be universally informative for cytotoxicity or apoptosis but that the use of multiple cell types would allow compounds to be binned by their pattern of response Xia et al. 2008). Upon completion of these assays, addi tional assays were added to the screening effort. In addition to assays selected by the Tox21 partners, the NTP compound library was screened in assays conducted at the NCGC as part of the Molecular Libraries Screening Initiative (NIH 2013a). The qHTS assays in which the NTP and/or U.S. EPA libraries were screened during Phase I are listed in Table 1. Figures 1-3 show the percentage of compounds classified as active in each of the phenotypic assays (NTP 1,408compound library); the pathway/ target assays (NTP 1,408compound library); and the nuclear receptor, DNA damage, cytochromep450, and miscellaneous assays (NTP 1,408 and EPA 1,462compound libraries). The substances were screened at  Xia et al. (2011). Briefly, concentration-response titration points for each compound were fit ted to a fourparameter Hill equation (Hill 1910), yielding half maximal concentration (AC 50 ) and maximal response (efficacy) val ues. The compounds were designated as class 1-4 according to the type of concentrationresponse curve observed (Inglese et al. 2006). Curve classes are heuristic measures of data confidence, classifying concentrationresponses on the basis of efficacy, the num ber of data points observed above background activity, and the quality of fit. To facilitate analysis, each curve class was combined with an efficacy cutoff and converted to a numeri cal curve rank ) such that more potent and efficacious compounds with higher quality curves were assigned a higher rank, and inactive (class 4) compounds were assigned a curve rank of 0. Curve ranks should be viewed as a numerical measure of com pound activity. Compounds with curve ranks > 4 or < -4 were considered as active activators or inhibitors, and compounds with other non zero curve ranks were considered inconclusive. The percentage of actives varied from as lit tle as 0.07% for an epigenetics cellbased assay [Locus DeRepression (LDR)] (Figure 1) to as much as 41% for a biochemical assay that eval uated the ability of compounds to interact with cytochrome P450 CYP1A2 (Figure 3). Within any single assay, the potency of the active com pounds (based on AC 50 values) varied as much as five orders of magnitude.
Although the focus of qHTS at the NCGC is on screening large numbers of com pounds in biochemical and cellbased assays of potential toxicological interest, the platform can also be used to explore genetic differences in sensitivity to toxicants. During Phase I, we evaluated differential sensitivity among a genetically defined panel of 81 human lympho blastoid cell lines [27 Centre d'Etude du Polymorphisme Humain (CEPH) trios (parents and offspring) assembled by the HapMap Consortium (http://hapmap. ncbi.nlm.nih.gov/)] using 240 compounds (12 concentrations, 0.26 nM to 46 µM), a selected subset of the NTP library. Caspase 3/7 activity, a marker of apoptosis, and intra cellular ATP, a measure of cell viability, were the end points evaluated. qHTS screening in the genetically defined population produced robust results, allowing for crosscompound, assay, and individual comparisons (Lock et al. 2012). The generation of highquality qHTS in vitro cytotoxicity data for these genetically defined cell lines on a large library of compounds demonstrated the potential of this methodology to assess the degree of inter individual variability in toxicity and to explore its genetic determinants. Results of phenotypic assays of the NTP 1,408-compound library showing the percentage of substances classified as active in each assay. Abbreviations: 12hLO, 12-human lipoxygenase; ALDH1A1, aldehyde dehydrogenase 1 family, member A1; AMPC, b-lactamase/d-alanine carboxypeptidase; AP1, activator protein 1; APE1, apurinic/apyrimidinic endonuclease 1; ATAD5, DNA damage response element; BRCA, breast cancer, early onset; CHO, Chinese hamster ovary; CRE, cAMP response element; ERK, extracellular signal-regulated kinase; HADH560, hydroxyacyl-coenzyme A dehydrogenase, type II; HEK, human embryonic kidney; hERG, human ether-a-go-go-related gene potassium channel; HPGD, hydroxyprostaglandin dehydrogenase; HRE, hypoxia response element; HSDB, hydroxysteroid (17-b) dehydrogenase; HSP90, heat shock protein 90kDa a; IL-8, interleukin 8; IMPase, inositol monophosphatase; i-RGS, G-protein signaling protein RGS12; JNK, c-Jun N-terminal kinase; LDH, l-lactate dehydrogenase; LDR, Locus DeRepression; NPS, neuropeptide S; O-Glc NAc transferase, O-Glc NAc N-acetylcysteine transferase (sOGT); PK, pyruvate kinase; PPAR-a, peroxisome proliferator-activated receptor a; ROR, retinoic acid-related orphan receptor; PRX, peroxiredoxins; SMN2, survival motor neuron protein; TDP1, tyrosyl-DNA phosphodiesterase; TNFa, tumor necrosis factor-a; TSH, thyroid stimulating hormone; YjeE:ADP binding, essential Escherichia coli protein of unknown function that binds adenosine diphosphate.

Active substances (%)
Phase I screening provided a valuable experience in the use of qHTS approaches for the toxicity screening of environmental compounds. The promise of this approach was clear, but limitations were also identified. Significant limitations on assay protocols are imposed by the use of 1,536well plates on a highly automated robotics platform [National Center for Advancing Translational Sciences (NCATS) 2013a]. Assay selection is often con strained by currently available technologies, but the NCGC has been able to adapt many assays to conform to the technological require ments of qHTS. Through Small Business Innovative Research and Small Business Technology Transfer grants and contracts (NIH 2013b, U.S. EPA 2013c), research col laborations, and communications with com mercial assay suppliers, the development of in vitro assays compatible with qHTS require ments has increased. Furthermore, to advance the capabilities of Tox21, there is a public nomination process for assays to be considered for implementation in qHTS (NCATS 2013c; NTP 2013d; U.S. EPA 2013d).
The results of the first qHTS cytotoxic ity and apoptosis assays suggested the need to decide on a preferred origin of cells for cellbased assays. A goal of Tox21 is to use human cell-based as opposed to rodent cellbased assays whenever possible to eliminate concerns about speciesdependent differences in response. In addition to considering spe cies and tissue of origin, the use of primary cells or mixed cell cultures versus established, commercially available individual cell lines was explored. From a biological perspective, primary cells and mixed cell cultures would be preferred, but they present challenges of avail ability, generally require special handling, are not easily adaptable to 1,536well assay condi tions, and are not used to establish reporter gene assays, which constitute the majority of current qHTS assays. An issue that has been extensively discussed is whether to restrict assays, including gene transactivation assays, to a single cell type to reduce the number of variables affecting data interpretation or, alter natively, to select each assay based solely on maximizing sensitivity and reproducibility. Because of the limited availability of reporter gene assays using a common cell type, Phase II will employ the latter approach.
There are other technical limitations in the current qHTS paradigm. There is cur rently no method for including metabolic activation in the qHTS screens because liver S9 mix is toxic to cells when used beyond a few hours and the current qHTS assay protocols cannot include aspiration steps. Thus, there is a critical need to develop other approaches for including xenobiotic metabo lism. These may include culturing primary hepatocytes alone or with a co cultured reporter gene assay, culturing threedimen sional liver model inserts (which are currently not applicable to high throughput) into wells along with a cocultured reporter gene assay, or using metabolically competent cell lines [e.g., HepaRG (Kanebratt and Andersson 2008)] as the target cell population. There are also targets of toxicological importance, such as the proteins involved in gap junction cell tocell communication or the orphan nuclear receptor constitutive androstane receptor, for which there are no existing in vitro assays amenable to qHTS.
The findings generated during Tox21 Phase I have demonstrated the applicabil ity of the qHTS approach for screening a large library of environmental compounds. Assays originally developed for drug discov ery can be used, directly or with modifica tion, to evalu ate cellular processes potentially involved in toxicity responses. Statistical approaches have been developed to analyze Active substances (%) volume 121 | number 7 | July 2013 • Environmental Health Perspectives the enormous amounts of data produced from qHTS screens. However, data analysis has not been straightforward. A surprising number of complications have been identified and approaches to deal with these complex issues are discussed below.
Taking NCGC assay throughput into account, the experience in Phase I, and the results of a comprehensive analysis of disease associated cellular pathways (e.g., Gohlke et al. 2009), the Phase II assay strategy is to initially focus on assays that measure the induction of stress response pathways (Simmons et al. 2009) and interactions with nuclear receptors. The selection of stress response assays (e.g., apoptosis, antioxidant response, cytotoxicity, DNA damage response, endoplasmic reticu lum stress response, heat shock, inflamma tory response, mitochondrial damage) is based on the premise that compounds that induce one or more stress response pathway are more likely to exhibit in vivo toxicity than those that do not. The human nuclear receptor assays (androgen; aryl hydrocarbon; estrogena; farnesoid X; glucocorticoid; liver X; peroxi some proliferatora, d, and g; progesterone; pregnane X; retinoid X; thyroidb; vitamin D) were selected because of the key roles they play in endocrine and metabolism pathways. The initial nuclear receptors assayed during Phase I at the NCGC used partial receptors that consisted of the ligandbinding domain and the Cterminal end (Huang et al. 2011a). However, because of concerns about potential differences in chemical response profiles when using a complete versus a partial receptor, the Phase II 10K library is being screened against both partial and complete receptors, at least for the androgen and estrogen receptors, in agonist and antagonist modes.
The primary limitation of this qHTS platform is that, although thousands of com pounds can be screened in a single assay, each assay is generally limited, in terms of biologi cal output, to one or two signals. In addi tion, most transcriptional activation assays are developed in established cell lines (and not always ones of human origin) rather than "normal" human cells. To potentially over come these limitations, Tox21 is investigat ing several different genomicbased platforms that would allow for the monitoring of gene expression patterns (signatures), induction or repression, (e.g., 200-1,000) in any cell type, including human primary and stem cells (undifferentiated and differentiated) in a 384well-plate format.
Also, in Tox21 Phase II, on the basis of the qHTS cytotoxicity results obtained with the 81 human lymphoblastoid cell lines (Lock et al. 2012), we (along with I. Rusyn and colleagues at the University of North Carolina-Chapel Hill) expanded the scope of this inter individual differential sensitiv ity project to evaluate approximately 1,100 different human lymphoblastoid cell lines, with densely sequenced genomes representing nine races of humankind, to 180 toxicants, using the CellTiter Glo® cell viability assay (Promega) to assess cytotoxicity. The large number of human cell lines used allows for an analysis of genetic determinants associated with differential cytotoxicity in vitro.
Informatics and lessons learned. In Phase I, an extensive set of concentrationresponse data was generated on approximately Figure 3. Results of nuclear receptor, DNA damage, cytochrome-p450, and miscellaneous assays of the NTP 1,408-and the U.S. EPA 1,462-compound libraries showing the percentage of substances classified as active in each assay. Abbreviations: AHR, aryl hydrocarbon receptor; AR, androgen receptor; ARE, antioxidant response element; ER, estrogen receptor; esrE, Escherichia coli strain K-12 substrain MG1655; FXR, farnesoid X receptor; GR, glucocorticoid receptor; HSP, heat shock protein; NFkB, nuclear factor-kB; PPAR, peroxisome proliferator-activated receptor; PXR, pregnane X receptor; RXR, retinoid X receptor; TR, thyroid hormone receptor; VDR, vitamin D receptor. 1,400 (only NTP compounds tested) or approximately 2,800 compounds (both NTP and U.S. EPA compounds tested) screened at the NCGC in approximately 70 qHTS assays (see Table 1 for which sets of com pounds were screened in which assays). In addition, in ToxCast™ Phase I, the U.S. EPA obtained concentration-response curves on 309 unique compounds tested across approxi mately 550 in vitro and lower organism in vivo assays by various contract and government laboratories (Chandler et al. 2011;Dix et al. 2007;Houck et al. 2009;Judson et al. 2010;Knight et al. 2009;Knudsen et al. 2011;Martin et al. 2009aMartin et al. , 2009bMartin et al. , 2010Martin et al. , 2011Padilla et al. 2012;Rotroff et al. 2010). The raw data from HTS studies is generated using a number of different readouts (e.g., fluores cence, luminescence, phenotypic). Regardless of the assay readout, the goal is the sameidentification of compounds that are active, not active, or inconclusive (i.e., based on the response, the compound is not clearly active or clearly inactive). In qHTS at the NCGC, the raw data generated at each chemical con centration tested are first normalized relative to the positive control response (i.e., 100% response) and the basal response in the solvent control DMSOonly wells (i.e., 0%) on the same 1,536well plate, and then corrected by applying a pattern correction algorithm using the compoundfree DMSO control plates. Outlier values are identified and removed based on the fit to the Hill equation, which is often used to describe sigmoidal biochemi cal phenomena [for a more extensive descrip tion of this process, see Inglese et al. (2006) and Shockley (2012)]. Traditional methods used to assess the significance of non linear regression analyses rely heavily on human inspection of individual residual plots or com parisons of the fit and the raw data, which is not practical in the qHTS analysis context with thousands of compounds in hundreds of assays. Furthermore, confounding effects such as autofluorescence of cellular constitu ents or the chemicals under study and cyto toxicity may complicate the data analysis and interpretation. The normalized and processed data set emerging from qHTS studies at the NCGC is very large, and a number of heuris tic approaches and statistical models have been developed to address a variety of HTS data structures (Inglese et al. 2006;Parham et al. 2009;Shockley 2012). When concentration-response data can be confidently modeled, AC 50 values are cal culated from curve fits to the three or four parameter Hill equation. In assay sets in which no upper asymptote (agonist assays) or lower asymptote (antagonist or cytotoxicity) can be defined, a lowest effective concentration is calculated, defined as the lowest concentra tion at which there is a statistically significant difference from the concurrent negative con trol. The results are corrected to remove arti facts due to cytotoxicity and parameters such as concentration for half maximal activity, maximum efficacy, and minimal response are used to make activity calls based on algorithms for specific assay types and platforms .
The extensive data being generated by Tox21, both in terms of the number of com pounds being screened and in the diversity of assays being used, is providing a unique opportunity for the development of novel approaches for making activity calls and for relating those calls to "truth" based on existing human and animal data. The determination of which approach is most appropriate depends on a priori knowledge of the assay in question, the purpose of the study, and the structure of the data. The collection of Tox21 data will be used to create a diagram of the biological network that responds to chemical perturba tions that will be linked to toxicological effects in animals and humans. To achieve this goal, assays that measure targets that encompass pathways rele vant to toxicity need to be used. However, there is no single comprehensive and uniform resource that covers all known annotations of pathways or any single platform that allows integrated browsing, retrieval, and analysis of information from the many exist ing individual webbased pathway resources. In response to this need, the NCGC is build ing an integrated pathway resource that hosts information from manually curated and pub licly available resources. The NCGC Human BioPlanet will allow easy browsing, visualiza tion, and analysis of the universe of human pathways. The main view of the BioPlanet maps all known human pathways on a three dimensional globe, where each spot represents a gene on a pathway. Selecting a gene on the globe will place all elements of the pathway in a detailed view window. Detailed descriptions of all genes in the selected pathway are shown below the threedimensional graphics. When multiple pathways are selected at the same time, the view will show all unique gene com ponents within selected pathways. Multiple pathways can be illustrated as one extended pathway that better shows the interaction between different biological processes. The BioPlanet will be searchable by any gene or pathway identifier and also by disease rele vance (prevalence of disease genes), toxicity relevance (occurrence of genes in toxicology litera ture), and availability of assays in PubChem. Thus, this platform will allow for an assessment of assay coverage across the approximately 1,100 human pathways and where new assays might be most useful.
The qHTS assays used at the NCGC focus on cellbased phenotypic or transactivation end points and do not measure directly the binding of a compound to a receptor or other cellular component. However, the wealth of data being generated in Tox21 Phase II will be used to both test the validity of existing docking and quantitative structure activity relationship (QSAR) models and for develop ing new ones. Given the public availability of the structures of the 8,193 unique compounds included in the Phase II > 10K library (U.S. EPA 2013e), we invite the scientific informat ics community to predict the activity of these compounds in the different nuclear recep tor and stress response pathway assays before public release of the data.
One critically important goal is to make all Tox21 data publicly accessible via various data bases, including the NTP's Chemical Effects in Biological Systems (CEBS; NTP 2013a), the U.S. EPA's Aggregated Computational Toxicology Resource (ACToR; Judson et al. 2012, U.S. EPA 2013a), PubChem, and the NCATS Tox21 Chemical Browser (NCATS 2013b), to encourage independent evaluations of Tox21 findings. Data will be made available after ensuring accurate linkage of the com pound to its correct structure, the results of the chemical analysis, and assay responses.
Will it work? Although we've made good progress in laying the groundwork to enable us to answer the question of whether Tox21 can fulfill the expectations to transform toxic ity testing, the area that requires the most work is one that we term targeted testing. This term encompasses everything from designing and carrying out confirmatory assays for a given biological outcome in a second related in vitro assay (i.e., an orthogonal assay); incorporating engineered human tissue models into Tox21; and confirming a response in a whole organ ism such as Caenorhabditis elegans, zebra fish, or rodents to evaluating methods for extrapo lating from in vitro concentration to in vivo dose levels. Perhaps the most important type of targeted testing that must be accomplished is the simple but huge intellectual effort needed to compare the output of Tox21 with what we know from our existing databases of ani mal and human toxicology. To date, data from ToxCast™ have been used to develop a number of prediction models and prioritiza tion schemes (Judson et al. 2008Kleinstreuer et al. 2011aKleinstreuer et al. , 2011bKleinstreuer et al. , 2013Martin et al. 2009aMartin et al. , 2009bMartin et al. , 2011Reif et al. 2010;Rotroff et al. 2013;Sipes et al. 2011). qHTS data generated at the NCGC have been used to develop structural feature models ), to profile the chemical modu lation of multiple human nuclear receptors (Huang et al. 2011a), to evaluate chemicals capable of interfering with mitochondrial func tion (Sakamuru et al. 2012), and to predict chemical structures that interact with cyto chrome P450 (Sun et al. 2012) among others. Although predictive models of phenotypic volume 121 | number 7 | July 2013 • Environmental Health Perspectives outcomes will require considerable effort to evaluate their reliability and relevance to support regulatory action, the technologies employed in the Tox21 program are actively being investigated for application to the priori tization of chemicals in testing programs by the U.S. EPA. For example, in the EDSP21 program (U.S. EPA 2011, 2013b), the short term goal is to use the technologies to priori tize chemicals for nomination for screening in the current EDSP assay battery, whereas the intermediate and longterm goals target the incorporation and ultimate replacement of the current assays with in silico and molecular based high throughput assays.
In support of Tox21, the NTP is evaluat ing techniques for mining its tissue archives for gene expression response profiles to expand our ability to link chemicals to genes, genes to pathways, and pathways to disease. The NTP archives contain stained histo pathology slides, paraffin tissue blocks, formalinfixed tissues and organs, and selected frozen tissue from over 2,000 experimental rodent studies, including toxicity, carcinogenicity, immuno toxicity, reproductive, and developmental studies. We have conducted studies to evaluate the extent to which gene expression signatures can be reliably derived from the molecular analysis of tissue samples stored as formalin fixed, paraffinembedded tissues (Merrick et al. 2012). Such signatures could contribute to a more comprehensive understanding of doseresponse relationships at the molecular level, the identification of useful targets for in vitro assays, to an evaluation of the correlation between in vitro test results and in vivo toxi cological outcomes, and to the development of predictive models of toxicity.
In addition, the NTP recently acquired DrugMatrix®, a toxicogenomics reference database, tissue archive, and informatics sys tem originally developed by Iconix (Mountain View, CA) in 2007. The NTP acquired this resource in order to expand our ability to develop predictive models for toxicological effects based on gene signatures, to provide an additional tool for linking in vitro data to in vivo gene signatures and disease outcomes, and to provide additional tissue samples for next generation sequencing-based investiga tions. This integrated database of rat gene expression profiles, pathology measures, phar macology assays, and literature information on 657 compounds, primarily drugs along with the linked automated toxico genomics analysis application, ToxFX® are publicly accessible (NTP 2013b(NTP , 2013c. ToxFX® is useful for formulating gene signatures of toxic ity, for identifying potentially useful targets for in vitro assays, for linking in vitro data to in vivo toxicological effects, and for evaluat ing the extent to which humans and rodents share common toxicity/disease pathways. The U.S. EPA's NCCT, in its virtual embryo (U.S. EPA 2013g) and virtual liver (U.S. EPA 2013h) projects, are building a knowledgebase of chemical effect networks to produce com putable models of key molecular, cellular, and circulatory systems in the human liver and the developing embryo, respectively.

Conclusion
We fully appreciate that Tox21 faces some very difficult issues: • "Perfect" assays do not exist.
• Coverage of all chemicals of interest is incomplete (i.e., volatiles). • A high throughput system for measuring the free concentration of a compound in vitro is not yet available. • Xenobiotic metabolism is lacking in virtu ally all in vitro assays. • Interactions between cells are poorly captured. • Distinguishing between statistical and bio logical significance is difficult. • Extrapolating from in vitro concentration to in vivo dose or blood levels is not straight forward. • Assessing the effects of chronic exposure conditions in vitro is not possible. • Identifying when a perturbation to a gene or pathway would lead to an adverse effect in animals or humans remains a challenge. • Achieving routine regulatory acceptance of the developed prediction models is years away. However, we are making progress in inte grating data from diverse technologies and end points into what is effectively a systems biol ogy approach to toxicology. This can only be accomplished when comprehensive knowledge is obtained with broad coverage of chemical and biological/toxicological space. The efforts described thus far reflect the initial stage of an exceedingly complicated program, one that will likely take decades to fully achieve its goals. However, even at this stage, the information obtained is enticing the inter national scientific community and, we believe, fore telling the future of toxicology.