Multiple-Objective Optimal Designs for Studying the Dose Response Function and Interesting Dose Levels

1 Background

Experiments are increasingly expensive to conduct and it is desirable to obtain maximal information at minimal cost. Researchers now want to have several research questions answered from a single study to save cost. Designing such multi-objective experiments can be challenging because an optimal design for one objective can perform poorly under another and further, not all objectives may be equally important. Contrary to a single-objective optimal design, the sought design has to incorporate the multiple objectives at the onset and provide user-specified efficiencies for making inferences for the more important objectives.

There is some work on constructing dual-objective optimal designs but there is little in the literature on construction and properties of optimal designs for 3 or more objectives with a concrete application. Our aims in this paper are to construct 3-objective optimal designs for a pharmaceutical application, investigate effectiveness of such designs over single-objective optimal designs, and study their robustness properties under mis-specification in the nominal values of the model parameters and a change in the optimality criterion. Our focus is to estimate interesting characteristics of an agent in a dose response study where some drug characteristics may be more meaningful or important than others. Our dose response model is the flexible 4-parameter logistic model widely used in many disciplines, such as in educational research, biological sciences, pharmaceutical sciences and agronomy, to name a few. Both our application and the model are illustrative in the sense that the methodology described here also applies to other models and criteria.

Our setup assumes that we have a nonlinear regression model defined on a given compact dose interval X. The model has a known mean structure with unknown parameters and errors are assumed to be normal, independent and identically distributed, each with mean 0 and constant variance σ2. The sample size n is assumed to be fixed in advance either by cost consideration or the number of subjects available for the study. Given the model and optimality criteria with possibly different degrees of importance, the research questions are (i) what is the optimal number of doses to be used in the study? (ii) what are the doses? and, (iii) what is the optimal number of subjects at each of these doses? Our focus is on approximate designs, which means that we now determine the optimal number of doses K, the location of each dose Di and the optimal proportion wi of subjects to assign to the dose Di in the study, i=1,…,K. In practice, after the optimal approximate design is identified, we assign nwi subjects to Di, such that nw1+nw2+⋯+nwK=n and each nwi is rounded to the nearest positive integer. The advantages of this approach versus the approach of finding optimal exact designs are well documented in Kiefer [1], among others. For example, one can use convex analysis theory and construct algorithms for finding different types of optimal approximate designs. One can also assess the proximity of any design to the optimal approximate design (without knowing the optimal approximate design) by providing an efficiency lower bound for the design [2]. Equivalence Theorems are also available to confirm the optimality of an approximate design among all designs on X.

We follow convention and measure the worth of a design by its information matrix defined as the negative of the expectation of the second derivatives of the log likelihood function with respect to the model parameters. Given an approximate design ξ, this matrix is the weighted sum of information matrices over the distinct dose levels of ξ, and the weights are the proportions of subjects at the dose levels. For nonlinear models, this matrix depends on the model parameters. Because the design criteria are formulated in terms of the information matrix, the resulting optimal designs depend on the nominal values of the model parameters. These optimal designs are termed locally optimal [3] and it is well known that they can be sensitive to the choices of the nominal parameter values. This suggests that when there are conflicting nominal values for the model parameters, it is desirable to have a design that is robust to mis-specification in the nominal values. Our work also investigates whether multiple-objective optimal designs are robust to changes in the optimality criteria or changes in the degrees of interest in the objectives.

Section 2 describes the methodology for searching optimal designs when there are 3 or more objectives of varying interests. In Section 3, we discuss optimality criteria for dose response studies in the context of the popular Hill model and its relationship with the 4-parameter logistic model. In Section 4, we revisit a dose response study with 7 anti-cancer drugs from Khinkis et al. [4], report the single-objective optimal designs for estimating all parameters in the mean function, the ED50 and the MED, and how they compared with the multiple-objective optimal design that accounts for these objectives simultaneously at the onset. Additionally, we show how an Equivalence Theorem can be used to confirm the optimality of a multiple-objective optimal design and study sensitivities of such an optimal design to mis-specification in the nominal values, changes in the design criteria and relative interests in the multiple objectives. Our work here suggests that multiple-objective optimal designs are generally more efficient for making inferences than single-objective optimal designs and more robust to mis-specification in the nominal values of the parameters than single-objective optimal designs. We offer a conclusion in Section 5.

2 Multiple-objective optimal designs

Multiple-objective optimal designs are appealing because many scientific studies have several objectives and these objectives may vary in importance. A properly constructed design allocates resources that ensures the more important objectives are attained with user-specified efficiencies at minimum cost. Such situations arise frequently in real studies. We give three examples. In a dose response study, there is interest to infer the mean response at a specific dose and to estimate the shape of the overall dose curve accurately, and one of these objectives may be more important than the other. Another example is in estimation problems. It is often the case that some parameters are more meaningful than others and so there is greater interest in estimating selected parameters more accurately. For instance, in the 2-parameter Michaelis-Menten model, the Michaelis-Menten constant is clearly more interesting to estimate than the other parameter because it controls the rate of an enzyme-kinetic reaction. The third example concerns model inadequacy and inference. A properly targeted multiple-objective optimal design can detect model inadequacies and provide accurate inference at the same time. For example, consider the Emax model, which is the same as the Michaelis-Menten model except that the substrate concentration variable in the model is raised to some power. With the power as the third parameter, the Emax model is more flexible and can better capture asymmetry in the mean dose response curve than the Michaelis-Menten model. Model inadequacy concern and accurate inference on the parameters for the Emax model can then be simultaneously incorporated at the design stage using a multiple-objective optimal design that requires the power parameter be estimated with a user-specified efficiency, say 90% and subject to this constraint, the design does as well as possible for estimating the other two parameters in the model.

Multiple-objective optimal designs date back to the early seventies, where the proposed methods for constructing such designs were largely either based on ad-hoc procedures or simply based on the hope that the design constructed for the most important objective will be adequate for the other objectives. Stigler [5], Lauter [6, 7] and Lee [8, 9] were early attempts to formalize the procedure. Most were concerned with polynomial regression problems. For instance, Studden [10] was concerned about model inadequacy and wanted to find a design that was robust to the degree of the assumed polynomial model. His method ensured that coefficients in the assumed model were estimated as accurately as possible, and at the same time, the design could also provide user-specified efficiencies for estimating coefficients that might be needed for a higher degree polynomial model. Subsequent work on finding dual-objectives optimal designs includes Dette [11, 12], Zhu et al. [13], Wong [14], Song and Wong [15], Tsai and Zen [16], Atkinson [17], McGree et al. [18], Tommasi [19] and, Padmanabhan and Dragalin [20]. A recent application is Zhang et al. [21] where they constructed dual-objective optimal designs for a mixture experiment.

The formulation of the multiple-objective optimal design problem invariably involves a constrained optimization problem where the goal is to find a design that simultaneously meets user-specified minimal efficiencies for the various criteria, with higher efficiencies sought for the more important criteria, and subject to these requirements, does as well as possible for the least important criterion. Of course the sought multiple-objective optimal design may not exist if the requirements are too stringent and the objectives are competitive, meaning that much efficiency of one type has to be given up for a small gain in another criterion. Ad-hoc methods generally seek to combine the multiple design criteria into a single criterion with the expectation that the resulting optimal design for the single combined criterion may be efficient for all the criteria. The common problems with such an approach include how to combine the criteria in a meaningful way and the unclear interpretation of the combined criterion.

A more formal method to search for multiple-objective optimal approximate designs when there are two objectives ϕ1 and ϕ2 was proposed by Cook and Wong [22]. Their method involves the following steps: (0) Decide which is the more important criterion, say ϕ1; (1) formulate each objective as a concave function of the information matrix; (2) specify the efficiency requirement for the sought design under ϕ1, for example, e1≥0.9; (3) form a compound criterion ϕ by taking a convex combination of the two objectives: ϕ=(1−λ)ϕ1+λϕ2 (which is still concave); (4) for a large number of fixed values for λ∈[0,1], use an algorithm to search the compound optimal design ξλ that maximizes the compound criterion; (5) compute the efficiencies of the compound optimal designs under the two criteria;: e1(ξλ) and e2(ξλ); (6) construct the efficiency plot by graphing both e1(ξλ) and e2(ξλ) versus values of λ over the interval [0,1]; (7) on the graph, identify λ∗, the value of λ that corresponds to the intersection point where e1(ξλ) meets the horizontal line e1(ξλ)=e1 and, (8) the sought constrained optimal design is ξλ∗ which is guaranteed to have at least e1-efficiency and does as well as possible under the second criterion. Efficiency definitions are given in Section 4.2 and examples of efficiency plots to search for dual-objective optimal designs are available in Cook and Wong [22] and Wong [23].

Clyde and Chaloner [24] extended the methodology to find Bayesian multiple-objective optimal designs for nonlinear models. However, there is no work to date that focuses on constructing an optimal design for 3 or more different objectives with a concrete application. We were also unable to find work that studies robustness properties of multiple-objective optimal designs to model mis-specifications or under a change in the optimality criterion. There are only a handful of related papers that either briefly considered finding an optimal design for a problem with 3 or more objectives or they addressed a different class of problems. For example, El-Monsef and Seyam [25] proposed optimal designs for model discrimination, parameter estimation, and estimation of a function of parameters. The paper combined the optimality criteria for the three objectives using a weighted compound criterion. However, despite the title of their paper, only the last half page outlined how one may construct a specific 3-objective optimal design. There were no practical details, examples and explanation on how to meaningfully select the weights in the combined design criterion for maximizing each efficiency. Another related work is Antognini and Zagoraiou [26], who considered a non dose-response setup and their goal was to determine an adaptive optimal allocation scheme for subjects to treatment groups under 3 criteria in a clinical trial. They wanted to balance the competing needs of ethics requirements, proper randomization and precise treatment efficacy estimation. In such problems, the number of treatment groups is fixed (i.e. the design space consists only of a few points) and so only the optimal proportion or optimal number of subjects to assign to each treatment has to be determined. Our optimization problems have a continuous dose interval and we need to determine the number of optimal dose levels, the optimal dose levels, and the optimal proportions or numbers of subjects to assign at the dose levels. Our constrained optimization problems are thus more difficult because they have more variables to optimize over a continuous multi-dimensional space. Earlier, Zhu and Wong [27] also found multiple-objective optimal designs but their setup was limiting; their interest was confined only to estimating percentiles in a two parameter logistic model and did not discuss different types of criteria, which we have here.

The dearth of work for finding an optimal design for 3 or more objectives in a dose response study with a concrete application can be partially explained by the complexity of the efficiency plot, which increases in dimension as the number of objectives increases. In a high-dimensional efficiency plot, the visual appreciation of the shapes of various efficiency plots becomes compromised and it becomes difficult to identify the correct vector λ∗ to be used in the compound optimality criterion. One may resort to an exhaustive search for the sought multiple-objective optimal design by first generating a complete list of compound optimal designs and then identifying the one that meets the user-specified efficiency requirements for the constrained optimal design. This task is generally laborious and time consuming because the algorithm may require a long time to find all compound optimal designs. Huang and Wong [28] hinted that a sequential method to tackle design problems with 3 or more objectives might work. They suggested to first consider two objectives at a time and determine the dual-objective optimal design. Then sequentially pair the rest of the objectives, two at a time, and compute the dual-objective optimal designs. The multiple-objective optimal design is then determined from the collection of generated dual-objective optimal designs. However they were unclear on how to systematically pair the objectives and work with them sequentially to determine the sought multiple-objective optimal design.

This paper presents a systematic approach to construct a multiple-objective optimal design for the 3 common objectives in a dose response study and the methodology can be directly applied to find other types of multiple-objective optimal designs in other problems. We provide an efficient algorithm for searching the multiple-objective optimal design that meets different user-specified efficiencies for the objectives and for evaluating efficiency of the generated design under various criteria. We also provide a concrete application to a dose response study and study robustness properties of the multiple-objective optimal design to mis-specification in nominal values for the model parameters and under a change of criterion. Bayesian multiple-objective optimal designs can also be found using our approach and we provide an example of such an optimal design with 10 dose levels found from our algorithm.

3 Objectives, models and algorithms for finding optimal designs in dose response studies

We assume that the continuous response variable from the jth subject treated at the ith dose Yij can be modeled by

where f(xi,Θ) is the mean response at dose xi, Θ is the vector of model parameters and σ2 is an unknown positive constant. In practice, each xi is usually expressed as log dose and the dose is selected from a given compact dose interval X, which can be multi-dimensional. Let ξ={(xi,wi)}1K denote the approximate design that takes nwi subjects at xi.

3.2 A common dose response model

We now consider a versatile and popular model commonly used in dose response studies and several other disciplines. The 4-parameter Hill model has a continuous outcome and its mean response is given by

(1)f(Di,a,b,c,d)=c+(d−c)(Dia)b1+(Dia)b.

Here f(Di,a,b,c,d) is the mean response of a continuous outcome at dose Di, a is the ED50, the dose that produces a response mid-way between the upper limit d, and the lower limit c. The parameter b denotes the Hill constant that controls the flexibility in the slope of the response curve.

Khinkis et al. [4] conducted a cell growth inhibition study in a laboratory to investigate the effectiveness of 7 anticancer drugs to shrink the tumor using the 4-parameter Hill model. Nominal parameter values for these drugs used in the study are displayed in Table 1. Figure 1 shows the mean response shapes of (2) for the 7 different sets of values of the parameters in Khinkis et al. [4], suggesting that the characteristics of the drugs are quite different.

Figure 1:

Expected response curves across the dose levels for the 7 drugs using nominal values of the parameters in Table 1 for Model (1).

Table 1:

Nominal parameter values for the 4-parameter logistic model for the 7 drugs.

Drug	Θ	Drug	Θ
TMTX	Θ1=(1.563,1.790,8.442,0.137)	AG2009	Θ5=(1.563,1.030,−4.851,0.137)
MTX	Θ2=(1.563,2.740,10.421,0.137)	AG337	Θ6=(1.563,1.540,1.169,0.137)
AG2034	Θ3=(1.563,0.825,0.653,0.137)	ZD1694	Θ7=(1.563,1.690,5.322,0.137)
AG2032	Θ4=(1.563,3.490,8.930,0.137)

Let Θ=(θ1,θ2,θ3,θ4). Model (1) may be re-parameterized as

(2)f(xi,Θ)=θ11+eθ2xi+θ3+θ4,

which is sometimes referred to as the 4-parameter logistic model. This form was used in Li and Majumdar [31] and is equivalent to the above form with θ1=d−c, θ2=−b, θ3=blog(a), θ4=c, θ1>0,θ2≠0, and −∞<ED50<∞, where xi=log(Di)∈[−M,M] for some sufficiently large value of M. When θ4=0, we have the 3-parameter logistic model or hyperbolic Emax model. The reduced 3-parameter logistic model was used in a Phase II clinical trial for ascertaining asthma indication in Bretz et al. [32]. Fitting these curves can be accomplished using commercial software packages or more specialized software using R-codes provided by Ritz and Streibig [33]. However, in dose-response studies, fitting the curves well is harder because there are usually only a few available doses to explore. This reinforces that selecting the right doses is an important design issue.

The 4-parameter logistic model has an advantage over the Hill model in that it tends to provide more stable parameter estimates [34]. In what is to follow, we use the 4-parameter logistic model (2) and redesign the study in Khinkis et al. [4] using optimal design theory and compare benefits of our locally multiple-objective optimal designs with their locally D-optimal designs.

If the approximate design ξ takes wi proportion of the total subjects at xi,i=1,…,K, a direct calculation shows the Fisher information matrix for model (2) using ξ is

(3)I(ξ;Θ)=nσ2∑i=1Kwig(xi)Tg(xi),

and g(x) is the gradient of the mean function f(x,Θ) evaluated at the nominal values of the parameters:

g(x)=11+eθ2x+θ3, −θ1xeθ2x+θ3(1+eθ2x+θ3)2, −θ1eθ2x+θ3(1+eθ2x+θ3)2,1.

When estimates Θˆ for Θ become available, the asymptotic variance of the estimated response at x is proportional to g(x)I(ξ,Θˆ)−1g(x)T and, as will be shown in the next section, this quantity plays an important role for finding the optimal design.

4 Dose response optimal designs

We now present single-objective optimal designs, multiple-objective optimal designs and investigate robustness properties of the latter designs to mis-specification in nominal parameter values and changes in the objectives. The nominal values in Table 1 were used to construct optimal designs on the dose interval [log(.001),log(1000)]=[−6.91,6.91]. For model (2), Li and Majumdar [31] proved that the locally D-optimal design has 4 dose levels and the design only depends on the nominal values for the parameters θ2 and θ3. Yang [35] generalized these results and showed that up to 4 dose levels are required to optimize the Fisher information matrix for model (2) regardless of the values of Θ. The implication is that if the optimality criterion is a function of the Fisher information matrix, all classical optimal designs for model (2) have at most 4 dose levels. The paper also showed that all classical optimal designs for model (2) do not depend on the parameters θ1 and θ4. Accordingly, in what is to follow, we use the same values of θ1 and θ4 and different values of θ2 and θ3 in Table 1 to construct optimal designs.

All optimal designs in this paper were found based on the Yang-Biedermann-Tang (YBT) algorithm that has been shown to converge to an optimal design for a large class of design problems [36]. The authors also used several examples and showed that their algorithm performed faster than current algorithms for finding single-objective optimal designs, including a traditional class of algorithms such as the V-algorithm. The YBT algorithm requires that the dose range to be discretized. If there are s parameters in the mean function, the algorithm uses a starting design with s+1 dose levels randomly selected from the discretized set of doses in X. At each iteration, the algorithm adds the dose that maximizes the sensitivity function to the current design to form a new design. The weights for the new design are found by optimizing the design criterion over a set of known dose levels using the Newton-Raphson method. The dose levels with zero weights are removed for the next iteration.

We discovered that there was a problem in the YBT algorithm when we applied it to search for multiple-objective optimal designs. If the randomly selected s+1 dose levels were far from the optimal dose levels, the YBT algorithm frequently required a lot more time to find the multiple-objective optimal design and sometimes it failed to do so. We modified the YBT algorithm by having it chose better initial dose levels via the V-algorithm [37]. This modification improved the search speed and generated the multiple-objective optimal design that the original YBT algorithm could not. We offer more details with examples and software implementation in Section 4.2. Our modified algorithm includes a function called MOPT to search and verify the multiple-objective optimal designs in this paper. The function is in an R-package called VNM [38] and the package can be freely downloaded from the R-archive. Interested readers may also write to the first author for the codes.

4.1 Single-objective optimal designs

We recall that a D-optimal design ξD maximizes the determinant of the information matrix I(ξ;Θ) over all designs on the specified dose interval and a c-optimal design provides the most accurate estimate for a user-selected function of the model parameters. For model (2), the ED50 as a function of the model parameters is given by ED50=argx{f(x,Θ)=12(θ1+2θ4)}=−θ3θ2. Let ED^50 be the maximum likelihood estimate of ED50 and let ED50′=(0,θ3θ22,−1θ2,0) be the derivative of ED50 with respect to Θ. The c-optimal design for estimating the ED50 minimizes Var(ED50^) and is given by

(4)ξED50=argminξ{ED50′I(ξ;Θ)−[ED50′]T},

where I(ξ;Θ)− is a generalized inverse of I(ξ;Θ).

Similar to estimating the ED50, the MED as a function of the model parameters in model (2) is given by MED=argx{f(x,Θ)=f(xmin,Θ)+δ}=log−δθ1+δ−θ3θ2 if θ2>0 or logθ1−δδ−θ3θ2 if θ2<0. Here xmin is the minimum dose level and δ is a user-specified clinically significant effect, with δ<0 when θ2>0 or δ>0 when θ2<0. For the given δ, the c-optimal design for estimating the MED minimizes Var(MED^) and is given by

(5)ξMED=argminξ{MED′I(ξ;Θ)−[MED]T},

where

MED′={−1(θ1+δ)θ2,θ3−log(−δθ1+δ)θ22,−1θ2,0,if θ2>01(θ1−δ)θ2,θ3−log(θ1−δδ)θ22,−1θ2,0,if θ2<0.

In the rest of the paper, we assume δ=−1 unless we mention otherwise.

Table 2 displays the single-objective optimal designs found from our modified algorithm. All optimal designs found by the algorithm are verified by an Equivalence Theorem. This is an important tool in optimal design theory that enables one to confirm if a design is optimal among all designs on the given dose interval. It is derived from the Frechet derivative of the convex(or concave) optimality criterion, and while all the Equivalence Theorems have similar forms, each criterion has its own Equivalence Theorem. For example, for D-optimality, the Equivalence Theorem states that if we have a homoscedastic model and the mean response function has s parameters, the design ξD is D-optimal if and only if

d(x,ξD)=g(x)I(ξD;Θ)−1gT(x)−s≤0

for all dose levels x in the dose interval X, with equality when x is a dose level of the design ξD.

Equivalence Theorems for other types of optimal designs are available in design monographs, see for example, Pukelsheim [39] and Atkinson et al. [40]. For example, in order to verify if a design ξ∗ is c-optimal for estimating the ED50 or c-optimal for estimating the MED, one checks whether one of the inequalities below is satisfied for all dose levels x in the dose interval X, with equality when x is a dose level of the design ξ∗:

(g(x)I(ξ∗;Θ)−[ED50′]T)2−ED50′I(ξ∗;Θ)−[ED50′]T≤0

or

(g(x)I(ξ∗;Θ)−[MED′]T)2−MED′I(ξ∗;Θ)−[MED′]T≤0.

In the literature, the function on the left hand side of the inequality is sometimes refereed to as the sensitivity function. Figure 2 shows the plot of the sensitivity function for each of the single-objective optimal designs when Θ1 is assumed as nominal values for Θ. Each plot shows the graph of the sensitivity function is bounded above by 0 with equality at the optimal dose levels and so confirms the optimality of the generated design shown in the first row of Table 2. All our optimal designs reported in this paper have been verified using an Equivalence Theorem.

Figure 2:

Plots of the sensitivity functions of the single-objective optimal designs when Θ1 is the nominal set of values for the model parameters in Model (2).

Table 2:

Single-objective optimal designs for different sets of Θ.

Θ	ξD	ξED50	ξMED
Θ1	−6.91−5.21−4.086.910.2500.2500.2500.250	−6.91−4.806.910.2760.5000.224	−6.91−4.616.910.5000.4540.046
Θ2	−6.91−4.18−3.436.910.2500.2500.2500.250	−6.91−3.806.910.2500.5000.250	−6.91−3.590.5000.500
Θ3	−6.91−2.000.506.910.2500.2500.2500.250	−6.91−0.876.910.2570.5000.243	−6.91−0.296.910.5000.4760.024
Θ4	−6.91−2.86−2.276.910.2500.2500.2500.250	−6.91−2.566.910.2500.5000.250	−6.91−2.390.5000.500
Θ5	−6.913.375.226.910.2500.2500.2500.250	1.435.046.910.1850.5000.315	−1.695.270.5000.500
Θ6	−6.91−1.43−0.086.910.2500.2500.2500.250	−6.91−0.756.910.2500.5000.250	−6.91−0.396.890.5000.4990.001
Θ7	−6.91−3.75−2.526.910.2500.2500.2500.250	−6.91−3.176.910.2530.5000.247	−6.91−2.841.490.5000.4910.009

Table 2 shows the optimal designs for different nominal sets for Θ and they vary depending on the nominal set of values and the study objective. For estimating Θ, the D-optimal designs are always equally supported at 4 dose levels including the lower and upper bounds of the dose interval and the middle two dose levels vary depending on Θ. Unlike the D-optimal designs, the c-optimal designs for estimating the ED50 may or may not include the extreme doses. Additionally, we observe that the c-optimal designs for estimating the MED are much more sensitive to the nominal values for the model parameters than the D-optimal designs and the c-optimal designs for estimating the ED50. Depending on Θ, the c-optimal design for estimating the MED can have 2 or 3 dose levels with different weights. Interestingly, the smallest allowable dose is mostly included in the c-optimal designs for the sets of nominal parameter values considered here. There are two cases (ξED50 and ξMED for Θ5), where the smallest allowable dose is not included in the c-optimal designs; however the lowest dose levels for these two optimal designs can be replaced with the smallest allowable dose without much loss in efficiencies. In practice, designs with more dose levels are desirable because they are likely to have non-singular information matrices and so can estimate all parameters in the model. Such optimal designs may also allow researchers to conduct a lack of fit test to assess model adequacy.

4.2 Multiple-objective optimal designs

We now apply our modified algorithm to search for multiple-objective optimal designs for estimating Θ, the ED50, and the MED simultaneously. Because an optimal design under one objective may not perform well under another, the implemented design must be selected carefully to provide satisfactory efficiencies for the various objectives. A common approach is to use a compound design criterion that combines the 3 optimality criteria using their efficiencies and find a design that maximizes the various efficiencies at the same time. For a model with s parameters in the mean function, recall that the D-efficiency of a design ξ is

EffD(ξ)=|I(ξ;Θ)||I(ξD;Θ)|1s.

The efficiencies of using ξ to estimate the ED50 and MED are, respectively given by

EffED50(ξ)=ED′50I(ξED50;Θ)−[ED′50]TED′50I(ξ;Θ)−[ED′50]T

and

EffMED(ξ)=MED′I(ξMED;Θ)−[MED′]TMED′I(ξ;Θ)−[MED′]T.

The above measures are all between 0 and 1 and have the following interpretation: if ξ has Eff(ξ)=e, then it needs 100(1/e−1)% more subjects to do as well as the optimal design. Multiple-objective optimal design for the objectives can be constructed by finding a design that maximizes the logarithm of a weighted product of the various efficiencies among all designs. Given a user-selected vector of weights λT=(λ1,λ2,λ3), the sought multiple-objective design ξM,λ is

ξM,λ=arg maxξ {λ1log(EffD(ξ))+λ2log(EffED50(ξ))+λ3log(EffMED(ξ))}          =arg maxξ {λ1slog(|I(ξ;Θ)|)−λ2log(Var(ED^50))−λ3log(Var(MED^))}.

Here each λi is non-negative and ∑i=13λi=1. Because a convex combination of concave functionals is also concave, we may directly use directional derivative considerations and show that for a fixed λ, the sensitivity function for the locally multiple-objective criterion is

(6)d(x,ξ)=λ1sg(x)I(ξ;Θ)−1gT(x)+λ2(g(x)I(ξ;Θ)−[ED′50]T)2ED′50I(ξ;Θ)−[ED′50]T+λ3(g(x)I(ξ;Θ)−[MED′]T)2MED′I(ξ;Θ)−[MED′]T−1.

Each summand in (6) has been properly scaled, and it is easy to see that if all but one of the weights is nonzero, (6) reduces to the Equivalence Theorem for the single-objective optimal design. The Equivalence Theorem states that for a given vector λ, the design ξM,λ is the multiple-objective optimal design if and only if for all doses x in the dose range X,

d(x,ξM,λ)≤0,

with equality when x is a dose level of the design ξM,λ.

All multiple-objective optimal designs were obtained using a modified version of the YBT algorithm and confirmed using the above Equivalence Theorem in (6). As noted earlier, without the modification, we were unable to find the multiple-objective optimal designs for some cases. One reason appears to be that the YBT algorithm sometimes begins its search using poor initial dose levels that are far from the optimal dose levels. The modification we made was to first run the V-algorithm r times using the sensitivity function (6) and then select the last s+1 generated dose levels as the initial dose levels for the YBT algorithm. The stopping criterion we used for both algorithms was that the maximum of |d(x,ξM,λt)|<0.001, implying that we were willing to accept ξM,λt, the design generated at the tth iteration as the multiple-objective optimal design if it satisfied the stopping criterion. Our experience suggests that r=10 usually works for finding the multiple-objective optimal design but sometimes it may fail. This seems to happen especially when the weight for the D-optimality criterion is small. For these cases, we suggest using r=30 or r=50 in our modified algorithm to choose better initial dose levels to search for the multiple-objective optimal designs. We provide two examples:

Example 1: We used the set of nominal values Θ1 in Table 1 to find the multiple-objective optimal designs using the YBT and our modified algorithms on the dose interval [log(.001),log(1000)]. The weights for the 3 criteria were λ1=0.05, λ2=0.05 and, λ3=0.90, respectively. When the YBT algorithm was ran, it failed to find the multiple-objective optimal design. For this example, we found |d(x,ξM,λt)| converged to a constant (0.048) as the algorithm iterated without end. When we applied our modified algorithm with r=10, the multiple-objective optimal design was found in 38 s.

Example 2: Miller et al. [29] assumed Θ=(16.8,−1,4.248,22), δ=5 with weights λ1=0.00, λ2=0.10 and, λ3=0.90 to find the multiple-objective optimal designs on the dose interval [log(.001),log(100)]. Again, the YBT algorithm failed to find the multiple-objective optimal design due to the same reasons as in the previous example: |d(x,ξM,λt)| converged to a constant (0.036). When our modified algorithm with r=10 was used, the multiple-objective optimal design was found in 20 s.

Table 3 shows the multiple-objective optimal designs from the YBT algorithm and our modified algorithm for the two examples. Figures 3 and 4 confirm the optimality of the two multiple-objective optimal designs found from our modified algorithm but not the designs found from the YBT algorithm.

Figure 3:

Plots of the sensitivity functions of the generated designs from the YBT and modified algorithms for Example 1.

Figure 4:

Plots of the sensitivity functions of the generated designs from the YBT and modified algorithms for Example 2.

Table 3:

Multiple-objective optimal designs for estimating Θ, ED50 and MED in Examples 1 and 2 found from the two algorithms.

	Example 1
YBT algorithm	−6.91−4.81−4.54−4.213.450.4810.1820.2210.0610.055
Modified algorithm	−6.91−4.71−3.976.900.4810.4130.0550.051
	Example 2
YBT algorithm	−6.912.413.474.600.4600.0840.4290.027
Modified algorithm	−6.912.303.374.600.4580.0740.4410.027

Table 4 shows the multiple-objective optimal designs for the different Θs found from our modified algorithm when we assumed λ1=λ2=λ3=1/3. They are all supported at 4 dose levels and always include the lower and upper bound of the dose interval. The middle two dose levels and the proportions of subjects at the doses of the multiple-objective optimal designs depend on the set of nominal values for Θ. Figure 5 is the plot of the sensitivity function of the multiple-objective design shown in Table 4 when Θ1 is assumed as the nominal values for Θ in model (2). The plot shows that the graph is bounded above by 0 with equality at the optimal dose levels and so confirms the optimality of the generated design.

Figure 5:

Plot of the sensitivity function of the multiple-objective optimal design when Θ=Θ1 for Model (2) and λ1=λ2=λ3.

Table 4:

Multiple-objective optimal designs for estimating Θ, the ED50 and the MED using different sets of nominal values for Θ when λ1=λ2=λ3=1/3.

Θ	ξM,λ	Θ	ξM,λ
Θ1	−6.91−4.89−4.186.910.3440.3230.1620.171	Θ5	−6.912.765.036.910.2710.1180.3990.212
Θ2	−6.91−4.04−3.576.910.3180.1870.3080.187	Θ6	−6.91−1.26−0.436.910.3160.1720.3250.187
Θ3	−6.91−1.520.046.910.3290.2260.2620.183	Θ7	−6.91−3.56−2.836.910.3210.1840.3100.185
Θ4	−6.91−2.79−2.386.910.3160.1830.3130.188

Table 5:

Efficiencies of various optimal designs under different objectives and various nominal values for Θ’s when λ1=λ2=λ3=1/3.

Θ	Design	EffD(ξ)	EffED50(ξ)	EffMED(ξ)	Θ	Design	EffD(ξ)	EffED50(ξ)	EffMED(ξ)
Θ1	ξD	1	0.599	0.511	Θ5	ξD	1	0.583	0.505
	ξED50	⋅	1	0.006		ξED50	⋅	1	0
	ξMED	⋅	0	1		ξMED	⋅	0.070	1
	ξM,λ	0.866	0.815	0.746		ξM,λ	0.866	0.786	0.696
Θ2	ξD	1	0.602	0.480	Θ6	ξD	1	0.596	0.474
	ξED50	⋅	1	0.001		ξED50	⋅	1	0.002
	ξMED	⋅	0	1		ξMED	⋅	0	1
	ξM,λ	0.894	0.762	0.645		ξM,λ	0.895	0.761	0.642
Θ3	ξD	1	0.595	0.493	Θ7	ξD	1	0.595	0.470
	ξED50	⋅	1	0		ξED50	⋅	1	0.002
	ξMED	⋅	0.161	1		ξMED	⋅	0.063	1
	ξM,λ	0.880	0.782	0.694		ξM,λ	0.891	0.766	0.657
Θ4	ξD	1	0.682	0.478
	ξED50	⋅	1	0
	ξMED	⋅	0.069	1
	ξM,λ	0.895	0.761	0.650

^[1]

How does a multiple-objective optimal design perform under a single-objective criterion? Table 5 displays these efficiencies and also shows how well single-objective optimal designs perform under a variation of criterion. As expected, single-objective optimal designs have efficiency 1 under their own criterion but do not perform well under different objectives. On the other hand, the multiple-objective optimal design ξM,λ with λ1=λ2=λ3=1/3, performs quite well for estimating Θ and the ED50 for the various sets of nominal parameter values. It provides lower efficiencies for estimating the MED but still clearly outperforms the other two single-objective optimal designs. We notice that the efficiencies of ξM,λ for the 3 objectives are not equal even though we had set λ1=λ2=λ3=1/3 for the 3 objectives. This is often the case suggesting that careful choice of the set for λ is important to capture the different efficiency requirements for each objective.

For multiple-objective optimal designs, the components in the weight vector λ represent the relative importance of each criterion but they can be rarely preselected to produce the targeted efficiencies, other than a sense that a larger weight for one objective should result in a higher efficiency under that objective. This means that determining in advance the correct weight vector λ to use in the compound criterion to find the multiple-objective optimal designs can be problematic. In particular, it is often difficult to predict how a change in the vector λ will translate to a change in the corresponding efficiencies, and this issue is frequently overlooked in such work. As an illustration, consider finding a dual-objective optimal design for estimating Θ and the ED50 in model (2) using the nominal set Θ2. In this case, λ is a scalar and the sought dual-objective optimal design ξλ,dual is

ξλ,dual=argmaxξλlog(EffD(ξ))+(1−λ)log(EffED50(ξ)).

Figure 6 shows the efficiency plots and visualize the effect of the choice of λ on the two efficiencies of the dual-objective optimal design. The weight λ represents the importance of estimating the model parameters relative to estimating the ED50. If D-optimality is more important, λ should be large and EffD(ξλ,dual) should be larger than EffED50(ξλ,dual). Table 6 lists selected values of λ and shows when λ=0.55, EffD(ξ0.55,dual)=0.912. When λ increases from 0.55 to 0.90, EffD(ξ0.90,dual)=0.995, implying that an increase of 64% in λ brings an increase of only 9% in D-efficiency. The figure also suggests that setting λ=0.45 rather than 0.5 maximizes both efficiencies equally.

Figure 6:

Efficiency plots of the dual-objective optimal designs when Θ=Θ2 in Model (2).

Table 6:

Dual-objective optimal designs for different values of λ.

λ	ξdual	EffD	EffED50
0.55	−6.91−4.05−3.596.910.2500.2470.2530.250	0.912	0.825
0.90	−6.91−4.13−3.466.910.2500.2510.2490.250	0.995	0.653
0.95	−6.91−4.15−3.436.910.2500.2510.2490.250	0.999	0.621
1.00	−6.91−4.18−3.436.910.2500.2500.2500.250	1.000	0.596

For 3 or more-objectives, it is harder to visualize the changes in the efficiencies of the generated design as the weights vary in the compound criterion from the high-dimensional efficiency plot. To this end, we generated multiple-objective optimal designs for each possible pair of values for λ1 and λ2 using a grid density 0.05, and evaluated the efficiencies of the resulting compound optimal designs under each of the 3 criteria. Figure 7 shows how the 3 efficiencies vary as the different values of the weights in λ change when Θ=Θ1. The figure shows that the multiple-objective optimal design provides equal efficiencies for the 3 objectives when λ1=0.20,λ2=0.35, and λ3=0.45, and it has EffD≈EffED50≈EffMED≈0.800.

Figure 7:

Efficiency plots of the multiple-objective optimal designs when Θ=Θ1 in Model (2).

In practice, practitioners first prioritize the importance of each objective and set efficiency requirements for each objective with higher efficiencies for the more important ones. For example, suppose Θ1 is the set of nominal parameter values and we want to find an multiple-objective optimal design that maximizes EffD subject to constraints that EffED50≥0.80 and EffMED≥0.70. Figure 7 shows such an optimal design exists and is given by the plot for λ1=0.40. By visual inspection, the required weights are λ1=0.40, λ2=0.40 and λ3=0.20, and the resulting D-efficiency is EffD=0.879 under the constraints that EffED50=0.838≥0.80 and EffMED=0.702≥0.70.