A Note on the Use of Optimal Control on a Discrete Time Model of Influenza Dynamics

A discrete time

1. Introduction.In April of 2009, the World Health Organization (WHO) announced the emergence of a novel A(H1N1) influenza strain [11].National and international public health agencies quickly took (often drastic) emergency measures and in June of 2009, the WHO and the US Centers for Disease Control (CDC) declared the outbreak to be a pandemic.
For last few decades, mathematical epidemiological models have been developed to understand the dynamics of infectious diseases [2,5,17].In particular, continuous time models have been used to study influenza outbreaks and their controls policies [3,12,13,16,28,30,33].Optimal control models have been provided to evaluate the usefulness of the policies put in place in many applications [4,21,23,29].The 184 P. A. GONZ ÁLEZ-PARRA, S. LEE, L. VEL ÁZQUEZ AND C. CASTILLO-CHAVEZ identification of optimal control strategies involving antiviral treatment and the isolation of infectious individuals have been studied using continuous time models [22,32].Recently, more attention have been focused on discrete epidemiological models [1,6,9,10,15,34].A discrete model is more convenient to compare data, which are collected in discrete time, with the output of a discrete model.
In this note, we proceed to identify optimal control policies aimed at minimizing the number of infected and dead individuals via the use of the most "cost-effective" policies involving social distancing and antiviral treatment within a discrete time epidemic framework that it is not just a discretization of a continuous-time model [6].Discrete optimal control theory is the main approach used in our analysis [8,14,18,19,20,24,31].The model's basic reproductive number is computed as well as final epidemic size relations (with and without controls).Only the implementation of intervention measures referred to as social distancing and antiviral treatment are explored in this note.Numerical simulations highlight the differences that result from the implementation of single versus dual intervention policies.
2. Discrete SAITR model.The total population under consideration is divided into susceptible (S), asymptomatic (A), infectious (I), treated (T ), recovered (R), and dead (D) (from influenza) classes of individuals.Births and deaths from natural causes are ignored since the focus is on single outbreaks.Treatment and social distancing are the only control policies explored in this note.The fraction of susceptible individuals at time t that remain susceptible at time t + 1 is modeled by the function where N t denotes the total population, β is the transmission rate, and m and ρ (0 < m, ρ ≤ 1) are the reductions in transmissibility for the asymptomatic and treated classes, respectively.The social distancing control function, x t , models the reduction in the number of contacts per unit of time ("generation").It is assumed that the social distancing control reduces "equally" the role of each infectious class on the transmission process.The fraction of individuals who get the disease but do not develop symptoms is given by q while δ denotes the proportion of diseaseinduced deaths per generation; the fraction of infected individuals who get treatment each generation is modeled by the antiviral treatment control function, τ t .It is assumed that asymptomatic and infectious individuals naturally recover with probability σ 1 (per generation) while treated individuals recover with probability σ 2 (per generation).The model (with two controls) is given by the following system of nonlinear discrete equations: In the absence of controls (x t ≡ 0 and τ t ≡ 0, for all t), the final size relationship is given by ln following the approach in [6] (details are presented in Appendix A).The basic reproductive number R 0 in this case is R 0 is a dimensionless quantity that accounts for the number of (initial) secondary cases generated by two classes: the infected (I) and the asymptomatic (A) in a population of primary susceptible individuals.The addition of controls replaces the expression in (2) by the inequality The following result is easily established.Result 2.1: If S ∞ is a solution of (2) and S wc ∞ satisfy the inequality (4) then S wc ∞ ≥ S ∞ , that is, the use of controls reduces the final epidemic size (an outline of the proof is found in Appendix A).
We observe that the fraction of the population that becomes infected during the course of a single epidemic outbreak in the absence of controls is 1 − S∞ N while the corresponding fraction with controls is denoted as 1 − S wc ∞ N .The final epidemic size decreases as a result of the implementation of social distancing measures or the application of antiviral treatment control measures.Numerical simulations that compute the final epidemic sizes with and without controls are used to corroborate Result 2.1.

3.
Optimal control problem.Our goal is to minimize the number of infected and dead individuals via the judicious (cost effective) use of social distancing and antiviral treatment measures over a finite interval [0, T f ].The objective functional F used to formulate the appropriate optimization problem is given by where with x = (x 0 , x 1 , ..., x T f −1 ) and τ = (τ 0 , τ 1 , ..., τ T f −1 ), the control variables.Further y t = (S t , A t , I t , T t , R t , D t ) T is the state variable with t = 0, 1, ..., T f .The weight constants B i , (i = 0, 1, 2, 3) are a measure of the relative cost of interventions over [0, T f ].In particular, B 2 and B 3 , are the relative costs associated with the implementation of social distancing and antiviral treatment, respectively.The use of these definitions and notations lead to the problem of finding control functions x and τ such that where U = {(x t , τ t ) : 0 ≤ x t ≤ x max , 0 ≤ τ t ≤ τ max , t = 0, 1, ..T f − 1}, subject to the state equations in Model (1) and appropriate initial conditions.Three different strategies are compared: • Strategy 3: Social distancing and antiviral treatment.
The relative impact of these strategies is "evaluated" from their effect on the final size relations under single or dual policies.It is worth observing again that, since there is no data on real costs, the model and the optimization process are based on "perceived" relative costs.Therefore, all simulation results are compared in terms of relative costs.The optimization problem is solved by using a discrete version of Pontryagin's maximum principle [18,19,23,31] (details are provided in Appendix B).The Hamiltonian associated with the problem is given by where x t , τ t are the control variables, and y t and λ t ∈ R 6 are the state and adjoint variables, respectively.The adjoint equations are where λ i t and y i t are the i-th component of λ t and y t , respectively.Finally, the optimality conditions are solved ( ∂Ht ∂xt = 0 and ∂Ht ∂τt = 0).The procedure to find optimal solutions is summarized in the following algorithm: Algorithm 1 Forward-Backward Algorithm 1: Initial guess for x 0 , τ 0 and condition y 0 are selected.4. Numerical results.The results of selected simulations generated by the numerical implementation of the strategies described in Section 3 are discussed in this section.The numbers of infected individuals generated under two scenarios (low to moderate R 0 values (1.3 -1.8) or high R 0 values (2.4 -3.2) in the absence of controls or in the presence of single or dual optimal controls) are compared.A sensitivity analysis is carried out on the robustness of these simulations in relation to the values of a priori selected constraints on the ranges of the bounds on the controls x t and τ t (x t ∈ [0, 0.2], τ t ∈ [0, 0.05]).The control upper bounds are interpreted as the maximum daily rates and we assume a smaller rate for the treatment control (per day) due to limitation of resources than the one for the social distancing control.The weight constants, B 2 and B 3 , are the relative costs associated with the implementation of social distancing and antiviral treatment, respectively.The weight constants are selected in part to facilitate computational issues.We notice that the numerical approach used to solve this discrete optimal control problem is sensitive to the weight constants on the controls.The sensitivity arises in part from the fact that all parameters in our discrete model represent daily rates, i.e. the time step is one day.Hence, we use small ranges for the weight constants in order to guarantee convergence to the optimal solutions (B 2 ∈ [0.002, 0.2] and B 3 ∈ [0.004, 0.4]).For most our simulations, we choose B 3 = 0.004 (treatment as the base line value) and B 2 (social distancing) which is assumed to be approximately ten times larger than ).We carried out simulations under the assumption that the costs associated with social distancing are higher than those associated with treatment.
As part of our sensitivity analysis, we found that the model results under two cases (B 2 > B 3 or B 2 < B 3 ) are perhaps robust to changes in the ordering of two weight constants (B 2 , B 3 ).The final time, 200 days, is chosen for all simulations.The baseline parameter values are given in Table 1.
4.1.Implication of social distancing and antiviral treatment.We compare the reduction in the final size and the proportion of daily infected cases that result from the implementation of Strategies 1 -3.In these simulations, the weight constants on two controls are B 3 = 0.004 and B 2 ∼ = 10B 3 .Results under two different values for R 0 , a low value (R 0 = 1.3) and a high value (R 0 = 2.4) are displayed in Figures 1 and 2, respectively.Figures 1A and 2A show the optimal control solution as a function of time for each strategy.Figures 1 (B-C) and 2 (B-C) compare the impact of each strategy on the cumulative proportion of infected individuals and the proportion of daily infected cases.
In Figures 1A and 2A, when R 0 is low (R 0 = 1.3), the treatment control must be implemented at the maximum rate while the moderate use of social distancing control is applied.However, when R 0 is high (R 0 = 2.4), the role of social distancing control becomes more important (the maximum use of the social distancing control must be implemented).In Figure 1B, dual policies generate strong reductions of almost 57% in the final epidemic size as well as 17% reduction under Strategy 1, and 50% under Strategy 2. The epidemic peak is reduced by more than 50% when dual policies are applied (Figure 1C). Figure 2 plots optimal controls and the resulting cumulative proportions of infected cases when R 0 = 2.4.The optimal control solution requires the application of the maximum effort during the first 50 and 75 days for Strategies 1 and 2. In the case of Strategy 3, the maximum effort for social distancing and treatment should be applied during the first 50 and 100 days respectively.The curves in Figure 2B show that the epidemic ends within 100 days of the start of the outbreak.Strategy 3 yields the largest reduction of 22% in the final epidemic size, but there is only a 9% and a 13% reductions under Strategies 1 and 2, respectively.
The impact of optimal strategies in terms of the final size as a function of R 0 is presented in Figure 3.Under a single policy, for small R 0 (up to ∼ 1.45), Strategy 2 is more effective in reducing the final epidemic size when compared to Strategy 1.However, the effect is reversed when R 0 ≥ 1.45.A dual policy has the strongest impact in the reduction of the final epidemic size for all values of R 0 13,25]).Under higher values of R 0 (R 0 ≥ 3), no policy seems significantly effective whenever the goal is to reduce the final epidemic size.

4.2.
Effects of weight constants.The role of weight constants is explored by assessing their quantitative impact in terms of the number of infected cases.The weight constants correspond to the relative costs of the effort carried out in the implementation of the optimal strategies.We study the impact of three different values for B 2 and B 3 on Strategies 1 (social distancing) and 2 (antiviral treatment), respectively.We set R 0 = 1.8, x max = 0.2, and τ max = 0.05.Figures 4 and 5 show the optimal control solutions computed under Strategies 1 and 2 as well as their impact on the cumulative proportion of infected individuals and daily infected cases.
Figure 4A shows that the optimal control solution requires the maximum value allowed for social distancing within the first 50 days of the epidemic when the weight constant is the smallest (B 2 = 0.002, relatively cheap).This high value for the social distancing control yields the highest reduction of 20% in the final epidemic size (Figure 4B).However, as the weight constant B 2 is increased to 0.2 (relatively expensive), the reduction in the final epidemic size decreases to 7.5%.Figure 4C shows the possibility of a longer delay in the peak of the epidemic as the value of the weight constant B 2 decreases.In the case when Strategy 2 is applied, similar results are obtained.As the weight constant increases, reductions in the final epidemic size are observed.For example, when B 3 = 0.004, the reduction in the final epidemic size is 25% but when B 3 = 0.4 the reduction is only 10% (Figure 5B).
The cumulative proportion of infected cases without controls together with those generated under Strategies 1 and 2, respectively, are plotted in Figures 6A and  6B. Figure 6A plots the results under Strategy 1 for three different values of the weight constant B 2 .Small values of B 2 = 0.002 result in a higher reduction in the final epidemic size for all ranges of R 0 .Figure 6B shows that for large values of R 0 (R 0 ≥ 2.5), reductions in the final epidemic size are not significantly different (B 3 = .004and B 3 = .04),as a consequence of the a priori limitations placed on the control upper bounds.

4.3.
The effect of upper bounds on the optimal control.We study the implementation of Strategies 1 and 2 under limited resources as a result of changes in the values of the controls upper bounds: x max ∈ [0.07, 0.2] and τ max ∈ [0.007, 0.05], respectively.We fix a moderate value of R 0 = 1.8 and set the values of the weight constants at B 3 = 0.004 and B 2 = 10B 3 .Figures 7A and 8A show the optimal control solution under Strategy 1 and 2, respectively.Figures 7(B-C) and 8(B-C) illustrates the effect of controls on the reduction of the final epidemic size and proportion of daily infected cases.Figure 7B shows that when the upper bound is relatively small (x max = 0.07), the reduction in the final epidemic size is less than 7% but when the upper bound is large (x max = 0.20) a reduction of 17% in the final epidemic size can be achieved.
Figure 7C demonstrates that a reduction of 50% or more in the epidemic peak can be achieved when the upper bound increases to the maximum value x max = 0.2.A similar behavior is observed when the a priori upper bound in the control linked to Strategy 2 is varied.Figure 8 shows that a maximum upper bound of τ max = 0.05 gives 22% reduction in the final epidemic size.In fact, when the upper bound is relatively small (τ max = 0.07), the final epidemic size is reduced but only by 4%.

5.
Conclusions.A discrete model is introduced in order to study single epidemic outbreaks in the context of influenza.The use of single and dual strategies (social distancing and antiviral treatment) results in reductions in the cumulative number of infected individuals.We have seen that dual strategies are more efficient at reducing the final epidemic size than single policies.
Our results show that under the implementation of a single policy, the social distancing strategy (Strategy 1) is more effective than the antiviral treatment strategy (Strategy 2) when R 0 > 1.5.Dual policies are always most effective and in this respect, our findings are consistent with those obtained recently using continuous time models [22].The simulation of the model with two controls (dual policy) showed that reduction in the prevalence of the diseases were more sensitive to changes in social distancing than in antiviral treatment.In the application of every policy, we find that the level of the control effort is higher at the beginning of the epidemic.Furthermore, for extremely high values of R 0 , R 0 ≥ 2.4, even under the implementation of maximum effort, it is observed that the selected policies do not make a significant difference.
Recent studies show that the 2009 influenza pandemic had stronger economical impact in Mexico [7,26].Unfortunately estimating the real costs associated with the selected policies (interventions measures) is difficult even in the context of the simple model in this note.Therefore, we have focused on the use of relative costs.However, even after we have chosen the interventions strategies (policies) ranking the relative costs is often a matter of debate.In addition we did not include the impact of time delays which arise from a multitude of factors including those tied in to the development or implementation of intervention (resource-limited policies).The 2009 influenza pandemic demonstrated that such delays could be critically important.Fortunately, the expected negative impact of these delays never materialized due to the relatively low severity associated with this novel influenza A(H1N1) strain.
Appendix A: Final epidemic size.Let us consider the model without control, then τ t = 0, x t = 0, and G t = e −β I t +mA t N hence Model (1) becomes to be Then from the first equation in (10) we get the previous equation becomes From the second equation in Model (1) we have and after some rearrangement of terms, we obtain Summing over k and let k → ∞ we get that But A 0 = 0 and S 0 ≈ N , therefore By adding equations S, A and I in Model (1) we obtain Rearrange the terms of the above equation to get and summing over k and let k → ∞ we have Substituting ( 12) into (13) yields Therefore, Substituting ( 12) and ( 14) into (11) gives hence we get the final size relation, ln which can be written as where the basic reproductive number is given by The proof of Result 2.1: The result can be verified easily for R 0 < 1.Let us consider R 0 > 1; f is a decreasing function when x < N R0 and moreover, N R0 is the unique critical point of f .Since f (N ) = 0, we have S ∞ < N R0 < N and f (S ∞ ) < 0. Hence, f < 0 if and only if S ∞ < x < N .By hypothesis f (S wc ∞ ) < 0, then we obtain S ∞ < S wc ∞ .
Appendix B. We focus on Strategy 3, that is, when both social distancing and antiviral treatment controls are implemented (dual policy).Pontryagin's Maximum Principle can be extended to discrete systems and the necessary conditions are obtained in a similar manner as the continuous models [19,23,29,31].The existence of optimal solutions are guaranteed because of the convexity of control functions in the objective functional and the regularity in the discrete system.The Hamiltonian is defined as , where The corresponding adjoint equations are (λ i t = ∂Ht where L t+1 = −λ 1 t+1 + qλ 2 t+1 + (1 − q)λ 3 t+1 with transversality conditions λ i (T f ) = 0, for i = 1, 2, ...6.
The optimality conditions are given by ∂Ht ∂xt = 0, and ∂Ht ∂τt = 0: and where G t is given by ( 17) and We find a solution of the nonlinear equation ( 20) by using Newton method [18,27].The value of the weight constant B2, which corresponds to the cost on social distancing is varied.For a small value of B2, the optimal solution allows the implementation of high values of social distancing that provides a high reduction in the final epidemic size (20%).For a large value of B2 = 0.2, smaller values of social distancing control must be implemented resulting in a non-significant reduction in the final epidemic size (7.5%).
Figure 5. Strategy 2: By increasing the cost on antiviral treatment, B3, the optimal solution permits the application of smaller value for treatment.We obtain an increase in the cumulative proportion and the proportion of daily infected cases.In contrast, when the cost is moderate, the optimal solution allows the implementation of the maximum permitted value of treatment with a strong impact in the reduction of the final epidemic size (22%).When the cost of social distancing is low, there is a significant reduction in the final epidemic size for every value of R0.However, when the cost is high (B2 = .2),there is a very small reduction in the final epidemic size (A).For treatment, under two values of B3, (B3 = .004and B3 = .04),there is not a considerable difference in the final epidemic size when R0 > 2.5 (B).

Figure 1 .
Figure1.For a low value of R0 = 1.3, the social distancing optimal control solution does not require the application of the allowable maximum values.For treatment, the optimal solution requires the implementation of the highest allowable values under each strategy.Dual policy (Strategy 3) has a significant reduction of almost 57% in the final epidemic size.

Figure 2 .
Figure2.For R0 = 2.4, the optimal solution requires the implementation of the highest permitted values for each control strategy.Strategy 3 causes a reduction of less than 22%.Even when there is a maximum control implementation, the effort is not enough and the reduction in the final epidemic size is less significant compared to the results obtained for low values of R0.

Figure 3 .
Figure 3.The comparison of final epidemic size vs.R0 under three strategies.The results show that Strategy 3 yields the highest reduction of the final epidemic size.However, for single policies Strategy 1 has more impact in the reduction of the final epidemic size than Strategy 2.

Figure 4 .
Figure 4. Strategy 1:The value of the weight constant B2, which corresponds to the cost on social distancing is varied.For a small value of B2, the optimal solution allows the implementation of high values of social distancing that provides a high reduction in the final epidemic size (20%).For a large value of B2 = 0.2, smaller values of social distancing control must be implemented resulting in a non-significant reduction in the final epidemic size (7.5%).

Figure 6 .
Figure 6.The final epidemic size vs.R0 for Strategies 1 and 2 by changing the weight constants for social distancing and antiviral treatment.When the cost of social distancing is low, there is a significant reduction in the final epidemic size for every value of R0.However, when the cost is high (B2 = .2),there is a very small reduction in the final epidemic size (A).For treatment, under two values of B3, (B3 = .004and B3 = .04),there is not a considerable difference in the final epidemic size when R0 > 2.5 (B).

Figure 7 .
Figure 7.The reduction of the final epidemic size is small (7%) with a small upper bound (xmax = 0.07) while we observe a stronger impact in the reduction of the final epidemic size (17%) with a larger upper bound (xmax = 0.2).

Figure 8 .
Figure 8.The impact of the control in Strategy 2 is reduced with a small upper bound, τmax = 0.007 (4% reduction of the final epidemic size).In the case of a lager upper bound τmax = 0.05, the final epidemic size is reduced by 22%.

Table 1 .
Definition of parameters and baseline values.