Modelling Optimal Control of In-Host HIV Dynamics Using Different Control Strategies

HIV is one of the major causes of deaths, especially in Sub-Saharan Africa. In this paper, an in vivo deterministic model of differential equations is presented and analyzed for HIV dynamics. Optimal control theory is applied to investigate the key roles played by the various HIV treatment strategies. In particular, we establish the optimal strategies for controlling the infection using three treatment regimes as the system control variables. We have applied Pontryagin's Maximum Principle in characterizing the optimality control, which then has been solved numerically by applying the Runge-Kutta forth-order scheme. The numerical results indicate that an optimal controlled treatment strategy would ensure significant reduction in viral load and also in HIV transmission. It is also evident from the results that protease inhibitor plays a key role in virus suppression; this is not to underscore the benefits accrued when all the three drug regimes are used in combination.


Introduction
There is an ever-changing need for new and useful treatment regimes that will provide assistance and relief in all aspects of the human condition. Subsequently, many researchers have embarked on the journey of analyzing the dynamics of various diseases affecting mankind with the aim of improving control and effect and finally eradicating the diseases from the population. Modelling and numerical simulations of the infectious diseases have been used as tools to optimize disease control. This is due to the fact that medical community has insufficient animal models for testing efficacy of drug regimes used in controlling infections. Human immunodeficiency virus (HIV) is one of the major problems that researchers have been working on for over three decades. According to the Joint United Nations Programme on HIV and AIDS (UNAIDS), there were 36.7 million people living with HIV/AIDS in 2016, 1.6 million of which live in Kenya [1]. Nonetheless, many treatment regimes for HIV have been approved by the US Food and Drug Administration. Highly Active Antiretroviral Therapy (HAART) is the latest combination in use for HIV treatment in most countries. HAART has been proven to be highly effective in viral suppression, prolongs life of the infected person, and also reduces the rate of HIV transmission. However, even over three decades since the first HIV cases were reported, the virus had no cure and hence various control methods for HIV/AIDS have been recommended. These controls range from preventive measures to treatment regimes. Preventive measures aim at reducing the number of new HIV infections, while treatment regimes target the already infected persons to increase their life expectancy and reduce the rate of HIV transmission. Various treatment strategies are still the subject of many ongoing clinical trials that are investigating their benefits versus risks aimed at determining the most optimal treatment for HIV. Unfortunately, various host-pathogen interaction mechanisms during HIV infection and progression to AIDS are still unknown. Consequently, many questions like which is the best combination, when is the best time to start treatment, and how the treatment should be administered are yet to be answered fully.
Mathematical modelling is one of the many important tools used in understanding the dynamics of disease transmission. It is also used in developing guidelines important in disease control. In HIV, mathematical models have provided a framework for understanding the viral dynamics and have been used in the optimal allocation of the various interventions against the HIV virions [2][3][4]. A fundamental goal of 2 Computational and Mathematical Methods in Medicine developing and applying the aforementioned mathematical models of HIV is to influence treatment decisions and construct better treatment protocols for infected patients. Most of the modern mathematical models that have been developed apply the optimal control theory. Optimal control theory is a branch of mathematics developed to find optimal ways of controlling a dynamical system [5]. It has been applied by mathematicians to assist in the analysis of how to control the spread of infectious diseases. The results are used in making key decisions that involve complex biological mechanism. In particular, it is used to determine the best dosage for various available vaccines or treatment in use for controlling infection. For instance, Gaff and Schaefer [6] applied optimal theory in evaluating mitigation strategy that would be highly effective in minimizing the number of people who get infected by an infection. The study applied both vaccinations and treatment as control variables for their various model. The results indicated that as much as treatment is paramount in controlling any infection, the most optimal method would be the combination of the two interventions. Furthermore, Bakare et al. [7] applied optimal control in an SIR model. The study illustrated the use of optimal control theory in establishing the optimal educational campaign and treatment strategies that would minimize the population of the infected persons as well as cutting the cost of controlling the various diseases. The results indicated that, for controlling infection, it is important to target the uninfected populations and apply measures that will prevent them from getting the infection.
In the literature, optimal control theory has been applied in the analysis of in-host HIV dynamics as well as in population-based HIV models. For instance, Yusuf and Benyah [14] applied optimal theory on HIV population model. The study aimed at determining the best method of controlling the spread of HIV/AIDS within a specified time frame. The study considered three control variables, that is, safe sex, education, and ARTs. The numerical results of the objective function for the model indicated that safe sex practice and early initiation of ARTs are the most optimal ways of mitigating the spread of HIV/AIDS. The study established that if the aforementioned strategies are well implemented, this would lead to an HIV-free nation in 10 years. In addition, for in-host model, optimal control theory has been applied in the search for optimal therapies for HIV infection.
Drugs such as fusion inhibitors (FIs), reverse transcriptase inhibitors (RTIs), and protease inhibitors (PIs) have been developed and applied in the various optimal control problems. Srivastava et al. [15] analyzed an initial infection model with reverse transcriptase inhibitors (RTIs). The study argued that, through the use of RTIs, an infected cell reverts back to susceptibility. However, this is unlikely since once a CD4 + Tcell is infected, it cannot recover. The only possible way is for it to remain latently infected but fail to produce infectious virus, since RTIs inhibit the reverse transcription process. Hattaf and Yousfi [16] analyzed two optimal treatments of HIV infection model. The study aimed at measuring the efficiency of RTIs and PIs. This was done by maximizing objective function aimed at increasing the number of the uninfected cells, decreasing the viral load, and minimizing the treatment cost. The results indicated that use of therapy is important in HIV control. It is also important to note that the study included two types of viruses, that is, the infectious virus and the noninfectious virus. Noninfectious virus is due to the use of PIs as a treatment regime.
Karrakchou et al. [17] applied optimal control theory on HIV. Like Hattaf and Yousfi [16], the study applied the two control strategies, that is, RTIs and the PIs. However, the study failed to put into account both the latently infected cells and the noninfectious virus that results due to the use of RTIs and PIs, respectively. Failure to include such important variables in the model underscores the adequacy of the model in representing the actual HIV in-host mechanism. In addition, Arruda et al. [12] applied optimal control theory in HIV immunology. The study used two control variables in fighting HIV with the inclusion of the CD8 + T-cells. However, the study has some shortcomings; for instance, the study suggested that activated CD8 + T-cells kill the HIV virions and also the infected cells. This is not the scenario, since the activated CD8 + T-cells are only able to kill infected CD4 + Tcells which in turn reduce the population of the HIV virions. Unfortunately, even with the aforementioned work done on HIV, the implementation of some of the recommendations has been proven to be inefficient and in most cases not economically viable, especially to the developing countries.
As per the literature cited, it is clear that as much as ARTs have been used for viral suppression, the optimal treatment schedule necessary to maintain low viral load is always an approximation. Until the time when HIV cure is found, physicians will try as much as possible to apply the control strategy that will inhibit viral progression while simultaneously holding the side effects of treatment to a minimum. Most of the treatment regimes have many side effects that must be maintained at a low level. For example, long-term use of protease inhibitors is associated with insulin intolerance, cholesterol elevation, and the redistribution of body fat. Therefore, there is a need to establish the optimal treatment strategy, that is, the one which both maximizes the patient's uninfected CD4 + T-cells and minimizes the harmful side effects due to the drugs.
This study has addressed some of the shortcomings noted from the in-host HIV dynamics models by applying three control variables representing the three drug regimes on the market, that is, the fusion inhibitor, reverse transcriptase inhibitors, and the protease inhibitors, in the in vivo HIV model. In addition, the study has incorporated the CD8 + T-cells in the model. For the analysis, the study will apply optimal control theory together with Pontryagin's Maximum Principle in solving the objective function with the aim of establishing the optimal treatment strategy.

Model Description.
In order for us to carry out optimal control processes, it is paramount to formulate a model that describes the basic interaction between the HIV virions and the body immune system. We develop a mathematical model The concentration of the noninfected CD4 + T-cells per cubic millimetre at any time ( ) The concentration of the infected CD4 + T-cells per cubic millimetre at any time The concentration of latently infected CD4 + T-cells per cubic millimetre at any time ( ) The concentration of HIV virions, copies/mL, at any time ( ) The concentration of the immature noninfectious virions, copies/mL, at any time ( ) The concentration of the CD8 + T-cells per cubic millimetre at any time ( ) The concentration of the activated CD8 + T-cells per cubic millimetre at any time Parameter Description The rate at which the noninfected CD4 + T-cells are produced per unit time.
The rate at which the noninfected CD4 + T-cells decay. The rate at which the CD4 + T-cells are infected by the virus.
The death rate of the infected CD4 + T-cells. The death rate of the latently infected CD4 + T-cells. The rate in which HIV virions are generated from the infected CD4 + T-cells.
The death rate of the infectious virus. The death rate of the noninfectious virions. The rate at which the infected cells are eliminated by the activated CD8 + T-cells.
The rate at which the CD8 + T-cells are produced per unit time.
The death rate of the CD8 + T-cells. The rate at which the CD8 + T-cells are activated by the presence of the virus and the infected CD4 + T-cells.
The rate at which the activated defense cells decay. The parameters for the model are as follows. The susceptible CD4 + T-cells are produced from the thymus at a constant rate , die at a constant per capita rate , and become infected by the HIV virions at the rate . However, due to the use of fusion inhibitor ( 1 ) which prevents the entry of the HIV virions into the CD4 + T-cells, a fraction 1 reverts back to susceptible class. In addition, when the infected CD4 + T-cells are exposed to the HIV virions in presence of reverse transcriptase inhibitor ( 2 ), the HIV virions RNA may not be reverse-transcribed. This results in a proportion 2 of the infected cells becoming latently infected. The infected cells are killed by the CD8 + T-cells at the rate and they die naturally at the rate , whereas latently infected cells die at the rate . This study assumes that the latently infected cells will die naturally and have no possibility of producing infectious virions nor becoming activated to become infectious. However, if the protease inhibitor ( 3 ) is used as a treatment strategy, it inhibits the production of protease enzyme, which is necessary for production of mature HIV virions. This therefore means that we have two kinds of HIV virions produced from infected CD4 + T-cells, that is, the infectious HIV virions and the immature noninfectious virions. The infectious HIV virions are produced at the rate (1 − 3 ) and die at the rate , while the noninfectious HIV virions are produced at the rate 3 and die at the rate . Furthermore, the CD8 + T-cells are produced naturally from the thymus at the rate , they die naturally at the rate , and they can also be activated to kill the infected cells at the rate . The activated CD8 + T-cells die naturally at the rate . It is very important to point out that the CD8 + T-cells are activated to kill the infected CD4 + T-cells and not the virus as suggested by Arruda et al. [12].
The summary for the model description is given as follows. The variables, parameters, and the control variables for the in-host model are described in Tables 1, 2, and 3, respectively.
From Figure 1 and the description above, we derive the following system of ordinary differential equations to describe the in vivo dynamics of HIV: Figure 1: A compartmental representation of the in vivo HIV dynamics with therapy.

Control variable
Description Purpose Are a class of antiretroviral drugs that work on the outside of the host CD4 + T-cell to prevent HIV from fusing with and infecting it.
Reverse transcriptase inhibitors Are a class of antiretroviral drugs used to treat HIV infection by inhibiting the reverse transcription process.
Are a class of antiviral drugs that are widely used to treat HIV/AIDS by inhibiting the production of protease enzyme necessary for the production of infectious viral particles.

Optimization Process
Control efforts are carried out to limit the spread of the disease and, in some cases, to prevent the emergence of drug resistance. Optimal control theory is a method that has been widely used to solve for an extremum value of an objective functional involving dynamic variables. In this section, we consider optimal control methods to derive optimal drug treatments as functions of time. The control variables as used in (1) are described as follows. The control 1 represents the effect of fusion inhibitors, which are the drugs that protect the uninfected CD4 + T-cells by preventing the entry of the virus into the CD4 + T-cells membrane. The control variable 2 simulates the effect of reverse transcriptase inhibitors.
These drugs hinder the reverse transcription process. The third control variable 3 simulates the effect of protease inhibitors, which prevent the already infected cells from producing mature infectious virions. The aforementioned controls represent effective chemotherapy dosage bounded between 0 and 1. The situation 1 ( ) = 2 ( ) = 3 ( ) = 1 represents total efficacy of the fusion inhibitors, reverse transcriptase inhibitors, and protease inhibitors, respectively, and 1 ( ) = 2 ( ) = 3 ( ) = 0 represents no treatment. It is worth noting that the aforementioned control variables are bounded Lebesgue-integrable functions. The study aims at maximizing the levels of the healthy CD4 + T-cells, as well as the levels of the CD8 + T-cells ( ), while minimizing the viral load ( ) and at the same time keeping cost and side effects of treatment at a minimum. With the above description, the following objective function (2) needs to be maximized: subject to the ordinary differential equations given in model (1).
( ), ( ), and ( ) are the solutions of the ODEs (1). The quantities 1 and 2 represent the cost associated with maximizing the number of CD4 + T-cells and the CD8 + Tcells, respectively, while 3 represents the cost associated with minimizing the viral load. In addition, 1 , 2 , and 3 are nonnegative constants representing the relative weights attached to the current cost of each treatment regime and is a fixed terminal time of the treatment program subject to the ordinary differential equations described in model (1). This study assumes that the cost of controls is of quadratic form. Furthermore, it is also based on the fact that there is no linear relationship between the effect of treatment on CD4 + T-cells and CD8 + T-cells and the HIV virions. Consequently, 1 , 2 , and 3 are Lebesgue-integrable; that is, they are piecewise continuous and integrable. The fundamental aim of this therapeutic strategy is to maximize the objective functional defined in (2) by increasing the number of the uninfected CD4 + T-cells and the CD8 + T-cells, decreasing the viral load ( ), and minimizing the harmful side effects and cost of treatment over the given time interval [0, ]. Therefore, we aim at determining the optimal controls * 1 , * 2 , and * 3 such that where is a set of all measurable controls defined by In the next section, we show the existence of an optimal control for system (1) and later derive the optimality system. This study will employ Pontryagin's Maximum Principle.

Characterization of the Optimal Control
The necessary conditions that an optimal control must satisfy come from Pontryagin's Maximum Principle [5].

Theorem 1. Suppose that the objective function
is maximized subject to the controls and state variables given in model (1) with Then there exist optimal controls ( * 1 , Proof. The existence of the solution can be shown using the results obtained in Fleming and Rishel [18], since for all , ∈ and all ∈ [0, 1], this condition is satisfied by the control set , 3 ) of the objective functional is concave on , (5) there exist constants 1 > 0, 2 > 0, and > 1 such that the integrand of the objective function ( , ) is bounded by ( , , , , , , , , where 1 depends on the upper bound on , , while 1 > 0 since 1 , 2 , 3 > 0 according to the definition.
Since all the above conditions are satisfied, we conclude that there exist optimal controls 1 * , 2 * , and 3 * .

6
Computational and Mathematical Methods in Medicine

Numerical Simulation
In this section, we investigate the effect of optimal strategy on HIV by applying Runge-Kutta forth-order scheme on the optimality system. The optimality system is obtained by taking the state system together with the adjoint system, the optimal control, and the transversality conditions. The dynamical behaviour of the models in relation to various control is also studied. The optimal strategy is achieved by obtaining a solution for the state system (1) and costate system (12). An iterative scheme is explored and used to determine the solution for the optimality system. The numerical method utilized is the forward-backward sweep method that incorporates iterative Runge-Kutta fourth-order progressive-regressive schemes. The progressive scheme is used in obtaining the solutions of the state ODEs given in (1) with the initial conditions, while the regressive scheme is applied in obtaining the solutions of the adjoint system given by (12) with transversality conditions given in (13). The controls are updated at the end of each iteration using the formula for optimal controls. We continue with the iterations until convergence is achieved. This is a two-point boundaryvalue problem, with separated boundary conditions at times 0 = 0 and = . This explains our choice in using the fourth-order Runge-Kutta scheme. For the numerical simulation, we take = 310 days or 10 months. This value represents the time in which treatment is stopped. Furthermore, the values of the weight function are taken as   values that are used in the numerical simulations of the in vivo model, while Table 5 consists of the proposed initial values of the state variables.
The initial values given in Table 5 are chosen in such a way that they reflect a patient during acute infection. This is in line with the WHO recommendations that stipulate that all people living with the HIV be put on ARTs irrespective of their CD4 + counts unlike in the past where the CD4 + count had to be less than 500 cells/mm 3 [19]. Figure 2 represents the various control strategies. It is evident that the control 1 remains at the maximum for the first two months and drops to zero onward, while control 2 remains at maximum for the first four and a half months and then drops to 30% the sixth and the ninth months and drops to the minimum after the 10th month. In addition, the control strategy 3 remains at a maximum for the first ninth months, only dropping to a minimum at the tenth month. From these results, we can see that protease inhibitor can be administered for a longer period of time.   Figure 3 shows the population of the CD4 + T-cells in different treatment strategy. In all the cases, it is evident that the introduction of the ARTs plays a significant role as far as controlling HIV is concerned. Nonetheless, it is clear that when fusion inhibitor ( 1 ) is used without other controls, the number of CD4 + T-cells reduces significantly and a longer time is taken before the number increases. In particular, the drug effectiveness seems to be felt after the first two months. We interpret the results to mean that it is difficult to control  the HIV virions by targeting their cell-entry mechanism. The use of protease inhibitor, however, leads to an increase in the number of the CD4 + T-cells. In addition, it is evident that a combination of the three drugs evokes a more pronounced CD4 + T-cells increase than in monotherapy or combination of two drugs. It is important to point out that CD4 + T-cell responses in number of cells gained were similar for patients treated with combination of two drugs therapies and patients treated with combination of three drugs therapies. Figure 4 presents the dynamics of the latently infected cells after the introduction of the various control strategies. It is evident that the latently infected cells are produced after the introduction of reverse transcriptase inhibitor to an HIV infected cell. Since the latently infected cells do not produce infectious virions, it is important to administer RTIs to an infected person. This will reduce the number of virions producing cells. Figure 5 shows the change in the population of the infected CD4 + T-cells with time in different control strategies.  From the simulated results, we see that use of ARTs plays a fundamental role, especially in controlling the rate of infection. Nonetheless, when the fusion inhibitors are introduced in the body, the number of the infected cells still increases for the first few months. This clearly shows that it is very difficult to control the HIV virions at the entry level. The reason would probably be based on the fact that HIV uses a complex series of steps to deliver its genome into the host cell cytoplasm while simultaneously evading the host immune response as shown in Figure 6. Figure 7 shows the change in the population of the HIV virions in different drug combination(s). It is evident that controls 1 and 2 are not as very effective as PIs in controlling viral progression. In particular, there is no significant difference when the control 1 is used and when no control is used at all. Researches such as [20] suggest that viruses blocked by entry inhibitors such as the fusion inhibitors are likely redistributed to plasma, where they artificially increase the number of HIV virions. This may probably be the reason why there is an indication of having  high number of viral loads even when control 1 is applied. In addition, the fusion inhibitor prevents the entry of the virions unlike the other two drugs that allow the entry of the HIV virions into the cells, confirming the absorption effect. Simulated results shows that protease inhibitor plays a significant role in reducing viral progression and it is the best single drug in use for viral suppression. This is in agreement with some of the works done in the field of in vivo HIV dynamics which have concluded that protease inhibitors are more effective than reverse transcriptase inhibitors and fusion inhibitors in terms of viral load reduction in HIV infected patient [21][22][23]. The simulated results also emphasize the importance of using a combination of the various ARTs when treating HIV. From Figure 8, it is evident that noninfectious viruses are produced after the introduction of the protease inhibitor in the body. Introduction of PIs to HIV infected cells generates a pool of immature HIV virions; this leads to the transfer  of noninfectious virus across the virological synapse. This therefore implies that the virus produced will not infect more susceptible CD4 + T-cells. Figure 9 shows the population of the CD8 + T-cells in different treatment strategy. Both the RTIs and PIs cause a substantial increase in the population of the CD8 + T-cells in HIV infected patients. However, it is evident that as much as these two drugs plays a major role, the combination of all the three controls produces a higher immune system reconstitution with sustained increases in circulating number of CD8 + T-cells. Figure 10 shows the population of the activated CD8 + Tcells. The activation process plays a major role in controlling the HIV virus particles. This is because the cells fight, destroy, and kill the infected CD4 + T-cells. This in turn reduces the number of HIV virions produced. From the simulated results, it is evident that, after the introduction of the ARTs, the number of activated CD8 + T-cells reduced significantly. The reduction may be attributed to the reconstituted immune system or due to the reduction of the retroviral activity on the cells [24]. However, the question we need to ask ourselves is whether this reduction has any clinical benefit. In the future, it is important to analyze the clinical benefit accrued from the reduction of the CD8 + T-cells activation process.

Conclusion
In this paper, we have analyzed a seven-dimension in vivo HIV model with inclusion of three drug combinations, that is, FIs, RTIs, and PIs. Optimal control theory is applied to determine the optimal treatment regime. The study applied Pontryagin's Maximum Principle in deriving the conditions for optimal control, which maximizes the objective function. The systems of ODEs, the state system, and the adjoint system were solved numerically by both forward and backward Runge-Kutta forth-order scheme. Results from the numerical simulations show that FIs and RTIs should be used within the four months and later the doctors should change the drugs and introduce another type, whereas the PIs can be used for a longer period of time without necessarily leading to major side effect. However, the inferiority of monotherapy compared with combination of therapies has been observed in the simulated result, especially in suppression of viral replication, CD4 + and CD8 + T-cells reconstitution, and controlling disease progression.
ARTs have been seen to play a significant role as far as viral suppression is concerned. Therefore, they should be recommended for all patients immediately after one is diagnosed as HIV-positive regardless of the CD4 + count. This supports the guidelines by WHO. However, the simulated results suggest that PI is possibly the best single drug and fusion inhibitor is the worst drug in terms of viral load and infected cells reduction. From the results, we recommend that RTIs be used as initial therapy for HIV. FI should be introduced to the patient after the RTIs but should never be used alone.
In the future, it is important to develop the model in such a way that it brings out the relationship between the number of the CD8 + T-cells and the CD4 + T-cells produced in the thymus.

Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.