Stochastic optimal therapy for enhanced immune response

doi:10.1016/j.mbs.2004.06.004

Mathematical Biosciences

Volume 191, Issue 2, October 2004, Pages 123-142

https://doi.org/10.1016/j.mbs.2004.06.004 Get rights and content

Abstract

Therapeutic enhancement of humoral immune response to microbial attack is addressed as the stochastic optimal control of a dynamic system. Without therapy, the modeled immune response depends upon the initial concentration of pathogens in a simulated attack. Immune response can be augmented by agents that kill the pathogen directly, that stimulate the production of plasma cells or antibodies, or that enhance organ health. Using a generic mathematical model of immune response to the infection (i.e., of the dynamic state of the system), previous papers demonstrated optimal (open-loop) and neighboring-optimal (closed-loop) control solutions that defeat the pathogen and preserve organ health, given initial conditions that otherwise would be lethal [Optimal Contr. Appl. Methods 23 (2002) 91, Bioinformatics 18 (2002) 1227] . Therapies based on separate and combined application of the agents were derived by minimizing a quadratic cost function that weighted both system response and drug usage, providing implicit control over harmful side effects.

Here, we focus on the effects that corrupted or incomplete measurements of the dynamic state may have on neighboring-optimal feedback control. Imperfect measurements degrade the precision of feedback adjustments to therapy; however, optimal state estimation allows the feedback strategy to be implemented with incomplete measurements and minimizes the expected effects of measurement error. Complete observability of the perturbed state for this four state example is provided by measurement of four of the six possible pairs of two variables, either set of three variables, or all four variables. The inclusion of state estimation extends the applicability of optimal control theory for developing new therapeutic protocols to enhance immune response.

Introduction

Infectious microbes trigger a dynamic response of the immune system, in which potentially uncontrolled growth of the invader (or pathogen ) is countered by various protective mechanisms. Initially, the innate immune system provides a non-specific tactical response, killing what pathogen it can, inducing inflammation and vasodilation that aids the defense, causing blood coagulation that slows the spread of infection to other parts of the body, and raising the alarm for more complete response. In the process, a humoral response is initiated, signaling the presence of extracellular ‘non-self’ organisms and activating B cells to become plasma cells that are specific to the intruders’ antigens. The plasma cells produce antibodies that bind to the antigens, mediating the destruction of pathogens by various modalities [3], [4], [5]. The adaptive immune system provides a strategic response that is tailored to the primary attack, producing B and T cells, as well as a host of molecules, that defeat specific intracellular pathogens by binding to infected cells and either killing them outright, inducing programmed cell death, or signaling other cells to finish the job. Additional B and T cells with narrowly focused memory also are produced; they can respond rapidly if invading microbes of the same type are encountered again. Innate, humoral, and adaptive immune responses are coupled, even though separate modes of operation can be identified.

Many models of immune response to infection have been postulated [6], [7], [8], [9], with recent emphasis on the human-immunodeficiency virus [10], [11], [12], [13], [14], [15]. Norbert Wiener and Richard Bellman appreciated and anticipated the application of mathematical analysis to treatment in a broad sense [16], [17], and Swan surveyed early optimal control applications to biomedical problems in [18], [19], [20]. Optimal control theory was postulated as an organizing principle for natural immune system behavior in [21], [22], [23], [24], and it is applied to HIV treatment in [25], [26]. Intuitive control approaches are presented in [27], [28], [29], [30]. The dynamics of drug response (pharmacokinetics) are modeled in [31], [32], and control theory is applied to drug delivery in [33], [34], [35], [36], [37], [38], [39], [40], [41].

In the remainder, we consider therapy that enhances humoral immune response to a pathogen, such as a toxin or extracellular bacterium. The options available for clinical treatment of the infection are to kill the invading microbes, to neutralize their harmful effects, to enhance the efficacy of immune response, to provide healing care to organs that are damaged by the microbes, or to employ some combination of therapies.

In prior studies, we examined remedial treatments with differing hypotheses about the initial pathogen concentration. If the initial concentration is known precisely [1], the optimizing control history maximizes efficacy of the drug while minimizing its side effects and cost. For the second study [2], a feedback strategy based on a linear perturbation model of response dynamics is derived to account for variations induced by unknown initial infection. The therapy is modified as a function of the difference between the optimal and observed dynamic states over the entire treatment period, assuming that the difference is measured without error. This feed-back approach is approximately optimal with zero-mean random disturbance inputs and perfect measurements of the state (e.g., with unmodeled variability of the infectious agent or small errors in the dynamic model itself that are modeled as ‘process noise’) [42]. If, however, the measurements used for feed-back therapy contain error or are incomplete, then the perturbed state must be estimated to account for the effects of imperfect knowledge.

In this paper, we incorporate a linear-optimal state estimator in the feedback therapy to minimize the effects of measurement error and to account for missing measurements. To provide a consistent pedagogical basis, we use the model of immune response that was employed in [1], [2]. The dynamic model and the deterministic optimal solutions are reviewed briefly. We introduce basic concepts of controllability and observability, illustrating how they apply to this problem. The feedback control law is recast as a stochastic neighboring-optimal control problem, whose solution can be partitioned into the control law derived in [2] plus the linear-optimal estimator (or Kalman–Bucy filter [43]) presented here. Examples of therapeutic effects in the presence of measurement error and incompleteness are given.

Section snippets

Immune response model

The mathematical model used in [1], [2] is an idealization of a generic humoral immune response that subsumes many details into aggregated effects [6]. There are four components in the model’s dynamic state: concentration of a foreign pathogen that expresses an identifying antigen (x₁), concentration of plasma cells that are specific to the antigen (x₂), concentration of antibodies that bind to the antigen (x₃) and mediate the destruction of the pathogen, and a measure of the health of an organ

Deterministic optimal therapeutic protocol

The optimal therapeutic protocol is derived by minimizing a treatment cost function, J, that penalizes large values of pathogen concentration, poor organ health, and excessive application of therapeutic agents. This multi-objective, positive-definite scalar cost function of many variables allows tradeoffs between important factors to be adjusted through the relative weighting of individual components. Concordant responses tend to reinforce each other while conflicting responses compete in the

Deterministic neighboring-optimal therapeutic protocol

Adjustments to the optimal therapy that account for different levels of infection can be based on the solution of a neighboring-optimal control problem [42]. State and control histories are represented as sums of the optimal histories derived from the iterative procedure, x*(t) and u*(t), and deviations from those histories, Δx(t) and Δu(t): $x (t) = x^{*} (t) + Δ x (t),$ $u (t) = u^{*} (t) + Δ u (t) .$ Eq. (7) can be expanded as, $\dot{x} (t) = {\dot{x}}^{*} (t) + Δ \dot{x} (t) = f {[x^{*} (t) + Δ x (t)], [u^{*} (t) + Δ u (t)], [w^{*} (t) + Δ w (t)]} \approx f [x^{*} (t), u^{*} (t)] + F (t) Δ x (t) + G (t) Δ u$

Controllability and observability of the immune response model

A principal goal is to reduce and eventually eliminate the pathogen concentration, preferably by the end of the treatment period. There are two means of doing this for the present model. The first is to deliver the state to a basin of attraction surrounding this model’s stable equilibrium point. Once in the proper neighborhood, the state can decay to its desired value through natural immune response without further therapy, the speed of convergence being dependent on the strength of natural

Stochastic neighboring-optimal therapeutic protocol

With unknown initial conditions or process uncertainty and measurements that contain error, the optimization problem is no longer deterministic, and a cost function that assumes perfect characterization of the system cannot be optimized with certainty. Rather than minimizing the deterministic cost function, J, subject to a deterministic dynamic constraint, we minimize the expected value of the cost, E(J), subject to a stochastic constraint. While the expected value has a rigorous definition [42]

Results of stochastic neighboring-optimal therapy

There are many combinations of measurements, control variables, cost function weights, and estimation statistics that could be considered. For illustration, we present five examples that display effects of measurement dimension and measurement error. In all cases, we assume that the control variable is the pathogen killer, u₁; hence, the result that assumes perfect measurement of the entire state (Fig. 4) presents a baseline against which the effects of estimation can be compared. While the

Conclusion

Optimal control theory has an important role to play in the development of new therapeutic protocols for treating infection. Prior studies demonstrated that numerical optimization of non-linear models combined with neighboring-optimal feedback of perfect information can define therapies for enhancing natural response of the immune system. Here, we show that similar results can be achieved when only imperfect or incomplete measurements are available. A principal requirement is that the

Acknowledgment

This research was supported by a grant from the Alfred P. Sloan Foundation. Mr. Ghigliazza received partial support from the Burroughs-Wellcome Fund for Biological Dynamics.

References (44)

M.A. Stafford et al.
Modeling plasma virus concentration and CD4+ T cell kinetics during primary HIV infection
J. Theor. Biol.
(2000)
G.W. Swan
Role of optimal control theory in cancer therapy
Math. Biosci.
(1990)
L.M. Wein et al.
Dynamic multidrug therapies for HIV: a control theoretic approach
J. Theor. Biol.
(1997)
L.M. Wein et al.
Mathematical analysis of antiretroviral therapy aimed at HIV-1 eradication or maintenance of low viral loads
J. Theor. Biol.
(1998)
R.F. Stengel et al.
Optimal control of innate immune response
Optimal Contr. Appl. Methods
(2002)
R.F. Stengel et al.
Optimal enhancement of immune response
Bioinformatics
(2002)
C.A. Janeway et al.
Immunobiology
(2001)
P.M. Lydyard et al.
Instant Notes in Immunology
(2000)
M. Thain et al.
The Penguin Dictionary of Biology
(2000)
A. Asachenkov et al.
Disease Dynamics
(1994)

A. Rundell, H. HogenEsch, R. DeCarlo, Enhanced modeling of the immune system to incorporate natural killer cells and...

A.S. Perelson

Immunology for physicists

Rev. Modern Phys.

(1997)

M.A. Nowak et al.

Virus Dynamics: Mathematical Principles of Immunology and Virology

(2000)

M.A. Nowak et al.

Antigenic oscillations and shifting immunodominance in HIV-1 infections

Nature

(1995)

A.S. Perelson et al.

HIV-1 dynamics in vivo: virion clearance rate, infected cell lifespan, and viral generation time

Science

(1995)

A.S. Perelson et al.

Mathematical analysis of HIV-1 dynamics in vivo

SIAM Rev.

(1999)

D. Wodarz et al.

The role of antigen-independent persistence of memory cytotoxic T lymphocytes

Int. Immun.

(2000)

D. Wodarz et al.

CD8 memory, immunodominance, and antigenic escape

Eur. J. Immun.

(2000)

N. Wiener

Cybernetics: or Control and Communication in the Animal and the Machine

(1948)

R.E. Bellman

Mathematical Methods in Medicine

(1983)

G.W. Swan

Optimal control applications in biomedical engineering – a survey

Opt. Contr. Appl. Methods

(1981)

G.W. Swan

Applications of Optimal Control Theory in Medicine

(1984)

Cited by (23)

Machine learning and mechanistic computational modeling of inflammation as tools for designing immunomodulatory biomaterials
2021, Immunomodulatory Biomaterials: Regulating the Immune Response with Biomaterials to Affect Clinical Outcome
The inflammatory response is a dynamic, complex process that involves multiple interacting cell types and molecular mediators. Proinflammatory, feed-forward processes allow for proper ramp-up of inflammation but can also drive a maladaptive, overly exuberant response that harms tissues. Antiinflammatory, negative feedback processes keep these feed-forward aspects of inflammation in check and often stimulate healing, but when produced inappropriately can either impair healing or, at the other extreme, drive excessive scaring and fibrosis. Biomaterials are used to repair injured tissues but may be suboptimal or even harmful if they exacerbate the inflammatory response in those tissues. In this chapter, we describe how computational approaches to modeling inflammation in silico—including some studies involving biomaterials—have helped us decipher some of the complexity of inflammation, suggesting applications of computational modeling to optimize the preclinical and clinical use of biomaterials.
Nonlinear optimal control of the acute inflammatory response
2020, Biomedical Signal Processing and Control
Citation Excerpt :
In this area one can note some results towards solving the control problem for the acute inflammatory response model. These results rely primarily on nonlinear model predictive control methods [21–24]. In this article a novel solution to the nonlinear optimal control problem of the acute inflammatory response model is proposed.
The article proposes a nonlinear optimal control approach for the dynamic model of the acute inflammatory response. This model describes the reaction of the human body to bacterial infection. Optimal control of the infusion of anti-inflammatory and anti-bacterial medication is critical for achieving more effective and less costly pharmaceutical treatments. To solve the related nonlinear optimal control problem the state-space description of the acute inflammatory response undergoes first approximate linearization around a temporary operating point which is recomputed at each iteration of the control algorithm. The linearization relies on Taylor series expansion and on the computation of the associated Jacobian matrices. For the approximately linearized model of the acute inflammatory response a stabilizing H-infinity feedback controller is designed. This controller stands for the solution of the optimal control problem under model uncertainties and external perturbations. To find the controller’s feedback gains an algebraic Riccati equation is solved at each time-step of the control method. The stability properties of the control scheme are proven through Lyapunov analysis.
Toward a model-free feedback control synthesis for treating acute inflammation
2018, Journal of Theoretical Biology
An effective and patient-specific feedback control synthesis for inflammation resolution is still an ongoing research area. A strategy consisting of manipulating a pro and anti-inflammatory mediator is considered here as used in some promising model-based control studies. These earlier studies, unfortunately, suffer from the difficultly of calibration due to the heterogeneity of individual patient responses even under similar initial conditions. We exploit a new model-free control approach and its corresponding “intelligent” controllers for this biomedical problem. A crucial feature of the proposed control problem is as follows: the two most important outputs which must be driven to their respective desired states are sensorless. This difficulty is overcome by assigning suitable reference trajectories to the other two outputs that do have sensors. A mathematical model, via a system of ordinary differential equations, is nevertheless employed as a “virtual” patient for in silico testing. We display several simulation results with respect to the most varied situations, which highlight the effectiveness of our viewpoint.
Immune therapeutic strategies using optimal controls with L<sup>1</sup> and L<sup>2</sup> type objectives
2017, Mathematical Biosciences
Citation Excerpt :
In particular, applying automatic control for biomedical therapeutic intervention has recently gained interest in such areas as glucose control for diabetes or in critically ill patients and for anesthesia depth control, as seen for example, in [5–8], and the references therein. The use of optimal control theory for biomedical applications can be seen in [9,10], and [11] for example; and, more recently, the use of model predictive control (MPC) in [12] and [13] which used the same mathematical model as used herein. MPC was also used in [14] and [15] for diabetes treatment strategies and [16] for an application to a multi-compartment respiratory system.
Therapeutic strategies to correct an excessive immune response to pathogenic infection is investigated as an optimal control problem. The control problem is formulated around a four dimensional mathematical model describing the inflammatory response to a pathogenic insult with two therapeutic control inputs which have either a direct pro- or anti-inflammatory effect in the given system. We use Pontryagin’s maximum principle and discuss necessary optimality conditions. We consider both an L¹ type objective functional as well as an L² type objective. For the former, the presence of singular control will be addressed. For each case, numerical simulations using a nonlinear programming optimization solver to acquire different drug treatment strategies are presented and discussed. The results provide insight for possible treatment strategies and the methods could be a relevant tool for future practice to assist in better prediction of clinical outcomes and subsequently better treatment for patients.
Solving Immunology?
2017, Trends in Immunology
Emergent responses of the immune system result from the integration of molecular and cellular networks over time and across multiple organs. High-content and high-throughput analysis technologies, concomitantly with data-driven and mechanistic modeling, hold promise for the systematic interrogation of these complex pathways. However, connecting genetic variation and molecular mechanisms to individual phenotypes and health outcomes has proven elusive. Gaps remain in data, and disagreements persist about the value of mechanistic modeling for immunology. Here, we present the perspectives that emerged from the National Institute of Allergy and Infectious Disease (NIAID) workshop ‘Complex Systems Science, Modeling and Immunity’ and subsequent discussions regarding the potential synergy of high-throughput data acquisition, data-driven modeling, and mechanistic modeling to define new mechanisms of immunological disease and to accelerate the translation of these insights into therapies.
New developments in the application of optimal control theory to therapeutic protocols
2016, Mathematical Biosciences
Optimal control theory is one of the most important tools in the development of new therapeutic protocols for treating infections. In this work, we present an algorithm able to deal with high-dimensional problems with bounded controls. The optimal solution is obtained by minimizing a positive-definite treatment cost function. Our method, based on Pontryagin’s Minimum Principle and the coordinate cyclic descent method, allows solving problems of varied nature. In this paper, and by way of example, therapeutic enhancement of the immune response to invasion by pathogenic attack is addressed as an optimal control problem. The generic mathematical model used describes the evolution of the disease by means of four non-linear, ordinary differential equations. The model is characterized by the concentration of pathogens, plasma cells, antibodies and a numerical value that indicates the relative characteristic of an organ damaged by disease. From a system theory point of view, drugs can be interpreted as control inputs. Therapies based on separate application of the agents are presented in previous studies. We shall present the more general problem in this paper, considering combined therapies and bounded controls. Finally, we present several numerical simulations.

View all citing articles on Scopus

View full text

Stochastic optimal therapy for enhanced immune response

Abstract

Introduction

Section snippets

Immune response model

Deterministic optimal therapeutic protocol

Deterministic neighboring-optimal therapeutic protocol

Controllability and observability of the immune response model

Stochastic neighboring-optimal therapeutic protocol

Results of stochastic neighboring-optimal therapy

Conclusion

Acknowledgment

J. Theor. Biol.

Math. Biosci.

J. Theor. Biol.

J. Theor. Biol.

Optimal control of innate immune response

Optimal Contr. Appl. Methods

Optimal enhancement of immune response

Bioinformatics

Immunobiology

Instant Notes in Immunology

The Penguin Dictionary of Biology

Disease Dynamics

Immunology for physicists

Rev. Modern Phys.

Virus Dynamics: Mathematical Principles of Immunology and Virology

Antigenic oscillations and shifting immunodominance in HIV-1 infections

Nature

HIV-1 dynamics in vivo: virion clearance rate, infected cell lifespan, and viral generation time

Science

Mathematical analysis of HIV-1 dynamics in vivo

SIAM Rev.

The role of antigen-independent persistence of memory cytotoxic T lymphocytes

Int. Immun.

CD8 memory, immunodominance, and antigenic escape

Eur. J. Immun.

Cybernetics: or Control and Communication in the Animal and the Machine

Mathematical Methods in Medicine

Optimal control applications in biomedical engineering – a survey

Opt. Contr. Appl. Methods

Applications of Optimal Control Theory in Medicine