A Multistage Feedback Control Strategy for Producing 1 , 3-Propanediol in Microbial Continuous Fermentation

In this paper, we consider a multistage feedback control strategy for producing 1,3-propanediol in microbial continuous fermentation. Both the dilution rate and the concentration of glycerol in the input feed are used as control variables, and these variables are further assumed to be in the form of a linear combination of biomass and glycerol concentrations. Unlike the general form of linear feedback control, the coefficients of linear combination are continuous functions with respect to time. Inspired by the control parameterization method, we use the piecewise-constant functions to approximate the coefficient functions; then we get the multistage feedback control law by solving nonlinear mathematical programming problems. Numerical results indicate the flexibility and effectiveness of our strategy.


Introduction
Nowadays, the production of 1,3-propanediol(1,3-PD) by microbial fermentation is very popular.There are three main ways to produce 1,3-PD by microbial fermentation: continuous culture, batch culture, and fed-batch culture.The main method for producing 1,3-PD by continuous culture is widely concerned [1], since it has the advantages of high production strength, stable production, and high automation.Compared with chemical synthesis, microbial fermentation for producing 1,3-PD is more attractive because it is easy to implement and does not generate toxic byproducts.Unfortunately, the production of 1,3-PD by microbial fermentation is not up to the standard of industry.Therefore, more and more scholars focus on optimizing the yield of 1,3-PD via microbial fermentation [2][3][4][5][6][7].
Previous studies on the optimal control of 1,3-PD only achieved the open-loop control, see, for example, [8][9][10].This is a discount in practical production.Based on our previous study, we find that feedback control is more in line with actual production process or experimental process by realizing closed-loop control.The linear feedback optimal control [11] has been widely studied in theory and applications.Different from the traditional approach to determine an optimal feedback control such as solving the well-known Hamilton-Jacobi-Bellman partial differential equation, the sensitivity penalization approach for computing robust suboptimal controllers [12], and the neighboring extremal approach [13,14], we consider the feedback control coefficient function with time dependence.In order to further increase the yield of 1,3-PD, we regard both dilution rate and the concentration of glycerol in the input feed as the controllers which are further assumed to be in the form of a linear combination of biomass and glycerol concentrations.Unlike the general form of linear feedback control, the coefficients of linear combination are continuous functions with respect to time.Inspired by the control parameterization method, we use the piecewise-constant functions to approximate the coefficient functions; then we get the multistage feedback control law by solving nonlinear mathematical programming problems.Finally, numerical results show the flexibility and effectiveness of our strategy.

Problem Statement
The dynamic model of the continuous culture process is based on the following assumptions.Assumption .The material composition in the fermentation tank does not change with the position of space, and the solution in the reactor is sufficiently well mixed so that the concentrations of reactants are uniform.
Assumption .The continuously added medium contains glycerin only, and the substance in the reactor is exported at dilution rate ().
Assumption .The materials in the fermentation are fully mixed in which the concentrations are even, and the concentrations of the reactants change only with the change of reaction time.
Under the above assumptions, the mass balance relationships for biomass, substrate, and products in the microbial continuous culture can be expressed as the following nonlinear dynamic system: and where  1 (),  2 (), where   = 0.67 is the maximum specific growth rate;   = 0.28 is the Monod saturation constant for substrate.Under anaerobic conditions at 37 ∘ C and pH=7.0, the values of other parameters used in (1) -( 7) are listed in Table 1.
In this paper, the dilution rate () and the concentration of glycerol in the input feed   0 () are chosen as the control variables ().It is obvious that the control variables are also constrained: where  * and  * are the lower and upper bounds of ().

Feedback Optimal Control Problem
In microbial fermentation, the most important factors to influence the final concentration of 1,3-PD are the concentrations of biomass and glycerol.And the linear state feedback control law is one of the most common feedback control structures [16].Thus, the feedback controller is considered as a linear combination form of the biomass and the glycerol concentrations, i.e., where The following bound constraints are imposed on the feedback control coefficients (): Substituting ( 13) into (8) gives where Consider system (16) with the initial condition of (8).Let (⋅ | ) denote the solution of system ( 16) on [0,   ].Then the constraint conditions (10) become Our goal is to present a state feedback control strategy to maximize the final concentration of 1,3-PD.We now consider the problem of choosing the feedback control coefficients   (),  = 1, 2, 3, 4, to minimize the total system cost subject to constraints (17).
Problem P is a nonlinear optimization problem in which a finite number of decision variables (the feedback control coefficients) needs to be optimized subject to a set of constraints.It is very difficult to solve Problem P, because each continuous inequality constraint in (17) actually constitutes an infinite number of constraints-one for each point in [0,   ].Hence, Problem P can be viewed as a semi-infinite optimization problem.Then, we will use a penalty method to transform Problem P [17].
The condition () ∈ ,  ∈ [0,   ] is equivalently transcribed into with Clearly, () = 0 if and only if () ∈ .However, the equality constraint ( 20) is nonsmooth at the points when h = 0. Consequently, standard optimization routines would have difficulties in dealing with this type of equality constraints.Let where the smoothing parameter  is a very small positive number, and   :  →  is defined by Obviously, G() is a smooth function in .Then, the objective function of Problem P can be reformulated as where  > 0 is the given penalty parameter.Problem P can be transformed into the following problem.
Problem Q. Choose  to minimize the penalty function ().
Similar to the work [18], we can get the following theorem.
Theorem 1.Let  *  be the optimal solution of Problem Q. Suppose that there exists an optimal solution  * of the original Problem P. en Theorem 1 guaranteed that any local solution of the approximate problem can be used for generating a corresponding local solution of the original problem when the smoothing penalty parameter is sufficiently small.

Control Vector Parameterization Technique and Particle Swarm Adaptive Algorithm
To solve Problem Q numerically, the control vector parameterization approach is applied [19,20], in which the feedback control variables Using the piecewise-constant policy, the feedback control variable   () is approximated by where  , is the value of ξ () in the th subinterval [ −1 ,   ), and   is defined as With the  ∈   , the differential equation ( 15) is of form where The initial condition remains: Let (⋅ | ) be the solution of the system (28) corresponding to the control parameter vector .And in this way, Problem Q can be approximated by a sequence of nonlinear programming problems, in which  is regarded as the decision vector.
We may now specify the approximate Problem Q(p) as follows.
Problem Q(p).Find a control parameter vector  ∈   to minimize the cost function.
Theorem 2. Let ξ * be the optimal control of the approximate Problem Q(p).Suppose that the original Problem Q has an optimal control  * .en, To solve the Problem P as mathematical programming problems, we require the gradient formulae for the function ().We shall derive the required formulae as follows [21].
Let the corresponding Hamiltonian function for the cost function be defined by with the boundary condition  (  ) = (0, 0, 0, 0, 0)  .(37) Using the similar arguments in [19], we obtain the gradient of  is During actual computation, the control parameterization is carried out on an partition of the interval [0,   ]; each component of (38) can be written in a more specific form: Based on the above control vector parameterization approach, we adopt a gradient-based adaptive refinement method [22] for solving Problem Q(p).The algorithm is adaptive so as to obtain economic and effective discretization grids.In this way, a high-quality solution can be obtained with low computational cost.
Define ] are the optimal objective function value and the optimal solution in iteration   , respectively.We hope to find a new discretization grid to make it better adapted to the solution.
Define the sensitivity of  For a given value then let If the following conditions hold, in which where  1 ,  2 , and  2 are given constants, and The main steps of this algorithm are as follows.
Step .By using particle swarm optimization algorithm to obtain the optimal objective function value  *  and the optimal solution  *  , in which ,  is the interval number corresponding to the interval cross powder at this time.
Step . .Update the inertia term according to the following formula: where  1 ,  2 obey the uniform distributions on [0, 1].
Step . .Bisecting each subinterval in Δ  to obtain the temporary grids Δ   and the corresponding control variables    .
Remark .In the above algorithm,  denote the total number of particles in the swarm. 1 and c 2 are the cognitive and social scaling parameters.  and   are the maximum and minimum inertia weights.  and   are vectors containing the maximum and minimum particle velocities.  is the maximum number of iteration. 1 and  2 are control factors. is the iteration index.

Numerical Results
In the microbial fermentation, we choose the boundary value of state vector as  * = [0.001,100, 0, 0, 0]  ,  * = [10, 2039, 939.5, 1026, 360.9]; the initial concentrations of biomass, glycerol, 1,3-PD, acetate, and ethanol are  the concentration of 1,3-PD at the terminal time is 752.7951mmol/L.The detailed evolution of time grids is illustrated in Figure 1, after six iterations, the optimal time grids is found, and in this case the results agree with experimental data.Figure 1 also shows the division of time grids in each iteration.
The feedback control parameters are shown in Figures 2-5, respectively.The dilution rate and the glycerol concentration in feed are shown in Figures 6 and 7.The concentration changes of biomass, glycerol, 1,3-PD, acetate, and ethanol under the optimal feedback control are shown in Figure 8.The computational results verify the effectiveness of this method.

Conclusions
In this paper, we have considered a feedback control strategy which is close-loop control for producing 1,3-PD in microbial continuous fermentation and developed a particle swarm adaptive algorithm to obtain the global solution.Numerical results show that the method is successful at producing highquality control strategies.