PRODUCTION PLANNING IN A THREE-STOCK REVERSE-LOGISTICS SYSTEM WITH DETERIORATING ITEMS UNDER A CONTINUOUS REVIEW POLICY

In this paper, we study a reverse supply chain with three stocks for a firm implementing the periodic-review as policy. The first stock consists of newly manufactured items, while the second one is devoted to remanufactured items. The third stock contains all the items returned from the market. We made the following assumptions: items are not as-good-as-new, deterioration and dynamic demands affect new and remanufactured items, and dynamic customer return rate. Using optimal control theory, explicit expressions of the optimal rates in all three stocks have been provided: those for manufacturing, remanufacturing, disposal, and inventory levels. In the case where items deterioration rates are assumed to be unknown, self-tuning optimal control has also been implemented using the recursive least-squares method. Numerical examples and sensitivity analyses illustrate the results.

1. Introduction. Nowadays, many companies are motivated to remanufacture/ reuse their returned products rather than to dispose of them. This is mainly due to environmental, social, and economical reasons. Environmental issues arise due to hazardous materials in products and large quantities of waste. Social issues occur, for example, as companies locate in foreign countries where business is conducted according to different rules. Economical issues are the additional income due to the presence in the market of new or almost new products.
It is more and more acknowledged that sustainability is no longer a choice, and that it has become a necessity. The United Nations defined "sustainability" as "meeting present needs without compromising the ability of future generations to meet their needs." Manufacturing firms have to design greener processes and products. Unnecessary squandering cannot be tolerated and waste should be reduced to a minimum.
Reverse-logistics systems deal with returned items which can be repaired, reconditioned, remanufactured, or recycled. Different aspects of reverse supply chains 1042 ABDELGHANI BOURAS AND LOTFI TADJ have been studied by researchers. In reviewing the literature, we noted that it was customary for researchers to assume that remanufactured items were as-good-asnew. This assumption may be true for some products, but it is not for all products. For example, printer toner is sometimes refilled but it does not become as-good-asnew. What would be the impact of assuming that remanufactured items were not as-good-as-new but rather as-good-as-old? Driven by this motivation, we study in this paper a reverse-logistics system where returned items are remanufactured, but are not brought to the as-good-as-new condition. Items whose condition is beyond repair are scraped. We assume that the manufacturing firm adopts a continuousreview policy. Under a certain objective function, the optimal manufacturing, remanufacturing, and disposal rates are obtained.
Our paper proceeds in the following order. The next section reviews up to date prior research. Section 3 contains the description of the model and Section 4 presents the analysis of the model using an optimal control approach. Section 4 also has numerical examples. The last section presents contributions and limitations of our study and further research avenues.
Before plunging into the literature, we would like to refer the reader to the paper by Bei and S. Linyan [4] who give the connotation of reverse logistics in a narrow and in a broad sense. They present and analyze the differences between reverse logistics and traditional forward logistics; and green logistics and closedloop supply chain. Also, we cite the paper by Guide and Van Wassenhove [25] who introduce closed-loop supply chains with a strong business perspective. They also describe how the research evolved during 5 different phases: The golden age of remanufacturing as a technical problem, from remanufacturing to valuing the reverse-logistics process, coordinating the reverse supply chain, closing the loop, and finally prices and markets.
Literature Reviews: In fact, so vast is the literature that several reviews have been published. De Brito et al. [8] describe over sixty case studies of reverse logistics activities in practice. Junior and Filho [33] examine and classify seventy-six papers on production planning and control, pinpointing the areas that lack research. Another recent literature review is that of Morgan and Gagnon [52] who systematically organize and review peer-reviewed journals up to July 2011 to assess progress in the scheduling of remanufacturing operations. Finally, Steeneck and Sarin [65] provide background information on reverse supply chains and conclude that it is important to integrate production planning and pricing models for reverse supply chains. Such literature reviews are beneficial not only because they determine the evolution of the research over recent years, but also because they provide some clear guidelines for the conduct of future research.
Real-Life Case Studies: Case studies on reverse logistics have also been widely reported in various industries and in different parts of the world. One of the latest case studies is that of Clottey et al. [6] who develop a generalized forecasting approach to determine the distribution of the returns of used products, as well as integrate it with an inventory model to enable production planning and control. The approach is validated in the remanufacturing operations of an electronics original equipment manufacturer (OEM) located in the Midwest. The OEM designs and manufactures components for telecommunication applications, medical systems, commercial imaging, and field-deployable defense applications. These components are in the form of embedded circuit boards and modular storage units. The OEM also remanufactures these components, and is also a contract remanufacturer of some of the products that these components are placed in.
Another recent case study is that of Kannan et al. [34] who develop a mixed integer linear model for a carbon footprint based reverse logistics network design. The proposed model aims at minimizing climate change, specifically, the CO 2 footprint, and it employs reverse logistics activities to recover used products, hence combining the location/transportation decision problem. The model is validated by examining a case study from the plastic sector in India.
Finally, Aitken and Harrison [1] examine the changes in governance structures that evolve as reverse logistics systems are developed. The UK car crash repair sector is used as a case study. The modular governance structure that developed through increased supplier capability coupled with higher levels of knowledge and information codifications are shown to be important factors in the establishment of a reverse logistics system.
Surveys: In their literature review, Prahinski and Kocabasoglu [58] argue that most research in reverse supply chains has relied on case studies and optimization models, and that opportunities exist to use survey-based research methods to explain current practices, predominant and critical issues, and managerial techniques used to manage the reverse supply chain. The last survey-based research that came to our attention is that of Nik Ab Halim et al. [54] who investigate the current level of reverse logistics adoption by Malaysian manufacturers, the reverse logistics activities performed and the strategic benefits of adopting reverse logistics, and the barriers faced by companies in implementing reverse logistics activities in their operation. The levels of adoption are found to be relatively very low due to lack of knowledge and awareness on reverse logistics concept by manufacturers, along with the perception that reverse logistics are expensive and require huge resources to implement.
Research Papers (Optimal Control Methodology): Finally, we want to report on the research papers that use the same methodology as ours in seeking the optimal problem solutions. Various quantitative models have been used in reverse logistics. We only intend to survey those research papers that use the optimal control methodology.
A one product recovery system for one product is investigated by Kiesmüller et al. [40]. Two stocks are considered, one inventory for returned and recoverable items and one for serviceable items. Return and demand rates are dynamic and a linear cost structure is assumed.
Minner and Kleber [50] generalize Kiesmüller et al. [40] by including a disposal option for returned items. Kleber et al. [43] generalize Kiesmüller et al. [40] to the case when there are different demand classes, e.g. different product qualities or different markets, while Kiesmüller [38] includes positive lead times.
Kiesmüller [37] addresses the control problem of a stochastic recovery system with two stocking points and different lead times for production and remanufacturing. A new approach which differs substantially from the existing ones is provided for the optimal control policy under a linear cost structure.
A similar problem is considered by Dobos [10] who assumes that the costs of the system consist of the quadratic holding costs for the two stocks and the quadratic manufacturing, remanufacturing and disposal costs.
Kiesmüller and Scherer [39] extend Kiesmüller et al. [40] to the case of stochastic demand and return rates. They assume the firm follows a period-review policy.
Nakashima et al. [53] use the optimal control methodology in a reverse logistics system under stochastic variability. By considering two types of inventory, they find the optimal production policy to minimize the expected average costs over periods.
Hedjar et al. [28] apply a receding horizon control strategy to a reverse logistics system with deteriorating items. Given the current inventory level, they determine the optimal production rates to be implemented at the beginning of each of the following periods over the control horizon.
Kleber [42] examines the financial impact of technological decisions firms face when introducing a new product. Foul and Tadj [21] study the same model as Dobos [10]. They further assume that items in either stock are subject to deterioration.
Gharbi et al. [22] consider a close-loop remanufacturing system repairing a singleequipment type with constant demand. Equipment disposal is not allowed. The remanufacturing resources respond to two types of demand: planned demand executed at the end of the expected life of each individual equipment and unplanned demand triggered by a major equipment failure. Unplanned demands are described by a stochastic variable. Major equipment repairs are processed in priority in order to put the equipment back in service as quickly as possible. Remanufacturing operations for planned demands can be executed at different rates. Pellerin et al. [56] extend the model of Gharbi et al. [22] by presenting an analytical framework for controlling the execution rate of repair and remanufacturing activities.
Foul et al. [20] study the model of Foul and Tadj [21] in the case of a continuous review policy. Al-Babtain [2] specializes the paper by Foul et al. [20] to the case of logistic demand and return rates.
Li et al. [45] consider a production/remanufacturing planning problem with a fixed period review, under stochastic demands and returns. They consider set up costs for activities related to manufacturing and disposal to compute parameters in the structure.
Foul [19] incorporates an advertisement investment component to the models of Foul et al. [20] and Foul and Tadj [21].
Filho [15] assumes demand for new products and return of used-products are random variables with known probability distributions. Chance-constraints are explicitly introduced into the problems formulation by considering physical constraints on inventory and production variables. A Linear-Quadratic-Gaussian (LQG) problem with chance-constraints is provided as an example of sub-optimal production planning policy for a closed-loop system.
Alshamrani [3] considers the adaptive control of a reverse logistic inventory system with unknown deterioration and disposal rates. The updating rules of both deterioration and disposal rates are derived from the conditions of asymptotic stability of the reference model. The adaptive controlled system is modeled by a nonlinear system of differential equations.
Kenné et al. [35] deal with a closed-loop reverse logistics network with machines subject to random failures and repairs. Three types of inventories are involved in this network. The objective of this research is to propose a manufacturing/remanufacturing policy that would minimize the sum of the holding and backlog costs for manufacturing and remanufacturing products. The optimality conditions are developed using the optimal control theory based on stochastic dynamic programming and a computational algorithm, based on numerical methods, is used to solve the optimal control problem.
Minner and Kiesmüller [49] present a joint buy-back pricing and manufacturing/remanufacturing decision model at the operationsmarketing interface that allows for dynamic parameters, e.g. product life cycles and seasonal aspects. The model allows the identification of beneficial opportunities for buying back and storing used products for immediate and future recovery. A solution algorithm to find the cost-minimizing manufacturing and remanufacturing policies as well as buy-back strategies for used products is developed.
Feng et al. [14] introduce a recovery system for perishable items considering production and remanufacturing capacity constraints. The system consists of two inventories, one for serviceable items and the other for returned and recoverable items.
Motivation of our Research: As mentioned in the Introduction, we believe that the assumption that returned items are always as-good-as-new is not a realistic assumption. Therefore, the reverse logistics system we consider in this paper assumes that remanufactured items are not as-good-as new. We also assume that demands for manufactured and remanufactured items are different, competing in different market segments. Remanufactured items might be alternatives of newly produced ones in case of shortage but at a lower price. Manufactured products could also be good but more expensive alternatives to repaired ones if any stockout occurs. For this reason, in our model, we are keeping the remanufactured items separately, in a different stock, from the manufactured items. The third stock is thus used to sort returned items that will be scrapped from those that will be remanufactured. Many references pointed to applications of the coexistence of new and repaired products in the same market. As illustrative examples, we cite tire manufacturing and retreading industry (Préjan [59]), remanufacturing electrical motors (Klausner et al. [41]), photocopier remanufacturing at Fuji Xerox (Kerr and Ryan [36]).
One more motivation for our work comes from a reference that came to our attention as we were revising this paper. Jaber and El Saadany [29] note that "One of the inventory problems that has been of interest to researchers is the production, remanufacture (repair) and waste disposal model, where used items are collected and remanufactured to "as-good-as new" state. The available models in the literature assume that customers' demand is satisfied from newly manufactured (produced) items and from remanufactured (repaired) items. This may be true in few industries, but not in other industries where customers do not consider "new" ("manufactured") and "remanufactured" ("repaired") items as being interchangeable. " This is the very remark that led us to study the present model. New and remanufactured items are subject to different demands in our and in their model. However, while Jaber and El Saadany [29] keep new and remanufactured items in the same stock, our model assume different stocks for each type of item. The other difference with our model is that Jaber and El Saadany [29] assume constant demand and return rates and use optimization techniques, while we assume dynamic demand and return rates and use optimal control theory.
3. Model description. We make in this paper the following assumptions: 1. there is a single-product with two different qualities 2. remanufactured items are perceived by customers to be of lower quality than newly manufactured items 3. demands for produced and remanufactured items are known, dynamic and different 4. return rate for used items is known and dynamic 5. unlimited storage capacity is available 6. planning horizon is finite 7. manufactured and remanufactured items are subject to deterioration while on the shelves, with different deterioration rates 8. the firm adopts a continuous-review policy.
We therefore consider a production-inventory system with three stocks, as depicted in Figure 1 below. Items in the first stock are called manufactured. Items in the second stock are called remanufactured. Items in the third stock are called returned. We will use throughout the paper the subscript "m" to indicate the quantity corresponds to the first (manufactured) stock, the subscript "r" to indicate the quantity corresponds to the second (remanufactured) stock and the subscript "R" to indicate the quantity corresponds to the third (returned) stock. So, for example, I m (t) represents the inventory level at time t in the first stock, while I r (t) and I R (t) represent the inventory levels at time t in the second and third stocks, respectively. For convenience, we list below the notation used.
State variables: I m (t) Inventory level of manufactured items at time t I r (t) Inventory level of remanufactured items at time t I R (t) Inventory level of returned items at time t Control variables: Manufacturing rate at time t P r (t) Remanufacturing rate at time t P d (t) Disposal rate at time t Targets: I m (t) Goal inventory level of manufactured items at time t I r (t) Goal inventory level of remanufactured items at time t I R (t) Goal inventory level of returned items at time t P m (t) Goal manufacturing rate at time t P r (t) Goal remanufacturing rate at time t P d (t) Goal disposal rate at time t  Initial inventory level of remanufactured items I 0 R Initial inventory level of returned items Let T denote the length of the planning horizon and consider the first stock. The production of new items at rate P m (t) increases the inventory level, while demand for new items at rate D m (t) and deterioration at rate θ m > 0 decreases the inventory level. Similarly, in the second stock, the remanufacturing of returned items at rate P r (t) increases the inventory level, while demand for remanufactured items at rate D r (t) and deterioration at rate θ r > 0 decreases the inventory level. Finally, the inventory level in the third stock increases by the returned items at rate R(t) and decreases due to disposal of defective items at rate P d (t) and to replenishment of second stock at rate P r (t). Thus, the state and control variables of the reverselogistics system satisfy the following differential equations: We are assuming that the firm has set some targets for its three inventory levels, as well as a target manufacturing rate, a target remanufacturing rate, and a target disposal rate. All these targets also satisfy the differential equations (1). Therefore, if we introduce the shift operator ∆f (t) = f (t)−f (t), equations (1) can be rewritten as There are costs incurred when any of the variables deviates from its goal. The objective of the firm is to minimize these costs during the planning horizon. Thus, the objective function can be formulated as follows: (3) represents the penalty at the end of the planning horizon.
The problem of the reverse supply chain is to derive the optimal manufacturing, remanufacturing, and disposal rates, subject to the dynamics (2). To analyze this problem, it is more convenient to use a matrix notation. Let us introduce the control vector and the state vector respectively. Then, the differential equations (2) can now be written in matrix-vector form

PRODUCTION PLANNING IN A THREE-STOCK REVERSE-LOGISTICS SYSTEM 1049
Also, let x 2 A = x T Ax. The objective function (3) can be rewritten as where Finally, the problem is written in compact form as follows In the next section, we use the Pontryagin maximum principle to obtain the optimal manufacturing, remanufacturing, and disposal rates.

Model Analysis.
Optimal control theory is a branch of mathematics developed to find optimal ways to control a dynamic or stochastic system, either in discrete or continuous time. The main applications of optimal control theory may be in engineering, but it has also found successful applications in the various areas of management science and operations research, such as finance, economics, production and inventory, marketing, maintenance and replacement, and consumption of natural resources; see for example, Sethi and Thompson [64].

Theoretical derivations.
The necessary conditions that must be satisfied by any optimal control (x * , u * ) are called the maximum principle and were developed by Pontryagin et al. [57]. The principle states that there exists an Adjoint function λ(t) such that where the function H, called the Hamiltonian, is defined by The first of the above three equations is called the control equation; the second is called the adjoint equation; the third is called the state equation. The optimal state and control variables are obtained by solving this system of simultaneous equations.
In order to specialize the above results to our problem (P), introduce the adjoint vector Λ(t) = [λ 1 (t) λ 2 (t) λ 3 (t)] T , and write the Hamiltonian Note that K −1 exists since we are assuming K m > 0, The state equation H Λ (x, u, Λ, t) = dx(t) dt , with x(0) = x 0 is equivalent to (4). We start manipulating these three equations in order to reach the optimal control and state variables. Substituting (7) into (4) yields Let z(t) = [x(t) Λ(t)] T . Then, (8) and (9) yield the differential equation where It is well-known that the solution to (10) is given by where m i are the eigenvectors of matrix Φ and Y i are the corresponding eigenvectors. However, to determine the six constants c i , i = 1, · · · , 6, we have three initial conditions: I m (0) = I 0 m , I r (0) = I 0 r , I R (0) = I 0 R , and three terminal conditions: , which means that the form (11) is not suitable. Another way to write the solution to (10) is and denote by Y the matrix whose columns are the corresponding eigenvectors Y i , i = 1, · · · , 6. Then, where Y (:, i) = Y i is the ith column of Y and Y −1 (i, :) is the ith row of Y −1 . The products Y (:, i)Y −1 (i, :) are commonly called the constituent matrices. Now, to compute z(t) by (12), we need to determine z(0) = [x(0) Λ(0)] T . Recall that However, using the final value Λ(T ) = H T x(T ), we can find Λ(0) as follows.
From (12), we have at t = T , which can be rewritten as where Φ i (T ), i = 1, 2, 3, 4, are 3 × 3 diagonal block matrices. Using the terminal condition, we have from which, it follows Some simple computations yield From (12) and (17), we have and Note that the three components of the state vector x(t) thus determined in (19) are the state variables ∆I m (t), ∆I r (t) and ∆I R (t). So, using (19) and upon simplification, we obtain the optimal state variables. Also, the co-state vector λ(t) thus obtained in (20) is used in conjunction with Equation (7) to obtain the optimal control vector u(t) whose components are the control variables. We summarize our findings in the following Theorem 4.1. The optimal state variables are given by and the optimal control variables are given by 4.2. Numerical example and sensitivity analysis. The results obtained are easily implemented. For illustration, consider a reverse supply chain with three stocks as the one described in this paper. The parameters of the system are described in Table 1 below. Since the production system is subject to many impacting factors, we have focused on fixing the deterioration rates for manufactured item at 1% and the deterioration rate for remanufactured items at 2%, and keeping a certain level of initial inventories for the three stocks. For these data, the optimal objective function value is found to be J * = 145, 775.82. Figure 2 shows the variations of the optimal solutions. As can be seen, all the shifted variables converge to zero. 1. The deviations of inventories from targets tend to zero as early as period 6.
They show a matching with the X-axis at 10 which reflects the achievement of the target related to the inventory. 2. The manufacturing and remanufacturing rates evolve normally to meet their goal rates over time. The deviation from targeted manufacturing rate is almost zero starting period 5 (∆P m ), while the remanufacturing rate evolves slowly towards its goal without achieving a very small deviation within the horizon time. 3. The disposal rate decreases naturally to reach its lowest level compared to the disposal goal rate by the end of the horizon. The deviation is shown very small, and stable starting period 6. The figure shows also that our model, with regard to different goals, reacts very strongly at the beginning of the horizon. A trend of increase or decrease, depending on the variable under study, is observed with a slow-down by mid horizon since almost all values are close to their targets.
Sensitivity analysis: Although the solution exhibited in Theorem 4.1 is explicit, it is nevertheless impossible to do a sensitivity analysis analytically, as the expressions found are highly nonlinear. However, sensitivity analysis can be conducted numerically. For instance, suppose we are interested in the effect of the length of the planning horizon and of the deterioration rates on the optimal objective function value. The results are summarized in Figure 3 and Table 2 below. We see that the optimal objective function value decreases as the length of the planning horizon increases. Figure 3 shows that the objective function reaches its steady state starting from T = 10. The value of J * remains almost unchanged for the upcoming periods. From a managerial point of view, this important remark shows that the company has to focus on managing the production system over a reasonable period of time and there is no need to lengthen the horizon.
We also see that the optimal objective function value decreases as any or both deterioration rates increases. To understand this, we have depicted in Figure 4 the variations of the optimal solutions for θ m = θ r = 0.9, with all other parameters kept as in Table 1. Comparing Figures 2 and 4, we see in Figure 4 that five of the six variables, namely, ∆I m , ∆I r , ∆I R , ∆P m , and ∆P d reach 0 by approximately period 4. The contribution of these variables to the objective function is thus 0 very early in the planning horizon. Although ∆P r takes more time to reach 0, overall, the objective function when θ m = 0.01 and θ r = 0.02 is smaller than the objective function when θ m = θ r = 0.9. The effect of other parameters can be assessed in a similar way.

5.
Conclusion and future research directions. In this paper we have used optimal control theory to derive the explicit expression of the optimal production rate, the optimal remanufacturing rate, and the optimal disposal rate in a reverse supply chain. We have obtained these results in a very general setting: (a) The demand rate for manufactured and remanufactured items and the customer return rate are assumed to be dynamic. (b) New as well as remanufactured items are subject to deterioration. (c) The firm adopts a continuous review policy. (d) In contrast to most of the existing research which assumes that returned items are as-good-as-new, we have assumed that returned items are not as-good-as-new. Numerical examples and sensitivity analyses were conducted in order to show the performance of the obtained solutions. The theoretical and the numerical results allow gaining insights into operational issues and demonstrating the scope for improving stock control systems. Of course, as with any research work, this study is not without limitations. One of the assumptions we made is that the firm implements a continuous-review policy. However, that may not be the case. We are currently further investigating this model in the case when the firm implements a periodic-review policy. Another research direction would be to use a predictive control strategy where, given the current inventory level, the optimal production rates to be implemented at the beginning of each of the following periods over the control horizon, are determined. Model predictive control (or receding-horizon control) strategies have gained wide-spread acceptance in industry. It is also well-known that these models are interesting alternatives for real-time control of industrial processes. In the case where the deterioration rates are unknown, the self-tuning predictive control can be applied. The proposed control algorithm estimates online these coefficients and feeds the controller to take the optimal production decision. Note that our state equations are linear and thus linear model predictive control (LMPC), which is widely used both in academic and industrial fields, can be used. Nonlinear model predictive control (NMPC) can be used in case one of the state equations is nonlinear, for example, if the demand rate depends on the stock on hand. An instance of nonlinear demand rate that is widely used in the literature is D(I(t)) = αI(t) β , α > 0, 0 < β < 1. NMPC has gained significant interest over the past decade. Various NMPC strategies that lead to stability of the closed-loop have been developed in recent years and key questions such as the efficient solution of the occurring open-loop control problem have been extensively studied. The case combining unknown coefficients and a nonlinear relationship between the demand rate and the on-hand inventory yields a very complex, highly nonlinear process for which there is no simple mathematical model. The use of fuzzy control seems particularly well appropriate then. Fuzzy control is a technique that should be seen as an extension to existing control methods and not their replacement. It provides an extra set of tools which the control engineer has to learn how to use where it makes sense. Nonlinear and partially known systems that pose problems to conventional control techniques can be tackled using fuzzy control.