ABSTRACT
Chapter 9 is entitled Stochastic Control in NHMS. An important class of problems in various populations is concerned with the control of the expected population structure or the control of the relative expected population structure. Both structures are crucially important for the well being and the constructive productivity in whatever way these are defined of these systems. In section 9.2 we study the problem of maintainability of a desired expected relative population structure by controlling the input flow into the population. A series of theorems are provided and it is proved that the maintainable sets of expected relative structures are convex sets with vertices which are functions of the transition probability matrices among the internal states and the rate of increase of the total memberships in the system. Then the theory is evolved by solving the next logical problem, that is, how to attain a desirable expected population structure which is also maintainable. An algorithm is provided for solving this problem in an optimal way, which is a mixture of dynamic and integer programming techniques. This algorithm works for any cost function that we wish to optimize. In section 9.4 we formulate and study a new aspect of the classical problem of the asymptotic behavior for a NHMS. Our interest is transferred in finding the set of all possible limiting expected relative population structures provided that we control the limiting vector of input probabilities. A basic theorem is proved in which the set of all possible limiting expected relative population structures is given as the convex set with vertices being functions of the limiting transition probability matrix among the internal states. In section 9.5 we relax basic assumptions of the two basic theorems of the previous section by allowing the inherent non-homogeneous Markov chain to converge to a periodic homogeneous Markov chain. We provide a series of theorems that address the problems posed in section 9.4 for the periodic case. Illustrative examples are provided which help significantly to clarify and explain the results and which will be useful to the practitioner. In section 9.7 we study the problems posed in the previous two sections for the asymptotic variability in NHMS, that is for the limiting vector of means variances and covariances. In section 9.8 we modify one of the basic assumption, that is, of assuming that the sequence of transition probabilities are given in chronological order. We generalize by assuming that at each step in time a transition probability matrix is chosen from a finite set of matrices by a compromise non-homogeneous Markov chain and we call this process non-homogeneous Markov systems in a stochastic environment (S-NHMS). A series of theorems are proved which evaluate the expected population structure as a function of the basic parameters of the process S-NHMS. Then the Chapter evolves by studying the maintainability aspect in a stochastic environment. Next we study strategies for attaining a desirable maintainable structure in an optimal way optimizing a cost function. An algorithm is provided to this respect and an illustrative potential application is described for the practitioner. We conclude the Chapter with a section with important Research Notes.
