Abstract
In this paper we present a practical method for robust parameter estimation in dynamic systems. In our study we follow the very successful approach for solving optimization problems in dynamic systems, namely the boundary value problem (BVP) approach. The suggested method combines multiple shooting for parameterizing dynamics, a flexible realization of the BVP principle, with a fast Gauss-Newton algorithm for solving the resulting constrained l 1 problem. We give an overview of the theoretical background as well as the details of a numerical implementation. We discuss why the Gauss-Newton algorithm, which is known to perform well mainly on well-conditioned problems, is appropriate for parameter estimation problems, while quasi-Newton methods have only limited use for parameter estimation. The method is implemented on the basis of the direct multiple shooting method as implemented in PARFIT, thus inheriting all basic properties of PARFIT such as numerical stability, reliability and efficiency. The new code has been successfully applied to real-life parameter estimation problems in enzyme and chemical kinetics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
E. Baake, M. Baake, H. G. Bock, and K. Briggs, “Fitting ordinary differential equations to chaotic data,” Physical Review A vol. 45, 1992.
R. H. Bartels and A. R. Conn, “An approach to nonlinear l 1 data fitting,” in Numerical Analysis Cocoyoc 1981,J. P. Hennart (Ed.), Lecture Notes in Mathematics, Springer Verlag, 1982, vol. 909, pp. 48–58.
R. H. Bartels, A. R. Conn, and J. W. Sinclair, “Minimization techniques for piecewise differentiable functions: The l 1 solution to an overdetermined linear system,” SIAM Journal on Numerical Analysis vol. 15, pp. 224–240, 1978.
I. Bauer, S. Körkel, H. G. Bock, and J. P. Schlöder, “Optimale nichtlineare Versuchsplanung f ür dynamische Systeme aus der chemischen Reaktionskinetik,” Optimale Versuchsplanung f ür nichtlineare Prozesse, schriftliche Projektpräsentation zum BMBF-Verbundvorhaben, DECHEMA e.V.: Frankfurt am Main, 1998.
I. Bauer, H. G. Bock, S. Körkel, and J. P. Schlöder, “Numerical methods for optimum experimental design in DAE systems,” Journal of Computational and Applied Mathematics vol. 120, pp. 1–25, 2000.
D. Birkes and Y. Dodge, Alternative Methods of Regression, John Wiley and Sons, 1993.
H. G. Bock, “Numerical treatment of inverse problems in chemical reaction kinetics,” in Modelling of Chemical Reaction Systems, K.H. Ebert, P. Deuflhard, and W. Jäger, (Eds.), Springer Series in Chemical Physics, Springer Verlag, 1981, vol. 18, pp. 102–125.
H. G. Bock, Randwertproblemmethoden zur Parameteridentifizierung in Systemen nichtlinearer Differentialgle-ichungen. Bonner Mathematische Schriften: Bonn, 1987, vol. 187.
H. G. Bock, E. Eich, and J. P. Schlöder, “Numerical solution of constrained least squares boundary value problems in differential-algebraic equations,” in Numerical Treatment of Differential Equations, K. Strehmel (Ed.), Teubner, 1988.
H. G. Bock, J. Kallrath, and J. P. Schlöder, “Least squares parameter estimation in chaotic differential equations,” Celestial Mechanics and Dynamical Astronomy,vol. 56, 1993.
H. G. Bock, E. A. Kostina, and J. P. Schlöder, “On the role of natural level functions to achieve global convergence for damped Newton methods,” in System Modelling and Optimization. Methods, Theory and Applications, M. J. D. Powell and S. Scholtes (Eds.), Kluwer, 2000, pp. 51–74.
A. Bommarius, M. Estler, A. Kluge, H. Werner, H. Vollmer, H. G. Bock, J. P. Schlöder, and E. Kostina, “Method to determine the process stability of enzymes,” Patent Application EP 1 067 198 A1, European Patent Office, Patentblatt,2,2001.
R. Boscovic, Theoria Philosophiae Naturalis,Vienna, 1758.
G. E. P. Box and G. C. Tiao, Bayesian Inference in Statistical Analysis,Wiley Classics Library, John Wiley and Sons, 1992.
T. F. Coleman and Y. Li, “A globally and quadratically convergent affine scaling method for linear l 1 problems,” Mathematical Programming vol. 56, pp. 189–222, 1992.
A. R. Conn and N. I. M. Gould, Trust-Region Methods, MPS/SIAM Series on Optimization, 2000.
P. Deuflhard, “A modified Newton method for the solution of ill-conditioned systems of nonlinear equations with applications to multiple shooting,” Numerical Mathematics,vol. 22, pp. 289–315, 1974.
R. Fletcher, Practical Methods of Optimization. 2nd edition, John Wiley and Sons, 1987.
R. Gabasov, F. M. Kirillova, and E. A. Kostina, “An adaptive method of solving l 1 extremal problems,” Zhurnal vychislitelnoy matematiki i matematicheskoy fiziki,vol. 38, no. 6, pp. 912–917, 1998 (in Russian, trans. to English: Computational Mathematics and Mathematical Physics, vol. 38, no. 9, pp. 1400–1411, 1998).
C. F. Gauss, Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium. F. Perthes and J. H. Besser, Hamburg, 1809.
C. F. Gauss, Theory of Combinations of Observations Least Subject to Errors, original with translation. SIAM, 1995.
J. Hald and K. Madsen, “Combined linear programming and quasi-Newton methods for non-linear L 1 optimization,” SIAM Journal on Numerical Analysis vol. 22, pp. 65–80, 1985.
P. J. Huber, Robust Statistics, John Wiley and Sons, 1981.
S. Körkel, Numerische Methoden f ür Optimale Versuchsplanungsprobleme bei nichtlinearen DAE-Modellen. Ph.D. thesis, Universität Heidelberg, available at http://www.ub.uni-heidelberg.de/archiv/2980, 2002.
E. A. Kostina, “The long step rule in the bounded-variable dual simplex method: Numerical experiments,” Mathematical Methods of Operations Research vol. 55, Issue 3, pp. 413–429, 2002.
E. A. Kostina and S. V. Prischepova, “A new algorithm for minimax and L 1-norm optimization,” Optimization (Journal on Mathematical Programming and Operations Research) vol. 44, pp. 263–289, 1998.
K. Madsen and H. B. Nielsen, “Afinite smoothing algorithm for linear l 1 estimation,” SIAM Journal on Optimization vol. 3, pp. 223–235, 1993.
R. A. McLean and G. A. Watson, “Numerical methods for nonlinear discrete l 1 approximation problems,” Inter-national Series for Numerical Mathematics vol. 59, pp. 169–183, 1980.
W. Murray and M. L. Overton, “ A projected Lagrangian algorithm for nonlinear l 1 optimization,” SIAM Journal on Scientific and Statistical Computing vol. 2, no. 2, pp. 207–224, 1981.
J. Nocedal and S. J. Wright, Numerical Optimization, Springer Verlag, 1999.
M. R. Osborne and G. A. Watson, “On an algorithm for discrete nonlinear L 1 approximation,” The Computer Journal vol. 14, pp. 184–188, 1971.
M. R. Osborne and G. A. Watson, “Nonlinear approximation problems in vector norm,” in Numerical Analysis Dundee 1977,G.A. Watson, (Ed.), Lecture Notes in Mathematics vol. 630, pp. 48–58, Springer Verlag, 1978.
V. H. Schulz, H. G. Bock, and M. C. Steinbach, “Exploiting invariants in the numerical solution of multipoint boundary value problems for DAE,” SIAM Journal on Scientific Computing vol. 19, no. 2, pp. 440–467, 1998.
G. A. Watson, “Towards more robust methods of data fitting,” IMA Bulletin vol. 21, pp. 187–192, 1985.
G. A. Watson, “Choice of norms for data fitting and function approximation,” Acta Numerica vol. 7, pp. 337–377, 1998.
P. Zwaga, Private Communication, 1977.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Kostina, E. Robust Parameter Estimation in Dynamic Systems. Optimization and Engineering 5, 461–484 (2004). https://doi.org/10.1023/B:OPTE.0000042035.67293.92
Issue Date:
DOI: https://doi.org/10.1023/B:OPTE.0000042035.67293.92