Object library of algorithms for dynamic optimization problems: benchmarking SQP and nonlinear interior point methods

Błaszczyk, Jacek ; Karbowski, Andrzej ; Malinowski, Krzysztof

International Journal of Applied Mathematics and Computer Science, Tome 17 (2007), p. 515-537 / Harvested from The Polish Digital Mathematics Library

Résumé

The main purpose of this paper is to describe the design, implementation and possibilities of our object-oriented library of algorithms for dynamic optimization problems. We briefly present library classes for the formulation and manipulation of dynamic optimization problems, and give a general survey of solver classes for unconstrained and constrained optimization. We also demonstrate methods of derivative evaluation that we used, in particular automatic differentiation. Further, we briefly formulate and characterize the class of problems solved by our optimization classes. The solution of dynamic optimization problems with general constraints is performed by transformation into structured large-scale nonlinear programming problems and applying methods for nonlinear optimization. Two main algorithms of solvers for constrained dynamic optimization are presented in detail: the sequential quadratic programming (SQP) exploring the multistage structure of the dynamic optimization problem during the solution of a sequence of quadratic subproblems, and the nonlinear interior-point method implemented in a general-purpose large-scale optimizer IPOPT. At the end, we include a typical numerical example of the application of the constrained solvers to a large-scale discrete-time optimal control problem and we use the performance profiles methodology to compare the efficiency and robustness of different solvers or different options of the same solver. In conclusions, we summarize our experience gathered during the library development.

Détail

Publié le : 2007-01-01

Zbl 1234.90022

EUDML-ID : urn:eudml:doc:207856

Bibliographie

[000] Arnold E. and Puta H. (1994): An SQP-type solution method for constrained discrete-time optimal control problems. In: Computational Optimal Control (R. Bulirsch and D. Kraft, Eds.), Birkhauser Verlag, Basel, Switzerland, pp. 127-136. | Zbl 0811.49025

[001] Arnold E., Tatjewski P. and Wolochowicz P. (1994): Two methods for large-scale nonlinear optimization and their comparison on a case study of hydropower optimization. Journal of Optimization Theory and Applications, Vol. 81, No. 2, pp. 221-248. | Zbl 0819.90091

[002] Benson H. Y., Shanno D. F. and Vanderbei R. J. (2001): Interior-point methods for nonconvex nonlinear programming: Filter methods and merit functions. Technical Report ORFE-00-06, Operations Research and Financial Engineering, Princeton University. Available athttp://www.princeton.edu/rvdb/tex/loqo4/loqo4_4.pdf | Zbl 1022.90017

[003] Benson H. Y., Shanno D. F. and Vanderbei R. J. (2002): A comparative study of large-scale nonlinear optimization algorithms. Technical Report ORFE-01-04, Operations Research and Financial Engineering, Princeton University. Available athttp://www.princeton.edu/rvdb/tex/loqo5/loqo5_5.pdf | Zbl 1066.90138

[004] Bertsekas D. P. (1982): Projected Newton methods for optimization problems with simple constraints. SIAM Journal on Control and Optimization, Vol. 20, No. 2, pp. 221-246. | Zbl 0507.49018

[005] Błaszczyk J., Karbowski A. and Malinowski K. (2002a): Object library of algorithms for unconstrained dynamic optimization problems. Proceedings of the 14-th National Conference on Automatic Control (KKA), Vol. I, Zielona Góra, Poland, pp. 451-456. | Zbl 1234.90022

[006] Błaszczyk J., Karbowski A. and Malinowski K. (2002b): Object library of algorithms for dynamic optimization problems without constraints or with simple bounds on control. Proceedings of the 8th IEEE International Conference on Methods and Models in Automation and Robotics, Vol. 1, Szczecin, Poland, pp. 257-262.

[007] Błaszczyk J., Karbowski A. and Malinowski K. (2003): Object library of algorithms for dynamic optimization problems with general constraints. Proceedings of the 9th IEEE International Conference on Methods and Models in Automation and Robotics, Vol. 1, Międzyzdroje, Poland, pp. 271-276.

[008] Bryson A. E. (1998): Dynamic Optimization. Menlo Park CA: Addison-Wesley, p. 550.

[009] Byrd R. H., Hribar M. E. and J. Nocedal (1999): An interior point algorithm for large scale nonlinear programming. SIAM Journal on Optimization, Vol. 9, No. 4, pp. 877-900. | Zbl 0957.65057

[010] Byrd R. H., Gilbert J. Ch. and Nocedal J. (2000): A trust region method based on interior point techniques for nonlinear programming. Mathematical Programming, Vol. 89, pp. 149-185. | Zbl 1033.90152

[011] Chamberlain R. M., Powell M.J.D., Lemarechal C. and Pedersen H.C. (1982): The watchdog technique for forcing convergence in algorithms for constrained optimization. Mathematical Programming Study, Vol. 16, pp. 1-17. | Zbl 0477.90072

[012] Dolan E. D. and More J. J. (2002): Benchmarking optimization software with performance profiles. Mathematical Programming, Vol. 91, No. 2, pp. 201-213. | Zbl 1049.90004

[013] Fiacco A. V. and McCormick G. P. (1968): Nonlinear Programming: Sequential Unconstrained Minimization Techniques. John Wiley and Sons, New York/London. | Zbl 0193.18805

[014] Findeisen W., Szymanowski J. and Wierzbicki A. (1980): Theory and Computational Methods of Optimization. Polish Scientific Publishers, Warsaw (in Polish). | Zbl 0455.49001

[015] Fletcher R. (1987): Practical Methods of Optimization. John Wiley and Sons, New York, NY, USA. | Zbl 0905.65002

[016] Fletcher R. (1995): An optimal positive definite update for sparse Hessian matrices. SIAM Journal on Optimization, Vol. 5, No. 1, pp. 192-218. | Zbl 0824.65038

[017] Fletcher R. and Leyffer S. (2002): Nonlinear programming without a penalty function. Mathematical Programming, Vol. 91, No. 2, pp. 239-269. | Zbl 1049.90088

[018] Fletcher R., Leyffer S. and Toint Ph. L. (2006): A brief history of filter methods. Technical Report ANL/MCS-P1372-0906, Mathematics and Computer Science Division, Argonne National Laboratory. Available at http://www.optimization-online.org/DB_FILE/2006/10/1489.pdf

[019] Franke R. (1994): Anwendung von Interior-Point-Methoden zur Losung zediskreter Optimalsteuerungsprobleme. M. S. thesis, Techniche Universitat Ilmenau, Fakultat fur Informatik und Automatsierung, Institut fur Automatisierungs- und Systemtechnik Fachgebiet Dynamik und Simulation okologischer Systeme, Ilmenau, Germany, (in German).

[020] Franke R. (1998): OMUSES - A tool for the optimization of multistage systems and HQP - A solver for sparse nonlinear optimization. Version 1. 5. Department of Automation and Systems Engineering, Technical University of Ilmenau, Germany. Available at ftp://ftp.systemtechnik.tu-ilmenau.de/pub/reports/omuses.ps.gz

[021] Franke R. and Arnold E. (1997): Applying new numerical algorithms to the solution of discrete-time optimal control problems. In: Computer-Intensive Methods in Control and Signal Processing: The Curse of Dimensionaly, (Warwick K. and Karny M., Eds.), Birkhauser Verlag, Basel, Switzerland, pp. 105-118.

[022] The solver Omuses/HQP for structured large-scale constrained optimization: Algorithm, implementation, and example application. Proceedings of the 6-th SIAM Conference on Optimization, Atlanta.

[023] Goldfarb D. and Idnani A. (1983): A numerically stable dual method for solving strictly convex quadratic programs. Mathematical Programming, Vol. 27, No. 1, pp. 1-33. | Zbl 0537.90081

[024] Gondzio J. (1994): Multiple centraly corrections in a primal-dual method for linear programming. Technical Report. 20, Department of Management Studies, University of Geneva, Geneva, Switzerland. Available at http://www.maths.ed.ac.uk/gondzio/software/correctors.ps

[025] Griewank A., Juedes D., Mev H., Utke J., Vogel O. and Walther A. (1999): ADOL-C: A package for the automatic differentiation of algorithms written in C/C++, Version 1. 8. 2, March 1999. Available at http://www.math.tu-dresden.de/adol-c/

[026] Karmarkar N. (1984): A new polynomial-time algorithm for linear programming. Combinatorica, Vol. 4, No. 4, pp. 373-395. | Zbl 0557.90065

[027] Luus R. (2000): Iterative Dynamic Programming. CRC Press, Inc., Boca Raton, FL, USA. | Zbl 1070.49001

[028] Mehrotra S. (1992): On the implementation of a primal-dual interior point method. SIAM Journal on Optimization, Vol. 2, No. 4, pp. 575-601. | Zbl 0773.90047

[029] Misc J. -P. (2003): Large scale nonconvex optimization. SIAM's SIAG/OPT Newetter Views-and-News, Vol. 14, No. 1, pp. 1-25. Available at http://fewcal.uvt.nl/sturm/siagopt/vn14_1.pdf

[030] Morales J. L., Nocedal J., Waltz R. A., Liu G. and Goux J. -P. (2001): Assessing the potential of interior methods for nonlinear optimization. Technical Report OTC 2001/4, Optimization Technology Center of Northwestern University. Available at http://www.ece.northwestern.edu/morales/PSfiles/assess.ps | Zbl 1062.65063

[031] Nocedal J. and Wright S. J. (1999): Numerical Optimization. Berlin: Springer-Verlag. | Zbl 0930.65067

[032] De O. Pantoja J.F.A. (1988): Differential dynamic programming and Newton's method. International Journal of Control, Vol. 47, No. 5, pp. 1539-1553. | Zbl 0669.49014

[033] Powell M. J. D. (1978): A fast algorithm for nonlinearly constrained optimization calculations. In: Numerical Analysis, Dundee (G. A. Watson, Ed.), Dundee: Springer-Verlag, pp. 144-157. | Zbl 0374.65032

[034] Powell M. J. D. (1985): On the quadratic programming algorithm of Goldfarb and Idnani. Mathematical Programming Study, Vol. 25, pp. 46-61. | Zbl 0584.90069

[035] Salahi M., Peng J. and Terlaky T. (2005): On Mehrotra-type predictor-corrector algorithms. Technical report, Advanced Optimization Lab, Department of Computing and Software, McMaster University, Hamilton, Ontario, Canada. Available at http://www.optimization-online.org/DB_FILE/2005/03/1104.pdf | Zbl 1165.90569

[036] Schittkowski K. (1980): Nonlinear Programming Codes: Information, Tests, Performance. Berlin: Springer-Verlag. | Zbl 0435.90063

[037] Schittkowski K. (1983): On the convergence of a sequential quadratic programming method with an augmented Lagrangian line search function. Mathematishe Operations Forschung und Statistik, Ser. Optimization, Vol. 14, No. 2, pp. 197-216. | Zbl 0523.90075

[038] Schwartz A. and Polak E. (1997): Family of projected descent methods for optimization problems with simple bounds. Journal of Optimization Theory and Applications, Vol. 92, No. 1, pp. 1-31. | Zbl 0886.90140

[039] Tenny M. J., Wright S. J. and Rawlings J. B. (2002): Nonlinear model predictive control via feasibility-perturbed sequential quadratic programming. Technical Report TWMCC-2002-02, Texas-Wisconsin Modeling and Control Consortium. Available at http://jbrwww.che.wisc.edu/jbr-group/tech-reports/twmcc-2002-02.pdf

[040] Tits A. L., Wachter A., Bakhtiari S., Urban T. J. and Lawrence C. T. (2002): A primal-dual interior-point method for nonlinear programming with strong global and local convergence properties. Technical Report TR 2002-29, Institute for Systems Research, University of Maryland. Available at http://www.ee.umd.edu/andre/pdiprev.ps | Zbl 1075.90078

[041] Ulbrich M., Ulbrich S. and Vicente L. N. (2004): A globally convergent primal-dual interior-point filter method for nonlinear programming. Mathematical Programming, Vol. 100, No. 2, pp. 379-410. | Zbl 1070.90110

[042] Vanderbei R. J. and Shanno D. F. (1997): An interior-point algorithm for non-convex nonlinear programming. Technical Report SOR-97-21, Statistics and Operations Research, Princeton University. Available at http://www.sor.princeton.edu/rvdb/ps/nonlin.ps.gz

[043] Wachter A. (2002): An Interior Point Algorithm forLarge-Scale Nonlinear Optimization with Applications in Process Engineering. Ph. D. dissertation, Department of Chemical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA. Available at http://www.research.ibm.com/people/a/andreasw/papers/thesis.pdf

[044] Wachter A. and Biegler L. T. (2000): Failure of global convergence for a class of interior point methods for nonlinear programming. Mathematical Programming, Vol. 88, No. 3, pp. 565-574. | Zbl 0963.65063

[045] Wachter A. and Biegler L. T. (2005): Line search filter methods for nonlinear programming: Motivation and global convergence. SIAM Journal on Optimization, Vol. 16, No. 1, pp. 1-31. | Zbl 1114.90128

[046] Wachter A. and Biegler L. T. (2006): On the implementation of a primal-dual interior-point filter line-search algorithm for large-scale nonlinear programming. Mathematical Programming, Vol. 106, No. 1, pp. 25-57. | Zbl 1134.90542

[047] Waltz R. A. and Plantenga T. (2006): KNITRO 5. 0 User's Manual. Available at http://www.ziena.com/docs/knroman.pdf.

[048] Wierzbicki A. (1984): Models and Sensitivy of Control Systems. Elsevier, Amsterdam.

[049] Wright S. J. (1993): Interior point methods for optimal control of discrete time systems. Journal of Optimization Theory and Applications, Vol. 77, No. 1, pp. 161-187. | Zbl 0796.49034

[050] Wright S. J. (1997): Primal-Dual Interior-Point Methods. SIAM, Philadelphia, PA. | Zbl 0863.65031

[051] Yakowz S. and Rutherford B. (1984): Computational aspects of discrete-time optimal control. Applied Mathematics and Computation, Vol. 15, No. 1, pp. 29-45 | Zbl 0554.90098