本文旨在研究一種用於求解非線性最佳控制問題的全空間拉格朗日-牛頓算法。這
類問題在計算科學和工程中的應用十分廣泛，例如軌道最佳化問題，工業機器人問題
等，這些問題也可以用數學公式轉化為等式約束優化問題。在此方法中，第一步是將
拉格朗日乘數引入目標函數從而得到拉格朗日函數，然後通過牛頓類方法找到一階必
要性最優條件（也稱為 KKT 條件）的臨界解，從而解決最佳化問題。牛頓型方法的
優點之一是收斂快速，前提是初始猜測足夠接近解。但是，通常很難獲得如此好的初
始猜測。當系統的非線性不平衡時，即使使用某些全局更新的技術，牛頓法也存在收
斂問題。拉格朗日-牛頓方法的缺點之一是需要構造 KKT 矩陣。KKT 系統的黑塞矩陣
的計算可能非常昂貴，例如使用有限差分近似法。為了提高牛頓方法的魯棒性，我們
提出了一種新的三級去耦預處理器。新算法的關鍵是在執行全局牛頓更新之前，在三
級解耦預處理階段，我們按順序校正拉格朗日乘數，控制變量和狀態變量。基於幾個
基準測試問題的數值結果表明，三級解耦預處理器有助於拉格朗日-牛頓算法的收斂，
並可以減少迭代次數。此外，我們報告了一系列比較研究，以研究採用全空間方法構
建黑塞矩陣的不同方法，包括解析方法，有限差分，自動微分和基於低秩更新的方法。
我們還通過數字顯示，全空間方法比 Matlab 工具箱中的優化器快數百倍，後者是使用
縮減空間的拉格朗日-牛頓方法實現的。

This thesis aims to study a full-space Lagrange-Newton algorithm for solving nonlinear optimal control problems. These problems represent a wide range of applications in computational science and engineering, such as trajectory optimization problems, industrial robots problems that can be also mathematically formulated as equality-constrained optimization problems.
In this method, the first step is to introduce the Lagrange multiplier into the objective function and then solve the optimization problem by finding the critical solution of the first-order necessary optimality condition (also known as the KKT condition) by the Newton-type method. One of the advantages of the Newton-type method is fast convergence provided that an initial guess is close to the solution. However, such a good initial guess is not easy to obtain. And Newton's method suffers from the convergence issue when the nonlinearity of the system is not well balanced even some globalization technique is used. One of the drawbacks of the Lagrange-Newton method is that the KKT matrix needs to construct. The computation of the Hessian matrix of the KKT system could be expensive for example using finite-difference approximation. To improve the robustness of Newton's method, we propose a new three-stage decoupling preconditioner. The key point of the new proposed algorithm is that before performing the global Newton update, in the three-stage decoupling preconditioning phase, we correct the Lagrange multiplier, control variables, state variables in order.
Our numerical results based on several benchmark test problems show that the three-stage decoupling preconditioner helpful for the convergence of the Lagrange-Newton algorithm and can reduce the number of iterations. Besides, we report a series of comparative study to investigate the full-space method with different approaches for constructing the Hessian matrix, including analytical approach, finite differences, automatic differentiation, and the low-rank update based method. We also show numerically that the full-space method is about a few hundred times faster than an optimizer in Matlab toolbox, which is implemented using the reduced-space Lagrange-Newton method.

[1] J.T. Betts. Survey of numerical methods for trajectory optimization. J. Guid. Contr.
Dynam., 21:193–207, 1998.
[2] F. L. Chernousko. Optimization in control of robots. Computational Optimal Control,
1994.
[3] D. Gaur and M. S. Prasad. Orbit transfer using optimal control theory. In 2019 3rd
International Conference on Recent Developments in Control, Automation Power
Engineering (RDCAPE), 27–31, 2019.
[4] D. L. Black and S. V. Gunn. Space nuclear propulsion. In Robert A. Meyers,
editor, Encyclopedia of Physical Science and Technology (Third Edition), 555 – 575.
Academic Press, New York, third edition edition, 2003.
[5] F. S. Forbes and P. A. Van Splinter. Liquid rocket propellants. In Robert A. Meyers,
editor, Encyclopedia of Physical Science and Technology (Third Edition), 741 – 777.
Academic Press, New York, third edition edition, 2003.
[6] R. G. Jahn and E. Y. Choueiri. Electric propulsion. In Robert A. Meyers, editor, En-
cyclopedia of Physical Science and Technology (Third Edition), 125 – 141. Academic
Press, New York, third edition edition, 2003.
[7] D. Altman. Rocket motors, hybrid. In Robert A. Meyers, editor, Encyclopedia of
Physical Science and Technology (Third Edition), 303 – 321. Academic Press, New
York, third edition edition, 2003.
[8] P. W. Keaton. Low-thrust rocket trajectories. Technical report, Los Alamos National
Lab., NM (USA), 1987.
[9] A.V. Rao. Trajectory optimization: A survey. In Optimization and Optimal Control
in Automotive Systems, 3–21. Springer, 2014.
[10] O. von Stryk and R. Bulirsch. Direct and indirect methods for trajectory optimiza-
tion. Annals Oper. Res., 37:357–373, 1992.
[11] J.T. Betts. Practical Methods for Optimal Control and Estimation Using Nonlinear
Programming. SIAM, Philadelphia, 2nd edition, 2010.
[12] F. Fahroo and I.M. Ross. Costate estimation by a Legendre pseudospectral method.
J. Guid., Contr. Dynam, 24:270–277, 2001.
[13] D.G. Hull. Optimal Control Theory for Applications. Springer-Verlag, New York,
2003.
[14] P. Williams. Jacobi pseudospectral method for solving optimal control problems. J.
Guid. Contr. Dynam., 27:293–297, 2004.
[15] J. Nocedal and S.J. Wright. Numerical Optimization. Springer-Verlag, New York,
2006.
[16] M.R. Sentinella and L. Casalino. Cooperative evolutionary algorithm for space tra-
jectory optimization. Celestial Mech. Dyn. Astron., 105:211–227, 2009.
[17] S. Subchan and R. Żbikowski. Computational Optimal Control: Tools and Practice.
John Wiley & Sons, 2009.
[18] A. Wuerl, T. Crain, and E. Braden. Genetic algorithm and calculus of variations-
based trajectory optimization technique. J. Spacecraft Rockets, 40:882–888, 2003.
[19] M. Pontani. Particle swarm optimization of ascent trajectories of multistage launch
vehicles. Acta Astronaut., 94:852–864, 2014.
[20] P. Lu and M.A. Khan. Nonsmooth trajectory optimization-an approach using con-
tinuous simulated annealing. J. Guid. Contr, Dynam., 17:685–691, 1994.
[21] Y.-S. Lo. A full space lagrange-newton-krylov algorithm for minimum time trajectory
optimization. Master’s thesis, Natianal Central University, 2012.
[22] J.E. Dennis Jr. and R.B. Schnabel. Numerical Methods for Unconstrained Optimiza-
tion and Nonlinear Equations. SIAM, Philadelphia, 1996.
[23] Stanley C Eisenstat and Homer F Walker. Globally convergent inexact newton meth-
ods. SIAM Journal on Optimization, 4(2):393–422, 1994.
[24] H. Yang, F.-N. Hwang, and X-.C. Cai. Nonlinear preconditioning techniques for full-
space Lagrange-Newton solution of PDE-constrained optimization problems. SIAM
J. Sci. Comput., 2016.