本論文將結合雙足機器人與機器學習，使用機器學習當中的監督式學習(Supervised Learning)方法為基礎，實現一種新創的雙足機器人系統建模方法。相較於傳統的解析方法，本方法因在其運算上的速度優勢，可望能夠提升機器人相關研究的運算效率。本論文主要討論一種通過深度神經網路(Deep Neural Network) 近似科列斯基分解形式(Cholesky-decompostion)雙足機器人系統的質量矩陣(Mass Matrix)的方法。在本論文中，針對一架在單足支撐階段具有5自由度的雙足機器人，基於Euler–Lagrange equation，建立其機器人系統在動態步行中的動力學模型，並利用解析方式推導其在單足支撐階段(Single Support Phase)的質量矩陣。其後利用所推導之解析質量矩陣訓練一個深度神經網路模型映射機器人系統的廣義座標與機器人系統的科列斯基分解形式質量矩陣。在經過 32 小時 50 分鐘的訓練後，能夠獲得一個在驗證階段的近似機器人系統之質量矩陣時有低於 5%的誤差的深度神經網路模型。為證明了此方法的優越性，本論文比較了三種由機器人在單足支撐階段的質量矩陣透過系統拘束方程式求得機器人在碰撞階段的拘束衝量的方法的計算速度。在拘束方程式中使用此種方法近似的質量矩陣進行運算，比使用解析質量矩陣快了5.375 毫秒。同時透過在實際機器人步態軌跡上的驗證，證明了此方法的實用性。

This thesis investigated a method that uses a deep neural network to encode the biped robot system. In detail, a supervised learning model is trained to approximate the mass matrix of the biped robot system in Cholesky-decomposed form. This method is expected to have higher computational efficiency than traditional system modeling methods. This thesis discusses a biped robot system with 5 degrees of freedom during the single support phase of walking. The ground truth of the supervised learning task is the biped robot system’s analytical mass matrix, derived from the Euler–Lagrange equation. A deep neural network is trained to map the generalized coordinates of the system to the mass matrix in Cholesky-decomposed form with reference to the analytical mass matrix. The training result shows that the neural network accurately encodes the biped robot system's mass matrix with an approximation error of less than 5% during the testing stage after 32 hours and 50 minutes of training. The discussion compares three ways to get the constraint impulse during the impact phase of bipedal walking via mass matrix. The comparative experiment result shows that computing with the Cholesky-decomposed mass matrix encoded by the deep neural network is 5.375 milliseconds faster than computing with the analytical mass matrix. This thesis also demonstrated the practicality of this method by verifying its approximation error on actual robot gait trajectories.

[1] I. Kato, S. Ohteru, H. Kobayashi, K. Shirai, and A. Uchiyama, “Information-power
machine with senses and limbs,” pp. 11–24, 1974.
[2] Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, and K. Fujimura,
“The intelligent asimo: System overview and integration,” in Proceedings of IEEE/
RSJ International Conference on Intelligent Robots and Systems, 2002, pp. 2478–
2483.
[3] G. Endo, J. Nakanishi, J. Morimoto, and G. Cheng, “Experimental studies of a
neural oscillator for biped locomotion with qrio,” in Proceedings of the IEEE International
Conference on Robotics and Automation, 2005, pp. 596–602.
[4] Y. Nakanishi, Y. Asano, T. Kozuki, et al., “Design concept of detail musculoskeletal
humanoid “kenshiro＂-toward a real human body musculoskeletal simulator,” in
Proceedings of 12th IEEE-RAS International Conference on Humanoid Robots,
2012, pp. 1–6.
[5] S. Zimmermann, R. Poranne, and S. Coros, “Go fetch!-dynamic grasps using boston
dynamics spot with external robotic arm,” in Proceedings of IEEE International
Conference on Robotics and Automation, 2021, pp. 4488–4494.
[6] G. Ficht and S. Behnke, “Bipedal humanoid hardware design: A technology review,”
Current Robotics Reports, vol. 2, no. 2, pp. 201–210, 2021.
[7] T. Sugihara, Y. Nakamura, and H. Inoue, “Real-time humanoid motion generation
through zmp manipulation based on inverted pendulum control,” in Proceedings of
IEEE International Conference on Robotics and Automation, 2002, pp. 1404–1409.
[8] X. Bajrami, A. Dermaku, A. Shala, and R. Likaj, “Kinematics and dynamics modelling
of the biped robot,” IFAC Proceedings Volumes, vol. 46, no. 8, pp. 69–73,
2013.
[9] M. W. Spong, S. Hutchinson, M. Vidyasagar, et al., Robot modeling and control.
Wiley New York, 2006, vol. 3.
[10] C. Hernández-Santos, E. Rodriguez-Leal, R. Soto, and J. Gordillo, “Kinematics and
dynamics of a new 16 dof humanoid biped robot with active toe joint,” International
Journal of Advanced Robotic Systems, vol. 9, no. 5, pp. 190–190, 2012.
[11] Q. Huang, K. Yokoi, S. Kajita, et al., “Planning walking patterns for a biped robot,”
IEEE Transactions on Robotics and Automation, vol. 17, no. 3, pp. 280–289, 2001.
[12] H. Yussof, M. Ohka, M. Yamano, and Y. Nasu, “Analysis of human-inspired biped
walk characteristics in a prototype humanoid robot for improvement of walking
speed,” in Proceedings of IEEE Second Asia International Conference on Modelling
& Simulation, 2008, pp. 564–569.
[13] M. Vukobratović and J. Stepanenko, “On the stability of anthropomorphic systems,”
Mathematical biosciences, vol. 15, no. 1-2, pp. 1–37, 1972.
[14] S. Kajita, O. Matsumoto, and M. Saigo, “Real-time 3d walking pattern generation
for a biped robot with telescopic legs,” in Proceedings of IEEE International
Conference on Robotics and Automation, 2001, pp. 2299–2306.
[15] H.-o. Lim, Y. Kaneshima, and A. Takanishi, “Online walking pattern generation
for biped humanoid robot with trunk,” in Proceedings of IEEE International Conference
on Robotics and Automation, 2002, pp. 3111–3116.
[16] K. Nishiwaki and S. Kagami, “Online walking control system for humanoids with
short cycle pattern generation,” The International Journal of Robotics Research,
vol. 28, no. 6, pp. 729–742, 2009.
[17] S. Kajita, F. Kanehiro, K. Kaneko, et al., “Biped walking pattern generation by
using preview control of zero-moment point,” in Proceedings of IEEE International
Conference on Robotics and Automation, 2003.
[18] T. Sato, S. Sakaino, and K. Ohnishi, “Real-time walking trajectory generation
method with three-mass models at constant body height for three-dimensional
biped robots,” IEEE Transactions on Industrial Electronics, vol. 58, no. 2, pp. 376–
383, 2010.
[19] S. Shimmyo, T. Sato, and K. Ohnishi, “Biped walking pattern generation by using
preview control based on three-mass model,” IEEE Transactions on Industrial
Electronics, vol. 60, no. 11, pp. 5137–5147, 2012.
[20] T. Katayama, T. Ohki, T. Inoue, and T. Kato, “Design of an optimal controller
for a discrete-time system subject to previewable demand,” International Journal
of Control, vol. 41, no. 3, pp. 677–699, 1985.
[21] T. Kwon and J. K. Hodgins, “Control systems for human running using an inverted
pendulum model and a reference motion capture sequence.,” in Proceedings
of Symposium on Computer Animation, 2010, pp. 129–138.
[22] S. Coros, P. Beaudoin, and M. Van de Panne, “Generalized biped walking control,”
ACM Transactions On Graphics, vol. 29, no. 4, pp. 1–9, 2010.
[23] R. Grzeszczuk, D. Terzopoulos, and G. Hinton, “Neuroanimator: Fast neural network
emulation and control of physics-based models,” in Proceedings of the 25th
annual conference on Computer graphics and interactive techniques, 1998, pp. 9–20.
[24] S. Kajita, F. Kanehiro, K. Kaneko, K. Yokoi, and H. Hirukawa, “The 3d linear
inverted pendulum mode: A simple modeling for a biped walking pattern generation,”
in Proceedings of IEEE/RSJ International Conference on Intelligent Robots
and Systems, 2001, pp. 239–246.
[25] C.-L. Shih and W. A. Gruver, “Control of a biped robot in the double-support
phase,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 22, no. 4,
pp. 729–735, 1992.
[26] H. Hemami and R. Farnsworth, “Postural and gait stability of a planar five link
biped by simulation,” IEEE Transactions on Automatic Control, vol. 22, no. 3,
pp. 452–458, 1977.
[27] C. A. D. Bezerra and D. E. Zampieri, “Biped robots: The state of art,” in Proceedings
of International Symposium on History of Machines and Mechanisms, 2004,
pp. 371–389.
[28] V. Kurtz, P. M. Wensing, and H. Lin, “Approximate simulation for template-based
whole-body control,” IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 558–
565, 2020.
[29] X. B. Peng, G. Berseth, K. Yin, and M. Van De Panne, “Deeploco: Dynamic locomotion
skills using hierarchical deep reinforcement learning,” ACM Transactions
on Graphics, vol. 36, no. 4, pp. 1–13, 2017.
[30] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy
optimization,” in Proceedings of International Conference on Machine Learning,
2015, pp. 1889–1897.
[31] Z. Wang, V. Bapst, N. Heess, et al., “Sample efficient actor-critic with experience
replay,” arXiv:1611.01224, 2016.
[32] I. Mordatch, K. Lowrey, and E. Todorov, “Ensemble-cio: Full-body dynamic motion
planning that transfers to physical humanoids,” in Proceedings of IEEE/RSJ
International Conference on Intelligent Robots and Systems, 2015, pp. 5307–5314.
[33] S. Auddy, S. Magg, and S. Wermter, “Hierarchical control for bipedal locomotion
using central pattern generators and neural networks,” in Proceedings of Joint
IEEE 9th International Conference on Development and Learning and Epigenetic
Robotics, 2019, pp. 13–18.
[34] S. Greydanus, M. Dzamba, and J. Yosinski, “Hamiltonian neural networks,” Advances
in Neural Information Processing Systems, vol. 32, 2019.
[35] M. Lutter, C. Ritter, and J. Peters, “Deep lagrangian networks: Using physics as
model prior for deep learning,” arXiv:1907.04490, 2019.
[36] K. Xu, D. Z. Huang, and E. Darve, “Learning constitutive relations using symmetric
positive definite neural networks,” Journal of Computational Physics, vol. 428,
pp. 110 072–110 072, 2021.
[37] G. Sutanto, A. Wang, Y. Lin, et al., “Encoding physical constraints in differentiable
Newton-Euler algorithm,” in Proceedings of Machine Learning Research,
2020, pp. 804–813.
[38] M. Lutter, K. Listmann, and J. Peters, “Deep lagrangian networks for end-toend
learning of energy-based control for under-actuated systems,” in Proceedings
of IEEE/RSJ International Conference on Intelligent Robots and Systems, 2019,
pp. 7718–7725.
[39] S.-M. Wang and R. Kikuuwe, “Neural Encoding of Mass Matrices of Articulated
Rigid-body Systems in Cholesky-Decomposed Form,” in Proceedings of IEEE International
Conference on Industrial Technology, 2022.
[40] T. Ando, T. Watari, and R. Kikuuwe, “Master-slave bipedal walking and semiautomatic
standing up of humanoid robots,” in Proceedings of IEEE/SICE International
Symposium on System Integration, 2020, pp. 360–365.
[41] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv:1711.05101,
2017.
[42] A. Paszke, S. Gross, S. Chintala, et al., “Automatic differentiation in pytorch,” in
Proceedings of 31st Conference on Neural Information Processing Systems, 2017.
[43] N. S. Keskar and R. Socher, “Improving generalization performance by switching
from adam to sgd,” arXiv:1712.07628, 2017.
[44] M. D. Zeiler, “Adadelta: An adaptive learning rate method,” arXiv:1212.5701, 2012.
[45] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv:
1412.6980, 2014.
[46] C. R. Harris, K. J. Millman, S. J. van der Walt, et al., “Array programming with
NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sep. 2020.