本論文的目的為藉由導入機器視覺與深度學習網路，控制六軸機械手臂與人臉保持固定距離且能對準額頭中央，並在有限範圍內主動跟隨人臉移動。此追蹤系統可應用於人與機器人的互動任務，在本論文中則是以考慮該手臂握持手機或平板與人臉保持上述距離與方向，即使人臉有些許移動或轉動，仍然可以追蹤人臉並維持設定好的距離與方向，方便使用者可以不用自己手持也能看手機或平板。為達成此目的，機械手臂端需經由人臉偵測與辨識臉部特徵，計算人臉與機械手臂末端的相對位置，並由逆向運動學完成人臉的追蹤任務。
在人臉偵測與臉部特徵辨識部分，此二功能皆透過深度學習達成，並且完成(1)取得面部特徵在影像中的位置；(2)計算特徵點與攝影機的相對位置。另外機械手臂的運動控制亦需以下程序。(1)建置虛擬環境；(2)計算模型的轉換矩陣；(3)求得目標點的座標，並以逆運動學控制機器手臂到目標點。綜合上述條件，便可在機械手臂自身機構限制內完成追蹤臉部任務。除了影像辨識與機械手臂的控制外，本研究也添加了手機應用程式供使用者微調機械手臂的位置，可透過手動或語音輸入來調整機械手臂進行上下左右偏移，亦可於手機上同步顯示當前機械手臂上的攝影機所拍攝的影像。本研究亦將此技術應用於目前疫情期間(2020 COVID-19)的自動額溫量測，當人靠近機械手臂前，便能偵測該人的臉部位置，並將測溫模組對準額頭為使用者量測額溫，若有發燒將會出現警示，如此便可免除人工量測的負擔。
本研究在Linux環境下使用機器人作業系統(Robot Operating System, 以下皆簡稱ROS)，由於ROS能共享資訊的特點，便可透過節點與節點間訊息的發送與接收，完成系統中不同軟硬體間的資訊溝通。本研究藉由ROS完成了筆記型電腦、六軸機械手臂、攝影機以及手機程式間資訊的傳遞與彙整，實現軟硬體協同應用。

The purpose of this thesis is to control the six-axis robotic arm to keep a fixed distance from the human face and aim at the center of the forehead with the aids of computer vision and deep learning. Furthermore, when the human face has a little movement, the robot can also follow it within the limited range. This tracking system can be applied to interactive tasks between humans and robots. In this thesis, it is considered that the arm holding a mobile phone or tablet maintains the above distance and direction to face the user’s face. Even if the face moves or rotates a little, it can still track the face with the set distance and direction, so that the user can watch the mobile phone or tablet without holding them. To achieve this, the robotic arm needs to detect the human face, recognize the facial features, and calculate the related positions between the face and the terminal of robotic arm. Finally, the robotic arm will implement the face tracking by using inverse kinematics.
In face detection and facial features recognition, both are achieved by deep learning technique, after that we also (1) obtain the position of facial features points in the image and (2) calculate the related positions between the facial features and the camera. In addition, the movement control of the robotic arm also needs the following process. (1) building a virtual environment; (2) calculating the transfer matrix of the model; (3) finding the coordinates of the target point and using inverse kinematics to control the robotic arm moving to the target point. This study also added a function in the mobile phone to fine-tune the pose of the robotic arm. The fine tune operation can be implemented by manual input or voice input. The mobile phone can also display the images taken by the current camera on the robot arm simultaneously. This technology can also be applied to the automatic forehead temperature measurement during the current epidemic situation (2020 COVID-19).
In this thesis, Robot Operating System (ROS) is used in the Linux operating system and completed the transmission and integration of information between laptop, robotic arm, camera, and mobile phone.
Keywords: Face detection, Facial landmark, 6 DOF robotic arm, Kinematics, Robot operating system, Coordinate transformation, Face tracking.

[1] TCS 臺灣傳播調查資料庫-台灣民眾手機使用習慣調查
http://www.crctaiwan.nctu.edu.tw/epaper/%E7%AC%AC90%E6%9C%9F2
0190815.htm，2020 年 5 月。
[2] goron smart plus
https://search.rakuten.co.jp/search/mall/goron+smart+plus/ ， 2020 年 5
月。
[3] Second generation Italian smart bed
https://www.hi-interiors.com/hi-bed/，2020 年 5 月。
[4] Gaming beds
https://www.bauhutte.jp/bauhutte-life/making-gaming-desk/ ， 2020 年 5
月。
[5] R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies
for accurate object detection and semantic segmentation," Proc. IEEE
Conference Computer Vision and Pattern Recognition, Columbus, Jun. 2014,
pp. 580-587.
[6] R. B. Girshick, "Fast R-CNN," Proc. International Conference on Computer
Vision Pattern Recognition, Santiago, Dec. 2015, pp. 1440-1448.
[7] S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: towards real time
object detection with region proposal networks," Proc. IEEE Transactions
on Pattern Analysis Machine Intelligence, 2017, vol. 39, pp. 1137-1149.
[8] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You only look once:
unified, real-time object detection," arXiv preprint, 2015, arXiv:1506.02640.
[9] W. Liu, et al., "SSD: single shot multiBox detector," Lecture Notes in
Computer Science, 2016, pp. 21–37.
[10]R. C. Luo, T. W. Lin, and Y. H. Tsai, "Analytical inverse kinematic solution
for modularized 7-DOF redundant manipulators with offsets at shoulder and
wrist," in Proc. IEEE/RSJ International Conference on Intelligent Robots
and System, Chicago, Sep. 2014, pp. 516-521.
[11]A. Khan, C. Xiangming, Z. Xingxing and W. L. Quan, "Closed form inverse
kinematics solution for 6-DOF underwater manipulator," International
Conference on Fluid Power and Mechatronics (FPM), Harbin, 2015, pp.
1171-1176.
61
[12]W. Chen, C. Chen, F. Lee and Y. Kung, "Digital hardware implementation
of the forward/inverse kinematics for a SCARA robot manipulator," IEEE
International Conference on Applied System Invention (ICASI), Chiba, 2018,
pp. 54-57.
[13]P. Beeson and B. Ames, "TRAC-IK: An open-source library for improved
solving of generic inverse kinematics," IEEE-RAS 15th International
Conference on Humanoid Robots (Humanoids), Seoul, 2015, pp. 928-935.
[14]S. Kumar, N. Sukavanam and R. Balasubramanian, "An optimization
approach to solve the inverse kinematics of redundant manipulator,
" International Journal of Information and System Sciences (Institute for
Scientific Computing and Information), 2010, vol. 6, no. 4, pp. 414-423.
[15]J. Vannoy and J. Xiao, "Real-time adaptive motion planning (RAMP) of
mobile manipulators in dynamic environments with unforeseen changes," in
Proc. IEEE Transactions on Robotics, Oct. 2008, vol. 24, pp. 1199-1212.
[16]J. Chaichawananit and S. Saiyod, "Solving inverse kinematics problem of
robot arm based on a-star algorithm," International Joint Conference on
Computer Science and Software Engineering (JCSSE), Khon Kaen, 2016,
pp. 1-6.
[17]H. Li, Z. Song, Z. Jiang, Y. Mo and W. Ni, "Solving inverse kinematics of
industrial robot based on bp neural network," IEEE 7th Annual
International Conference on CYBER Technology in Automation, Control,
and Intelligent Systems (CYBER), Honolulu, 2017, pp. 1167-1171.
[18]S. Klemm, J. Oberlander, A. Hermann, A. Roennau, T. Schamm, J. M. ¨
Zollner, and R. Dillmann, "RRT*-connect: faster, asymptotically optimal
motion planning," Proc. IEEE International Conference on Robotics and
Biomimetics, Zhuhai, Dec. 2015, pp. 1670-1677.
[19]J. Demby’s, Y. Gao and G. N. DeSouza, "A study on solving the inverse
kinematics of serial robots using artificial neural network and fuzzy neural
network," IEEE International Conference on Fuzzy Systems (FUZZ-IEEE),
New Orleans, 2019, pp. 1-6.
[20]J. Xiao, W. Han and A. Wang, "Simulation research of a six degrees of
freedom manipulator kinematics based on Matlab toolbox," in Proc.
International Conference on Advanced Mechatronic Systems, Xiamen, Mar.
2017, pp. 376-380.
[21]Deep Neural Networks
62
https://docs.opencv.org/master/d2/d58/tutorial_table_of_content_dnn.html
，2020 年 5 月。
[22]F. Haoqiang and E. Zhou. "Approaching human level facial landmark
localization by deep learning," Image Vis. Comput., Mar. 2016, vol. 47, pp.
27-35.
[23]L. Xiao, "A review of solutions for perspective-n-point problem in camera
pose estimation," Journal of Physics: Conference Series., Sep. 2018, vol.
1087.
[24]J. J. Kuffner Jr and S. M. LaValle, "RRT-Connect: An efficient approach to
single-query path planning," Proc. IEEE International Conference on
Robotics and Automation, San Francisco, Aug. 2000, pp. 995-1001.
[25]ASUS – GU501
https://www.asus.com/tw/ROG-Republic-Of-Gamers/ROGGU501/?SearchKey=gu501/，2020 年 5 月。
[26]達明機器人
https://www.tm-robot.com/zh-hant/regular-payload/，2020 年 5 月。
[27]Intel® RealSense™ Depth Camera D435i
https://www.intelrealsense.com/zh-hans/depth-camera-d435i/ ，2020 年 5
月。
[28]ROS - Wiki
http://wiki.ros.org/ROS/Tutorials，2020 年 5 月。
[29]J. Redmon, A. Farhadi, "YOLOv3: An incremental improvement," arXiv
preprint, 2018, arXiv:1804.02767.
[30]K. Simonyan and A. Zisserman, "Very deep convolutional networks for
large-scale image recognition," CoRR, Apr. 2015.
[31]K. He, X. Zhang, S. Ren and J. Sun, "Deep residual learning for image
recognition," IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), Las Vegas, Nov. 2016, pp. 770-778.
[32]I. H. Choi and Y. G. Kim, "Head pose and gaze direction tracking for
detecting a drowsy driver," International Conference on Big Data and
Smart Computing, Bangkok, 2014, pp. 241-244.
[33]Camera Calibration and 3-D Vision - MATLAB & Simulink
https://www.mathworks.com/help/vision/ref/cameracalibrator-app.html ，
2020 年 5 月。
63
[34]Z. Zhang, "A flexible new technique for camera calibration," IEEE
Transactions on Pattern Analysis and Machine Intelligence, Nov. 2000, vol.
22, no. 11, pp. 1330-1334.
[35]B. Amos, B. Ludwiczuk, M. Satyanarayanan, "Openface: a general-purpose
face recognition library with mobile applications," CMU-CS-16-118, CMU
School of Computer Science, June 2016.
[36]D. Jacques and H. R. Scheunemann, "A kinematic notation for lower-pair
mechanisms based on matrices," Trans ASME J. Appl. Mech, 23, 1955, pp.
215-221.
[37]J. Peng, W. Xu, Z. Wang and D. Meng, "Analytical inverse kinematics and
trajectory planning for a 6DOF grinding robot," IEEE International
Conference on Information and Automation (ICIA), Yinchuan, 2013, pp.
834-839.
[38]std_msgs - ROS Wiki
http://wiki.ros.org/std_msgs，2020 年 5 月。
[39]Melexis-MLX90621
https://www.melexis.com/en/product/mlx90621/far-infrared-sensor-arrayhigh-speed-low-noise，2020 年 5 月。
[40]Intel RealSense Help Center D400 Series
https://support.intelrealsense.com/hc/enus/community/posts/360037076293-Align-color-and-depth-images，2020
年 5 月。
[41]Datasheet – Intel
https://www.intel.com/content/dam/support/us/en/documents/emergingtechnologies/intel-realsense-technology/Intel-RealSense-D400-SeriesDatasheet.pdf，2020 年 6 月。
[42]Depth Post-Processing for Intel® RealSense™ Depth Camera D400 Series
https://dev.intelrealsense.com/docs/depth-post-processing，2020 年 6 月。