在新冠肺炎疫情爆發的時代中，由於政府政策的規範，人們常常因為聊天、吃飯以及拍照等因素暫時性地脫下口罩，然而卻忘記在進入下一個場合時再次將口罩正確地戴上。本論文沿用原系統[1]設計的臉部身分辨識基礎，並結合深度學習技術進行設計與改良，實現一套新系統可以檢測人們配戴口罩狀況、辨識真假人臉、以及估測人們的性別與年齡，並強化系統對抗口罩遮蔽的能力。原系統[1]設計基礎中建有身份辨識的功能，人們只要經過註冊，儘管臉部受到部分遮蔽，如：口罩、墨鏡、帽子等遮蔽物，系統依舊能夠透過人臉資料庫辨識出註冊人臉。本論文繼續擴展原系統[1]的功能，針對人們的口罩配戴狀態進行檢測，可以判斷該人是否正確配戴口罩，並依據身份是否為註冊人臉之不同進行更深入的分析，對於辨識「是註冊人臉資料庫中的人臉」，進行活體檢測，防止有心人士以偽造人臉（如照片或圖片）闖關，增進本系統對於「註冊人臉」的加強把關；對於「非註冊人臉」則進行性別及年齡估測，增加本系統對於畫面中陌生人的紀錄資料，並可以對陌生人之資料進行後續分析。
本系統主要包含三個部分。第一部分為口罩配戴狀況檢測，首先藉由模型檢測出畫面中的臉部區域並辨識出每張臉分別是屬於哪一種口罩配戴狀況。接著原系統[1]的身份辨識資訊會與口罩配戴狀況資訊進行匹配結合，此時本系統將了解每張人臉的身份與口罩配戴狀況。第二部分為活體檢測，透過身份辨識資訊了解畫面中出現的「註冊人臉」，並經由口罩配戴狀況資訊了解臉部未遮蔽的區域，取得臉部未被遮蔽的特徵辨識真假人臉。第三部分為性別及年齡估測，透過身份辨識資訊了解畫面中出現的「非註冊人臉」，並經由口罩配戴狀況資訊了解臉部未遮蔽的區域，取得臉部未被遮蔽的特徵估計該人臉的性別及年齡。本系統可以達到10 FPS的整體運算效能，實現即時辨識的效果。

[1] 儲健峰, "基於AI技術之對抗部分遮蔽的即時臉部辨識系統," 碩士論文, 國立中央大學, 2021.

In the new era of the pneumonia epidemic, due to government policies, people often take off their masks temporarily for chatting, eating, and taking pictures, but they forget to put their masks on properly in the next occasion. In this thesis, we design a system by improving the original system [1] that detects people’s mask-wearing status, identifies live and fake faces, estimates people's gender and age, and enhances the system's resistance to mask obscuration. The original system [1] was designed with an identification feature that allows the system to recognize faces through a face database even if they are partially obscured, such as masks, sunglasses, hats, and other occlusions, as long as the faces are registered. In this thesis, we continue to extend the functionality of the original system [1] to detect people’s mask-wearing status, determine whether people are wearing masks correctly, and perform a deeper analysis to identify "registered face" for liveness detection, to prevent people from using fake faces (e.g. photos or pictures) to pass. The system can apply gender and age estimation of "unregistered face" to increase the system's record of strangers on the screen. Then we can use the record of strangers to analysis.
The system consists of three main parts. The first part is the detection of masking status. First, the model detects the face region on the screen and determines which masking state each face belongs to. Then let the identity information in the original system [1] and the mask-wearing status be matched such that the system will recognize the identity and the mask-wearing status for each face. The second part is the liveness detection to understand the "registered face" appearing on the screen by the identity information, and the unmasked area of the face by the mask-wearing status information to obtain the unmasked features of the face and identify the live and fake faces. The third part is the estimation of gender and age. The gender and age of the face are estimated by understanding the "unregistered face" that appears on the screen through the recognition information and the uncovered area of the face through the mask-wearing status information. The system can achieve an overall computing performance of 10 FPS for real-time recognition.

[1] 儲健峰, "基於AI技術之對抗部分遮蔽的即時臉部辨識系統," 碩士論文, 國立中央大學, 2021.

[1] 儲健峰, "基於AI技術之對抗部分遮蔽的即時臉部辨識系統," 碩士論文, 國立中央大學, 2021.
[2] B. Karthikeyan and S. Gowri, "A real-time face mask detection using ssd and mobilenetv2," The 4th International Conference on Computing and Communications Technologies, 2021, pp. 144-148.
[3] A. Bochkovskiy, C.-Y. Wang, and H.-Y. M. Liao, "Yolov4: optimal speed and accuracy of object detection," arXiv preprint arXiv:2004.10934, 2020.
[4] G. Jocher, et al., "Ultralytics/yolov5," 2022, doi: https://doi.org/10.5281/zenodo.3908559.
[5] G.-H. Yang, et al., "Face mask recognition system with yolov5 based on image recognition," The 6th International Conference on Computer and Communications, 2020, pp. 1398-1404.
[6] P. Anthony, B. Ay, and G. Aydin, "A review of face anti-spoofing methods for face recognition systems," The International Conference on INnovations in Intelligent SysTems and Applications, 2021, pp. 1-9.
[7] A. K. Singh, P. Joshi, and G. C. Nandi, "Face recognition with liveness detection using eye and mouth movement," The International Conference on Signal Propagation and Computer Technology, 2014, pp. 592-597.
[8] M. Killioğlu, M. Taşkiran, and N. Kahraman, "Anti-spoofing in face recognition with liveness detection using pupil tracking," The 15th International Symposium on Applied Machine Intelligence and Informatics, 2017, pp. 87-92.
[9] T. Dhawanpatil and B. Joglekar, "A review spoof face recognition using lbp descriptor," The 1st International Conference on Smart System, Innovations and Computing, 2018, pp.661-668.
[10] X.-Y. Tan, et al., "Face liveness detection from a single image with sparse low rank bilinear discriminative model," The European Conference on Computer Vision, 2010, pp. 504–517.
[11] G. Albakri and S. Alghowinem, "The effectiveness of depth data in liveness face authentication using 3d sensor cameras," Sensors, 2019.
[12] Z.-T. Yu, et al., "Deep learning for face anti-spoofing: a survey," arXiv preprint arXiv:2106.14948, 2021.
[13] Y.-J. Liu, A. Jourabloo, and X.-M. Liu, "Learning deep models for face anti-spoofing: binary or auxiliary supervision," The Conference on Computer Vision and Pattern Recognition, 2018, pp. 389-398.
[14] 林韻柔, "基於深度跨域轉換之人臉影像防偽辨識," 碩士論文, 國立交通大學, 2020.
[15] R. Caruana, "Multitask learning," Machine Learning, 1997, pp. 41-75.
[16] S. Ruder, "An overview of multi-task learning in deep neural networks," arXiv preprint arXiv:1706.05098, 2017.
[17] R. Ranjan, et al., "An all-in-one convolutional neural network for face analysis," The 12th IEEE International Conference on Automatic Face & Gesture Recognition, 2017, pp. 17-24.
[18] B. Yoo, et al., "Deep facial age estimation using conditional multitask learning with weak label expansion," IEEE Signal Processing Letters, 2018, pp. 808-812.
[19] H. Han, et al., "Heterogeneous face attribute estimation: a deep multi-task learning approach," IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, pp. 2597-2609.
[20] M. S. Ejaz and M. R. Islam, "Masked face recognition using convolutional neural network," The International Conference on Sustainable Technologies for Industry 4.0, 2019, pp. 1-6.
[21] D. S. Trigueros, L. Meng, and M. Hartnett, "Enhancing convolutional neural networks for face recognition with occlusion maps and batch triplet loss," Image and Vision Computing, 2018, pp. 99-108.
[22] E. Osherov and M. Lindenbaum, "Increasing cnn robustness to occlusions by reducing filter support," The IEEE International Conference on Computer Vision, 2017, pp. 550-561.
[23] "配備廣角鏡頭的羅技C930e 1080p 商務網路攝影機 ," [Online]. Available: https://www.logitech.com/zh-hk/products/webcams/c930e-business-webcam.960-000976.html. [Accessed: June, 2022].
[24] "AORUS 15 (Intel 9th Gen) 特色重點 ," [Online]. Available: https://www.gigabyte.com/tw/Laptop/AORUS-15--Intel-9th-Gen#kf. [Accessed: June, 2022].
[25] "Face mask detection dataset," [Online]. Available: https://makeml.app/datasets/mask. [Accessed: June, 2022].
[26] P. Kottarathil, "Face mask detection dataset," Kaggle Data, 2020. [Online]. Available: https://www.kaggle.com/prasoonkottarathil/face-mask-lite-dataset. [Accessed: June, 2022].
[27] A. Cabani, et al., "Maskedface-net–a dataset of correctly/incorrectly masked face images in the context of covid-19," arXiv preprint arXiv:2008.08016, 2020.
[28] J. Redmon, et al., "You only look once: unified, real-time object detection," arXiv preprint arXiv: 1506.02640, 2015.
[29] "YOLOv5 is Here: State-of-the-Art Object Detection at 140 FPS," [Online]. Available: https://blog.roboflow.com/yolov5-is-here/. [Accessed: June, 2022].
[30] V. A. Adibhatla, et al., "Applying deep learning to defect detection in printed circuit boards via a newest model of you-only-look-once," Mathematical Biosciences and Engineering, 2021, pp. 4411-4428.
[31] Y.-H. Zhang, et al., "Celeba-spoof: large-scale face anti-spoofing dataset with rich annotations," arXiv preprint arXiv:2007.12342, 2020.
[32] M.-X. Tan and Q.-V. Le, "Efficientnet: rethinking model scaling for convolutional neural networks," arXiv preprint arXiv:1905.11946, 2019.
[33] M.-X. Tan, et al., "Mnasnet: platform-aware neural architecture search for mobile," arXiv preprint arXiv: 1807.11626, 2018.
[34] S. Shao, et al., "Objects365: a large-scale, high-quality dataset for object detection," The IEEE/CVF International Conference on Computer Vision, 2019, pp. 8429-8438.
[35] 陳蓁晏, "以手機輔助視障人士之室內物件偵測及拿取," 碩士論文, 國立中央大學, 2022.
[36] J.-C. Cheng, et al., "Exploiting effective facial patches for robust gender recognition," Tsinghua Science and Technology, 2019, vol. 24, no. 3, pp. 333-345.