基於深度學習之室內盲人引導輔助系統｜國立中央大學博碩士論文系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	王婷 Ting Wang
論文名稱：	基於深度學習之室內盲人引導輔助系統 A Deep-learning-based Indoor Navigation Assistance System for Visually Impaired Persons
指導教授：	蘇木春 Mu-Chun Su
口試委員:
學位類別：	碩士 Master
系所名稱：	資訊電機學院 - 資訊工程學系 Department of Computer Science & Information Engineering
論文出版年：	2020
畢業學年度：	108
語文別：	中文
論文頁數：	74
中文關鍵詞：	深度學習、導盲系統、室內引導、偵測系統、視障者
外文關鍵詞：	deep learning, navigation systems for the blind persons, indoor navigation, detection system, visually impaired
相關次數：	點閱：15 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

視障者要獨立行走在一個陌生且複雜的室內公共空間中，是一件很困難的事情，因此，要如何獲取環境資訊以讓視障者可以在無需旁人輔助的情況下到達目的地，是個重要的研究課題。因此，本論文使用影像處理技術與深度學習做結合，開發出一套室內盲人引導輔助系統，讓視障者能在陌生的環境中獨立行走。
本篇論文所開發出的室內引導盲人輔助系統包含三部分:(1)指標偵測；使用YOLOv3模型搭配深度影像，偵測出室內常見的指標以及地上危險指標，並且計算其距離位置、(2)文字偵測與辨識:使用PSENET模型搭配OCR文字辨識API，偵測與識別指示標示牌中的文字、和(3)方向與指示標示牌資訊配對:使用YOLOv3模型偵測出區域，以配對指示標示牌中的箭頭與資訊。綜合上述三部分之功能，在取得指示標示牌影像後，可以識別此標示牌之資訊，可以透過語音提示的方式來輔助視障者掌握前方陌生的環境資訊。
本系統研發的目的在於輔助視障者在陌生環境公共空間中，提供眼前的資訊，以幫助視障者抵達目的地。實驗結果顯示，平均偵測到的指標準確率達到93%，方向與指示標誌牌資訊配對的精確度為86%，有此可以證明本系統具備一定程度之可用性。

關鍵字：深度學習、導盲系統、室內引導、偵測系統、視障者

It is difficult for the visually impaired to walk independently in a strange and complicated indoor public space. Therefore, how to obtain environmental information so that the visually impaired can reach the destination without the assistance of others is an important research topic. Therefore, this paper uses image processing technology combined with deep learning to develop an indoor blind navigation assistance system that allow the visually impaired to walk independently in unfamiliar environments.
The indoor blind navigation assistance system developed in this paper consists of three parts: (1) Sign detection: using the YOLOv3 model with depth images to detect common indoor signs and ground hazard indicators, and calculate their distance and location, (2) Word detection and recognition: using PSENET model with the OCR text recognition API to detect and recognize the text in the indicator plate. (3) direction and indicator information pairing: Using YOLOv3 model to detect the area and match the direction indicating arrows and information. Combining the functions of the above three parts, after obtaining the image of the front signboards, the information of the environment can be identified, and the system can assist the visually impaired to grasp the unfamiliar environment information in front via voiced signals.
The purpose of the development of this system is to assist visually impaired people to provide information in front of them in a public space in an unfamiliar environment to help the visually impaired reach their destination. The experimental results show that the average accuracy of the detected indicators reaches 93%, and the accuracy of direction and sign information matching is 86%. From the experimental results, it proves that the system has a certain degree of usability.

Keywords: deep learning, navigation systems for the blind persons, indoor navigation, detection system, visually impaired

摘要    i
ABSTRACT    ii
致謝    iv
目錄    v
圖目錄    vii
表目錄    ix
第一章、緒論    1
1-1 研究動機    1
1-2 研究目的    2
1-3 論文架構    3
第二章、相關研究    4
2-1 盲人引導輔具    4
2-1-1 傳統輔具    4
2-1-2 科技輔具產品    6
2-2 室內定位與導航技術    9
2-3 物件偵測模型    13
2-3-1 YOLO    13
2-3-2 YOLOv2    16
2-3-3 YOLOv3    18
2-4 PSENET文字偵測模型    20
第三章、研究方法    22
3-1 硬體介紹    22
3-1-1 NAVIDIA Jetson AGX Xavier嵌入式系統    22
3-1-2 Zed Camera    23
3-2 系統介紹    26
3-2-1 系統架構    26
3-2-2 系統設計    27
3-2-3 系統流程    28
3-3 識別指標模組    30
3-3-1 指標偵測    30
3-3-2 文字偵測與辨識    33
3-3-3 方向與指示標示資訊配對    35
第四章、實驗設計與結果    40
4-1 標誌偵測實驗    40
4-1-1 評估方式    40
4-1-2 IOU閥值實驗結果與分析    43
4-2 標誌偵測資料集解析度影響之實驗    46
4-2-1 實驗設計    46
4-2-2 實驗結果與分析    47
4-3 方向標誌配對實驗    49
4-3-1 評估方式    49
4-3-2 PSENET與YOLOv3比較    51
4-3-3 影像閥值化實驗結果與分析    54
4-4 識別指標模組測試    55
第五章、結論與未來展望    57
5-1 結論    57
5-2 未來展望    58
參考文獻    59


                                

[1] "伊甸福利基金會," [Online]. Available: http://light.eden.org.tw/service_assure_dissemination.htm. [Accessed 22 6 2020].
[2] 余. 唐. 曾亮, "關懷無障礙設施系列報導(一)," [Online]. Available: http://www.arch.net.tw/modern/month/338/338-2.htm. [Accessed 22 6 2020].
[3] "大專校院資源教室輔導人員特教知能(初階)研習講義26," [Online]. Available: http://sencir.spc.ntnu.edu.tw/site/c_file/a_download/t_key/1178. [Accessed 22 6 2020].
[4] "清大盲友會," [Online]. Available: https://blind.cs.nthu.edu.tw/guide/guide-1.html. [Accessed 9 6 2020].
[5] "財團法人愛犬基金會," [Online]. Available: https://www.tfb.org.tw/web/service/service.jsp?no=CP1497412035614. [Accessed 9 6 2020].
[6] L. Chang, "這個新的AR技術能透過讓物品說話幫助盲人辨識方位," [Online]. Available: https://chinese.engadget.com/2018/11/29/ar-headset-tech-helps-blind-navigate/. [Accessed 9 6 2020].
[7] "HoloLens (第一代)硬體," 2019. [Online]. Available: Available: https://docs.microsoft.com/zh-tw/hololens/hololens1-hardware. [Accessed 22 6 2020].
[8] 蔡騰輝, "改善傳統視障輔具南台科大推出物聯網手杖," 2018. [Online]. Available: https://www.digitimes.com.tw/iot/article.asp?cat=158&id=0000543796_PVJ8XV6N8X4UGB7O7LM49. [Accessed 9 6 2020].
[9] 楊晨欣, "Google 新APP要幫視覺障礙人士看世界," [Online]. Available: https://www.bnext.com.tw/article/49067/google-lookout-app-for-visual-impaired-people. [Accessed 9 6 2020].
[10] C. Yang and H. SHao, "Wifi-based indoor positioning," IEEE Communications Magazine, pp. 150-157, 2016.
[11] S. Willis and A. (Sumi) Helal., "RFID information grid for blind navigation and wayfinding," ISWC, vol. 5, pp. 37-37, 2005.
[12] 黃凱祥, "應用於魔物館Beacon微定位技術:以國立台灣歷史博物館為例," 博物館與文化, no. 15, pp. 5-29, 2018.
[13] 藍介洲, "Beacon 系統應用於視障者室內與戶外定向與引導效用之使用者經驗研究," 身心障礙研究, vol. 16, no. 3-4, 2018.
[14] K. Nikhil , I. Sai Pavan Kalyan ,Jetty Sagar ,M. Sai Rohit , and M. Nesasudha, "Li-FI Smart Indoor Navigation System for Visually Impaired People," 2019 2nd International Conference on Signal Processing and Communication (ICSPC), pp. 187-192, 2019.
[15] B. Li, JP Muñoz, X. Rong, Q. Chen, J. Xiao, Y. Tian, A. Arditi ,and M. Yousuf,, "Vision-Based Mobile Indoor Assistive," IEEE Transactions on Mobile Computing, vol. 18, no. 3, pp. 702-714, 2019.
[16] A. Hub, J. Diepstraten ,and T.Ertl, "Augmented Indoor Modeling for Navigation Support for the Blind," in Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility, 2004.
[17] B. And, S. Baglio, and V. Marletta, "Intelligent sensing solutions for AAL," Sensors, no. 162, pp. 321-324, 2004.
[18] J. Xiao, S. Joseph, X. Zhang, B. Li, X. Li ,and J. Zhang, "An Assistive Navigation Framework for the Visually Impaired," IEEE Transactions on Human-Machine Systems Issue5, vol. 45, no. 5, pp. 635-640, 2015.
[19] J. Redmon and A. Farhadi, "You only look once: Unified, real-time object detection," Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 779-788, 2016.
[20] J. Redmon and A. Farhadi, "YOLO9000: better, faster, stronger," CoRRabx/1612.08242, 2016.
[21] J. Redmon and A. Farhadi, "Yolov3: An incremental improvement," arXivpreprint arXiv:1804.02767, 2018.
[22] X. Li, W. Wang, W. Hou, R.-Z. Liu, T. Lu ,and J. Yang, "Shape Robust Text Detection with Progressive Scale Expansion Network," Proc. IEEE Conf. Comput. Vis. Pattern Recognit., pp. 9336-9345, 2019.
[23] "NVIDA JETSON AGX XAVIER," NVIDIA, [Online]. Available: https://www.nvidia.com/zh-tw/autonomous-machines/jetson-agx-xavier/. [Accessed 6 2020].
[24] "STEREOLABS," [Online]. Available: https://www.stereolabs.com/developers/. [Accessed 6 2020].
[25] liuheng92, "tensorflow_PSENet," [Online]. Available: https://github.com/liuheng92/tensorflow_PSENet. [Accessed 6 9 2019].

簡易檢索 / 詳目顯示

相關論文