| 研究生: |
呂澤阜 Tse-Fu Lu |
|---|---|
| 論文名稱: |
資料擴增於即時眼寫辨識的卷積神經網路 Convolutional Neural Network with Data Augmentation for Real-time EOG Eye-writing Recognition |
| 指導教授: |
蔡章仁
Jang-Zern Tsai |
| 口試委員: | |
| 學位類別: |
碩士 Master |
| 系所名稱: |
資訊電機學院 - 電機工程學系 Department of Electrical Engineering |
| 論文出版年: | 2019 |
| 畢業學年度: | 107 |
| 語文別: | 中文 |
| 論文頁數: | 78 |
| 中文關鍵詞: | 眼電圖 、眼寫 、眨眼偵測 、深度學習 、卷積網路 、數據擴增 |
| 外文關鍵詞: | EOG, eye-writing, blink detection, deep learning, CNN, data augmentation |
| 相關次數: | 點閱:11 下載:0 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
本研究實現了一種基於卷積網路(Convolutional neural network, CNN) 的眼寫系統。 透過由眼電圖法(Electro-OculoGraphy, EOG)擷取眼睛移動產生的電訊號,採用End-to-End的架構,直接將訊號送入網路並且即時的輸出結果,無須再對眼電訊號進行繁雜的特徵抽取。本研究提出的系統除了硬體成本低廉,並且能夠在一般平價的品牌電腦上即時運行。
除了書寫之外,本研究提出的系統也能在各種控制場合做為人機介面,利用EOG訊號做為輸入指令,控制物件。本研究利用此系統實作一個範例,能夠直接依照著手寫筆劃書寫10個阿拉伯數字、26個英文大寫字母,以及自定義的空白(space)及退格(backspace)的眼寫系統,並且在測試時擁有95.7%的準確率。在文末也透過混淆矩陣(confusion matrix)來探討較容易使系統誤判的情況,並提供設計指令/符號的一些訣竅與建議。
此外,本研究還提出了即時的適應性眨眼偵測演算法用以偵測眨眼及計算眨眼次數來分隔符號、局部最大值偵測法用以縮放訊號來標準化訊號、及EOG訊號的兩種數據擴增方式以使數據集涵蓋更多可能的眼寫軌跡。
In this study, a Convolutional neural network (CNN) based eye-writing system is implemented. With the end-to-end structure, we acquire electric signal generated by eye moving through Electro-OculoGraphy (EOG), then feed the signal into the network directly without extracting complex features from the signal and network will output the symbol of recognition immediately. This system also has low hardware cost and it can run in most budget PC at real time.
Besides writing symbols or characters, this system also can be used to facilitate a Human-Machine Interface (HMI). In order to demonstrate the system proposed in this study, we used this system to implement an eye-writing system as an example which with 10 Arabic numerals and 26 English capital letters, all of which are written in accordance with handwritten strokes, as well as the space and backspace with specially designed writing patterns. This system attains a 95.7% accuracy in the test.
In the end of this paper, we discuss some conditions where it is easier to misjudge the system by the confusion matrix, and provide some tips and suggestions for instructions/symbols design.
In addition, this study also proposes a real-time adaptive blink detection algorithm which can detect and count eye blinks in the time domain with low computation, a local maximum based standardization algorithm and two method of data augmentation of EOG signal.
[1] 百度百科. "世界五大絕症." 百度百科. https://baike.baidu.com/item/%E4%B8%96%E7%95%8C%E4%BA%94%E5%A4%A7%E7%BB%9D%E7%97%87 (accessed 6/26, 2019).
[2] F. Fang and T. Shinozaki, "Electrooculography-based continuous eye-writing recognition system for efficient assistive communication systems," PLoS One, vol. 13, no. 2, 2018.
[3] 羅揚清, 蔡清標, 林恭平, and 廖光淦. "《淺談漸凍人─肌萎縮性脊髓側索硬化症》." http://www.mnda.org.tw/disease_info.php (accessed 6/26, 2019).
[4] J.-Z. Tsai, C. K. Lee, C.-M. Wu, J. J. Wu, and K. P. Kao, "A feasibility study of an eye-writing system based on electro-oculography," Journal of Medical and Biological Engineering, vol. 28, pp. 39-46, 2008.
[5] C.-K. Lee, "注音符號眼寫系統之可行性研究," 中央大學電機工程學系學位論文, 2005.
[6] 余長憲, "眼寫電話控制系統," 國立中央大學電機工程學系學位論文, 2006.
[7] 張嘉安, "眼寫鍵盤和眼寫滑鼠," 國立中央大學電機工程學系學位論文, 2006.
[8] 陳才士, "可攜式眼寫電腦控制及仿口語系統," 國立中央大學電機工程學系學位論文, 2007.
[9] L. Kwang-Ryeol, C. Won-Du, K. Sungkean, and I. Chang-Hwan, "Real-time "Eye-Writing" recognition using electrooculogram," IEEE Trans Neural Syst Rehabil Eng, vol. 25, no. 1, pp. 37-48, 2017.
[10] F. Simini, A. Touya, A. Senatore, and J. Pereira, "Gaze tracker by electrooculography (EOG) on a head-band," in 10th International Workshop on Biomedical Engineering, Kos, Greece, 2011, pp. 1-4.
[11] T. Yagi, Y. Kuno, K. Koga, and T. Mukai, "Drifting and blinking compensation in electro-oculography (EOG) eye-gaze interface," in IEEE International Conference on Systems, Man and Cybernetics, Taipei, Taiwan, 2006, vol. 4, pp. 3222-3226.
[12] A. Sammaiah, B. Narsimha, E. Suresh, and M. S. Reddy, "On the performance of wavelet transform improving Eye blink detections for BCI," in 2011 International Conference on Emerging Trends in Electrical and Computer Technology, Nagercoil, India, 2011: IEEE, pp. 800-804.
[13] M. S. Reddy, B. Narasimha, E. Suresh, and K. S. Rao, "Analysis of EOG signals using wavelet transform for detecting eye blinks," in 2010 International Conference on Wireless Communications & Signal Processing (WCSP), Suzhou, China, 2010, pp. 1-4.
[14] A. M. S. Ang, Z. G. Zhang, Y. S. Hung, and J. N. F. Mak, "A user-friendly wearable single-channel EOG-based human-computer interface for cursor control," in 7th International IEEE/EMBS Conference on Neural Engineering (NER), Montpellier, France, 2015, pp. 565-568.
[15] M. S. Hossain, K. Huda, and M. Ahmad, "Command the computer with your eye-An electrooculography based approach," in 8th International Conference on Software, Knowledge, Information Management and Applications (SKIMA 2014), Dhaka, Bangladesh, 2014, pp. 1-6.
[16] A. Chaudhuri, A. Dasgupta, S. Chakrborty, and A. Routray, "A low-cost, wearable, portable EOG recording system," in 2016 International Conference on Systems in Medicine and Biology (ICSMB), Kharagpur, India, 2016: IEEE, pp. 102-105.
[17] A. J. Golparvar and M. K. Yapici, "Graphene-coated wearable textiles for EOG-based human-computer interaction," in 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), Las Vegas, NV, USA, 2018: IEEE, pp. 189-192.
[18] N. Meziane, J. G. Webster, M. Attari, and A. J. Nimunkar, "Dry electrodes for electrocardiography," Physiological measurement, vol. 34, no. 9, p. R47, 2013.
[19] T. Instruments. INA12x Precision, Low-Power Instrumentation Amplifiers [Online] Available: http://www.ti.com/lit/ds/symlink/ina128.pdf
[20] ARDUINO. "ARDUINO-Home page." https://store.arduino.cc/usa/ (accessed 6/27, 2019).