融合自然語言處理與 Transformer 模型之中文至自然手語翻譯系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	朱禹丞 Yu-Cheng Chu
論文名稱：	融合自然語言處理與 Transformer 模型之中文至自然手語翻譯系統 A Chinese-to-Natural Sign Language Translation System Integrating NLP and Transformer Models
指導教授：	蘇木春 Mu-Chun Su
口試委員:
學位類別：	碩士 Master
系所名稱：	資訊電機學院 - 資訊工程學系 Department of Computer Science & Information Engineering
論文出版年：	2025
畢業學年度：	113
語文別：	中文
論文頁數：	89
中文關鍵詞：	中文轉換、手語翻譯、自然手語、自然語言處理、Transformer
外文關鍵詞：	Chinese-to-Sign Translation, Sign Language Translation, Natural Sign Language, Natural Language Processing, Transformer
相關次數：	點閱：181 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

根據2024年底衛福部統計資料顯示，台灣登記在案的聽覺機能障礙者（Hearing Impairment）人數已達138,966人。這些聽障人士在日常生活中，主要依賴台灣自然手語（Taiwan Sign Language, TSL）進行溝通。然而，一般民眾在學習手語時，通常是透過手語書或課程學習文法式、結構化的手語，其語言表達方式與自然手語差異甚大，造成聽障人士與聽人的溝通往往出現理解困難與落差。提升一般人對自然手語的理解與應用，有助於促進更順暢的雙向溝通，進而實踐資訊無障礙的理念。

為此，本文提出一種將書面中文自動轉換為自然手語的系統，協助聽人能更有效地表達出聽障人士熟悉的自然手語句型。系統採用基於 Transformer 架構的深度學習方法，透過平行語料訓練模型，自動學習中文與自然手語之間的對應關係。此外，本研究結合自然語言處理（NLP）工具，對輸入中文句進行詞性標註、依存句法分析、成分句法分析與斷詞處理，以協助模型掌握語法與語意特徵，提升轉換品質。

實驗結果顯示，本系統能有效處理多種中文語句類型，並能正確調整語序與詞彙，生成貼近自然手語表達邏輯的輸出內容。此方法未來可應用於手語教學輔助系統、跨語言溝通平台或公共服務場域，具備實質的社會應用潛力與推廣價值。

According to statistics from Taiwan's Ministry of Health and Welfare at the end of 2024, there are 138,966 registered individuals with hearing impairments. In their daily lives, these individuals primarily rely on Taiwan Sign Language (TSL) for communication. However, most hearing individuals learn sign language through books or structured courses, which are based on formalized and grammatical versions of sign language that differ significantly from natural TSL. This discrepancy often leads to communication gaps and misunderstandings between hearing and deaf individuals. Enhancing the general public’s understanding and use of natural sign language can facilitate smoother two-way communication and promote the concept of information accessibility.

To address this issue, this study proposes a system that automatically translates written Mandarin Chinese into natural TSL expressions, helping hearing individuals convey messages in a form more familiar to the deaf community. The system employs a deep learning approach based on the Transformer architecture, learning the mapping between Mandarin and natural sign language through parallel corpora. Additionally, natural language processing (NLP) tools are integrated to perform part-of-speech tagging, dependency parsing, constituency parsing, and word segmentation on the input text, enabling the model to better capture syntactic and semantic features and improve translation quality.

Experimental results show that the system can effectively handle various types of Chinese sentences, accurately adjust word order and vocabulary, and generate output aligned with the logic of natural TSL expression. This approach holds practical potential for applications such as sign language education tools, cross-linguistic communication platforms, and public service systems.

目錄
頁次
摘要 iv
Abstract v
誌謝 vii
目錄 ix
一、 緒論 1
1.1 研究動機 .................................................................. 1
1.2 研究目標 .................................................................. 2
1.3 論文架構 .................................................................. 3
二、 背景知識以及文獻回顧 4
2.1 文獻回顧 .................................................................. 4
2.1.1 深度學習應用於翻譯之相關研究 ........................... 4
2.1.2 翻譯文本至手語之相關研究 ................................. 7
2.2 背景知識 .................................................................. 10
2.2.1 台灣手語 ......................................................... 10
2.2.2 手語種類 ......................................................... 11
2.2.3 自然語言處理介紹 ............................................. 12
2.2.4 Transformer 介紹................................................ 13
2.2.5 參數高效微調技術：LoRA................................... 15
2.2.6 BLEU 分數....................................................... 16
viii
目錄
2.2.7 ROUGE 分數 .................................................... 18
三、 研究方法 20
3.1 資料集 ..................................................................... 20
3.1.1 手語與口語平行語料 .......................................... 20
3.1.2 手語辭彙庫 ...................................................... 21
3.2 系統架構 .................................................................. 22
3.3 前處理 ..................................................................... 23
3.3.1 資料擴增 ......................................................... 23
3.3.2 CKIP 斷詞........................................................ 25
3.3.3 NLP 分析......................................................... 26
3.4 模型架構 .................................................................. 28
3.4.1 MarianMT、T5 與 mT5 模型架構........................... 28
3.4.2 語言結構增強的模型設計 .................................... 30
3.4.3 基於 LoRA 的 Adapter 架構實作 ............................ 32
3.5 結構提示設計 ............................................................ 34
3.6 後處理 ..................................................................... 35
3.7 使用者介面 ............................................................... 36
四、 實驗設計與結果 38
4.1 實驗流程與評估指標 ................................................... 38
4.2 實驗環境與參數設定 ................................................... 39
4.2.1 不同模型結果 ................................................... 40
4.2.2 不同 Embedding 方式比較 .................................... 42
4.2.3 不同 Prompt 方式比較 ......................................... 43
4.2.4 Adapter 架構添加比較......................................... 44
4.2.5 擴增資料集比較 ................................................ 48
ix
目錄
4.3 結果呈現與分析 ......................................................... 50
4.3.1 模型效能驗證 ................................................... 50
4.3.2 錯誤案例分析 ................................................... 56
五、 總結 60
5.1 結論 ........................................................................ 60
5.2 未來展望 .................................................................. 61
參考文獻 62
附錄 A CKIP 詞性標註對照表 66
附錄 B HanLP 標記對照表 69
                                

參考文獻
[1] 衛生福利部. “身心障礙者人數按等級及類別分.” Accessed: May 12, 2025. [Online]. Available: https://www.mohw.gov.tw/dl - 69414 - ee6c345a - 38a4 - 4851 - 9c0d -
28b8c0fd3948.html.
[2] 中華民國啟聰協會. “台灣手語介紹及手語 QA.” Accessed: May 12, 2025. [Online].
Available: https://www.deaf.org.tw/OnePage.aspx?mid=51&id=46.
[3] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and
I. Polosukhin, “Attention Is All You Need,” 2017, arXiv:1706.03762.
[4] D. Jurafsky and J. H. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd. 2025, Online manuscript released January 12, 2025. [Online]. Available: https://web.stanford.edu/~jurafsky/slp3/.
[5] CKIPLab. “ckiplab/ckip-transformers: Transformer-based Chinese NLP models by CKIP.”
Accessed: May 12, 2025. [Online]. Available: https://github.com/ckiplab/ckip-transformers.
[6] H. He and J. D. Choi, “The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders,” in Proc. 2021 Conf. Empirical Methods Natural Lang.
Process., Online and Punta Cana, Dominican Republic: Association for Computational
Linguistics, Nov. 2021, pp. 5555–5577. [Online]. Available: https://aclanthology.org/
2021.emnlp-main.451.
[7] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Proc. 28th Int. Conf. Neural Inf. Process. Syst. - Vol. 2, Cambridge, MA, USA:
MIT Press, 2014, pp. 3104–3112.
[8] D. Bahdanau, K. Cho, and Y. Bengio, “Neural Machine Translation by Jointly Learning
to Align and Translate,” 2016, arXiv:1409.0473.
[9] M.-T. Luong, H. Pham, and C. D. Manning, “Effective Approaches to Attention-based
Neural Machine Translation,” 2015, arXiv:1508.04025.
[10] M. Junczys-Dowmunt et al., “Marian: Fast Neural Machine Translation in C++,” 2018,
arXiv:1804.00344.
62
[11] Y. Liu, J. Gu, N. Goyal, X. Li, S. Edunov, M. Ghazvininejad, M. Lewis, and L. Zettlemoyer, “Multilingual Denoising Pre-training for Neural Machine Translation,” 2020,
arXiv:2001.08210.
[12] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and
P. J. Liu, “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” 2019, arXiv:1910.10683.
[13] Y. Ouargani and N. E. Khattabi, “Advancing Text-to-GLOSS Neural Translation Using
a Novel Hyper-parameter Optimization Technique,” 2023, arXiv:2309.02162.
[14] S. Egea Gómez, E. McGill, and H. Saggion, “Syntax-aware Transformers for Neural
Machine Translation: The Case of Text to Sign Gloss Translation,” in Proc. 14th Workshop Building and Using Comparable Corpora (BUCC 2021), R. Rapp, S. Sharoff, and
P. Zweigenbaum, Eds., Online (Virtual Mode): INCOMA Ltd., Sep. 2021, pp. 18–27.
[Online]. Available: https://aclanthology.org/2021.bucc-1.4/.
[15] D. Zhu, V. Czehmann, and E. Avramidis, “Neural Machine Translation Methods for
Translating Text to Sign Language Glosses,” in Proc. 61st Annu. Meeting Assoc. Comput. Linguistics (Volume 1: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki,
Eds., Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 12 523–
12 541. DOI: 10.18653/v1/2023.acl-long.700. [Online]. Available: https://aclanthology.
org/2023.acl-long.700/.
[16] S. M. Abdullah, A. Paul, S. Rayana, A. Kabir, and Z. Masud, “State-of-the-Art Translation of Text-to-Gloss using mBART: A case study of Bangla,” 2025, arXiv:2504.02293.
[17] N. C. Camgoz, S. Hadfield, O. Koller, H. Ney, and R. Bowden, “Neural Sign Language Translation,” in Proc. 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recognit.,
2018, pp. 7784–7793. DOI: 10.1109/CVPR.2018.00812.
[18] T. Hanke, M. Schulder, R. Konrad, and E. Jahn, “Extending the Public DGS Corpus
in Size and Depth,” in Proc. LREC2020 9th Workshop Representation and Process.
Sign Lang., Marseille, France: European Language Resources Association (ELRA), May
2020, pp. 75–82. [Online]. Available: https://aclanthology.org/2020.signlang-1.12/.
[19] R. Konrad et al., MEINE DGS – annotiert. Öffentliches Korpus der Deutschen Gebärdensprache, 3. Release / MY DGS – annotated. Public Corpus of German Sign Language,
3rd release, Version 3.0, 2020. DOI: 10 . 25592/ dgs . corpus - 3 . 0. [Online]. Available:
https://doi.org/10.25592/dgs.corpus-3.0.
[20] SignTube. “台灣手語南北差異 2.” Accessed: May 12, 2025. [Online]. Available: https:
//www.youtube.com/shorts/_qzWN4ZCYTA.
[21] 立法院. “國家語言發展法.” Accessed: May 12, 2025. [Online]. Available: https://law.
moj.gov.tw/LawClass/LawAll.aspx?pcode=H0170143.
63
[22] K. Toutanova, D. Klein, C. D. Manning, and Y. Singer, “Feature-rich part-of-speech
tagging with a cyclic dependency network,” in Proc. 2003 Conf. North Amer. Chapter
Assoc. Comput. Linguistics on Human Lang. Technol., Association for Computational
Linguistics, 2003, pp. 173–180.
[23] S. Kübler, R. McDonald, and J. Nivre, Dependency Parsing. San Rafael, CA, USA: Morgan & Claypool Publishers, 2009.
[24] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” 2019, arXiv:1810.04805.
[25] T. B. Brown et al., “Language Models are Few-Shot Learners,” 2020, arXiv:2005.14165.
[26] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a Method for Automatic Evaluation of Machine Translation,” in Proc. 40th Annu. Meeting Assoc. Comput. Linguistics,
Philadelphia, Pennsylvania, USA: Association for Computational Linguistics, Jul. 2002,
pp. 311–318. DOI: 10.3115/1073083.1073135. [Online]. Available: https://aclanthology.
org/P02-1040/.
[27] C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” in Text Summarization Branches Out, Barcelona, Spain: Association for Computational Linguistics,
Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013/.
[28] 臺北市勞動力重建運用處. “臺北市手語翻譯培訓教材.” Accessed: May 26, 2025.
[Online]. Available: https://fd.gov.taipei/News_Content.aspx?n=33AD377CE79CDD67&
sms=777D61CE7A1EE646&s=87A01B9BF9B38744.
[29] 教育部. “常用手語辭典.” Accessed: May 26, 2025. [Online]. Available: https://special.
moe.gov.tw/signlanguage/.
[30] 教育部國民及學前教育署. “學齡前 2 至 6 歲教保服務人員手語手冊.” Accessed:
May 26, 2025. [Online]. Available: https://www.ece.moe.edu.tw/ch/special_education/
skill/skill_0002/.
[31] 手語語言學台灣研究中心. “台灣手語線上辭典.” Accessed: May 26, 2025. [Online].
Available: https://twtsl.ccu.edu.tw/TSL/.
[32] L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, and
C. Raffel, “mT5: A massively multilingual pre-trained text-to-text transformer,” 2021,
arXiv:2010.11934.
[33] X. Ming. “text2vec: A Tool for Text to Vector.” Accessed: Jul. 15, 2025. [Online]. Available: https://github.com/shibing624/text2vec.
[34] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen,
“LoRA: Low-Rank Adaptation of Large Language Models,” 2021, arXiv:2106.09685.
[35] CKIPLab. “Ckip-transformers Pos Table.” Accessed: May 21, 2025. [Online]. Available:
https://github.com/ckiplab/ckip-transformers/blob/master/docs/main/tag/pos.csv.
64
[36] hankcs. “HanLP Documentation: Universal Dependencies.” Accessed: May 21, 2025.
[Online]. Available: https://hanlp.hankcs.com/docs/annotations/dep/ud.html.
[37] hankcs. “HanLP Documentation: Constituency Parsing Chinese Tree Bank.” Accessed:
May 21, 2025. [Online]. Available: https : / / hanlp . hankcs . com / docs / annotations /
constituency/ctb.html.

簡易檢索 / 詳目顯示

相關論文