基於深度學習與Hybrid N-grams之英文語法錯誤更正系統

簡易檢索 / 詳目顯示

回結果列表

研究生：	陳宜陞 Yi-Sheng Chen
論文名稱：	基於深度學習與Hybrid N-grams之英文語法錯誤更正系統 A Grammatical Error Correction System based on the Integration of Deep Learning and Hybrid N-grams
指導教授：	蘇木春 Mu-Chun Su
口試委員:
學位類別：	碩士 Master
系所名稱：	資訊電機學院 - 資訊工程學系 Department of Computer Science & Information Engineering
論文出版年：	2019
畢業學年度：	107
語文別：	中文
論文頁數：	93
中文關鍵詞：	英文語法更正、深度學習、混和N元語法、英文語法檢查
外文關鍵詞：	grammatical error correction, deep learning, hybrid n-gram, grammatical error detection
相關次數：	點閱：11 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

在英文使用者當中有過半數的人是非母語的英文使用者，對於這些人來
說如何快速且有效的檢查自己的文章有沒有語法錯誤是一件相當重要的事
情。 Natural Language Processing 一直是計算機科學領域中一門相當重要的議題，文法錯誤更正 Grammatical Error Correction 是其中的一項主要研究議題之一。這幾年來，已有多種文法錯誤更正的解決方案陸續被提出來，各有其優缺點。
本論文結合深度學習與混和 N元語法 Hybrid N-gram 來為文法錯誤
更正問題提出另一種解決方案。此解決方案由三種類神經網路所組成： 1
混和 N元語法語意分類器、 2 混合 N元語法轉換器和 3 混和 N元語
法反轉換器。此系統會先判斷輸入的英文句子是否具有混和 N元語法，接
著，再檢查與更正語法錯誤，最後才反轉換混和 N元語法並重組回英文句
子。藉此三階段的方式，達到利用混和 N元語法檢查英文語法的效果。
本論文將使用 StringNet及 CoNLL2013兩種資料集，來驗證所題方法之
有效性。會針對三種類神經網路，分別進行不同網路結構及資料前處理方法
的效果比較及分析。

More than half of English-speaking users are non-native English speakers. For these people, how to quickly and effectively check whether there are grammatical errors in their articles is quite important. Natural Language Processing has always been a very important topic in the field of computer science. Grammatical Error Correction is one of the main research topics. Over the past few years, different approaches to grammatical error correction have been proposed. Each approach has its own advantages and disadvantages. This thesis tries to combine deep learning with mixed N-grams to propose an alternative solution to the problem of grammatical error correction. This solution consists of three types of neural networks: (1) a hybrid N-gram semantic classifier, (2) a hybrid N-gram grammar converter, and (3) a hybrid N-gram grammar converter. This system will first determine whether an English sentence has a mixed N-gram, then check and correct its grammatical error, and finally transform the corrected N-gram back into its corresponding correct English sentence. In this three-stage way, the effect of using the hybrid N-gram to check the English grammar is achieved. Finally, this thesis will use StringNet and CoNLL2013 data sets to verify the performance of the proposed method. The effects of different network structures and data pre-processing methods will be compared and analyzed for three types of neural networks.

摘要    i
ABSTRACT    ii
致謝    iv
目錄    v
圖目錄    vii
表目錄    x
第一章、緒論    1
1-1 研究動機    1
1-2 研究目的    3
1-3 論文架構    4
第二章、相關研究    5
2-1 英文語法錯誤更正研究    5
2-1-1 Method of grammatical error detection and correction    10
2-1-2 NLTK    12
2-1-3 Error tag data    13
2-1-4 Hybrid N-grams    14
2-1-5 Semantic Analysis    15
2-2 神經網路模型相關研究    16
2-2-1 RNN    16
2-2-2 LSTM    17
2-2-3 Bidirectional LSTM    21
2-2-4 Seq2Seq model    22
2-2-5 CNN    24
2-2-6 Convolutional Seq2Seq with Attention    26
第三章、研究方法    30
3-1 Error Tag 與 Hybrid N-grams 之差異    30
3-2 英文語法錯誤更正演算法    32
3-2-1 Hybrid N-grams語意分類器    33
3-2-2 Hybrid N-grams轉換演算法    39
3-2-3 Hybrid N-grams to sentence N-grams轉換演算法    43
3-2-4 Corrected N-grams recombine    44
第四章、實驗設計與結果    47
4-1 實驗設計    47
4-2 Hybrid N-grams語意分類器模型訓練    47
4-2-1 Word level CNN 語意分類器    47
4-2-2 POS level CNN 語意分類器    50
4-2-3 POS level MLP 分類器    55
4-2-4 實驗分析    56
4-3 Hybrid N-grams轉換演算法模型訓練    57
4-3-1 Bi-LSTM Seq2Seq model    57
4-3-2 Masked Bi-LSTM Seq2Seq model    60
4-4 Hybrid N-grams to sentence N-grams轉換演算法模型訓練    63
4-4-1 Bi-LSTM Seq2Seq model    63
4-4-2 Multi-input Bi-LSTM Seq2Seq model    65
4-5 英文語法錯誤更正演算法之推廣性比較    67
4-5-1 CoNLL 2013 Shared Task 5種錯誤類別更正效果    67
第五章、結論與未來展望    72
5-1 結論    72
5-2 未來展望    73
參考文獻    74


                                

[1] "Natural language processing," [Online]. Available: https://en.wikipedia.org/wiki/Natural_language_processing. [Accessed : 05 - Jun - 2019].
[2] "List of languages by total number of speakers," [Online]. Available: https://en.wikipedia.org/wiki/List_of_languages_by_total_number_of_speakers. [Accessed : 05 - Jun - 2019].
[3] M. Soni and J. S. Thakur, "A Systematic Review of Automated Grammar Checking in English Language," arXiv preprint arXiv:.00540, 2018.
[4] "CoNLL-2013 Shared Task: Grammatical Error Correction," [Online]. Available: https://www.comp.nus.edu.sg/~nlp/conll13st.html. [Accessed : 11 - Jun - 2019].
[5] "CoNLL-2014 Shared Task: Grammatical Error Correction," [Online]. Available: https://www.comp.nus.edu.sg/~nlp/conll14st.html. [Accessed : 12 - Jun - 2019].
[6] H. T. Ng, S. M. Wu, T. Briscoe, C. Hadiwinoto, R. H. Susanto, and C. Bryant, "The CoNLL-2014 Shared Task on Grammatical Error Correction," Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pp. 1-14, 2014.
[7] A. Rozovskaya, K.-W. Chang, M. Sammons, and D. Roth, "The University of Illinois System in the CoNLL-2013 Shared Task," Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task, pp. 13-19, 2013.

[8] "Web 1T 5-gram Version 1 - Linguistic Data Consortium - LDC Catalog," [Online]. Available: https://catalog.ldc.upenn.edu/LDC2006T13. [Accessed : 13 - Jun - 2019].
[9] N.-L. Tsao and D. Wible, "A Method for Unsupervised Broad-Coverage Lexical Error Detection and Correction," Proceedings of the NAACL HLT Workshop on Innovative Use of NLP for Building Educational Applications, pp. 51-54, 2009.
[10] D. Dahlmeier, H. T. Ng, and S. M. Wu, "Building a Large Annotated Corpus of Learner English:The NUS Corpus of Learner English," Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 22-31, 2013.
[11] W. B. Cavnar and J. M. Trenkle, "N-Gram-Based Text Categorization," Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval. Vol. 161175., 1994.
[12] A. Rozovskaya and D. Roth, "Algorithm Selection and Model Adaptation for ESL Correction Tasks," Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pp. 924-933, 2011.
[13] M. Junczys-Dowmunt and R. Grundkiewicz, "The AMU System in the CoLL-2014 Shared Task: Grammatical Error Correction by Data-Intensive and Feature-rich Statistical Machine Translation," Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pp. 25-33, 2014.
[14] D. Bahdanau, K. Cho, and Y. Bengio, "Neural Machine Translation By Jointly Learning To Align And Translate," arXiv preprint arXiv:1409.0473, 2014.

[15] E. Loper and S. Bird, "NLTK: The Natural Language Toolkit," 2002.
[16] S. Bird, E. Klein, and E. Loper, "Natural Language Processing with Python," [Online]. Available: http://www.nltk.org/book/. [Accessed : 15 - Jun - 2019].
[17] "British Natinal Corpus," [Online]. Available: https://en.wikipedia.org/wiki/British_National_Corpus. [Accessed : 23 - Jun - 2019].
[18] D. Wible and N.-L. Tsao, "StringNet as a Computational Resource for Discovering and Investigating Linguistic Constructions," Proceedings of the NAACL HLT Workshop on Extracting and Using Constructions in Computational Linguistics, pp. 25-31, 2010.
[19] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-Based Learning Applied to Document Recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, November 1998.
[20] Y. Kim, "Convolutional Neural Networks for Sentence Classification," Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746-1751, 2014.
[21] "Recurrent neural network," [Online]. Available: https://en.wikipedia.org/wiki/Recurrent_neural_network. [Accessed : 15 - Jun - 2019].
[22] "Understanding LSTM Networks," [Online]. Available: https://colah.github.io/posts/2015-08-Understanding-LSTMs/. [Accessed : 15 - Jun - 2019].

[23] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[24] M. Schuster and K. K. Paliwal, "Bidirectional Recurrent Neural Networks," IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997.
[25] A. Graves, N. Jaitly, and A.-r. Mohamed, "Hybrid Sreech Recognition with Deep Bidirectional LSTM," 2013 IEEE workshop on automatic speech recognition and understanding, pp. 273-278, 2013.
[26] I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to Sequence Learning with Neural Networks," Advances in neural information processing systems, pp. 3104-3112, 2014.
[27] "Computer, respond to this email.," Google AI Blog, [Online]. Available: https://ai.googleblog.com/2015/11/computer-respond-to-this-email.html. [Accessed : 16 - Jun - 2019].
[28] V. Mnih, N. Heess, A. Graves, K. Kavukcuoglu, and G. DeepMind, "Recurrent Models of Visual Attention," Advances in neural information processing systems, pp. 2204-2212, 2014.
[29] "IBM Deep learning architectures," [Online]. Available: https://developer.ibm.com/articles/cc-machine-learning-deep-learning-architectures/. [Accessed : 20 - Jun - 2019].
[30] "Deep learning for complete beginners: convolutional neural networks.," [Online]. Available: https://cambridgespark.com/content/tutorials/convolutional-neural-networks-with-keras/index.html/. [Accessed : 20 - Jun - 2019].
[31] "CS231n Convolutional Neural Networks for Visual Recognition," [Online]. Available: https://cs231n.github.io/convolutional-networks/. [Accessed : 20 - Jun - 2019].
[32] J. Gehring, M. Auli, D. Grangier, D. Yarats, Y. N. Dauphin, and F. A. Research, "Convolutional Sequence to Sequence Learning," Proceedings of the 34th International Conference on Machine Learning-Volume 70., pp. 1243-1252, 2017.
[33] Y. N. Dauphin, A. Fan, M. Auli, and D. Grangier, "Language Modeling with Gated Convolutional Networks," Proceedings of the 34th International Conference on Machine Learning-Volume 70., pp. 933-941, 2017.
[34] K. He, X. Zhang, S. Ren, J. Sun, and M. Research, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE conference on computer vision and pattern recognition., pp. 770-778, 2016.
[35] "從《Convolutional Sequence to Sequence Learning》到《Attention Is All You Need》," [Online]. Available: https://zhuanlan.zhihu.com/p/27464080. [Accessed : 25 - Jun - 2019].
[36] bentrevett, "Convolutional Sentiment Analysis," [Online]. Available: https://github.com/bentrevett/pytorch-seq2seq. [Accessed : 28 - Jun - 2019].
[37] P. Yang, X. Sun, W. Li, S. Ma, W. Wu, and H. Wang, "SGM: Sequence Generation Model for Multi-Label Classification," Proceedings of the 27th International Conference on Computational Linguistics., pp. 3915-3926, 2018.

[38] bentrevett, "Neural Machine Translation by Jointly Learning to Align and Translate," [Online]. Available: https://github.com/bentrevett/pytorch-seq2seq/. [Accessed : 30 - Jun - 2019].
[39] bentrevett, "Packed Padded Sequences, Masking and Inference," [Online]. Available: https://github.com/bentrevett/pytorch-seq2seq/. [Accessed : 30 - Jun - 2019].
[40] "NLTK 3.4.3 documentation," [Online]. Available: https://www.nltk.org/. [Accessed : 30 - Jun - 2019].
[41] 蘇木春、張孝德, 機器學習：類神經網路、模糊系統以及基因演算，第四版, 全華科技圖書, 民國一百零六年.

簡易檢索 / 詳目顯示

相關論文