跳到主要內容

簡易檢索 / 詳目顯示

研究生: 李哲豪
Che-Hao Li
論文名稱: 使用句對模型在文章中抓取相關資訊用於問題生成
Using Sentence Pair Model to Capture Relevant Information from Document for Question Generation
指導教授: 蔡宗翰
Tsung-Han Tsai
口試委員:
學位類別: 碩士
Master
系所名稱: 資訊電機學院 - 資訊工程學系
Department of Computer Science & Information Engineering
論文出版年: 2019
畢業學年度: 107
語文別: 中文
論文頁數: 41
中文關鍵詞: 問題生成閱讀理解序列到序列注意力機制複製機制句對模型深度學習
外文關鍵詞: Question Generation, Reading Comprehension, Sequence to Sequence, Attention Mechanism, Copy Mechanism, Sentence Pair Model, Deep Learning
相關次數: 點閱:11下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 近幾年問題生成的研究發展迅速,過去以句子的語法結構定義規則生成問題,隨著深度學習成熟的技術,現今機器能理解語意並自動產生適當的問題。

    問題生成的目標是給定一段文字訊息與答案,產生相對應的問題,與機器閱讀理解任務類似,因此閱讀理解的資料集常被用在問題生成任務中。以往問題生成模型的輸入並非整篇文章,而是包含答案的句子,但有些問題的內容和答案不在同一個句子,可能是依據其他句子資訊產生該答案的問題,於是本論文提出一個新架構,由句對模型和問題生成模型所組成,利用句對模型處理文章結構,將每一句資訊與包含答案的句子進行匹配,計算各自的相關程度並且重新賦予句子權重,接著傳送到問題生成模型產生最終的問題。句對模型主要目的是從整篇文章中自動找尋和答案有關的內容進而產生適合的問題。

    實驗結果表示,我們的系統能有效處理文章結構,相比只有問題生成模型的系統,在中文和英文的資料集都有更好的表現。


    In recent years, question generation (QG) has developed rapidly. In the past, using rules that are based on syntactic structure to generate questions. Nowadays, the machine can understand semantic and automatically generate appropriate questions with a proven technique of deep learning.

    Question generation aims to generate corresponding questions from a given passage and answer. It is similar to machine reading comprehension (RC) task. Therefore, reading comprehension dataset is often used to question generation task. The input of the previous question generation model is the sentence containing the answer rather than the whole article. However, the content of some questions and its answers are not in the same sentence. The question may be based on other information in sentences. Then, our paper proposed a new framework which consists of sentence pair model and question generation model. Using the sentence pair model to process article structure. Its method is matching each sentence and the sentence containing the answer to compute the respective degree of correlation to reweight sentences and then produce questions by question generation model. The main purpose of sentence pair model is to automatically find the content related to the answer from the article.

    Experiment results show that our system can handle article structure. In contrast to a system with only question generation model, our system has better performance in Chinese and English dataset.

    摘要ii Abstract iii 誌謝v 目錄 vii 圖目錄 ix 表目錄 x 一、緒論1 1.1 研究背景.................................................................. 1 1.2 研究動機.................................................................. 3 1.3 章節概要.................................................................. 4 二、相關研究5 2.1 自然語言處理(Natural Language Processing, NLP)............ 5 2.2 深度學習(Deep Learning) ............................................ 5 2.3 自然語言生成(Natural Language Generation, NLG)........... 6 2.4 注意力機制(Attention Mechanism) ................................ 7 2.5 問題生成(Question Generation, QG) .............................. 7 2.6 自然語言推理(Natural language Inference, NLI) ............... 9 三、系統架構10 3.1 資料前處理............................................................... 11 3.2 詞嵌入..................................................................... 11 3.3 句子編碼器............................................................... 11 3.3.1 Attention-Based Convolutional Neural Network ........ 12 3.3.2 Match-LSTM .................................................... 14 3.4 訊息編碼器............................................................... 15 3.5 問題解碼器............................................................... 17 四、實驗方法與討論20 4.1 資料集..................................................................... 20 4.1.1 台達閱讀理解資料集(Delta Reading Comprehension Dataset, DRCD)................................................... 20 4.1.2 史丹佛問題答案資料集(Stanford Question Answering Dataset, SQuAD)................................................... 20 4.2 評估方法.................................................................. 21 4.2.1 BLEU ............................................................. 21 4.2.2 F-score ............................................................ 22 4.3 參數設定.................................................................. 22 4.4 實驗結果.................................................................. 23 4.5 討論........................................................................ 26 五、結論與未來工作28 參考文獻29

    [1] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+ questions for
    machine comprehension of text,” arXiv preprint arXiv:1606.05250, 2016.
    [2] X. Du, J. Shao, and C. Cardie, “Learning to ask: Neural question generation for
    reading comprehension,” arXiv preprint arXiv:1705.00106, 2017.
    [3] C. C. Shao, T. Liu, Y. Lai, Y. Tseng, and S. Tsai, “Drcd: a chinese machine reading
    comprehension dataset,” arXiv preprint arXiv:1806.00920, 2018.
    [4] I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural
    networks,” in Advances in neural information processing systems, pp. 3104–3112,
    2014.
    [5] D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning
    to align and translate,” arXiv preprint arXiv:1409.0473, 2014.
    [6] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE
    Transactions on Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997.
    [7] M. Heilman and N. A. Smith, “Good question! statistical ranking for question generation,”
    in Human Language Technologies: The 2010 Annual Conference of the
    North American Chapter of the Association for Computational Linguistics, pp. 609–
    617, Association for Computational Linguistics, 2010.
    [8] Q. Zhou, N. Yang, F. Wei, C. Tan, H. Bao, and M. Zhou, “Neural question generation
    from text: A preliminary study,” in National CCF Conference on Natural Language
    Processing and Chinese Computing, pp. 662–671, Springer, 2017.
    [9] C. Gulcehre, S. Ahn, R. Nallapati, B. Zhou, and Y. Bengio, “Pointing the unknown
    words,” arXiv preprint arXiv:1603.08148, 2016.
    [10] J. Gu, Z. Lu, H. Li, and V. O. Li, “Incorporating copying mechanism in sequenceto-
    sequence learning,” arXiv preprint arXiv:1603.06393, 2016.
    [11] L. Song, Z. Wang, W. Hamza, Y. Zhang, and D. Gildea, “Leveraging context information
    for natural question generation,” in Proceedings of the 2018 Conference
    of the North American Chapter of the Association for Computational Linguistics:
    Human Language Technologies, Volume 2 (Short Papers), pp. 569–574, 2018.
    [12] Z. Wang, H. Mi, W. Hamza, and R. Florian, “Multi-perspective context matching
    for machine comprehension,” arXiv preprint arXiv:1612.04211, 2016.
    [13] V. Kumar, G. Ramakrishnan, and Y.-F. Li, “A framework for automatic question
    generation from text using deep reinforcement learning,” arXiv preprint arXiv:
    1808.04961, 2018.
    [14] Z. Tu, Z. Lu, Y. Liu, X. Liu, and H. Li, “Modeling coverage for neural machine
    translation,” arXiv preprint arXiv:1601.04811, 2016.
    [15] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic
    evaluation of machine translation,” in Proceedings of the 40th annual meeting on
    association for computational linguistics, pp. 311–318, Association for Computational
    Linguistics, 2002.
    [16] C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” Text Summarization
    Branches Out, 2004.
    [17] Y. Kim, H. Lee, J. Shin, and K. Jung, “Improving neural question generation using
    answer separation,” arXiv preprint arXiv:1809.02393, 2018.
    [18] S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning, “A large annotated corpus
    for learning natural language inference,” arXiv preprint arXiv:1508.05326, 2015.
    [19] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation,
    vol. 9, no. 8, pp. 1735–1780, 1997.
    [20] T. Rocktäschel, E. Grefenstette, K. M. Hermann, T. Kočiskỳ, and P. Blunsom, “Reasoning
    about entailment with neural attention,” arXiv preprint arXiv:1509.06664,
    2015.
    [21] W. Yin, H. Schütze, B. Xiang, and B. Zhou, “Abcnn: Attention-based convolutional
    neural network for modeling sentence pairs,” Transactions of the Association for
    Computational Linguistics, vol. 4, pp. 259–272, 2016.
    [22] S. Wang and J. Jiang, “Learning natural language inference with lstm,” arXiv
    preprint arXiv:1512.08849, 2015.
    [23] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations
    of words and phrases and their compositionality,” in Advances in neural
    information processing systems, pp. 3111–3119, 2013.
    [24] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv
    preprint arXiv:1412.6980, 2014.

    QR CODE
    :::