近年來有許多回答問題及產生問題的研究應用於閱讀理解中，並獲得相當不錯的成效。回答問題研究的目標是通過閱讀文章來回答問題，而產生問題研究希望根據文章產生多樣性問題且可作為閱讀測驗的考題。然而，產生問題有時會產生超出文章本身的範圍，以致於機器無法產生解答。本論文為了解決上述問題-答案不成對的問題，提出一解決方案。用戶只要輸入文章及問題類型，系統即可輸出具有品質的問題-答案對，其產生的問題和答案必須關注在相同的輸入文章句子上，並具備流暢度、相關性及正確性。
在本論文中，我們採用基於注意力機制的序列到序列模型，並於訊息編碼中加入階層式輸入。為了解決從龐大字典中產生問題及答案而造成無法收斂，所以輸出問題及答案採用動態字典，除包含常用單詞之外，還有隨文章不同而動態改變的單詞，讓訓練能夠收斂並大幅提升模型能力。所訓練的模型除了能產生問題-答案對之外，也能作為回答問題及產生問題，且效能優於檢索式模型。

In recent years, many studies in question answering and question generation have been applied in reading comprehension with satisfactory performance. Question answering aims to answer questions by reading documents, and question generation attempts to generate diverse questions from given documents and as reading test questions. However, some questions generated in question generation studies are beyond the scope of the given documents themselves, so that the machine cannot find an appropriate answer. In this thesis, we propose an approach to resolve the above question-answer unpaired situations. Given a document and the question type, the system can output a question-answer pair with quality. The question and answer generated must focus on the input document with fluency, relevance, and correctness.
In this thesis, we use the attention-based sequence-to-sequence model and add hierarchical input to the model encoder. In order to solve the problem that the model generating question and answer from huge vocabulary cannot converge, the output question and answer adopts a dynamic vocabulary, which includes not only commonly used words, but also words dynamically changing with the document. Such approach makes the training to converge and improves model capabilities. The trained model can generate question-answer pairs as well as do question answering and question generation. The resulting performance is better than the retrieval-based model.

[1] Kuyten, P., Bickmore, T., Stoyanchev, S., Piwek, P., Prendinger, H., & Ishizuka, M. (2012, September). Fully automated generation of question-answer pairs for scripted virtual instruction. In International Conference on Intelligent Virtual Agents (pp. 1-14). Springer, Berlin, Heidelberg.
[2] Sukhbaatar, S., Weston, J., & Fergus, R. (2015). End-to-end memory networks. In Advances in neural information processing systems (pp. 2440-2448).
[3] Weston, J., Bordes, A., Chopra, S., Rush, A. M., van Merriënboer, B., Joulin, A., & Mikolov, T. (2015). Towards ai-complete question answering: A set of prerequisite toy tasks. arXiv preprint arXiv:1502.05698.
[4] Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016). Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
[5] Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., & Deng, L. (2016). MS MARCO: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
[6] Du, X., Shao, J., & Cardie, C. (2017). Learning to Ask: Neural Question Generation for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Vol. 1, pp. 1342-1352).
[7] Wang, W., Yang, N., Wei, F., Chang, B., & Zhou, M. (2017). Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Vol. 1, pp. 189-198).
[8] Tan, C., Wei, F., Yang, N., Du, B., Lv, W., & Zhou, M. (2017). S-net: From answer extraction to answer generation for machine reading comprehension. arXiv preprint arXiv:1706.04815.
[9] Luong, T., Pham, H., & Manning, C. D. (2015). Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (pp. 1412-1421).
[10] Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing, 45(11), 2673-2681.
[11] Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010). Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association.
[12] Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
[13] Karpathy, A. (2015). The unreasonable effectiveness of recurrent neural networks. Andrej Karpathy blog. http://karpathy.github.io/2015/05/21/rnn-effectiveness/
[14] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112).
[15] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
[16] OpenNMT, http://opennmt.net/
[17] TensorFlow, https://www.tensorflow.org/
[18] Wang, T., Yuan, X., & Trischler, A. (2017). A Joint Model for Question Answering and Question Generation. arXiv preprint arXiv:1706.01450.
[19] Indurthi, S., Raghu, D., Khapra, M. M., & Joshi, S. (2017). Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers (Vol. 1, pp. 376-385).
[20] WikiAnswers, http://knowitall.cs.washington.edu/oqa/data/wikianswers/
[21] Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002, July). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics.
[22] Lin, C. Y. (2004). Rouge: A package for automatic evaluation of summaries. Text Summarization Branches Out.
[23] Good, I. J. (1953). The population frequencies of species and the estimation of population parameters. Biometrika, 40(3-4), 237-264.
[24] Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.
[25] Pagliardini, M., Gupta, P., & Jaggi, M. (2017). Unsupervised learning of sentence embeddings using compositional n-gram features. arXiv preprint arXiv:1703.02507.
[26] Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
[27] Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5), 378.
[28] Benesty, J., Chen, J., Huang, Y., & Cohen, I. (2009). Pearson correlation coefficient. In Noise reduction in speech processing (pp. 1-4). Springer Berlin Heidelberg.