用於不斷發展的分類法之具備新穎性檢測的分層文本分類技術

簡易檢索 / 詳目顯示

回結果列表

研究生：	楊若函 Jo-Han Yang
論文名稱：	用於不斷發展的分類法之具備新穎性檢測的分層文本分類技術 Hierarchical text classification with novelty detection for evolving taxonomies
指導教授：	蔡宗翰 Richard Tzong-Han Tsai
口試委員:
學位類別：	碩士 Master
系所名稱：	資訊電機學院 - 資訊工程學系 Department of Computer Science & Information Engineering
論文出版年：	2021
畢業學年度：	109
語文別：	英文
論文頁數：	36
中文關鍵詞：	自然語言處理、階層式文字分類、階層式新類偵測
外文關鍵詞：	Nature Language Processing, Hierarchical Text Classification, Hierarchical Novelty Detection
相關次數：	點閱：28 下載：0
分享至:	分享至facebook 分享至twitter

查詢本校圖書館目錄查詢臺灣博碩士論文知識加值系統勘誤回報

在深度學習的領域中，分類任務的技術越趨成熟，而近年來相關研
究人員也陸續投身於具備新穎性檢測的階層式分類方法。我們在此篇
論文提出了一個利用分解信心值和連接條件機率，達到具新穎性檢測
的階層式分類模型，且訓練過程中不需加入額外的新類別資料，而基
準模型包含了自頂向下方法及攤平方法。將我們提出的模型與基準模
型相互比較，從結果可以得知，我們的模型除了有效提升已知類別的
準確度外，於尋找新的分類上也更加精準。此外針對階層式新類偵測
的任務，論文中提出了一個新的算分方法，目的是同時考慮新類偵測
以及階層式分類兩個任務，使其能更精確地顯示出模型的效能。

With the development of classification methods based on deep learning,
hierarchical classification tasks with new class detection began to attract researchers’ attention. In this paper, we propose a hierarchical classification
with a novelty detection model by decomposing confidence and concatenating conditional probability, which can be trained without labeled novelty data.
We compare it with a baseline model that combines the topdown method and
flatten method. From the results, we found that our model can improve the
classification accuracy of known categories and find instances belonging to
new categories more effectively. We propose a new evaluation metric for the
hierarchical novelty detection task. It considers both novelty detection and
hierarchical classification so that it is able to express the performance of the
model more obviously.

Contents
中文摘要 i
Abstract ii
謝誌 iii
Contents iv
List of Figures vi
List of Tables vii
1 Introduction 1
2 Related Work 4
2.1 Pre­trained Language Model . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 Hierarchical Classification . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Novelty detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.4 Hierarchical evaluation measure . . . . . . . . . . . . . . . . . . . . . . 8
3 Method 11
3.1 Encoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Hierarchical Decomposed Network . . . . . . . . . . . . . . . . . . . . . 12
3.3 Concatenating Conditional Probability . . . . . . . . . . . . . . . . . . . 15
iv
4 Experiment 17
4.1 Dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.2 Evaluation Metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.3 Evaluation of our model and baseline . . . . . . . . . . . . . . . . . . . 20
4.4 Ablation Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5 Conclusion 23
Bibliography 24
                                

[1] J. Devlin, M.W. Chang, K. Lee, and K. Toutanova, “Bert: Pretraining of deep
bidirectional transformers for language understanding,” 2019.
[2] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradientbased learning applied to
document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324,
1998.
[3] L. S. Larkey and W. B. Croft, “Combining classifiers in text categorization,” in
Proceedings of the 19th Annual International ACM SIGIR Conference on Research
and Development in Information Retrieval, ser. SIGIR ’96. New York, NY, USA:
Association for Computing Machinery, 1996, p. 289–297. [Online]. Available:
https://doi.org/10.1145/243199.243276
[4] D. Gao, W. Yang, H. Zhou, Y. Wei, Y. Hu, and H. Wang, “Deep hierarchical classification for category prediction in ecommerce system,” 2020.
[5] G.R. Xue, D. Xing, Q. Yang, and Y. Yu, “Deep classification in largescale
text hierarchies,” in Proceedings of the 31st Annual International ACM SIGIR
Conference on Research and Development in Information Retrieval, ser. SIGIR ’08.
24
New York, NY, USA: Association for Computing Machinery, 2008, p. 619–626.
[Online]. Available: https://doi.org/10.1145/1390334.1390440
[6] D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and outofdistribution examples in neural networks,” 2018.
[7] P. F. Brown, P. V. deSouza, R. L. Mercer, V. J. D. Pietra, and J. C. Lai, “Classbased
ngram models of natural language,” Comput. Linguist., vol. 18, no. 4, p. 467–479,
Dec. 1992.
[8] T. Mikolov, M. Karafiát, L. Burget, J. H. Cernocký, and S. Khudanpur, “Recurrent
neural network based language model,” in INTERSPEECH, 2010.
[9] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,
L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in
Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio,
H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran
Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper/
2017/file/3f5ee243547dee91fbd053c1c4a845aaPaper.pdf
[10] C. N. Silla and A. A. Freitas, “A survey of hierarchical classification across different
application domains,” Data Mining and Knowledge Discovery, vol. 22, pp. 31–72,
2010.
[11] S. Kumar, J. Ghosh, and M. Crawford, “Hierarchical fusion of multiple classifiers
for hyperspectral data analysis,” Pattern Anal. Appl., vol. 5, pp. 210–220, 06 2002.
25
[12] W. Liu, X. Wang, J. D. Owens, and Y. Li, “Energybased outofdistribution detection,” 2021.
[13] Y.C. Hsu, Y. Shen, H. Jin, and Z. Kira, “Generalized odin: Detecting outofdistribution image without learning from outofdistribution data,” 06 2020, pp.
10 948–10 957.
[14] S. Kiritchenko and F. Famili, “Functional annotation of genes using hierarchical text
categorization,” Proceedings of BioLink SIG, ISMB, 01 2005.
[15] Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao,
Q. Gao, K. Macherey, J. Klingner, A. Shah, M. Johnson, X. Liu, Łukasz Kaiser,
S. Gouws, Y. Kato, T. Kudo, H. Kazawa, K. Stevens, G. Kurian, N. Patil, W. Wang,
C. Young, J. Smith, J. Riesa, A. Rudnick, O. Vinyals, G. Corrado, M. Hughes, and
J. Dean, “Google’s neural machine translation system: Bridging the gap between
human and machine translation,” 2016.
[16] K. Lee, K. Lee, K. Min, Y. Zhang, J. Shin, and H. Lee, “Hierarchical novelty detection for visual object recognition,” 2018

簡易檢索 / 詳目顯示

相關論文