中文摘要
審閱健保提供者的醫療診斷記錄可以進行細緻的分析，為小型診所和大型醫療機構的物流管理、品質控制和成本效益產生有價值的見解。
然而，大多數醫生仍然使用紙本病歷，而現有的電子病歷（如放射學報告）主要以非結構化格式編寫。
這是一個耗時的過程，因為具有醫學領域知識的醫療人員必須閱讀整個文檔才能完全理解非結構化、自由格式的文本，無論是手寫的還是數位化的格式。
對於非醫學相關人員來說，這是一項不可能完成的任務，因為現在的醫學報告傾向於詳細記錄當下情況，並附有許多技術註釋和速記術語。
除此之外，含糊不清的術語如: 排除、疑似診斷或鑑別診斷常用於表示器官或疾病的狀況。
但手術結果數據通常報告為原始發病率和死亡率，這不一定能有效且直觀地反映出手術治療的有效性與否。
若是有一種能夠區分正常病例和異常病例並識別支持結論的關鍵句子的工具，可能會減少查看病歷的時間成本。
此外，它還可以幫助沒有足夠醫學知識的普通檢閱人員更精準、正確地理解病例報告。
NLP是自然語言處理的縮小，它是一種能用於自動文檔分類的新技術。它能將非結構化文檔轉換為結構化格式以利於後續信息提取的數值分析的工作。因此，我們的目標有兩個：1）使用 NLP 挖掘診斷報告中的消息，以區分報告案例是正常還是異常 2）找出哪些句子最為關鍵，可能導致出結果。現在已經有很多最先進的方法來實踐 NLP，例如樸素貝葉斯、長短期記憶 (LSTM)、BERT…等。
通過使用 BERT 分類器模型，在醫學主題詞 (MeSH) 數據集（n = 3,955個帶標記的報告）上，識別病例是否異常的分類任務，我們的校驗分數及用於校驗的數據數（n = 792 手動標記，正常病例的 f1-score = 0.97，異常案例的f1-score = 0.94）和測試分數及用於測驗的數據數（n = 395 手動標記，正常案例的 f1-score = 0.98，異常案例的 f1-score = 0.96）。而且在使用了sentence-BERT之後，我們可以得到每個句子和整個報告之間的相似度。這有助於找出哪個句子可能屬於正常或異常。

Abstract
Reviewing the medical diagnosis records of a health care provider allows the meticulous analysis for generating invaluable insights into logistics management, quality control, and cost effectiveness for both small clinics and large health organizations. However, the majority of physicians still use paper medical records and the existing electronic medical records such as radiology reports are scripted mainly in an unstructured format. It is a time-consuming process as the medical examiner with domain knowledge has to read entire document in order to be able to fully understand the unstructured, free-form texts, which are in no matter hand-written or digital formats. It is an impossible task to non-medical examiners since the medical reports nowadays tend to record the situation in detail with many technical notes and terms in shorthand. In addition, the ambiguous terms “rule out”, “suspected diagnosis”, or “differential diagnosis” are more commonly used to denote the conditions of the organs or the diseases. But surgical outcome data are generally reported as raw morbidity and mortality, which do not necessarily reflect the level of effectiveness of surgical treatment. A tool, which can distinguish normal from abnormal cases and identify the key sentences supporting the conclusion, can possibly reduce the time cost in reviewing medical records. Furthermore, it can also help common examiners who have not enough medical knowledge to understand the reports more precisely and correctly.
NLP, which stands for natural language processing, is a novel technology for automatic document classification. It is the work horse to transform unstructured documents into a structured format to enable numerical analysis for information extraction. Thus, our goals are two folds: 1) to use NLP for mining the messages from diagnosis reports to distinguish the report cases are normal or abnormal 2) to find which sentences may lead to the consequences. There have been plenty of state-of-art methods to practice NLP such as Naïve-Bayes, Long Short-Term Memory (LSTM), Bidirectional Encoder Representations from Transformer (BERT) etc.
By using BERT classifier model, the identifying the case is abnormal or not task on Medical Subject Headings (MeSH) dataset (n = 3,955 annotated reports) we got validate (n = 792 labeled manually, f1-score of Normal cases = 0.97, f1-score of Abnormal cases = 0.94) and test score (n = 395 labeled manually, f1-score of Normal case= 0.98, f1-score of Abnormal cases = 0.96). And with sentence-BERT, we can get the similarity between each sentence and whole reports. It would help finding which sentence may belong to normal or abnormal.

[1] Producing NLP-based On-line Contentware Francis Wolinski, Frantz Vichot, Olivier Gremont 16 Sep 1998 https://arxiv.org/abs/cs/9809021
[2] Automated Triaging of Adult Chest Radiographs with Deep Artificial Neural Networks.Mauro Annarumma*, Samuel J. Withey*, Robert J. Bakewell, Emanuele Pesce, Vicky Goh, Giovanni Montana * M.A. and S.J.W. contributed equally to this work.Published Online:Jan 22 2019 https://doi.org/10.1148/radiol.2018180921
[3] An empirical study of the naive Bayes classifier I. Rish T.J. Watson Research Cente https://www.cc.gatech.edu/~isbell/reading/papers/Rish.pdf
[4] Modelling Radiological Language with Bidirectional Long Short-Term Memory Networks Savelie Cornegruta, Robert Bakewell, Samuel Withey, Giovanni Montana 27 Sep 2016 https://arxiv.org/abs/1609.08409
[5] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova 11 Oct 2018 https://arxiv.org/abs/1810.04805
[6] CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, Jayne Seekins, David A. Mong, Safwan S. Halabi, Jesse K. Sandberg, Ricky Jones, David B. Larson, Curtis P. Langlotz, Bhavik N. Patel, Matthew P. Lungren, Andrew Y. Ng 21 Jan 2019 https://arxiv.org/abs/1901.07031
[7] Deep neural network improves fracture detection by clinicians Robert Lindsey, Aaron Daluiski, Sumit Chopra, View ORCID ProfileAlexander Lachapelle, Michael Mozer, Serge Sicular, Douglas Hanel, Michael Gardner, Anurag Gupta, Robert Hotchkiss, and Hollis Potter PNAS November 6, 2018 115 (45) 11591-11596 first published October 22, 2018 https://doi.org/10.1073/pnas.1806905115
[8] Insights into the problem of alarm fatigue with physiologic monitor devices: a comprehensive observational study of consecutive intensive care unit patients Barbara J Drew , Patricia Harris , Jessica K Zègre-Hemsey , Tina Mammone , Daniel Schindler , Rebeca Salas-Boni , Yong Bai , Adelita Tinoco , Quan Ding , Xiao Hu PMID: 25338067 PMCID: PMC4206416 DOI: 10.1371/journal.pone.0110274 https://pubmed.ncbi.nlm.nih.gov/25338067/
[9] Interpretation of plain chest roentgenogram Suhail Raoof 1, David Feigin 2, Arthur Sung 3, Sabiha Raoof 4, Lavanya Irugulpati 5, Edward C Rosenow 3rd PMID: 22315122 https://pubmed.ncbi.nlm.nih.gov/22315122/
[10] A general natural-language text processor for clinical radiology.
C Friedman, P O Alderson, J H Austin, J J Cimino, and S B Johnson 1994 Mar-Apr PMCID: PMC116194 PMID: 7719797 doi: 10.1136/jamia.1994.95236146 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC116194/
[11] Document-level classification of CT pulmonary angiography reports based on an extension of the ConText algorithm Brian E.ChapmanaSeanLeecHyunseok PeterKangbWendy W.Chapmana October 2011 https://doi.org/10.1016/j.jbi.2011.03.011
[12] Deep Learning to Classify Radiology Free-Text Reports Matthew C. Chen, Robyn L. Ball, Lingyao Yang, Nathaniel Moradzadeh, Brian E. Chapman, David B. Larson, Curtis P. Langlotz, Timothy J. Amrhein, Matthew P. Lungren Published Online: Nov 13 2017 https://doi.org/10.1148/radiol.2017171115
[13] Can Artificial Intelligence Reliably Report Chest X-Rays?: Radiologist Validation of an Algorithm trained on 2.3 Million X-Rays Preetham Putha, Manoj Tadepalli, Bhargava Reddy, Tarun Raj, Justy Antony Chiramal, Shalini Govil, Namita Sinha, Manjunath KS, Sundeep Reddivari, Ammar Jagirdar, Pooja Rao, Prashant Warier 19 Jul 2018 https://arxiv.org/abs/1807.07455
[14] Automatic Lung Cancer Prediction from Chest X-ray Images Using Deep Learning Approach Worawate Ausawalaithong, Sanparith Marukatat, Arjaree Thirach, Theerawit Wilaiprasitporn 31 Aug 2018 https://arxiv.org/abs/1808.10858
[15] Supervised and unsupervised language modelling in Chest X-Ray radiological reports Ignat Drozdov ,Daniel Forbes,Benjamin Szubert,Mark Hall,Chris Carlin,David J. Lowe Published: March 10, 2020 https://doi.org/10.1371/journal.pone.0229963
[16] Revisiting Unreasonable Effectiveness of Data in Deep Learning Era Chen Sun, Abhinav Shrivastava, Saurabh Singh, Abhinav Gupta 10 Jul 2017 https://arxiv.org/abs/1707.02968
[17] “Unsupervised Topic Modeling in a Large Free Text Radiology Report Repository,” Saeed Hssanpour and Curtis P. Langlotz,Journal of Digital Imaging, Feb 2016, vol 29, no 1, pp 59-62. doi: 10.1007/s10278-015-9823-3
[18] Efficient Estimation of Word Representations in Vector Space Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean 16 Jan 2013 https://arxiv.org/abs/1301.3781
[19] PadChest: A large chest x-ray image dataset with multi-label annotated reports Aurelia Bustos, Antonio Pertusa, Jose-Maria Salinas, Maria de la Iglesia-Vayá 22 Jan 2019 https://arxiv.org/abs/1901.07441
[20] Convolutional Neural Networks for Sentence Classification Yoon Kim 25 Aug 2014 https://arxiv.org/abs/1408.5882
[21] Attention Is All You Need Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin 12 Jun 2017 https://arxiv.org/abs/1706.03762
[22] HuggingFace's Transformers: State-of-the-art Natural Language Processing Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, Alexander M. Rush 9 Oct 2019 https://arxiv.org/abs/1910.03771
[23] MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs Alistair E. W. Johnson, Tom J. Pollard, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih-ying Deng, Yifan Peng, Zhiyong Lu, Roger G. Mark, Seth J. Berkowitz, Steven Horng 21 Jan 2019 https://arxiv.org/abs/1901.07042
[24] Siamese Neural Networks for One-shot Image Recognition Gregory Koch Master of Science Graduate Department of Computer Science University of Toronto 2015 http://www.cs.toronto.edu/~gkoch/files/msc-thesis.pdf
[25] Bidirectional LSTM-CRF Models for Sequence Tagging Zhiheng Huang, Wei Xu, Kai Yu 9 Aug 2015 https://arxiv.org/abs/1508.01991
[26] BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova 11 Oct 2018 https://arxiv.org/abs/1810.04805
[27] DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter Victor Sanh, Lysandre Debut, Julien Chaumond, Thomas Wolf 2 Oct 2019 https://arxiv.org/abs/1910.01108
[28] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers, Iryna Gurevych 27 Aug 2019 https://arxiv.org/abs/1908.10084