基於深度學習的自然場景文字分析相關研究在近年來十分盛行，文字
區域偵測更是其中的重要環節。現今文字偵測大多以字串為標記單位，然而
字串中可能包含不同語言的文字，標記時較不易確認該字串文字所屬語言。
本研究提出以字元為單位的偵測方式，不僅能準確標記所屬語言，也讓辨識
時能採用相對應語言模型以達到更好的效果。對於辨識模型而言，字串需要
考量不規則的文字走向，且字串辨識模型通常需要較大量的訓練資料與訓
練時間。反觀字元辨識則不太需要考慮文字走向，訓練模型相對簡單省時，
且面對多語言自然場景文字時能更有彈性地根據語言特性，選擇適合的辨
識單位與方法。本研究使用高解析度網路架構，以字元為偵測單位，標記字
元區域並點出字元中心，且利用多個通道進行語言分類。由於真實資料集字
元標記的缺乏，我們提出針對字元的弱監督式學習方法，使得網路在缺乏字
元標記的情況下也能在偵測字元的表現有明顯的效果提升。在多語言分類
上，不管是偵測後用個別分類器亦或是在偵測的同時進行語言辨識皆有一
定的效果，驗證了字元辨識的可行性。我們實驗以拉丁文(英數字)、中文、
日文、韓文為範例，分析本設計的可行性與合理性。

In recent years, scene text analysis based on deep learning techniques draw
a lot of research attention. Text detection in natural scenes is an important step of
scene text analysis and most of the existing text detection designs are based on
string detection. However, a string may contain words of different languages so it
is not easy to mark the language to which the string belongs accurately. Scene text
recognition using string-level annotations need to consider the effect of irregular
orientations and require a lot of training data and training time. Conversely,
character-based recognition methodologies do not need to consider orientations,
which simplifies the training processes. Multilingual natural scene text
recognition may be benefited from the flexibility of selecting suitable recognition
models according to different language characteristics. In this research, we use a
high-resolution network architecture to label word regions and point out the
centers of characters, and also employ multiple channels for substring language
classification. Due to the lack of character-level annotations in real datasets, we
propose a weakly supervised learning approach for characters, enabling the
network to improve the detection of characters significantly. The performance of
multi-language recognition is verified by using individual classifiers after
detection or by performing language recognition at the same time. The feasibility
of the proposed design is verified by showing the character detection of different
languages, including Latin, Chinese, Japanese, and Korean, as examples.

[1] Baoguang Shi, Mingkun Yang , Xinggang Wang, Pengyuan Lyu, Cong Yao , and Xiang Bai. “ ASTER: An Attentional Scene Text Recognizer with Flexible Rectification” CVPR 2016
[2] Canjie Luo, Lianwen Jin, Zenghui Sun. “A Multi-Object Rectified Attention Network for Scene Text Recognition.” CVPR2019
[3] Jaderg, Simonyan, Vedaldi, Zisserman. “Reading text in the wild with convolutional neural network”,IJCV2016
[4] Minghui Liao,Baoguang Shi,Xiang Bai, “Textboxes++ : A Single-shot Oriented Scene Text Detector.” TIP2018
[5] Kaiming He,Georgia Gkioxari,Piotr Dollar,Ross Girshick, “Mask R-CNN”, Facebook AI Research
[6] Yuliang Liu, Sheng Zhang, Lianwen Jim, Lele Xie, Yaqiang Wu, Zhepeng Wang. “Omnidirectional Scene Text Detection with Sequential-free Box Discretization.” Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, 2019
[7] X. Zhou,C. Yao,H.Wen,Y. Wang,S. Zhou,W.He, J.Liang “EAST: An efficient and Accurate Scene Text Detector.”,CVPR 2017
[8] Jonathan Long, Evan Shelhamer, Trevor Darrell. “Fully Convolution Networks for Semantic Segmentation.” CVPR2015
[9] Minghui Liao, Zhaoyi Wan,Cong Yao,Kai Chan,Xiang Bai, “Real-time Scene Text Detection with Differentiable Binarization.” AAAI2020
[10] Jian Ye, Zhe Chen, Juhua Liu, Bo Du. “TextFuseNet: Scene Text Detection with Richer Fused Features.” IJCAI2020
[11] Youngmin Baek, Bado Lee, Dongyoon Han, Sangdoo Tun, Hwalsuk Lee. “Character Region Awareness For Text Detection.” CVPR2019
[12] Linjie Xing, Zhi Tian , Weilin Huang and Matthew R. Scott. “Convolutional Character Networks.” ICCV 2019
[13] Olaf Ronneberger, Philipp Fischer, Thomas Brox. “U-Net: Convolutional Networks for Biomedical Image Segmentation.” MICCAI2015
[14] Zhi-Hua Zhou《A Brief Introduction to Weakly Supervised Learning》journal National Science Review 2018.1
[15] Shangbang Long,Xin He,Cong Yao, “Scene Text Detection and Recognition: The Deep Learning Era”, IJCV2021
[16] Ankush Gupta, Andrea Vedakdi and Andrew Zisserman. “SynthText in the Wild Dataset.” CVPR2016
[17] Max Jaderberg, Karen Simonyan, Andrea Vedaldi and Andrew Zisserman. “Reading Text in the Wild with Convolution Neural Networks.” International journal of computer vision 2016
[18] Prasun Roy, Saumik Bhattacherya, Subhankar Ghosh, Umapada Pal. “STEFANN: Scene Text Editor using Font Adaptive Neural Network.” CVPR’20
[19] Shangbang Long, Cong Yao “UnrealText: Synthesizing Realistic Scene Text Images from the Unreal World” CVPR2020
[20] PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing, ACM Transactions on Graphics journal 2009
[21] Ke Sun,Bin Xiao,Dong Liu,Jingdong Wang. “Deep High-Resolution Representation Learning for Human Pose Estimation”, CVPR2019
[22] Mingxing Tan,Quoc V. Le. “EfficientNet: Rethinking Model Scaling for Convolutioin Neural Networks.”
[23] Nibal Nayef, Fei Yin, Imen Bizid, Hyunsoo Choi, Yuan Feng, “Icdar2017 Robust Reading Challenge on Multi-lingual Scene Text Detection and Script Identification”, 14th Iapr International Conference on Document Analysis and Recognition, 2017.
[24] Chee Kheng Chng,Chee Seng Chan. “Total-Text:A Comprehensive Dataset for Scene Text Detection and Recognition.” ICDAR2017