影像中的文字為重要的感興趣區域(regions of interest)，在影像中定位文字供後續處理能夠幫助該影像相關資訊的擷取，並有利於許多有趣應用的開發。近年來語義分割和通用物件檢測框架技術已被文字偵測任務所廣泛採用，兩者在實作中有各自的優勢與缺點。本研究提出結合兩者優點的文字偵測機制，其中包含一個主要文字串偵測網路輔以一個文字精修網路。主要網路利用語意分割的方式並搭配FPN (Feature Pyramid Network)與ASPP (Atrous Spatial Pyramid Pooling)等技術，強化特徵提取效果，藉此偵測文字區域與邊框，將其視為主要結果且具備高召回率。我們接著使用以區域檢測框架為基礎的精修網路再次分析可能的文字區域，將主要結果中較不確定區域以精修網路協助判斷，最後再使用非極大值抑制技術(Non-Maximum Suppression, NMS)得到最終的文字區域偵測結果。實驗結果顯示本研究能有效的在複雜場景中偵測文字，並藉此探討兩種不同架構的深度學習網路在目標應用中的使用方式。

Texts appearing in images are often regions of interest and locating such areas for further analysis may help to extract image-related information and facilitate many interesting applications. Pixel-based segmentation and region-based object classification are two methodologies for locating text areas in images and have their own pros and cons. In this research, we proposed a text detection scheme consisting a main pixel-based classification network and a supplemented region proposal network. The main network is a Fully Convolutional Network (FCN) employing Feature Pyramid Network (FPN) and Atrous Spatial Pyramid Pooling (ASPP) to identify text areas and borders with higher recall. Certain areas are further processed by the supplemented refinement network, i.e., a simplified Connectionist Text Proposal Network (CTPN) with higher precision. Non-Maximum Suppression (NMS) is then applied to form suitable text-lines. The experimental results show feasibility of the proposed text-detection scheme.

[1] N. Dalal, B. Triggs, “Histograms of oriented gradients for human detection.” IEEE International Conference Computer Vision and Pattern Recognition (CVPR), 2005.
[2] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, D. Hassabis, "Mastering the game of Go with deep neural networks and tree search," Nature,vol. 529(7587), pp.484-489, 2016.
[3] M. Schuster and K. K. Paliwal, “Bidirectional recurrent neural networks,” IEEE Transactions on Signal Processing, vol. 45, no. 11, pp. 2673–2681, 1997.
[4] D.G. Lowe, “Object recognition from local scale-invariant features,” Proceedings of the International Conference on Computer Vision: 1150–1157. 1999.
[5] P. Viola, M. Jones, “Rapid object detection using a boosted cascade of simple features,” Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001
[6] C. Cortes, V. Vapnik, Support-vector networks. Machine Learning. 1995, 20 (3): 273–297.
[7] Yoav Freund, Robert Schapire, “Experiments with a New Boosting Algorithm,” Machine Learning: Proceedings of the Thirteenth International Conference, 1996.
[8] Y.-F. Pan, X. Hou, and C.-L. Liu, “Hybrid approach to detect and localize texts in natural scene images,” IEEE Trans. Image Processing (TIP), vol. 20, pp. 800–813, 2011.
[9] K. Wang, B. Babenko, and S. Belongie, “End-to-end scene text recognition,” in IEEE International Conference on Computer Vision (ICCV), 2011.
[10] B. Epshtein, O. Eyal, W. Yonatan, "Detecting text in natural scenes with stroke width transform." IEEE International Conference Computer Vision and Pattern Recognition (CVPR), 2010.
[11] C. Yao, X. Bai, W. Liu, Y. Ma, Z. Tu, “Detecting texts of arbitrary orientations in natural images.” IEEE International Conference Computer Vision and Pattern Recognition (CVPR), 2012.
[12] W. Huang, Z. Lin, J. Yang, J. Wang, “Text localization in natural images usingstroke feature transform and text covariance descriptors.” IEEE International Conference on Computer Vision (ICCV), 2013.
[13] J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image and vision computing (IVC), vol. 22, pp. 761–767, 2004.
[14] L. Neumann, K. Matas, “Text localization in real-world images using eficiently pruned exhaustive search.” IEEE International Conference on Document Analysis and Recognition (ICDAR), 2011.
[15] L. Neumann, K. Matas, “Real-time scene text localization and recognition.” IEEE International Conference Computer Vision and Pattern Recognition (CVPR), 2012.
[16] W. Huang, Q. Yu, X. Tang, "Robust scene text detection with convolution neural network induced mser trees." European Conference on Computer Vision (ECCV), 2014.
[17] W. Huang, Z. Lin, J. Yang, and J. Wang, “Text localization in natural images using stroke feature transform and text covariance descriptors,” in IEEE International Conference on Computer Vision (ICCV), 2013.
[18] C. L. Zitnick and P. Dolla´r, “Edge boxes: Locating object proposals from edges,” in European Conference on Computer Vision (ECCV), 2014.
[19] He, T., Huang, W., Qiao, Y., Yao, J.: Text-attentional convolutional neural networks for scene text detection. IEEE Trans. Image Processing (TIP) 25, 2529–2541, 2016.
[20] L. Sun, Q. Huo, W. Jia, and K. Chen, “A robust approach for text detection from natural scene images,” Pattern Recognition, vol. 48, pp. 2906–2920, 2015.
[21] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[22] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition", 2015.
[23] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna “Rethinking the Inception Architecture for Computer Vision,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[24] G. Huang, Z. Liu, L. van der Maaten, K. Q. Weinberger “Densely Connected Convolutional Networks,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[25] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
[26] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 2015.
[27] X. Zhou, C. Yao, H. Wen, Y. Wang, S. Zhou, W. He, J. Liang “EAST: An Efficient and Accurate Scene Text Detector,” 2017 IEEE Conference on Computer Vision and Pattern Recognition.
[28] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241. Springer, 2015.
[29] K.-H. Kim, S. Hong, B. Roh, Y. Cheon, and M. Park, “PVANET: Deep but lightweight neural networks for realtime object detection.” arXiv preprint arXiv:1608.08021, 2016.
[30] L. Huang, Y. Yang, Y. Deng, and Y. Yu, “Densebox: Unifying landmark localization with end to end object detection.” arXiv preprint arXiv:1509.04874, 2015.
[31] M. Liao, B. Shi, X. Bai “TextBoxes++: A Single-Shot Oriented Scene Text Detector,” arXiv preprint arXiv:1801.02765, 2018.
[32] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, A. C. Berg “SSD: Single Shot MultiBox Detector,” arXiv preprint arXiv:1512.02325
[33] L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs,” IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 40, NO. 4, APRIL 2018
[34] T. Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, S. Belongie “Feature Pyramid Networks for Object Detection,” 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[35] Z. Tian, W. Huang, T. He, P. He, and Y. Qiao. Detecting text in natural image with connectionist text proposal network. In ECCV, 2016.
[36] F. Yu and V. Koltun, “Multi-scale context aggregation by dilated convolutions,” arXiv preprint arXiv:1511.07122, 2015.
[37] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications” arXiv preprint arXiv:1704.04861, 2017.
[38] D. Deng, H. Liu, X. Li, D. Cai, “PixelLink: Detecting Scene Text via Instance Segmentation,” AAAI18 – Vision.
[39] G. J. Brostow, J. Shotton, J. Fauqueur, R. Cipolla, “Segmentation and Recognition Using Structure from Motion Point Clouds,” Computer Vision – ECCV 2008 pp 44-57.
[40] D. Karatzas, F. Shafait, S. Uchida, M. Iwamura, L. Gomez, S. Robles, J. Mas, D. Fernandez, J. Almazan, L.P. de las Heras, "ICDAR 2013 Robust Reading Competition", In Proc. 12th International Conference of Document Analysis and Recognition, 2013, IEEE CPS, pp. 1115-1124.
[41] “ICDAR2017 Competition on Multi-lingual scene text detection and script identification,” https://rrc.cvc.uab.es/?ch=8
[42] A. Shrivastava, A. Gupta, R. Girshick, “Training Region-based Object Detectors with Online Hard Example Mining,” arXiv preprint arXiv:1604.03540, 2016.
[43] P. He, W. Huang, T. He, Q. Zhu, Y. Qiao, X. Li, “Single shot text detector with regional attention,” ICCV, 2017.
[44] L. Deng, Y. Gong, Y. Lin, J. Shuai, X. Tu, Y. Zhang, Z. Ma, M. Xie “Detecting Multi-Oriented Text with Corner-based Region Proposals,” arXiv preprint arXiv:1804.02690, 2018.
[45] Image Source https://yinguobing.com/separable-convolution/#fn2
[46] Y. Baek, B. Lee, D. Han, S. Yun, H. Lee, “Character Region Awareness for Text Detection” arXiv preprint arXiv:1904.01941, 2019.
[47] X. Liu, D. Liang, S. Yan, D. Chen, Y. Qiao, J. Yan, “FOTS: Fast Oriented Text Spotting with a Unified Network” arXiv preprint arXiv:1801.01671v2, 2018.
[48] P. Lyu, M. Liao, C. Yao, W. Wu, X. Bai, “Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes” arXiv preprint arXiv:1807.02242, 2018.
[49] J. Ma, W. Shao, H. Ye, L. Wang, H. Wang, Y. Zheng, X. Xue, “Arbitrary-Oriented Scene Text Detection via Rotation Proposals” arXiv preprint arXiv: 1703.01086, 2017.