本論文提出一多模態階層式分類器應用於大類別的植物種子辨識，我們首先擷取種子影像的不同模態特徵，包括LBP紋理、HOG特徵、GLCM特徵、BLOB幾何特徵、色彩統計特徵等五種模態特徵。每一種模態各自使用一個非監督的自組織映射神經網路進行分群，將特徵相近的種子聚類為同一群，不同的模態特徵可能有不同的分群結果。我們規劃了一個投票機制用以從五個模態分群結果篩選出模糊類別，做為第一階層的初分類結果。接著我們使用每一類種子的特徵映射聚類座標，以One-hot編碼方式，將五組座標合成一組二值座標向量作為新特徵向量，建立每一類種子的機率神經網路分類模型，進行第二個階層的植物種子辨識。我們以多模態階層式分類器對311類種子樣本進行實驗，最高能達到86%的辨識率，而以TOP-5為輸出辨識率更可高達95%。與深度神經網路中的ResNet50進行比較，結果可高出24%，運算時間與所使用資源量也遠少於深度神經網路。

This paper proposes a multi-modal hierarchical classifier for the identification of large-scale plant seeds. We first capture different modal features of seed images, including LBP texture, HOG features, GLCM features, BLOB geometric features, color statistical features, etc. Five types of modal features. Each modality uses an unsupervised self-organizing map neural network for clustering seeds with similar features into the same group, and different modality features may have different clustering results. We have planned a voting mechanism to screen out fuzzy categories from the five modal grouping results as the first classification results of the first class. Then we use the feature mapping cluster coordinates of each type of seeds, and use the one-hot encoding method to synthesize five sets of coordinates into a set of binary coordinate vectors as new feature vectors to establish a probabilistic neural network classification model for each type of seed. Carry out the second class of plant seed identification. We use multi-modal hierarchical classifier experiments on 311 types of seed samples, and the highest recognition rate can reach 86%. The recognition rate can be as high as 95% with TOP-5 as the output. Compared with ResNet50 in the deep neural network, the result can be 24% higher. The calculation time and the amount of resources used are far less than the deep neural network.

[1] L.-M. Hsu, Y.-Z. Lin, Y.-Z. Liu, and C.-Y. Wang, "Distribution of exotic weeds in coastal areas of southern and eastern Taiwan during summer", PLANT PROTECTION BULLETIN-TAIPEI-, vol. 48, no. 2, p. 129, 2006.
[2] 徐玲明, "外來雜草種子的檢疫, 偵測及鑑定", 中華民國雜草學會會刊, vol. 32, no. 1, pp. 11-21, 2011.
[3] M. Kumar, S. Gupta, X.-Z. Gao, and A. Singh, "Plant Species Recognition Using Morphological Features and Adaptive Boosting Methodology", IEEE Access, vol. 7, pp. 163912-163918, 2019.
[4] A. Arefi, A. M. Motlagh, and A. Khoshroo, "Recognition of weed seed species by image processing", Journal of Food, Agriculture and Environment, vol. 9, no. 1, pp. 379-383, 2011.
[5] I. Cinar and M. Koklu, "Classification of Rice Varieties Using Artificial Intelligence Methods", International Journal of Intelligent Systems and Applications in Engineering, vol. 7, no. 3, pp. 188-194, 2019.
[6] I. Chatnuntawech, K. Tantisantisom, P. Khanchaitit, T. Boonkoom, B. Bilgic, and E. Chuangsuwanich, "Rice classification using spatio-spectral deep convolutional neural network", arXiv preprint arXiv:.11491, 2018.
[7] 許鐈云, 陳宗禮, 黃國益, 郭育妏, and 黃卯昌, "水稻種子影像辨識系統之研究", 植物種苗, vol. 17, no. 4, pp. 1-18, 2015.
[8] N. Pandey, S. Krishna, and S. Sharma, "Automatic seed classification by shape and color features using machine vision technology", International Journal of Computer Applications Technology and Research, vol. 2, no. 2, pp. 208-213, 2013.
[9] A. V. Vlasov and A. S. Fadeev, "A machine learning approach for grain crop's seed classification in purifying separation", in Journal of Physics: Conference Series, vol. 803, no. 1, p. 012177: IOP Publishing, 2017.
[10] G. TekalignTujo, G. Dileep Kumar, D. ElifeneshYitagesu, and B. MeseretGirma, "A Predictive Model to Predict Seed Classes using Machine Learning", International Journal of Engineering Research and Technology, vol. 6, no. 08, pp. 334-344, 2017.
[11] Y. N. Dauphin and Y. Bengio, "Big neural networks waste capacity", arXiv preprint arXiv:1301.3583, 2013.
[12] M. M. Najafabadi, F. Villanustre, T. M. Khoshgoftaar, N. Seliya, R. Wald, and E. Muharemagic, "Deep learning applications and challenges in big data analytics", Journal of Big Data, vol. 2, no. 1, p. 1, 2015.
[13] R. M. Haralick, S. R. Sternberg, and X. Zhuang, "Image analysis using mathematical morphology", IEEE transactions on pattern analysis and machine intelligence, no. 4, pp. 532-550, 1987.
[14] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. J. J. o. a. i. r. Kegelmeyer, "SMOTE: synthetic minority over-sampling technique", Journal of artificial intelligence research, vol. 16, pp. 321-357, 2002.
[15] L. Zhang and D. Zhang, "Evolutionary cost-sensitive extreme learning machine", IEEE transactions on neural networks and learning systems, vol. 28, no. 12, pp. 3045-3060, 2016.
[16] C. Elkan, "The foundations of cost-sensitive learning", in International joint conference on artificial intelligence, vol. 17, no. 1, pp. 973-978: Lawrence Erlbaum Associates Ltd, 2001.
[17] B. Krawczyk, L. L. Minku, J. Gama, J. Stefanowski, and M. Woźniak, "Ensemble learning for data stream analysis: A survey", Information Fusion, vol. 37, pp. 132-156, 2017.
[18] H. R. Bonab and F. Can, "A theoretical framework on the ideal number of classifiers for online ensembles in data streams", in Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, pp. 2053-2056, 2016.
[19] E. Menahem, A. Shabtai, L. Rokach, and Y. Elovici, "Improving malware detection by applying multi-inducer ensemble", Computational Statistics & Data Analysis, vol. 53, no. 4, pp. 1483-1494, 2009.
[20] L. Deng and J. C. Platt, "Ensemble deep learning for speech recognition", in Fifteenth Annual Conference of the International Speech Communication Association, 2014.
[21] P. Rebentrost, M. Mohseni, and S. Lloyd, "Quantum support vector machine for big data classification", Physical review letters, vol. 113, no. 13, p. 130503, 2014.
[22] D. Peralta, S. Del Río, S. Ramírez-Gallego, I. Triguero, J. M. Benitez, and F. Herrera, "Evolutionary feature selection for big data classification: A mapreduce approach", Mathematical Problems in Engineering, vol. 2015, 2015.
[23] J. Dean and S. Ghemawat, "MapReduce: simplified data processing on large clusters", Communications of the ACM, vol. 51, no. 1, pp. 107-113, 2008.
[24] P. Martre et al., "Multimodel ensembles of wheat growth: many models are better than one", Global change biology, vol. 21, no. 2, pp. 911-925, 2015.
[25] C. Tebaldi and R. Knutti, "The use of the multi-model ensemble in probabilistic climate projections", Philosophical transactions of the royal society A: mathematical, physical and engineering sciences, vol. 365, no. 1857, pp. 2053-2075, 2007.
[26] S. S. Al-Amri, N. Kalyankar, and S. Khamitkar, "Image segmentation by using edge detection", International journal on computer science and engineering, vol. 2, no. 3, pp. 804-807, 2010.
[27] M. Hofmann, P. Tiefenbacher, and G. Rigoll, "Background segmentation with feedback: The pixel-based adaptive segmenter", in 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pp. 38-43: IEEE, 2012.
[28] M. Babaee, D. T. Dinh, and G. Rigoll, "A deep convolutional neural network for video sequence background subtraction", Pattern Recognition, vol. 76, pp. 635-649, 2018.
[29] E. Dougherty, Mathematical morphology in image processing. CRC press, 2018.
[30] A. D. El Maliani, M. El Hassouni, Y. Berthoumieu, and D. Aboutajdine, "Color texture classification method based on a statistical multi-model and geodesic distance", Journal of Visual Communication and Image Representation, vol. 25, no. 7, pp. 1717-1725, 2014.
[31] T. Ojala, M. Pietikainen, and D. Harwood, "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions", in Proceedings of 12th International Conference on Pattern Recognition, vol. 1, pp. 582-585: IEEE, 1994.
[32] T. Ojala, M. Pietikainen, T. Maenpaa, and m. intelligence, "Multiresolution gray-scale and rotation invariant texture classification with local binary patterns", in IEEE Transactions on pattern analysis, vol. 24, no. 7, pp. 971-987, 2002.
[33] M. A. Rahim, M. S. Azam, N. Hossain, and M. R. Islam, "Face recognition using local binary patterns (LBP)", Global Journal of Computer Science and Technology, 2013.
[34] S. Malathi and C. Meena, "An efficient method for partial fingerprint recognition based on local binary pattern", in 2010 INTERNATIONAL CONFERENCE ON COMMUNICATION CONTROL AND COMPUTING TECHNOLOGIES, pp. 569-572: IEEE, 2010.
[35] H.-D. Cheng, X. Jiang, and J. Wang, "Color image segmentation based on homogram thresholding and region merging", Pattern recognition, vol. 35, no. 2, pp. 373-393, 2002.
[36] K. S. Tan and N. A. M. Isa, "Color image segmentation using histogram thresholding–Fuzzy C-means hybrid approach", Pattern Recognition, vol. 44, no. 1, pp. 1-15, 2011.
[37] M. J. Swain and D. H. Ballard, "Color indexing", International journal of computer vision, vol. 7, no. 1, pp. 11-32, 1991.
[38] A. Lieb, "Color indexing and searching for images," ed: Google Patents, 2008.
[39] R. Hettiarachchi and J. F. Peters, "Voronoï region-based adaptive unsupervised color image segmentation", Pattern Recognition, vol. 65, pp. 119-135, 2017.
[40] N. Z. N. Rashid, M. Y. Mashor, and R. Hassan, "Unsupervised color image segmentation of red blood cell for thalassemia disease", in 2015 2nd International Conference on Biomedical Engineering (ICoBE), pp. 1-6: IEEE, 2015.
[41] S. Hinz, "Fast and subpixel precise blob detection and attribution", in IEEE International Conference on Image Processing 2005, vol. 3, pp. III-457: IEEE, 2005.
[42] P. Telagarapu, M. N. Rao, and G. Suresh, "A novel traffic-tracking system using morphological and Blob analysis", in 2012 International Conference on Computing, Communication and Applications, pp. 1-4: IEEE, 2012.
[43] T. Ganokratanaa and S. Pumrin, "The vision-based hand gesture recognition using blob analysis", in 2017 International Conference on Digital Arts, Media and Technology (ICDAMT), pp. 336-341: IEEE, 2017.
[44] N. Zulpe and V. Pawar, "GLCM textural features for brain tumor classification", International Journal of Computer Science Issues, vol. 9, no. 3, p. 354, 2012.
[45] X. Zhang, J. Cui, W. Wang, and C. Lin, "A study for texture feature extraction of high-resolution satellite images based on a direction measure and gray level co-occurrence matrix fusion algorithm", Sensors, vol. 17, no. 7, p. 1474, 2017.
[46] B. Sebastian V, A. Unnikrishnan, and K. Balakrishnan, "Gray level co-occurrence matrices: generalisation and some new features", arXiv preprint arXiv:.1205.4831, 2012.
[47] N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection", in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), vol. 1, pp. 886-893: IEEE, 2005.
[48] Y. Zhao, Y. Zhang, R. Cheng, D. Wei, and G. Li, "An enhanced histogram of oriented gradients for pedestrian detection", IEEE Intelligent Transportation Systems Magazine, vol. 7, no. 3, pp. 29-38, 2015.
[49] S. B. Maind and P. Wankar, "Research paper on basic of artificial neural network", International Journal on Recent and Innovation Trends in Computing and Communications of the ACM, vol. 2, no. 1, pp. 96-100, 2014.
[50] F. Rosenblatt, "The perceptron: a probabilistic model for information storage and organization in the brain", Psychological review, vol. 65, no. 6, p. 386, 1958.
[51] S. S. Dahikar and S. V. Rode, "Agricultural crop yield prediction using artificial neural network approach", International journal of innovative research in electrical, electronics, instrumentation and control engineering, vol. 2, no. 1, pp. 683-686, 2014.
[52] A. Graves, A.-r. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks", in 2013 IEEE international conference on acoustics, speech and signal processing, pp. 6645-6649: IEEE, 2013.
[53] E. Hodo et al., "Threat analysis of IoT networks using artificial neural network intrusion detection system", in 2016 International Symposium on Networks, Computers and Communications (ISNCC), pp. 1-6: IEEE, 2016.
[54] D. F. Specht, "Probabilistic neural networks", Neural networks, vol. 3, no. 1, pp. 109-118, 1990.
[55] T. Kohonen, "The self-organizing map", Proceedings of the IEEE, vol. 78, no. 9, pp. 1464-1480, 1990.
[56] C.-H. Chen, M.-Y. Lin, and X.-C. Guo, "High-level modeling and synthesis of smart sensor networks for Industrial Internet of Things", Computers and Electrical Engineering, vol. 61, pp. 48-66, 2017.
[57] A. Presley and D. H. Liles, "The use of IDEF0 for the design and specification of methodologies", in Proceedings of the 4th industrial engineering research conference, 1995.
[58] R. David, "Grafcet: A powerful tool for specification of logic controllers", IEEE transactions on control systems technology, vol. 3, no. 3, pp. 253-268, 1995.
[59] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks", in Advances in neural information processing systems, pp. 1097-1105, 2012.
[60] K. Simonyan, A. Vedaldi, and A. Zisserman, "Deep inside convolutional networks: Visualising image classification models and saliency maps", arXiv preprint arXiv:1312.6034, 2013.
[61] C. Szegedy et al., "Going deeper with convolutions", in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1-9, 2015.
[62] K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition", in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.